LM Studio 0.4.0

Summary

Today we are thrilled to share LM Studio 0.4.0, the next generation of LM Studio. This release introduces parallel requests with continuous batching for high throughput serving, all-new non-GUI deployment option, new stateful REST API, and a refreshed user interface. LM Studio 0.4.0 highlights include: Deploy LM Studio's core on cloud servers, in CI, or anywhere without GUI. Parallel requests to the same model with continuous batching (instead of queueing). New stateful REST API endpoint: /v1/chat that allows using local MCPs. Refreshed application UI with chat export, split view, developer mode, and in-app docs. Read on for more details! Deploy on servers, deploy in CI, deploy anywhere Today we're introducing llmster: it's the core of the LM Studio desktop app, but packaged to be server-native, without reliance on the GUI. We've rearchitected our software to separate the GUI from the core functionality, allowing llmster to run as a standalone daemon. This means llmster can be run completely independently of the app and deployed anywhere: Linux boxes, cloud servers, your GPU rig, or even Google Colabs. It can of course still be run on your local machine without the GUI, for those who prefer terminal-based workflows. How to install llmster Linux / Mac curl -fsSL https://lmstudio.ai/install.sh | bash Windows irm https://lmstudio.ai/install.ps1 | iex Using llmster Start the daemon: lms daemon up Download a model: lms get <model> Start the local server: lms server start Open an interactive session: lms chat Update your runtime: lms runtime update llama.cpp (and lms runtime update mlx on macOS) Parallel Requests Alongside LM Studio 0.4.0, our llama.cpp engine is graduating to version 2.0.0. With it we're introducing support for concurrent inference requests to the same model. Your browser does not support the video tag. Run parallel requests in the app with Split View Max Concurrent Predictions and Unified KV Cache You will find 2 new load options in the model loader dia...