2.2 KiB
LMStudio
Allows running LLMs locally.
Considered the most accessible tool for local LLM deployment, particularly for users with no technical background.
TL;DR
Focused on single-user scenarios without built-in rate limiting or authentication.
Offers highly mature and stable OpenAI-compatible API.
Supports full streaming, embeddings API, experimental function calling for compatible models, and limited multimodal support.
Supports GGUF and Hugging Face Safetensors formats.
Has a built-in converter for some models, and can run split GGUF models.
Implements experimental tool calling support following the OpenAI function calling API format.
Models trained on function calling (e.g., Hermes 2 Pro, Llama 3.1, and Functionary) can invoke external tools through
the local API server. However, tool calling should not yet be considered suitable for production.
Streaming tool calls or advanced features like parallel function invocation are not currently supported.
Some models show better tool calling behavior than others.
The UI eases defining function schemas and test tool calls interactively
Considered ideal for:
- Beginners new to local LLM deployment.
- Users who prefer graphical interfaces over command-line tools.
- Developers needing good performance on lower-spec hardware (especially with integrated GPUs).
- Anyone wanting a polished professional user experience.
Setup
brew install --cask 'lm-studio'