Tag
5 catalogue entries use this topic; each Project's license policy is labeled separately.
Run open language models locally with one command
Ollama reduced local model hosting from an afternoon of CUDA troubleshooting to a single pull command. It handles model download, quantisation selection, GPU offloading, and memory management, and exposes an OpenAI-compatible endpoint on localhost.
Inference engine that runs LLMs on almost any hardware
llama.cpp is the layer most local model tooling is built on. It is a C and C++ inference engine with no Python runtime, aggressive quantisation support, and hand-tuned kernels for a remarkably wide range of hardware.
Speech recognition that runs anywhere, with no runtime
whisper.cpp is a C and C++ port of the Whisper speech recognition model with no Python dependency, quantised weights, and hardware acceleration across Apple Silicon, CUDA, and plain CPU.
Local voice dictation with AI cleanup for macOS
VoiceInk transcribes speech on the machine itself using a local Whisper model, then runs the raw transcript through a cleanup pass that removes filler and applies punctuation.
Desktop application for running language models locally
LM Studio made local models approachable for people who do not want a terminal. Browse a catalogue, download a quantisation that fits the machine, and chat, with hardware requirements explained in plain language.