Skip to content
#

llama-cpp-alternative

Star

Here is 1 public repository matching this topic...

Language: Python
Filter by language
All 3 C 1 C++ 1 Python 1

konjoai / squish

Star 21

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

macos rust metal self-hosted quantization mlx inference-engine kv-cache apple-silicon openai-api int4 llm local-llm local-ai ollama qwen llm-infernece on-device-inference ollama-alternative llama-cpp-alternative
  • Updated Sep 28, 2026
  • Python

Add this topic to your repo

To associate your repository with the llama-cpp-alternative topic, visit your repo's landing page and select "manage topics."

Learn more