A small, on-device model is fast and private, but sometimes wrong. At Cactus we post-train models to know when they are wrong: we ship probes inside the checkpoint that score every answer with a confidence between 0 and 1, returned as structured data (never parsed out of the answer text). Answer on-device when confidence is high; you can re-route to a bigger model when it's low: if confidence < 0.85: answer = ask_a_bigger_model(prompt) We start the rollout with Gemma 4 E2B Hybrid, all builds live in the Cactus Hybrid collection on Hugging Face. Gemma 4 E2B hybrid, the smallest Gemma model, matches Gemini 3.1 Flash-Lite on most benchmarks by routing only 15–35% of queries to the Gemini 3.1 Flash-Lite and running the remnant itself. Benchmark Handoff to match Flash-Lite (FP16) At 4-bit At 3-bit ChartQA 15–20% 25–30% 40–50% MMBench 30–35% 40–45% 50–55% LibriSpeech 25–30% 35–40% 55–65% GigaSpeech 30–35% 40–45% 50–55% MMAU 30–35% 35–40% 50–55% MMLU-Pro 45–55% ~90% n/a N/B: Quantisation quality is measured on Cactus Quants which performs well at uniform quantization. Developers are encouraged to benchmark for Unsloth, GGUF, and MLX quantization independently. # pip install cactus-compute import json from cactus.bindings.cactus import cactus_complete, cactus_init from cactus.cli.download import download_bundle lm = cactus_init(str(download_bundle("Cactus-Compute/gemma-4-E2B-it"))) result = cactus_complete( lm, [{"role": "user", "content": "What is the capital of France?"}], json.dumps({"max_tokens": 512, "auto_handoff": False}), None, lambda *_: None, ) print(result["response"].strip()) print("confidence:", result["confidence"]) # pip install mlx-lm import re from mlx_lm import load, generate model, tokenizer = load( "Cactus-Compute/gemma-4-e2b-it-hybrid-mlx", tokenizer_config={"trust_remote_code": True}, ) messages = [{"role": "user", "content": "What is the capital of France?"}] answer = generate( model, tokenizer, prompt=tokenizer.apply_chat_template(messages, add_gener...
First seen: 2026-07-22 22:52
Last seen: 2026-07-23 15:05