unlimited-ocr-mlx — Unlimited-OCR on Apple Silicon

Repo: github.com/will702/unlimited-ocr-mlx · Finding: FINDINGS.md · AI index: llms.txt

Model: mlx-community/Unlimited-OCR-mxfp8 — MXFP8 Unlimited-OCR with model_type=unlimited-ocr for mlx-vlm ≥ 0.6.

Upstream quant: sahilchachra/unlimited-ocr-mxfp8-mlx (same weights; old deepseekocr shim breaks on current mlx-vlm).

Fallback: PyTorch MPS via infer_mac.py (older path; slower on this Mac).

Requirements

• Apple Silicon Mac
• Python 3.12 + uv
• ~6 GB free unified memory for MXFP8
• First MLX run downloads ~3.7 GB of quantized weights

Setup

cd unlimited-ocr-mlx   # or your local clone path
uv venv .venv --python 3.12
source .venv/bin/activate
uv pip install -r requirements.txt

Optional local mirror:

python prepare_mlx_model.py

Run OCR (recommended)

source .venv/bin/activate
python infer_mlx.py path/to/doc.pdf

# Faster smoke / lower DPI
python infer_mlx.py samples/smoke.pdf --dpi 150

# Keep layout det boxes
python infer_mlx.py path/to/doc.pdf --grounding

Default Hub ID is mlx-community/Unlimited-OCR-mxfp8. Outputs: ./outputs/result.md

Verified on Apple M5 / 16 GB: smoke PDF OCR in ~2.2s, peak ~5.8 GB, text correct.

Quantization notes

Variant	Notes
MXFP8 (default)	Best balance: lighter/faster than FP16, better reported CER
Int8	Similar speed/memory, slightly worse CER than MXFP8
Int4 / MXFP4	Smaller/faster but worse CER — avoid unless you accept quality loss

Why the config fix?

sahilchachra’s Hub pack sets model_type: deepseekocr for older mlx-vlm. Current mlx-vlm (0.6+) has a real unlimited-ocr module; the shim produces garbage. The mlx-community pack ships with model_type: unlimited-ocr already.

PyTorch MPS fallback

source .venv/bin/activate
python prepare_mac_model.py   # once (~6.7 GB baidu weights + CUDA→MPS patch)
export PYTORCH_ENABLE_MPS_FALLBACK=1
python infer_mac.py path/to/doc.pdf --device mps --dtype bf16

About mlx-dspark

mlx-dspark is lossless speculative decoding for text LLMs (Qwen/Gemma), not OCR VLMs. It cannot wrap Unlimited-OCR.

Notes

• Book → LaTeX conversion is out of scope for this setup; OCR markdown is the deliverable.
• Do not use upstream CUDA / vLLM / SGLang on this Mac.