Skip to content

FINDINGS.md

Latest commit

 

History

History
62 lines (42 loc) · 2.27 KB

FINDINGS.md

File metadata and controls

62 lines (42 loc) · 2.27 KB
Raw

Finding: Unlimited-OCR MXFP8 needs model_type: unlimited-ocr on mlx-vlm ≥ 0.6

Summary

sahilchachra/unlimited-ocr-mxfp8-mlx ships with:

  • config.json → "model_type": "deepseekocr"
  • processor_config.json → "processor_class": "DeepseekOCRProcessor", "sft_format": "deepseek"

That shim was correct for older mlx-vlm (before a native Unlimited-OCR module existed). On mlx-vlm 0.6+, which registers unlimited-ocr → mlx_vlm.models.unlimited_ocr, loading the Hub pack as deepseekocr produces degenerate / repetitive garbage (hits max_tokens, wrong BPE stream).

Fix

Patch before mlx_vlm.load:

// config.json
"model_type": "unlimited-ocr"

// processor_config.json
"processor_class": "UnlimitedOCRHFProcessor"
"sft_format": "unlimitedocr"

Weights are unchanged (MXFP8). Only routing / processor class changes.

Working inference (mlx-vlm 0.6.8, Apple M5)

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("path/to/patched-mxfp8")
config = load_config("path/to/patched-mxfp8")
prompt = apply_chat_template(processor, config, "Free OCR.", num_images=1)
out = generate(model, processor, prompt=prompt, image="page.png",
               max_tokens=4096, temperature=0.0, repetition_penalty=1.05)
# Post-process SentencePiece glyphs if needed: Ġ → space, Ċ → newline

Default prompt document parsing. often emits EOS immediately on this path; Free OCR. works reliably.

Measured (smoke PDF, M5 / 16 GB)

Backend Wall time Peak mem Result
PyTorch MPS bf16 ~17 s ~7 GB class Correct
MLX MXFP8 (patched unlimited-ocr) ~2.2 s ~5.8 GB Correct
MLX MXFP8 (stock deepseekocr shim) ~37 s @ 4096 tok ~5.2 GB Garbage

Published pack

Patched configs + same MXFP8 weights:

Credit