| id | TD-029 | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| title | On-device weight RL / fine-tuning is out of scope — RAM and the Intel Arc training toolchain block it | ||||||||
| status | accepted | ||||||||
| date | 2026-06-22 | ||||||||
| supersedes | |||||||||
| superseded_by | |||||||||
| tags |
|
The recurring "let's RL the model so it gets better at our repos" idea. This TD records why training the weights on this laptop is rejected — so the same proposal isn't re-litigated — and what the realistic adaptation path is if it's ever wanted. The headline reasons aren't the usual ones ("costly", "catastrophic forgetting"); on this box two harder blockers come first.
This machine has 30 GiB unified RAM and inference already sits near the ceiling (TD-006: ~18 GB weights + KV + OS). Training adds, on top of the weights:
| Path | Resident footprint | On 30 GiB? |
|---|---|---|
| Full FT of a 30B (fp16) | weights + grads + Adam(2×) + activations ≈ 4–6× weights → hundreds of GB | ❌ no |
| RL/PPO on a 30B | policy + reference + reward/value models resident together + the above | ❌ no |
| QLoRA SFT of a 30B | 4-bit base (~18 GB) + LoRA grads/optim + activations |
RL is strictly heavier than SFT (multiple models resident); if even QLoRA SFT of the 30B is borderline, on-device RL on it is not in question.
Carbon is an inference orchestrator (TD-016). Its engines do not train:
- llama.cpp (IPEX + SYCL): inference only.
- OpenVINO GenAI: inference only.
- IPEX-LLM: inference, and Intel archived it Jan 2026 (per the README).
A real training path on the Arc 140T means PyTorch on Intel GPU (torch-xpu / IPEX training),
which is immature and unvalidated on this exact silicon. So even the "feasible" option (LoRA SFT
of a small model) is a toolchain build-out, not a config flag — a different kind of project from
"thin orchestrator over host inference".
Do not implement on-device weight RL or fine-tuning. If model adaptation is ever genuinely needed, the realistic path is offline: LoRA/QLoRA SFT (not RL) of a small model on a bigger box / CUDA, merge, export to GGUF/OV IR, and serve the result through Carbon's normal inference path. Carbon's role stays serving, not training.
The interesting optimization target for a local agent isn't the weights anyway — it's the system around the model (TD-030): context selection, tool strategy, memory. That's tractable on this laptop with the real tasks already run; weight RL is not. "The model is the processor; optimize the OS."
- Positive: avoids a doomed GPU-training subsystem on unsupported silicon; keeps Carbon an inference orchestrator.
- Negative: the served model's raw weights never specialize to the user's repos — accepted; the TD-030 trace-learning loop captures the specialization that matters, with the model byte-identical.