Skip to content

Latest commit

 

History

History
76 lines (57 loc) · 3.67 KB

File metadata and controls

76 lines (57 loc) · 3.67 KB
id TD-029
title On-device weight RL / fine-tuning is out of scope — RAM and the Intel Arc training toolchain block it
status accepted
date 2026-06-22
supersedes
superseded_by
tags
rl
fine-tuning
lora
training
intel-arc
toolchain
ram-ceiling
scope

TD-029: On-device weight RL / fine-tuning is out of scope

Context

The recurring "let's RL the model so it gets better at our repos" idea. This TD records why training the weights on this laptop is rejected — so the same proposal isn't re-litigated — and what the realistic adaptation path is if it's ever wanted. The headline reasons aren't the usual ones ("costly", "catastrophic forgetting"); on this box two harder blockers come first.

Blocker 1 — RAM

This machine has 30 GiB unified RAM and inference already sits near the ceiling (TD-006: ~18 GB weights + KV + OS). Training adds, on top of the weights:

Path Resident footprint On 30 GiB?
Full FT of a 30B (fp16) weights + grads + Adam(2×) + activations ≈ 4–6× weights → hundreds of GB ❌ no
RL/PPO on a 30B policy + reference + reward/value models resident together + the above ❌ no
QLoRA SFT of a 30B 4-bit base (~18 GB) + LoRA grads/optim + activations ⚠️ tight→impossible, and it's SFT, not RL

RL is strictly heavier than SFT (multiple models resident); if even QLoRA SFT of the 30B is borderline, on-device RL on it is not in question.

Blocker 2 — the Intel Arc training toolchain (the Carbon-specific killer)

Carbon is an inference orchestrator (TD-016). Its engines do not train:

  • llama.cpp (IPEX + SYCL): inference only.
  • OpenVINO GenAI: inference only.
  • IPEX-LLM: inference, and Intel archived it Jan 2026 (per the README).

A real training path on the Arc 140T means PyTorch on Intel GPU (torch-xpu / IPEX training), which is immature and unvalidated on this exact silicon. So even the "feasible" option (LoRA SFT of a small model) is a toolchain build-out, not a config flag — a different kind of project from "thin orchestrator over host inference".

Decision

Do not implement on-device weight RL or fine-tuning. If model adaptation is ever genuinely needed, the realistic path is offline: LoRA/QLoRA SFT (not RL) of a small model on a bigger box / CUDA, merge, export to GGUF/OV IR, and serve the result through Carbon's normal inference path. Carbon's role stays serving, not training.

Rationale

The interesting optimization target for a local agent isn't the weights anyway — it's the system around the model (TD-030): context selection, tool strategy, memory. That's tractable on this laptop with the real tasks already run; weight RL is not. "The model is the processor; optimize the OS."

Consequences

  • Positive: avoids a doomed GPU-training subsystem on unsupported silicon; keeps Carbon an inference orchestrator.
  • Negative: the served model's raw weights never specialize to the user's repos — accepted; the TD-030 trace-learning loop captures the specialization that matters, with the model byte-identical.

Related

  • TD-030 (what we optimize instead), TD-031 (the loop that makes it possible), TD-028 / TD-003 (the pattern: record the dead-end so it isn't re-proposed).