Skip to content

karpathy-pattern.mdx

Latest commit

 

History

History
72 lines (53 loc) · 4.41 KB

karpathy-pattern.mdx

File metadata and controls

72 lines (53 loc) · 4.41 KB
Raw
title Karpathy's LLM Wiki Pattern and llmwiki
sidebarTitle Karpathy Pattern
description How llmwiki implements Andrej Karpathy's LLM Wiki pattern: compile raw sources into durable, interlinked, citation-traceable knowledge before agents query it.

Andrej Karpathy's LLM Wiki pattern is a simple but important shift: do not make every agent rediscover knowledge from raw data at query time. Compile the knowledge first, then let agents query, inspect, and reuse the compiled artifact.

llmwiki is a concrete implementation of that pattern. It turns raw source material into a persistent markdown wiki with typed pages, wikilinks, citations, review state, freshness metadata, retrieval indexes, and export formats.

The Core Idea

Traditional RAG treats the source corpus as the primary artifact. Each question retrieves chunks, feeds them to a model, and asks the model to reconstruct the relevant relationships on the fly. That works well for ad-hoc retrieval, but the discovered structure is usually thrown away after the answer.

The LLM Wiki pattern treats compiled knowledge as the primary artifact. Sources are still the ground truth, but they are first transformed into durable pages:

  • Concepts get stable pages.
  • Related pages link to each other.
  • Shared ideas from multiple sources merge instead of competing as duplicate chunks.
  • Citations point back to the source material.
  • Later queries operate over the compiled wiki, not only the raw files.

How llmwiki Implements It

llmwiki makes the pattern operational with a two-phase compile pipeline:

Each changed source is analyzed for the concepts it contains. Extraction finishes across the source set before pages are written, so the compiler can see cross-source overlaps. The compiler writes markdown pages for concepts, entities, comparisons, and overview pages. Concepts that appear in more than one source are merged into one page with combined provenance. llmwiki updates the index, embeddings, activity journal, viewer snapshot, context packs, MCP tools, and export formats around the compiled wiki.

Why Compile Before Querying

Compiling first creates leverage that raw retrieval does not naturally provide:

  • Durability - pages remain on disk, can be inspected, reviewed, edited, exported, and versioned.
  • Accumulation - saved query answers and newly ingested sources enrich the wiki over time.
  • Citation tracing - paragraphs and claims point back to source files and line ranges.
  • Reviewability - risky pages can be held as candidates before they become live context.
  • Freshness tracking - source changes can mark compiled pages stale or orphaned.
  • Interoperability - compiled knowledge can be exported to JSON, JSON-LD, GraphML, Marp, llms.txt, and Open Knowledge Format.
llmwiki still uses retrieval. The difference is that retrieval runs over the compiled wiki: semantic chunk search, BM25 reranking, and wikilink-graph expansion select evidence from a structured artifact.

When This Pattern Fits

The pattern is strongest when the same knowledge will be reused: research corpora, engineering docs, team handbooks, source packs, standards, decision records, support knowledge, or long-running agent projects.

It is less useful for throwaway questions over fast-changing data where there is no benefit to preserving structure. In those cases, plain search or traditional RAG may be enough.

RAG vs. llmwiki

Traditional RAG llmwiki
Primary artifact Raw chunks Compiled wiki pages
Query behavior Reconstructs context each time Reuses prior compiled structure
Knowledge growth Usually ephemeral Compounds through pages and saved queries
Cross-source concepts Often duplicate or compete Merge into shared pages
Agent interface Retrieved snippets Context packs, MCP tools, viewer, exports
Provenance Chunk references Paragraph and claim citations
Freshness Usually external Built into lint/status/viewer/context surfaces

llmwiki is complementary to RAG, not a rejection of it. Use RAG for ad-hoc lookup. Use llmwiki when compiled, inspectable, citation-traceable knowledge is the thing you want to keep.