Skip to content

docs(use-dkg): large-content guidance — RDF literal limit, chunk/externalize, roadmap#1527

Open
branarakic wants to merge 1 commit into
mainfrom
docs/large-content-guidance
Open

docs(use-dkg): large-content guidance — RDF literal limit, chunk/externalize, roadmap#1527
branarakic wants to merge 1 commit into
mainfrom
docs/large-content-guidance

Conversation

@branarakic

Copy link
Copy Markdown
Contributor

Docs-only. The public-facing half of the oversize-literal remediation (internal spec: OT-RFC-56 in the private spec repo — deliberately not linked from here).

Adds docs/use-dkg/large-content.md:

  • the 60,000-byte RDF literal limit and the honest why (Blazegraph ~64KB hard cap → cross-backend storability; replication health),
  • what users see (OVERSIZED_RDF_LITERAL, HTTP 400, with subject/predicate/bytes),
  • the two supported patterns today: ordered chunking, or externalize with URI + SHA-256 + byte size + excerpt (hash keeps it verifiable; for KAs the hash is covered by the on-chain commitment),
  • a roadmap note: first-class external content storage (pointer triples + content-addressed blobs) is planned.

Indexed in docs/SUMMARY.md + docs/use-dkg/README.md. Follow-up (remediation PR stack): producer rejection error messages cite this page.

🤖 Generated with Claude Code

…ize patterns, roadmap

Public-facing page for the 60,000-byte RDF literal limit (OVERSIZED_RDF_LITERAL):
why it exists (cross-backend storability + replication health), the two
supported patterns today (ordered chunking; externalize with URI + SHA-256 +
size + excerpt), and the roadmap note for first-class external content
storage. Producer rejection errors will cite this page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Writes containing an oversized literal are rejected with a structured
`OVERSIZED_RDF_LITERAL` error (HTTP 400 on the API routes). The error names the
subject, predicate, and byte size so you can locate the offending value. This
applies to publishes, updates, imports, shared-memory writes, and context-graph

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Context-graph metadata is documented as guarded, but the create path can bypass the 60 KB check

What's wrong
The new page turns the oversized-literal behavior into a public contract for context-graph names/descriptions, but the context-graph create flow does not use the 60,000-byte guard before inserting those metadata triples. That means callers can see different behavior from what the docs promise, and on mixed backends the metadata may still hit backend-specific failure behavior instead of a consistent API rejection.

Example
POST /api/context-graph/create with a 60,001-byte description follows the context-graph create path. On the embedded Oxigraph store, there is no 60,000-byte validation in that path, so it can be accepted instead of returning the documented structured OVERSIZED_RDF_LITERAL HTTP 400.

Suggested direction
Align the docs and implementation: either remove context-graph registration metadata from this guarantee, or validate generated name/description quads before insertion and map OVERSIZED_RDF_LITERAL to the same structured 400 response.

For Agents
Either narrow the docs to the write paths that actually call assertQuadLiteralsMutf8Safe, or add the same literal-size validation to context-graph create/register metadata before store.insert. Preserve the documented HTTP 400 body shape and prove it with a route-level context-graph create test using an oversized description/name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants