Skip to content

Document live views#479

Open
puzpuzpuz wants to merge 5 commits into
mainfrom
puzpuzpuz_live_view
Open

Document live views#479
puzpuzpuz wants to merge 5 commits into
mainfrom
puzpuzpuz_live_view

Conversation

@puzpuzpuz

@puzpuzpuz puzpuzpuz commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Documents the new live view feature from questdb/questdb#6939 (OSS) and questdb/questdb-enterprise#1087 (Enterprise). A live view incrementally maintains a window-function query over a single WAL-backed base table and can be queried like a regular table.

New pages:

  • concepts/live-views.md: what live views are and why, how they work (refresh vs flush cadences, in-memory tier, freshness model), supported window functions, anchored windows, backfill, base-table lifecycle, monitoring, limitations, tradeoffs, and Enterprise features (permissions, replication, backup/restore).
  • query/sql/create-live-view.md: CREATE LIVE VIEW reference covering FLUSH EVERY, IN MEMORY, PARTITION BY, BACKFILL, anchored windows, query constraints, and error messages.
  • query/sql/drop-live-view.md: DROP LIVE VIEW reference.
  • configuration/live-views.md: the cairo.live.view.* server settings.

Updated pages:

  • query/functions/meta.md: adds the live_views() catalogue function and the L (live view) table_type discriminator.
  • query/sql/show.md: adds SHOW CREATE LIVE VIEW.
  • security/rbac.md: adds the CREATE LIVE VIEW and DROP LIVE VIEW permissions.
  • concepts/views.md, configuration/overview.md, operations/backup.md, sidebars.js: cross-links, index rows, and navigation.

Facts (config property names/defaults, live_views() columns, SHOW CREATE output, validation errors) were verified against the source on the puzpuzpuz_live_view branch of both repositories.

Follow-up (not in this PR)

The questdb/sql-parser grammar needs new keywords for web-console highlighting/autocomplete: live, flush, backfill, anchor, and daily (memory is already covered by open PR #27; every and view already exist).

@puzpuzpuz @claude
Document live views (OSS and Enterprise)
c2fb238 
Adds documentation for the new live view feature: incrementally maintained
window-function results over a single WAL-backed base table, queried like a
regular table.

New pages:
- concepts/live-views.md: what live views are, how they work (refresh vs
  flush cadences, in-memory tier, freshness), supported window functions,
  anchored windows, backfill, base-table lifecycle, monitoring, limitations,
  tradeoffs, and Enterprise features.
- query/sql/create-live-view.md: CREATE LIVE VIEW reference (FLUSH EVERY,
  IN MEMORY, PARTITION BY, BACKFILL, anchored windows, constraints, errors).
- query/sql/drop-live-view.md: DROP LIVE VIEW reference.
- configuration/live-views.md: cairo.live.view.* server settings.

Updated pages:
- query/functions/meta.md: live_views() catalogue function; table_type 'L'.
- query/sql/show.md: SHOW CREATE LIVE VIEW.
- security/rbac.md: CREATE LIVE VIEW / DROP LIVE VIEW permissions.
- concepts/views.md, configuration/overview.md, operations/backup.md,
  sidebars.js: cross-links, index rows, and navigation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown

🚀 Build success!

Latest successful preview: https://preview-479--questdb-documentation.netlify.app/docs/

Commit SHA: fbf5029

📦 Build generates a preview & updates link on each commit.

@puzpuzpuz puzpuzpuz changed the title Document live views (OSS and Enterprise) Document live views Jul 9, 2026
@puzpuzpuz puzpuzpuz self-assigned this Jul 9, 2026
@puzpuzpuz puzpuzpuz added the documentation Improvements or additions to documentation label Jul 9, 2026
@puzpuzpuz puzpuzpuz mentioned this pull request Jul 9, 2026
Open
puzpuzpuz and others added 2 commits July 9, 2026 18:55
@puzpuzpuz @claude
Fix live-views docs review findings
9ec2c1b 
Address internal-contradiction and precision issues in the live-views pages,
verified against OSS PR questdb/questdb#6939:

- Relabel the comparison-table "Freshness control" row to "Freshness /
  durability"; FLUSH EVERY is a durability knob, not a freshness one.
- Make the flagship read example an unfiltered SELECT *, and note that
  timestamp-filtered reads trail by up to one FLUSH EVERY interval.
- Remove "truncating" from the base-table invalidation bullet: TRUNCATE is
  freeze-and-continue (walked past like DROP PARTITION / TTL), not an
  invalidation.
- Qualify the query constraint as a top-level ORDER BY; the window ORDER BY
  inside OVER(...) is required.
- Narrow "all view-level clauses precede AS" to the four pre-AS clauses; note
  OWNED BY follows the query.
- Add a note explaining the non-determinism error names "materialized view"
  because it reuses the shared guard (accurate server output).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@puzpuzpuz @claude
Use EMA/VWEMA terminology in live-views supported functions
a9eb6fc 
QuestDB's window-function docs name this capability EMA / VWEMA (modes of
avg()); "EWMA" appeared only here. Align the terminology for consistency
and searchability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jerrinot

jerrinot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

the "Live Views" page x-links Mat Views, but the Mat View page should cross link back to Live Views. As a user I find these 2 concepts very similar and they should cross-link each other.

@jerrinot

jerrinot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

the doc says:

A live view falling behind sustained ingestion stays correct but grows stale. There is no automatic throttle. Monitor lag_seqtxn and lag_micros in live_views() to detect a view that cannot keep up.

but there is no guidance about the semantics of lag_seqtxn. what does it represent? what number is good? how do I act on it?

@jerrinot

jerrinot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

the doc says:

Dropping, renaming, or changing the type of a referenced column invalidates the view.
Renaming or dropping the base table invalidates the view.
[...]
An invalidated view keeps serving its existing data and reports the reason in live_views(). It stops refreshing.

but it's missing a guidance: what should I do with an invalid live-views? how can I un-invalidate it?

@jerrinot

jerrinot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

the doc says:

No live-view-on-live-view. A live view cannot be the base of another live view.

Can I create a mat-view from a live view? and can I chain another live-view on such mat view? How about regular (non-materialized) views on top of live views? Can I create a live-view on top of such view?

@puzpuzpuz

puzpuzpuz commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@jerrinot I've addressed the review feedback in fbf5029:

  • Added reciprocal links from the materialized-view page, including guidance on when to choose a live view.
  • Expanded monitoring guidance for lag_seqtxn: it counts base WAL transactions rather than rows, 0 means the durable tier is caught up, temporary lag between flushes is normal, and sustained/increasing lag is the actionable signal. Also clarified that lag_micros is time since the last successful flush rather than a base/view timestamp difference, and documented remediation steps.
  • Documented that invalidation is permanent and requires saving the definition, repairing the base schema, then dropping and recreating the live view; BACKFILL is needed when retained base history should be rebuilt.
  • Replaced the narrow live-on-live limitation with a composition matrix. In particular: regular views can query live views; materialized views and other live views cannot use a live view as their base; regular views cannot be a live-view base; materialized views can be a live-view base, but a full materialized-view rebuild invalidates the dependent live view.
  • Added the missing base-object validation errors to the CREATE LIVE VIEW reference.

The production Docusaurus build passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@puzpuzpuz puzpuzpuz

Labels

documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@puzpuzpuz @jerrinot