Design: workflow-oriented tool redesign + MCP 2025-11-25 feature adoption

[Macawls]

Why this issue

v1.5.0 shipped Umami Cloud support, which opens the obvious next question: which of the ~30+ Umami API endpoints should we expose as MCP tools? A naive answer is "roughly 13 new tools, one per endpoint we don't cover yet" (daterange, realtime, events/series, events/stats, event-data/events, event-data/fields, event-data/values, event-data/stats, sessions, sessions/stats, sessions/weekly, funnel, retention, journey, …).

After reviewing Anthropic's "Writing tools for agents" and the MCP 2025-11-25 spec, I think that's the wrong instinct. Good MCP servers are not JSON-RPC proxies over an API — they're workflow surfaces designed around how an agent actually subdivides a task. Filing this to align on the shape before writing code.

---

1. Audit — where we are today

1a. Tools already exposed (5)

Tool	Umami endpoint	Notes
get_websites	GET /websites	Thin wrapper
get_stats	GET /websites/:id/stats	Thin wrapper; no filters; no compare
get_pageviews	GET /websites/:id/pageviews	Thin wrapper; no filters; no compare; no timezone
get_metrics	GET /websites/:id/metrics	Thin wrapper; no filters
get_active	GET /websites/:id/active	Thin wrapper

1b. Endpoints we don't expose (grouped by category)

Website stats family

• GET /websites/:id/daterange — available data range (mindate / maxdate)
• GET /websites/:id/metrics/expanded — metrics + engagement fields
• GET /websites/:id/events/series — event counts over time

Events family

• GET /websites/:id/events — raw event rows, paginated
• GET /websites/:id/events/stats — aggregated event counts with compare
• GET /websites/:id/event-data/events — event names + properties + counts
• GET /websites/:id/event-data/fields — property schema discovery
• GET /websites/:id/event-data/values?event=&propertyName= — value distribution for a property on an event
• GET /websites/:id/event-data/stats — event-data summary

Sessions family

• GET /websites/:id/sessions — paginated session list
• GET /websites/:id/sessions/stats
• GET /websites/:id/sessions/weekly — hour-of-weekday heatmap
• GET /websites/:id/sessions/:sid + /activity + /properties — per-session drill-down
• GET /websites/:id/session-data/properties
• GET /websites/:id/session-data/values?propertyName=

Realtime

• GET /realtime/:id — 30-minute rolling snapshot (countries, urls, referrers, events, timeseries). Much richer than /active.

Reports (POST, JSON body)

• POST /reports/funnel — conversion drop-off across ordered steps
• POST /reports/retention — cohort return-rate
• POST /reports/journey — path analysis between start/end
• POST /reports/attribution — marketing attribution
• POST /reports/breakdown — multi-dimensional segmentation
• POST /reports/goal — pageview / event conversion
• POST /reports/revenue — currency + conversion tracking
• POST /reports/utm — campaign performance
• POST /reports/performance — Core Web Vitals

Filter params supported by every stats-like endpoint that our tools ignore: url, referrer, title, query, host, os, browser, device, country, region, city, language, tag, event, distinctId, utm_source, utm_medium, utm_campaign, utm_content, utm_term, plus compare and timezone on time-series endpoints. All missing from our tool schemas.

1c. Symptoms that the current design hasn't absorbed the workflow

Re-reading tools.json with fresh eyes, the descriptions are doing too much work:

• get_websites says "Always call this FIRST before any analytics queries" — a workflow hint that shouldn't need to live in a description
• get_stats says "verify that X days ago is after createdAt — if not, adjust start_date to createdAt" — asking the LLM to do calendar math to avoid empty results
• Every dated tool accepts "ISO 8601 date strings or Unix timestamps in milliseconds (13 digits)" — two formats to choose between, plus a warning about timestamp length
• get_metrics description enumerates 20 metric types in prose because the enum list isn't self-explanatory

These aren't description problems — they're tool-shape problems showing through the description.

---

2. What the research says

2a. Anthropic — "Writing tools for agents"

Direct lessons applicable here:

• "Build a few thoughtful tools targeting specific high-impact workflows" rather than exposing every endpoint. Excessive tools distract agents from efficient strategies.

• "Even small refinements to tool descriptions can yield dramatic improvements." Descriptions should clarify niche terminology and use unambiguous parameter names.

• Offer a response_format enum (e.g. concise / detailed) so the agent can control token spend for downstream calls.
• Error messages should be "actionable improvements" that guide the agent toward a fix — pagination, filter hints, valid IDs — not opaque status codes.
• Pagination, filtering, and truncation with sensible defaults remain important even as context windows grow.
• Mirror how humans subdivide tasks, not raw technical operations.

2b. MCP Specification 2025-11-25 — features we aren't using

The current tools.json schema is essentially {name, description, inputSchema}. The spec defines considerably more:

Feature	What it does	Current status
title	Human-readable tool name distinct from machine name	Not set
outputSchema	JSON Schema describing the structured return shape	Not set
structuredContent	Structured JSON alongside the content text block, enabling strict validation and type info for clients	Not returned
annotations.readOnlyHint	Marks the tool as non-mutating	Not set (all ours are read-only)
annotations.idempotentHint	Same input → same output	Not set
annotations.openWorldHint	Signals whether the tool reaches beyond a well-known system	Not set
annotations.destructiveHint	Marks destructive operations	N/A (we have none)
icons	Per-tool icons for UIs	Not set
isError: true in result	Canonical way to surface API errors — the LLM sees a human-readable message and can self-correct	We return raw JSON-RPC errors, which per the spec are less likely to be recoverable
Resources	First-class; let clients read data without a tool call	Not used. The website list is a natural umami://websites/{id} resource
Prompts	First-class; parametric workflows	We ship prompts.json but underuse them — no workflow prompts like "weekly traffic summary"

---

3. Proposed redesign

3a. Replace the 5 thin wrappers with ~7 workflow tools

1. get_websites — kept. Also expose each website as an MCP resource (umami://websites/{id}) so clients that support resources can read the list without burning a tool call.

2. get_overview — the primary analytics tool. Covers "how is my site doing?" in a single call by composing /daterange + /stats (with comparison) + /active + /realtime + /pageviews timeseries + top metric breakdowns.

inputSchema:
  website_id: string (required)
  period: enum [last_hour, today, yesterday, last_7_days, last_30_days, this_month, last_month, custom] (default: last_7_days)
  start_date / end_date: ISO 8601 (required only if period=custom)
  sections: multiselect [stats, timeseries, top_pages, top_referrers, top_countries, top_browsers, realtime] (default: [stats, timeseries, top_pages, top_referrers])
  response_format: enum [concise, detailed] (default: concise)

Internally uses /daterange to auto-clamp the requested period against the site's actual data range, so "last_30_days" works even on a 5-day-old site without the LLM doing arithmetic. Single call answers roughly 80% of questions.

3. get_metric_breakdown — dedicated dimension explorer. Replaces the bare get_metrics with something that actually supports the filter params the API offers.

inputSchema:
  website_id
  dimension: enum [path, referrer, browser, os, device, country, region, city, language, screen, event, host, title, tag, utm_source, utm_medium, utm_campaign, entry, exit, channel]
  period / custom dates
  filters: object { path?, referrer?, country?, browser?, os?, device?, host?, event?, tag?, utm_* }
  compare: boolean
  limit: int (default 20)

4. explore_events — one tool covering the entire event-data/* family with a mode enum that mirrors the actual discovery workflow:

mode: enum
  - list_events     -> GET /event-data/events
  - list_fields     -> GET /event-data/fields
  - values_for      -> GET /event-data/values (requires event + property_name)

Drives the natural flow: "what events exist? → which properties do they have? → what values does plan take on signup?"

5. get_funnel — POST /reports/funnel with ordered steps[] + window + filters.

6. get_retention — POST /reports/retention with cohort config.

7. get_journey — POST /reports/journey between defined start/end steps.

3b. Independent improvements (apply regardless of which tools exist)

These are non-breaking and should ship first:

• Tool annotations on every tool: readOnlyHint: true, idempotentHint: true, openWorldHint: false (the data is scoped to the configured Umami instance).
• title fields on every tool for human-readable display (e.g. "Umami — Website Overview").
• outputSchema on every tool. Define concrete schemas for Website, Stats, Metric, PageView, etc. so clients can validate and LLMs know the shape without parsing prose.
• Return structuredContent alongside the text block, per the spec. Text stays for backwards compatibility; structured JSON is the primary channel.
• Drop Unix-ms date acceptance. ISO 8601 only, plus the period preset enum for natural-language shortcuts. One format, zero ambiguity.
• Actionable errors via isError: true. Replace generic API error 404: {website X} with:
  • Website abc123 not found. Call get_websites to list valid IDs.
  • start_date 2020-01-01 is before the site's data range (mindate: 2025-01-01). Adjust to 2025-01-01 or use period=last_30_days.
• MCP prompts for common workflows:
  • weekly_summary(website_id) — renders a full traffic summary prompt
  • diagnose_drop(website_id, period) — compares against previous period and highlights losses
  • funnel_diagnosis(website_id, funnel_name) — narrates drop-off
• Resources — expose each website as umami://websites/{id}. Clients with resource support avoid the tool-call roundtrip.

---

4. Out of scope (revisit on user demand)

• Admin writes (create/update/delete websites, teams, users, reports). This is a read-focused analytics surface.
• Raw per-row GET /events and GET /sessions listings — they drown an LLM in tokens. If an LLM needs them, paginated + heavily-filtered access through explore_events / a new search_sessions is a better shape.
• Niche report types: attribution, breakdown, revenue, performance. Add on request.
• Per-session drill-down (/sessions/:sid/activity, /properties). Add when someone asks for investigative workflows.

---

5. Shipping order

1. Non-breaking MCP feature adoption. Annotations, title, outputSchema, structuredContent, actionable isError responses. No tool surface change. Low risk, compounding value for every tool that exists now and every tool we add later.
2. Value bump. Ship get_overview and explore_events. Add filter / compare / timezone params to the existing tools.
3. Reports. Add get_funnel, get_retention, get_journey. Requires a doRequest variant that accepts a POST body.
4. Deprecate + remove get_stats / get_pageviews / get_active once get_overview has covered their ground in production for a release or two.

---

6. Open questions

1. Shape of get_overview. Is one composite tool with a sections multiselect the right call, or should it stay as 4-5 narrower tools (get_stats, get_timeseries, get_realtime, get_top) that the LLM composes itself? The composite is fewer roundtrips and carries the daterange-clamping logic server-side; the narrow set is easier to describe and cache.
2. Which Tier-3 reports are worth the surface area? funnel / retention / journey feel obviously useful. attribution / breakdown / revenue / utm / performance are a judgement call.
3. Backwards compatibility. The thin-wrapper tools are already in the wild. Deprecate with a release-long warning period, or break at v2.0.0?

Feedback welcome — especially on (1). I'd rather argue about the shape now than ship twice.

---

References

• Writing tools for agents — Anthropic
• Model Context Protocol specification 2025-11-25 — Tools
• Umami API documentation