Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .agents/knowledge/BACKLOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Knowledge backlog

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An initial scan of potentially missing knowledge entries seeded from my own personal agentic sessions and some initial PR investigations.

Each entry would be codifying potentially ambiguous or stylistic decision-making, and should be reviewed/discussed individually. I believe it to be the most important tool our team has in curbing the last ~20% of scenarios the base models+harnesses cannot do alone.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe .BACKLOG.md or __BACKLOG.md if we don't intend agents to read this, but may not matter.


Candidate agent-only entries, grouped by contributor-docs coverage. Each lands as its own reviewed PR.

## Not in the docs

- **casing-and-pluralization** — how API names map to Terraform names; when `api_name` is required.
- **output-only-test-assertions** — server-populated fields must have their values asserted in tests.
Comment thread
c2thorn marked this conversation as resolved.
Outdated
- **pr-mined-lessons** — recurring review catches and per-service quirks surfaced from merged PRs.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **pr-mined-lessons** — recurring review catches and per-service quirks surfaced from merged PRs.

(up to you) IMO cut as we'd use historical analysis to determine new rules of this type to cover, rather than the outputs of historical analysis being a single rule.


## Overlaps the docs (entry would add the judgment layer)

- **permadiff-decision-path** — choosing between `output`, `default_from_api`, diff suppression, or a real
fix; mechanics are in `docs/content/develop/diffs.md`.
- **data-source-idioms** — pitfalls beyond the procedure in `docs/content/develop/add-handwritten-datasource.md`.
- **test-adequacy-traps** — cases `docs/content/test/test.md` doesn't cover (identical-config update steps,
missing import-and-recheck).

## Mostly covered by the docs (revisit only if agents misread them)

- **immutability-nuances** — `docs/content/reference/field.md`, `docs/content/best-practices/immutable-fields.md`.
- **sensitive-and-write-only** — `docs/content/reference/field.md`.

## Lives elsewhere

- **failure-troubleshooting-catalog** — `.agents/skills/operations/troubleshooting_reference.md`; migrates
here if it outgrows the skill.
44 changes: 44 additions & 0 deletions .agents/knowledge/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Agent knowledge base

The **funnel to the knowledge an agent needs**: [`index.md`](index.md) routes to the right source for a
given decision. Agents read the index at decision points and open only what the task needs — never the
whole base (loading everything degrades reasoning and wastes context).

Most knowledge already lives in the **contributor documentation** (`docs/content/`), and that stays the
single source of truth for everything it covers — the index points there. **Agent-only entries** exist in
this directory only for knowledge that has no home in the docs: judgment rules that were never written
down, pitfall catalogs, and (later) lessons proposed by agents from completed tasks. If something is
covered by the contributor docs but covered badly, the fix is improving the docs — not writing a copy here.

Planned entries: [`BACKLOG.md`](BACKLOG.md).

## Entry format

One Markdown file, one topic, ≤120 lines, YAML frontmatter:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the frontmatter a well-known format or did you define it? Will agents do progressive disclosure with the format if it's custom?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the frontmatter a well-known format

Having YAML frontmatter is a well-known format. Gemini skills/rules use it (name+description), and so does the GCP Open Knowledge Format. The exact contents of the frontmatter were my invention, but can be fairly flexible if we find it's better to add or remove parts

Will agents do progressive disclosure with the format if it's custom?

Progressive disclosure doesn't depend on the frontmatter at all in these files. Right now frontmatter-driven loading only applies to skills + rules, and those are both location and frontmatter dependent.

Right now the mechanism is driven by a harness-level rule to read the index.

If Antigravity adopts an official shared knowledge system (not just local memories), we could switch to whatever that format is. They'll likely reuse name+description. The rest of the custom fields are mostly for record keeping.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By well-known format I mean like OKF or skills i.e. https://agentskills.io/specification#frontmatter!

Right now the mechanism is driven by a harness-level rule to read the index.

So we expect the agent to proactively read the index into context, and then choose to follow the index for those topics on-demand?


```yaml
---
name: enums-vs-strings # kebab-case, unique, matches filename
description: Model an API enum as Enum (strict) or String (forward-compatible). # the index line; <=140 chars
topics: [field] # index section(s) that list it
task_types: [field-add, new-resource]
source: docs/content/best-practices/validation.md # provenance: doc, PR, or "authored"
status: certified # certified (human-reviewed) | draft (proposed, unreviewed)
last_verified: 2026-07-09
---
```

Body rules: **rule + the why**, never bare imperatives; a real example per rule; a "do NOT use for" line
where misuse is likely; reference the contributor docs rather than duplicating them.

## Curation rules

- **Nothing enters unreviewed.** Entries land via PR like code, **one entry at a time** — each one codifies
team judgment and deserves a real review conversation. Agent-proposed entries ship as `status: draft` in
the task PR that motivated them; a human review flips them to `certified`.
- **Never bulk-rewrite.** Edit entry by entry. Do not have a model re-summarize or reorganize the base
wholesale — that is the documented failure mode (drift and self-contradiction).
- **Contradictions are quarantined.** A proposed entry that contradicts an existing one is not merged; a
human adjudicates: update or retire the old entry, or discard the proposal.
- **Keep the index true.** Index lines come from entry frontmatter (name + description). Adding or changing
an entry updates the index in the same commit.
36 changes: 36 additions & 0 deletions .agents/knowledge/field/enums-vs-strings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Entries in .agents/knowledge/ are intended to cover decision points or otherwise tacit knowledge that the contributor documentation does not cover.

A question that we'll need to ask ourselves: should this exist as an agent-only knowledge entry, or promoted to the contributor documentation?

I would say in most new cases, it'll be more efficient to have new topics/decision points exist as new entries outside of the contributor documentation. These entries are intended to be optimized for LLM ingestion and also fed by future session learnings.

Over time, I expect the % of documentation readers to lean towards mostly agents, whether initiated by a human author or not.

name: enums-vs-strings
description: Model an API enum as Enum (strict, plan-time) or String (forward-compatible) - the deliberate trade-off.
topics: [field]
task_types: [field-add, new-resource]
source: docs/content/best-practices/validation.md + the schema-review skill's original trade-off note
status: draft
last_verified: 2026-07-09
---

# Enum vs. String

**The trade-off (decide deliberately, per field):**
- **`Enum`** when the value set is exhaustive for a clearly-defined domain and new values are extremely
unlikely. Buys strict plan-time validation — users fail fast instead of mid-apply.
- **`String`** (with allowed values named in the description and a link to API docs) when the API is
likely to add values. **Why:** an Enum's value list lives in the provider; when the API adds a value,
every user needs a provider upgrade before they can use it — and imports of resources using the new
value break. A String stays forward-compatible.

```yaml
- name: 'severity'
type: Enum
enum_values:
- 'LOW'
- 'MEDIUM'
- 'HIGH'
```

**Rules for Enum:**
- Do **not** include `UNSPECIFIED`-style values in `enum_values`.
Comment thread
c2thorn marked this conversation as resolved.
Outdated
- Enums validate against the list automatically; a custom `validation` **overrides** that default — if you
add one, it must re-verify the enum values itself.

**Rule for String-instead-of-enum:** list the allowed values in the field description with a link to the
API docs — otherwise users only discover an invalid value when the API rejects it mid-apply.
56 changes: 56 additions & 0 deletions .agents/knowledge/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Knowledge index

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lightweight index for agents to deliberately read so they are aware of all existing documentation available to them.

The intention is that it performs better than the agent sporadically deciding to grep for existing documentation via specific terms or phrases.

It could still decide to NOT consult any documentation (without more forceful guardrails), but this should cover awareness at the very least.


Read this index at decision points; open only the source the task needs. Format and curation rules:
[README.md](README.md).

## Contributor documentation (`docs/content/`)

**Reference (what the schema properties mean):**

- Field properties (`output`, `immutable`, defaults, `sensitive`, validation, enums, arrays, `api_name`, conflicts) — `docs/content/reference/field.md`
- Resource-level properties (URLs, timeouts, async, custom code hooks) — `docs/content/reference/resource.md`
- Resource metadata — `docs/content/reference/metadata.md`
- Samples format — `docs/content/reference/sample.md`
- Update test expectations — `docs/content/reference/update-test-changes.md`
- Make commands — `docs/content/reference/make-commands.md`

**Procedures (how to do a task):**

- Add or change a field — `docs/content/develop/add-fields.md`
- Add a resource — `docs/content/develop/add-resource.md`
- Add a handwritten data source — `docs/content/develop/add-handwritten-datasource.md`
- Add IAM support — `docs/content/develop/add-iam-support.md`
- Custom code (expanders, flatteners, hooks) — `docs/content/develop/custom-code.md`
- Fix diffs and permadiffs — `docs/content/develop/diffs.md`
- Client-side fields — `docs/content/develop/client-side-fields.md`
- Promote beta → GA — `docs/content/develop/promote-to-ga.md`
- Generate the providers / set up the environment — `docs/content/develop/generate-providers.md`, `docs/content/develop/set-up-dev-environment.md`

**Testing:**

- Write acceptance tests (create, update, import) — `docs/content/test/test.md`
- Run tests — `docs/content/test/run-tests.md`

**Contributing (PRs, review, release notes):**

- Contribution process end to end — `docs/content/contribution-process.md`
- Create a PR — `docs/content/code-review/create-pr.md`
- Write release notes — `docs/content/code-review/release-notes.md`
- Review a PR — `docs/content/code-review/review-pr.md`

**Documentation:**

- Add resource documentation — `docs/content/document/add-documentation.md`
- Handwritten docs style guide — `docs/content/document/handwritten-docs-style-guide.md`

**Best practices (judgment calls with team positions):**

- Immutable fields / ForceNew — `docs/content/best-practices/immutable-fields.md`
- Deletion policy — `docs/content/best-practices/deletion-policy.md`
- Labels and annotations — `docs/content/best-practices/labels-and-annotations.md`
- Client-side validation — `docs/content/best-practices/validation.md`
- Common resource patterns (singletons) — `docs/content/best-practices/common-resource-patterns.md`

## Agent-only entries

- **enums-vs-strings** — Model an API enum as Enum (strict, plan-time) or String (forward-compatible): the deliberate trade-off. — [field/enums-vs-strings.md](field/enums-vs-strings.md)
14 changes: 4 additions & 10 deletions .agents/skills/utils/schema-review/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: schema-review
description: "Stub skill for reviewing Magic Modules schemas. Currently covers Enum vs. String trade-offs."
description: "Skill for reviewing Magic Modules schemas. Currently covers the Enum vs. String decision via the knowledge base."
---

# `schema-review`
Expand All @@ -14,17 +14,11 @@ description: "Stub skill for reviewing Magic Modules schemas. Currently covers E
## Execution Steps

### 1. Enum vs. String Trade-off
When adding a field that is defined as an Enum in the API, you must decide between `Enum` and `String` in Magic Modules.

#### The Trade-off:
* **Prefer `Enum`** when you want strict, plan-time validation of values to fail fast.
* **Prefer `String`** (with allowed values documented) when the API is dynamic or likely to add values. This prevents Terraform from crashing with a validation error for users when the API updates *before* the provider catches up.
When adding a field that is defined as an Enum in the API, you must decide between `Enum` and `String` in Magic Modules. Read the knowledge entry [.agents/knowledge/field/enums-vs-strings.md](../../../knowledge/field/enums-vs-strings.md) and apply its trade-off.

> [!NOTE]
> This skill is a **stub** and will be expanded over time with more schema review checklist items (such as `api_name` overrides, array flattening, output-only fields, etc.).
> This skill will be expanded over time with more schema review checklist items. The judgment content lives in [.agents/knowledge/](../../../knowledge/index.md) — this skill references it and does not hold its own copy.
### 2. Verification & Handoff
Instructions on how the agent should verify the command succeeded, and what workflow or rule it should return to next.

* Proceed with your field addition or PR review based on this stylistic choice.
* Reference the `modernization_roadmap.md` for upcoming expansions to this checklist.
* Proceed with your field addition or PR review based on the decision.
2 changes: 2 additions & 0 deletions .agents/skills/workflows/add_list_resource/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ description: "Opt resource into MMv1 list-resource generation by setting `genera

This workflow produces a single PR scoped to **one product** that flips `generate_list_resource: true` on every eligible MMv1 resource in that product, generates the downstream code, runs the generated list-query tests, and opens the PR. Do **one product per PR**, with as many eligible resources as pass.

Consult `.agents/knowledge/index.md` for the topics this task touches and open the relevant sources.

## Prerequisites

* You are in the `magic-modules` root directory.
Expand Down
1 change: 1 addition & 0 deletions .agents/skills/workflows/default/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ This document outlines the structured 5-step lifecycle for formal implementation

### 2. Triage
* Gather context on the change or bug. Plan the change (New feature or bug fix) within schema or logic.
* Consult `.agents/knowledge/index.md` for the topics the change touches and open the relevant sources.
* Execute the `triage` skill (located in `.agents/skills/operations/triage/`) to perform this work.
* **Transfers to Step 3:** Approved implementation plan and file paths file.

Expand Down
1 change: 1 addition & 0 deletions .agents/skills/workflows/new_resource/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,6 @@ There are two ways to generate the initial YAML definition for the resource:
- If the subagent reports test failures, enter the **Default Workflow** ([default/SKILL.md](../default/SKILL.md)) at **Step 5 (Fix)** to plan remediation.

### 2B. Manual Path
* Consult `.agents/knowledge/index.md` for the topics the resource touches and open the relevant sources.
* Follow the standard process to draft the YAML definition in `mmv1/products/...` based on API documentation and repository patterns.
* **Handoff:** Once the YAML is drafted, enter the **Default Workflow** ([default/SKILL.md](../default/SKILL.md)) at **Step 3 (Generate)** to compile the provider and continue with testing.
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ here, upstream — never in the downstream repos.
provider task.
- **`.agents/skills/`** — reusable skills the workflows compose (generation, testing, log parsing,
troubleshooting). Each skill's `SKILL.md` frontmatter states when to use it.
- **`.agents/knowledge/`** — the curated knowledge base (initial seeding in progress). When present,
consult its `index.md` at decision points and open only the entries the task needs.
- **[.agents/knowledge/index.md](.agents/knowledge/index.md)** — the knowledge index: a short map of
where provider knowledge lives. Consult it at the start of any task; open only the sources the task
needs.
- **`.agents/archive/`** — parked tracks (currently TGC). Not maintained; do not use as reference.

## Ground rules
Expand Down
Loading