failure-routing.md

Failure Routing (Auto-fix Loops)

This guide explains how to configure Visor to automatically remediate failures and re-run steps until convergence, using safe, deterministic routing.

What You Can Do

• Retry failed steps with backoff (fixed or exponential)
• Run remediation steps on failure (e.g., lint-fix, npm ci)
• Jump back to an ancestor step on failure or success (goto)
• Compute remediation and targets dynamically with safe JS (run_js, goto_js)
• Protect against infinite loops with per-scope loop caps and per-step attempt counters
• Use the same semantics inside forEach branches (each item is isolated)

Outputs Surface (Routing JS)

When writing run_js/goto_js, you have three accessors:

• outputs['x'] — nearest value for check x in the current snapshot and scope.
• outputs_raw['x'] — aggregate value for x (e.g., the full array from a forEach parent).
• outputs.history['x'] (alias: outputs_history['x']) — all historical values for x up to this snapshot.

Precedence for outputs['x'] in routing sandboxes:

• If running inside a forEach item of x, resolves to that item’s value.
• Else prefer an ancestor scope value of x.
• Else the latest committed value of x in the snapshot.

Quick example using outputs_raw in goto_js:

checks:
  list:
    type: command
    exec: echo '["a","b","c"]'
    forEach: true

  decide:
    type: script
    depends_on: [list]
    content: 'return { n: (outputs_raw["list"] || []).length }'
    on_success:
      goto_js: |
        // Branch by aggregate size, not per-item value
        return (outputs_raw['list'] || []).length >= 3 ? 'bulk-process' : null;

  bulk-process: { type: log, message: 'bulk mode' }

Tip: outputs_raw is also available in provider templates (AI/command/log/memory), mirroring routing JS.

Quick Examples

Retry + goto on failure:

version: "2.0"
routing: { max_loops: 5 }
steps:
  setup: { type: command, exec: "echo setup" }
  build:
    type: command
    depends_on: [setup]
    exec: |
      test -f .ok || (echo first try fails >&2; touch .ok; exit 1)
      echo ok
    on_fail:
      goto: setup
      retry: { max: 1, backoff: { mode: exponential, delay_ms: 400 } }

on_success jump-back once + post-steps:

steps:
  unit: { type: command, exec: "echo unit" }
  build:
    type: command
    depends_on: [unit]
    exec: "echo build"
    on_success:
      run: [notify]
      goto_js: |
        // Jump back only once using history length as proxy for attempt count
        return outputs.history['build'].length === 1 ? 'unit' : null;
  notify: { type: command, exec: "echo notify" }

Note on outputs access:

• outputs['step-id'] returns the latest value for that step in the current snapshot.
• outputs.history['step-id'] returns the cross-loop history array.
• outputs_history['step-id'] is an alias for outputs.history['step-id'] and is available in routing JS and provider templates. See Output History for more details.

forEach remediation with retry:

steps:
  list: { type: command, exec: "echo '[\\"a\\",\\"b\\"]'", forEach: true }
  mark: { type: command, depends_on: [list], exec: "touch .m_{{ outputs.list }}" }
  process:
    type: command
    depends_on: [list]
    exec: "test -f .m_{{ outputs.list }} || exit 1"
    on_fail:
      run: [mark]
      retry: { max: 1 }

Configuration Keys

Top-level defaults (optional):

routing:
  max_loops: 10         # per-scope cap on routing transitions
  defaults:
    on_fail:
      retry: { max: 1, backoff: { mode: fixed, delay_ms: 300 } }

Per-step actions:

• on_fail:
  • retry: { max, backoff: { mode: fixed|exponential, delay_ms } }
  • run: [step-id, …]
  • goto: step-id (ancestor-only)
  • goto_event: event to simulate during goto (e.g., pr_updated)
  • run_js: JS returning string[]
  • goto_js: JS returning string | null
  • transitions: declarative transition rules (see below)
• on_success:
  • run, goto, goto_event, run_js, goto_js, transitions (same types and constraints as above)
• on_finish (forEach checks only):
  • run, goto, goto_event, run_js, goto_js, transitions (same types and constraints as above)

Declarative Transitions

The transitions field provides a declarative alternative to goto_js. It is an array of rules evaluated in order; the first matching rule wins:

steps:
  validate:
    type: ai
    on_success:
      transitions:
        - when: "outputs['validate'].score >= 90"
          to: publish
        - when: "outputs['validate'].score >= 70"
          to: review
        - when: "true"  # default fallback
          to: reject

Each rule has:

• when: JavaScript expression evaluated in the same sandbox as goto_js
• to: target step ID, or null to explicitly skip goto
• goto_event: optional event override (same as goto_event above)

Helper functions available in when expressions:

• any(arr, pred) - returns true if any element matches predicate
• all(arr, pred) - returns true if all elements match predicate
• none(arr, pred) - returns true if no element matches predicate
• count(arr, pred) - returns count of elements matching predicate

When transitions is present, it takes precedence over goto_js and goto.

Semantics

• Retry: re-run the same step up to retry.max; backoff adds fixed or exponential delay with deterministic jitter.
• Run: on failure (or success), run listed steps first; if successful, the failed step is re-attempted once (failure path).
• Goto (ancestor-only): jump back to a previously executed dependency, then continue forward. On success, Visor re-runs the current step once after the jump.
• Loop safety: routing.max_loops counts all routing transitions (runs, gotos, retries). Exceeding it aborts the current scope with a clear error. For a hard cap on repeated executions of the same step, see Execution Limits.
• forEach: each item is isolated with its own loop/attempt counters; *_js in on_finish context receives forEach metadata (see Available Context in on_finish).

Fan‑out vs. Reduce (Phase 5)

You can control how routing targets behave when invoked from a forEach context:

• fanout: map — schedule the target once per item (runs under each item scope).
• fanout: reduce (or reduce: true) — schedule a single aggregation run (default/back‑compat).

Where it applies:

• on_success.run / on_success.goto
• on_fail.run / on_fail.goto
• on_finish.run / on_finish.goto (when defined on the forEach producer)

Example — per‑item side‑effects via routing:

checks:
  list:
    type: command
    exec: echo '["a","b","c"]'
    forEach: true
    on_success:
      run: [notify-item]

  notify-item:
    type: log
    fanout: map  # ← run once for each item from 'list'
    message: "Item: {{ outputs['list'] }}"

Example — single aggregation after forEach:

checks:
  extract:
    forEach: true
    on_success:
      run: [summarize]

  summarize:
    type: script
    fanout: reduce  # ← single run
    content: |
      const arr = outputs_raw['extract'] || [];
      return { total: arr.length };

Goto Event Override (goto_event)

You can instruct a goto jump to simulate a different event so that the target step is filtered as if that event occurred. This is useful when you need to re-run an ancestor step under PR semantics from a different context (e.g., from an issue comment or internal assistant flow).

Key points:

• Add goto_event: <event> alongside goto or use it with goto_js.
• Valid values are the same as on: triggers (e.g., pr_updated, pr_opened, issue_comment, issue_opened).
• During the inline goto execution, Visor sets an internal event override:
  • Event filtering uses the overridden event for the target step.
  • if: expressions see event.event_name derived from the override (e.g., pr_* → pull_request, issue_comment → issue_comment, issue_* → issues).
• After the jump, the current step is re-run once; the override applies only to the inline target and that immediate re-run.
• goto remains ancestor-only.

Example: After security succeeds, jump back to overview and re-run security, evaluating both as if a PR update happened:

steps:
  overview:
    type: ai
    on: [pr_opened, pr_updated]

  security:
    type: ai
    depends_on: [overview]
    on: [pr_opened, pr_updated]
    on_success:
      goto: overview           # ancestor-only
      goto_event: pr_updated   # simulate PR updated during the jump

Dynamic variant (only jump on first success):

steps:
  quality:
    type: ai
    depends_on: [overview]
    on: [pr_opened, pr_updated]
    on_success:
      goto_js: |
        // Jump back only once using history length as proxy for attempt count
        return outputs.history['quality'].length === 1 ? 'overview' : null
      goto_event: pr_updated

When to use goto_event vs. full re-run:

• Use goto_event for a targeted, in-process jump to a specific ancestor step with PR semantics.
• Use a higher-level “internal re-invoke” (e.g., synthesize a pull_request synchronize event and call the action entrypoint) when you need to re-run the entire PR workflow chain from an issue comment trigger.

Dynamic JS (safe, sync only)

• goto_js / run_js are evaluated in a sandbox with:
  • Read-only context: { step, outputs, outputs_history, outputs_raw, output, memory, event, forEach }
  • outputs contains current values, outputs.history contains arrays of all previous values (see Output History)
  • outputs_raw provides aggregate/parent values (e.g., full array from forEach parent)
  • memory provides read/write access to the memory store (get, set, has, getAll, increment, clear)
  • log() function available for debugging (outputs with Debug: prefix)
  • Pure sync execution; no IO, no async, no timers, no require/process.
  • Time and size limits (short wall time; small code/output caps) — evaluation failures fall back to static routing.

Note: The on_finish context is richer and additionally includes attempt, loop, pr, files, and env.

Return types:

• goto_js: a string (step id) or null/undefined to skip.
• run_js: a string[] (may be empty). Duplicates are removed preserving order.

Guardrails & Tips

• Keep max_loops small (5-10). Add retries sparingly and prefer remediation over blind loops.
• Restrict goto to ancestors to preserve dependency semantics and avoid hard-to-reason paths.
• For expensive remediations, put them behind run_js conditions keyed to output or outputs.history values.
• Use outputs.history['check-name'].length as a proxy for attempt count in goto_js/run_js.
• In CI, you can override defaults with CLI flags (future): --on-fail-max-loops, --retry-max.

on_finish Hook (forEach Aggregation & Routing)

The on_finish hook is a special routing action that triggers once after a forEach check completes all of its dependent checks across all iterations. This is the ideal place to aggregate results from forEach iterations and make routing decisions based on the collective outcome.

When on_finish Triggers

• Only on checks with forEach: true
• Triggers after all dependent checks complete all their iterations
• Does not trigger if the forEach array is empty
• Executes even if some iterations failed (you decide how to handle failures)

Execution Order

forEach check executes once → outputs array [item1, item2, ...]
↓
All dependent checks execute N times (forEach propagation)
  - dependent-check runs for item1
  - dependent-check runs for item2
  - ...
↓
on_finish.run executes (checks run sequentially in order)
↓
on_finish.run_js evaluates (dynamic check selection)
↓
on_finish.goto_js evaluates (routing decision)
↓
If goto returned, jump to ancestor check and re-run current check

How It Differs from on_success and on_fail

Hook	Triggers When	Use Case
on_fail	Check fails	Handle single check failure, retry, remediate
on_success	Check succeeds	Post-process single check success
on_finish	All forEach dependents complete	Aggregate all forEach iteration results, decide next step

Key difference: on_finish sees the complete picture of all forEach iterations and all dependent check results, making it perfect for validation and aggregation scenarios.

Available Context in on_finish

The on_finish hooks have access to the complete execution context:

{
  step: { id: 'extract-facts', tags: [...], group: '...' },
  attempt: 1,              // Current attempt number for this check
  loop: 0,                 // Current loop number in routing
  outputs: {
    'extract-facts': [...],   // Array of forEach items
    'validate-fact': [...],   // Array of ALL dependent results
    history: { ... }          // Alias for outputs_history
  },
  outputs_history: {
    'extract-facts': [[...], ...], // Cross-loop history
    'validate-fact': [[...], ...], // All results from all iterations
  },
  outputs_raw: {
    'extract-facts': [...],   // Aggregate/parent values
    'validate-fact': [...],
  },
  forEach: {
    items: 3,              // Number of forEach items
    last_wave_size: 3,     // Items in the last wave (when forEach parent)
    last_items: [...],     // The forEach items array (when forEach parent)
    is_parent: true        // Indicates this check is a forEach parent
  },
  memory: {                // Memory access functions
    get: (key, ns?) => ...,
    set: (key, value, ns?) => ...,
    has: (key, ns?) => ...,
    getAll: (ns?) => ...,
    increment: (key, amount?, ns?) => ...,
    clear: (ns?) => ...
  },
  pr: {                    // PR metadata
    number: 123,
    title: '...',
    author: '...',
    branch: '...',
    base: '...'
  },
  files: [...],            // Changed files
  env: { ... },            // Environment variables (filtered for safety)
  event: { name: '...' }   // Event that triggered execution
}

Configuration

checks:
  extract-facts:
    type: ai
    forEach: true
    # ... regular check configuration ...

    on_finish:
      # Optional: Run additional checks to aggregate results
      run: [aggregate-validations]

      # Optional: Dynamically compute additional checks to run
      run_js: |
        return error ? ['log-error'] : [];

      # Optional: Static routing decision
      goto: previous-check

      # Optional: Dynamic routing decision
      goto_js: |
        const allValid = memory.get('all_facts_valid', 'fact-validation');
        const attempt = memory.get('fact_validation_attempt', 'fact-validation') || 0;

        if (allValid) {
          return null;  // Continue normal flow
        }

        if (attempt >= 1) {
          return null;  // Max attempts reached, give up
        }

        // Retry with correction context
        memory.increment('fact_validation_attempt', 1, 'fact-validation');
        return 'issue-assistant';  // Jump back to ancestor

      # Optional: Override event for goto target
      goto_event: pr_updated

Common Patterns

Pattern 1: Validation with Retry

Aggregate validation results and retry if any fail:

checks:
  extract-claims:
    type: ai
    forEach: true
    transform_js: JSON.parse(output).claims
    on_finish:
      run: [aggregate-results]
      goto_js: |
        const allValid = memory.get('all_valid', 'validation');
        const attempt = memory.get('attempt', 'validation') || 0;

        if (allValid || attempt >= 2) {
          return null;  // Success or max attempts
        }

        memory.increment('attempt', 1, 'validation');
        return 'generate-response';  // Retry

  validate-claim:
    type: ai
    depends_on: [extract-claims]
    # Validates each claim individually

  aggregate-results:
    type: script
    content: |
      const results = outputs.history['validate-claim'];
      const allValid = results.every(r => r.is_valid);
      memory.set('all_valid', allValid, 'validation');
      return { total: results.length, valid: results.filter(r => r.is_valid).length };

Pattern 2: Conditional Post-Processing

Run different post-processing based on forEach results:

checks:
  scan-files:
    type: command
    forEach: true
    exec: "find . -name '*.ts'"
    on_finish:
      run_js: |
        const hasErrors = outputs.history['analyze-file'].some(r => r.errors > 0);
        return hasErrors ? ['generate-fix-pr'] : ['post-success-comment'];

  analyze-file:
    type: command
    depends_on: [scan-files]
    exec: "eslint {{ outputs['scan-files'] }}"

Pattern 3: Multi-Dependent Aggregation

Aggregate results from multiple dependent checks:

checks:
  extract-facts:
    type: ai
    forEach: true
    on_finish:
      run: [aggregate-all-validations]
      goto_js: |
        const securityValid = memory.get('security_valid', 'validation');
        const technicalValid = memory.get('technical_valid', 'validation');
        const formatValid = memory.get('format_valid', 'validation');

        if (!securityValid || !technicalValid || !formatValid) {
          return 'retry-with-context';
        }
        return null;

  validate-security:
    type: ai
    depends_on: [extract-facts]
    # Runs N times, validates security aspects

  validate-technical:
    type: ai
    depends_on: [extract-facts]
    # Runs N times, validates technical aspects

  validate-format:
    type: ai
    depends_on: [extract-facts]
    # Runs N times, validates format/style

  aggregate-all-validations:
    type: script
    content: |
      // Access ALL results from ALL dependent checks
      const securityResults = outputs.history['validate-security'];
      const technicalResults = outputs.history['validate-technical'];
      const formatResults = outputs.history['validate-format'];

      memory.set('security_valid', securityResults.every(r => r.is_valid), 'validation');
      memory.set('technical_valid', technicalResults.every(r => r.is_valid), 'validation');
      memory.set('format_valid', formatResults.every(r => r.is_valid), 'validation');

      return { aggregated: true };

Error Handling

• If on_finish.run checks fail, the forEach check is marked as failed
• If goto_js throws an error, the engine falls back to static goto (if present)
• Clear error messages are logged for debugging
• Loop safety: on_finish.goto counts toward max_loops

Best Practices

1. Use for Aggregation: Perfect for collecting and analyzing results from all forEach iterations
2. Memory for State: Store aggregated results in memory for use in routing decisions and downstream checks
3. Fail Gracefully: Handle both success and failure scenarios in goto_js
4. Limit Retries: Use attempt counters to prevent infinite loops
5. Log Decisions: Use log() in JS to debug routing decisions
6. Validate First: Run aggregation checks before routing to ensure data is ready

Debugging

// In on_finish.goto_js
log('forEach stats:', forEach);
log('All results:', outputs.history['dependent-check']);
log('Current attempt:', attempt, 'loop:', loop);

const results = outputs.history['validate-fact'];
log('Valid:', results.filter(r => r.is_valid).length);
log('Invalid:', results.filter(r => !r.is_valid).length);

Full Examples

See the repository examples:

• examples/routing-basic.yaml
• examples/routing-on-success.yaml
• examples/routing-foreach.yaml
• examples/routing-dynamic-js.yaml
• examples/fact-validator.yaml - Complete on_finish example with validation and retry