pnpm agent-eval

The scripted rubric behind the AI-first claim: can this environment go from zero to a validated, deploy-ready app using only the public CLI? Any agent or CI runs it and gets a score.

pnpm agent-eval          # ~1 min, 9 steps
pnpm agent-eval --full   # adds the convex + browser smoke (~4 min)
pnpm agent-eval --json   # machine-readable result line

The steps

  • doctor: environment ready.
  • scaffold a reshaped crm demo (supabase, mcp, --fields, --table/--labels).
  • install + typecheck the generated app (tsc-clean).
  • deploy dry-run (plans + builds).
  • gallery spec: bare-name scaffold + typecheck.
  • generate: extractJson parses fenced/nested/bare + buildPrompt shape.
  • mcp-server: stdio handshake + tools/list exposes scaffold/generate/deploy.
  • upgrade-app: dry-run round-trip on a fresh scaffold.
Self-cleaning; exits 0 only on a perfect score. Current: 9/9 in under a minute.