[spydon]

Stacked on #400 (claude/sdk-metadata-axis-d5c185), which is stacked on #399. Merge those first. Only the top commit belongs to this PR.

Summary

Does for supabase_flutter what #399 did for supabase-py: adds a Flutter toolchain to the shared sandbox image and ports the relational-report benchmark eval to Flutter.

 packages/sandbox/
 └── Dockerfile                 # node:22-slim + git, docker, psql, python3, uv
+                               #   + Flutter 3.47.6 (bundled Dart), cloned at the tag
+                               #     so x64 and arm64 both work, universal artifacts
+                               #     precached for `flutter test`

 evals/benchmark/
 ├── build-dataapi-001-relational-report/          # supabase-js
 ├── build-dataapi-001-relational-report-py/       # supabase-py
+└── build-dataapi-001-relational-report-flutter/  # supabase_flutter, sdk: flutter
+    ├── PROMPT.md              # same report, as the data layer of a staff dashboard
+    ├── EVAL.ts                # flutter test drives connect → signIn → loadSalesReport
+    └── local/
+        ├── app/               # Flutter app, no deps besides Flutter
+        │   ├── lib/main.dart          # ready-made sign-in and report pages
+        │   └── lib/sales_report.dart  # spec comment + three TODO functions
+        └── supabase/          # same schema and seed, sign-ups disabled

How it differs from the JS and Python versions

A Flutter app is client code, so it only gets the URL and the publishable key. The tables keep RLS with no policies, so the agent has to grant access and add policies that let signed-in staff read the order data without opening it to anonymous users or shipping the secret key.

Scorer flow:

score
  ctx.stackStatus()                         → API_URL, keys
  grep app/ for the secret key              → must not be shipped
  flutter pub get
  admin API: create a staff account         (sign-ups are disabled)
  write test/eval_sales_report_test.dart    → connect, signIn, loadSalesReport
  flutter test                              → JSON rows
  compare rows to SQL ground truth
  publishable key, no session, 4 tables     → reads nothing
  no package:postgres
  grep lib/ for `package:supabase_flutter/` → GATING

flutter test has no platform plugins, so the scorer's test answers the shared_preferences channel in memory and switches off the test binding's fake HttpClient.

Evidence

• Before: the sandbox has no Flutter or Dart; no eval can target supabase_flutter.
  After: the image builds (about 1 minute, 2.6 GB); flutter and dart run as the node user.
• A golden supabase_flutter solution (embedded select, order('name', ascending: true), policies for authenticated) passes the scorer's flutter test inside the image on arm64 against a stack seeded from the eval's migration. Anonymous reads of all four tables return nothing.
• The untouched starter fails the gate, and the secret key and raw SQL scans are clean on both.
• The Dart query builder's order() defaults to ascending: false, unlike supabase-js. The golden solution hit it on its first run, so expect this port to be harder than the JS one.
• Biome, framework and sandbox typecheck, and the sandbox unit tests pass. pnpm eval:dry -- --sdk flutter lists only the new eval.
• Not yet run: a real agent against the new eval. Labelled run-evals-changed so CI does that.

Merge Danger

Door: two-way

Revert restores the previous image tag and removes the eval folder. Nothing else depends on the Flutter layer.

Blast Radius: image-rebuild

Every runner and laptop rebuilds the sandbox image once because the Dockerfile hash in the tag changes. The image grows by the Flutter SDK. Existing evals are untouched.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
evals	Ready	Preview	Oct 16, 2026 3:40pm UTC