Skip to content

fix(cache): record session-cache completeness per provider - #1303

Merged
iamtoruk merged 1 commit into
mainfrom
fix/per-provider-cache-completeness
Sep 12, 2026
Merged

iamtoruk merged 1 commit into
mainfrom
fix/per-provider-cache-completeness

Conversation

@iamtoruk

Copy link
Copy Markdown
Member

Replaces the conservative guard from #1229 with the per-provider model the review preferred but deferred.

  • The problem: SessionCache.complete is one whole-cache boolean, but a scan is scoped on two axes. fix(parser): avoid marking partially scanned caches complete (#912) #1229 kept a provider-scoped run from stamping it by running a second, unfiltered discoverAllSessions() in the stamp path. On a multi-provider machine nothing else writes the flag, so scoped users never go warm: every run re-enters cold hydration and pays a full discovery across all providers (Codex reads up to 64KB per uncached rollout). That hits the Claude-scoped refresh in usage-aggregator.ts, which exists precisely to avoid touching other providers.
  • Completeness now lives on ProviderSection.complete. A scoped run marks only the provider it walked; an unscoped run marks every section and the whole-cache flag, which is the only thing that can vouch for providers with no section at all.
  • A section with no flag of its own falls back to the whole-cache flag, so an existing cache reads exactly as it did in both states (true = all complete, false/absent = none). The two new envelope fields are additive and ignored by older readers, so CACHE_VERSION is unchanged and nobody's session cache is thrown away. DAILY_CACHE_VERSION is untouched.
  • Date axis: a run that skipped uncached sources because their mtime predates dateRange.start stores that start as the section's completeFrom floor instead of claiming all of history. A watermark rather than a plain boolean because nearly every parse is ranged - the 365-day history parse skips everything older, and withholding completeness outright would leave any machine with older sessions permanently cold. Readers pass the requested range start; a wider request than the floor re-enters cold hydration and lowers the floor as it goes.
  • Discovery no longer fails open: discoverAllSessionsWithFailures reports which providers threw, and one that returned an empty list because it failed is never marked complete. Per-file and per-provider parse isolation is unchanged.
  • Discovery passes in a scoped run: 2 before (one filtered, one unfiltered over every provider), 1 after. No new parse, no extra file I/O.
  • Verification: npx tsc --noEmit clean; full npm test 3938 passed, 5 skipped, 0 failed across 290 files. tests/scoped-run-completeness.test.ts rewritten (dead CODEX_HOME block dropped - the codex provider resolves its dir at import time and tests/setup/env-isolation.ts already isolates it) with cases for scoped marking, three-run convergence with a cold-hydration counter, the date floor, a throwing provider, and both legacy-cache states.

Known gap: a provider skipped mid-parse on a permission error is still marked complete, as it is today. Left alone deliberately - changing it would leave a Full-Disk-Access-denied machine cold on every launch, which is a separate call.

A scan is scoped on two axes, provider and date range, but completeness
was one whole-cache boolean. #1229 stopped a scoped run from stamping it
by adding a second, unfiltered discoverAllSessions to the stamp path,
which left scoped users on a multi-provider machine permanently cold and
paid an extra full discovery on every run.

Completeness now lives on the provider section, with an optional
completeFrom floor for the sources a date range filtered out. The
whole-cache flag keeps its meaning (an unscoped scan finished) and is
what a section with no flag of its own falls back to, so existing caches
read exactly as before and no version bump is needed. Discovery reports
which providers threw, and a provider that reported nothing because it
failed is never marked complete.
@iamtoruk
iamtoruk merged commit 5783c6d into main Sep 12, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant