Skip to content

test(portal-clis): isolate valid flag controls from live services - #581

Merged
MadsLorentzen merged 2 commits into
MadsLorentzen:masterfrom
rudycelekli:test/offline-valid-cli-controls
Oct 18, 2026
Merged

MadsLorentzen merged 2 commits into
MadsLorentzen:masterfrom
rudycelekli:test/offline-valid-cli-controls

Conversation

@rudycelekli

Copy link
Copy Markdown
Contributor

Three existing positive validation controls launch actual search CLIs without a response fixture: two in LinkedIn and one in Freehire. They assert only that the error is not BAD_ARG, so they remain green when the child attempts an external request and subsequently fails for a network reason. The Freehire comment incorrectly claimed that checking the absence of an error made the case offline.

Serve the LinkedIn controls a saved two-card response over loopback HTTP, and point Freehire's control at its existing loopback API override. Assert exit 0, empty stderr, actual freshness/pagination parameters and output; LinkedIn also checks IDs and client-side result limits. The four Danish validation suites already stop valid-value controls at required-argument guards, so they need no change.

Credit: portapulito captured the real LinkedIn guest response in #153, closed as an accidental personalized-fork submission. The JSON-encoded excerpt preserves its HTML bytes; source commit, original/excerpt hashes and capture date are recorded beside it. Freehire uses an explicitly synthetic empty JSON protocol response because this test concerns option/request construction, not portal content parsing.

Validation on Bun 1.4.3 / TypeScript 5.9.3:

  • Before: unchanged six flag-validation suites pass, but a deny-and-record observer in their actual CLI children records 3 nonloopback request attempts (LinkedIn 2, Freehire 1). Every attempt is denied before transport; no live portal request was allowed.
  • After: the complete six reference CLI suites pass (341 tests, 0 failures) under an OS loopback-only sandbox, and the helper-child observer records 0 nonloopback attempts. All six typechecks and repository guards pass.
  • Evaluator controls: deleting the actual freshness-parameter construction makes both LinkedIn controls fail and the Freehire control fail; the source is restored afterward.

This isolates these three controls. A JavaScript preload is not whole-CI OS egress enforcement; our complete local qualification separately uses an OS network sandbox. Existing public CI runs of unchanged upstream tests should not be described as intrinsically network-free on this basis. No production code, runtime dependencies, credentials or profile data changed.

At exact head 86bdf759, the complete 579-test Python suite and all three repository guards also pass.

@MadsLorentzen MadsLorentzen left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified on the real path and merge-ready; one ask before it lands: add a line under [Unreleased] > Fixed in CHANGELOG.md (one sentence in the file's house style, PR number in parentheses). Merge on push.

rudycelekli added a commit to rudycelekli/ai-job-search that referenced this pull request Oct 17, 2026
@rudycelekli

Copy link
Copy Markdown
Contributor Author

Addressed in 931e0d88393b: added the requested one-sentence entry under [Unreleased] → Fixed, with the PR number in parentheses.

The follow-up changes only CHANGELOG.md; changelog structure, skill lint, framework-version and security guards passed locally.

The LinkedIn and Freehire positive controls spawned live search CLIs and only checked for the absence of BAD_ARG, so attempted portal requests still left a green suite. Use loopback responses and assert successful output plus the constructed freshness, pagination and result-limit contract.

LinkedIn cards are verbatim excerpts from portapulito's actual captured response in PR MadsLorentzen#153; fixture provenance and original hashes are retained. Four Danish validation suites already stop at required-argument guards and need no change. This isolates these controls rather than claiming a whole-CI egress policy.
@MadsLorentzen
MadsLorentzen force-pushed the test/offline-valid-cli-controls branch from 931e0d8 to 78bb90b Compare October 18, 2026 04:55
@MadsLorentzen
MadsLorentzen merged commit 2017521 into MadsLorentzen:master Oct 18, 2026
18 checks passed
@MadsLorentzen

Copy link
Copy Markdown
Owner

Merged. The three master controls really did attempt linkedin.com and freehire.me requests and stayed green with the parameter construction deleted; the new controls fail on that mutation and record zero non-loopback attempts. Fixture verified verbatim against the #153 capture.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants