Skip to content

Make source build dependency upgrades more liberal - #24273

Merged
MikeMcQuaid merged 1 commit into
Homebrew:mainfrom
HaraldNordgren:keep-compatible-build-dependencies
Oct 17, 2026
Merged

MikeMcQuaid merged 1 commit into
Homebrew:mainfrom
HaraldNordgren:keep-compatible-build-dependencies

Conversation

@HaraldNordgren

@HaraldNordgren HaraldNordgren commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

I know building from source isn't a supported configuration (see #24261), so rather than adding a dedicated source-build feature, this extends the rule #15927 introduced for bottles: an outdated dependency that's known to be compatible isn't upgraded. Bottles use the versions recorded in the bottle's tab. Source builds now use compatibility_version, which formulae already declare to reduce the number of dependency upgrades users need.

It's still a big quality-of-life improvement on machines without bottles. On my Intel macOS 12 machine, brew install ccache spent three hours building llvm before I cancelled it. That was only because asciidoctor needs ruby, which needs rust to build, which needs llvm, and all three were outdated.

Before:

==> Would install 1 formula:
ccache 4.14.1_2
==> Would install 10 dependencies for ccache:
blake3       1.8.7
fmt          12.2.0
hiredis      1.5.0
xxhash       0.8.4
asciidoctor  2.0.26
cpp-httplib  0.61.0
doctest      2.5.3
span-lite    0.11.0
tl-expected  1.3.1
make         4.4.1
==> Would upgrade 9 dependencies for ccache:
llvm      22.1.8   -> 23.1.3_1
rust      1.96.1   -> 1.99.0_1
ruby      3.1.1    -> 4.0.7_1
isl       0.20     -> 0.28
automake  1.18.1_1 -> 1.19
libtool   2.4.6_1  -> 2.6.2
mpfr      4.0.1    -> 4.2.2
libmpc    1.1.0    -> 1.4.1
gcc       8.2.0    -> 16.2.0

After:

==> Would install 1 formula:
ccache 4.14.1_1
==> Would install 10 dependencies for ccache:
asciidoctor  2.0.26
cpp-httplib  0.61.0
doctest      2.5.3
span-lite    0.11.0
tl-expected  1.3.1
blake3       1.8.7
fmt          12.2.0
hiredis      1.4.1_1
xxhash       0.8.4
make         4.4.1
==> Would upgrade 6 dependencies for ccache:
ruby     3.1.1   -> 4.0.7_1
isl      0.20    -> 0.28
libtool  2.4.6_1 -> 2.6.2
mpfr     4.0.1   -> 4.2.2
libmpc   1.1.0   -> 1.4.1
gcc      8.2.0   -> 16.2.0

llvm, rust and automake are no longer upgraded because their installed compatibility_version matches. gcc is still upgraded because my 2018 gcc keg predates compatibility_version. The "after" run used HOMEBREW_NO_INSTALL_FROM_API=1 (hence the slightly older versions from my tap) because the API doesn't include compatibility_version until it's regenerated with the fix below.

Changes:

  • When building from source, keep an outdated build dependency whose installed compatibility_version matches the current formula, along with the dependencies it was built against (otherwise e.g. upgrading llvm would break the kept rust). brew upgrade still upgrades it.
  • Upgrade it anyway when building a bottle (--build-bottle, which brew test-bot also passes when installing dependencies) or if this install would incompatibly upgrade one of the dependencies it was built against.
  • Load compatibility_version from the API. FormulaStruct didn't declare it, so formulae loaded from the API always returned nil: brew ruby -e 'p Formula["rust"].compatibility_version' prints nil, but 1 with HOMEBREW_NO_INSTALL_FROM_API=1.
  • Let Dependency#installed? check minimum_compatibility_version without also needing a minimum_version. Nothing passed it before.

  • Have you followed our Contributing guidelines?
  • Have you checked for other open Pull Requests for the same change?
  • Have you explained what your changes do? Performance claims (e.g. "this is faster") must include brew benchmark results.
  • Have you explained why you'd like these changes included, not just what they do?
  • For bug fixes, have you given step-by-step brew commands to reproduce the bug?
  • Have you written new tests (excluding integration tests)? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) locally?

  • I did not use AI/LLM to create this PR, or I disclosed the tool/model below and reviewed its output; I did not attribute commits to AI and will answer maintainer questions and review comments myself without AI/LLM.

Claude Code (Claude Opus 5.5) helped trace the dependency resolution code and write the change and specs. I verified it with brew style, brew typecheck, the dependency specs and the affected formula_installer and formulary examples locally, and by checking the dependency plan for brew install ccache on my machine (above).


Summary by CodeRabbit

  • New Features
    • Added support for formula compatibility versions when checking dependencies.
    • Compatible installed build dependencies can be reused when their runtime dependencies remain unchanged. They’re included for upgrade when needed to build a bottle or when a related dependency is also being upgraded.

@MikeMcQuaid MikeMcQuaid left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Prepared to merge something like this. Main concern here as-is is that we probably want to avoid this behaviour change when building bottles from source? Fine when explicitly or due to missing bottles having to build from source.

@HaraldNordgren

Copy link
Copy Markdown
Contributor Author

Thanks! Prepared to merge something like this. Main concern here as-is is that we probably want to avoid this behaviour change when building bottles from source? Fine when explicitly or due to missing bottles having to build from source.

Can you explain more what you would want, do you mean enable the new behavior with a flag?

@HaraldNordgren
HaraldNordgren force-pushed the keep-compatible-build-dependencies branch from 84a673d to c4b959b Compare October 16, 2026 19:39
@carlocab

Copy link
Copy Markdown
Member

Thanks! Prepared to merge something like this. Main concern here as-is is that we probably want to avoid this behaviour change when building bottles from source? Fine when explicitly or due to missing bottles having to build from source.

Can you explain more what you would want, do you mean enable the new behavior with a flag?

I think Mike means when the build uses brew install --build-bottle, instead of only just brew install --build-from-source.

@MikeMcQuaid

Copy link
Copy Markdown
Member

I think Mike means when the build uses brew install --build-bottle, instead of only just brew install --build-from-source.

Yes, thanks Carlo.

New behaviour fine:

  • --build-from-source
  • implicitly building from source due to missing bottle

New behaviour to be avoided:

  • --build-bottle

@HaraldNordgren

HaraldNordgren commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Yes, but isn't that what this PR already does?

Comment thread Library/Homebrew/formula_installer.rb Outdated
Comment thread Library/Homebrew/dependency.rb
@HaraldNordgren
HaraldNordgren force-pushed the keep-compatible-build-dependencies branch from c4b959b to 9432c02 Compare October 17, 2026 09:27
- Since Homebrew#15927, installing from a bottle doesn't upgrade outdated
  dependencies the bottle is known to work with. Building from source
  still upgrades every outdated build dependency, so e.g. `ccache` on
  Intel macOS rebuilds `rust` and `llvm` just to build `ruby` for
  `asciidoctor`.
- When building from source, keep an outdated build dependency whose
  installed `compatibility_version` matches the current formula, along
  with the dependencies it was built against. `brew upgrade` still
  upgrades it.
- Upgrade it anyway when building a bottle or if this install would
  incompatibly upgrade one of the dependencies it was built against.
- Load `compatibility_version` from the API: `FormulaStruct` dropped
  it, so formulae loaded from the API always returned `nil`.
- Let `Dependency#installed?` check `minimum_compatibility_version`
  without also needing a `minimum_version`.
@HaraldNordgren
HaraldNordgren force-pushed the keep-compatible-build-dependencies branch from 9432c02 to bfd4847 Compare October 17, 2026 09:40
@carlocab

Copy link
Copy Markdown
Member

Yes, but isn't that what this PR already does?

Does it also keep the existing/old behaviour for brew install --build-from-source --build-bottle (i.e. both flags are passed)? Because IIRC that's what test-bot does.

@HaraldNordgren

Copy link
Copy Markdown
Contributor Author

Does it also keep the existing/old behaviour for brew install --build-from-source --build-bottle (i.e. both flags are passed)?

The check uses build_bottle? alone so --build-bottle should keep the old behaviour whether or not --build-from-source is also passed.

@MikeMcQuaid
MikeMcQuaid marked this pull request as ready for review October 17, 2026 12:22

@MikeMcQuaid MikeMcQuaid left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@MikeMcQuaid
MikeMcQuaid added this pull request to the merge queue Oct 17, 2026
@coderabbitai

coderabbitai Bot commented Oct 17, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 45665bd6-b295-42ec-8f8c-6bd3d956f69f
📥 Commits

Reviewing files that changed from the base of the PR and between aac1fb8 and bfd4847.

📒 Files selected for processing: 7
  • Library/Homebrew/api/formula_struct.rb
  • Library/Homebrew/dependency.rb
  • Library/Homebrew/formula_installer.rb
  • Library/Homebrew/formulary.rb
  • Library/Homebrew/test/dependency_spec.rb
  • Library/Homebrew/test/formula_installer_spec.rb
  • Library/Homebrew/test/formulary_spec.rb

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

📝 Walkthrough

The API formula structure now includes a compatibility version, which the formula loader assigns to generated formula classes. Dependency checks use compatibility versions, and formula installation can retain eligible installed build dependencies during dependency expansion.

Changes: Compatibility versions and build dependencies

Layer / File(s) Summary
Compatibility version metadata
Library/Homebrew/api/formula_struct.rb, Library/Homebrew/formulary.rb, Library/Homebrew/test/formulary_spec.rb
The API formula structure adds a nullable compatibility version. The formula loader assigns this value to generated formula classes, and a test checks the loaded value.
Compatibility-based installed checks
Library/Homebrew/dependency.rb, Library/Homebrew/test/dependency_spec.rb
Dependency#installed? can check a minimum compatibility version when no minimum version is set. Tests cover matching and non-matching compatibility versions.
Build dependency retention
Library/Homebrew/formula_installer.rb, Library/Homebrew/test/formula_installer_spec.rb
Dependency expansion can retain eligible installed build dependencies and their runtime dependency names. It repeats expansion when those runtime dependencies conflict with upgrades. Tests cover retention, upgrades, and bottle builds.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to bfd48

The change lets source builds keep compatible outdated build dependencies, and no merge-blocking issue was identified in the supplied context. Normal CI should confirm the result before merge.

Security architecture risk: 🔵 Low · up to bfd48

The change preserves existing metadata authentication and installation controls. A narrow failure-containment risk remains when a runtime dependency is rebuilt with different options while an older build tool is retained. No security exploit was established.

Retained concerns

  • Low · reliability · inferred: Runtime revalidation uses installed? without checking required options, although a dependency can enter the installation plan because those options are missing. The installer subsequently applies the requested options. If that rebuild changes the runtime ABI without changing compatibility_version, an older retained build tool can become unusable after its runtime dependency is replaced, before the parent source build runs. Initial satisfaction checks and individual installation recovery do not cover this successful-but-incompatible replacement. The failure is conditional on formula-specific option behavior; it is not a demonstrated security exploit.
Security review details

Security Blast Radius

  • inferred — The demonstrated exposure is the dependency graph and installed tools within the affected Homebrew installation. Metadata now influences whether older build tools remain in use; the inspected changes do not establish a new remote entrypoint or privilege-escalation path.

Trust Boundaries and Controls

  • observed — The formula API uses the existing signed-payload fetch path. Verification requires PS512 with the bundled Homebrew public key; verification failure terminates loading, and the payload-cache reader also verifies signatures. These controls predate this PR, so missing external producer evidence is not evidence of an unauthenticated metadata boundary.

Resilience and Maintainability Implications

  • inferred — Existing locking and exception recovery limit concurrent mutation and failed individual installations, but do not establish compatibility after a successful option-changing runtime rebuild. That remaining risk concerns local failure containment rather than a verified attacker-controlled bypass.

Hardening Proposals

  • proposed — Evaluate retained tools against the planned runtime dependency build, including required options, rather than treating the currently installed compatibility generation alone as sufficient evidence for every replacement.
Pre-merge checks: Passed 4
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly summarises the main change: making source build dependency upgrades more selective by retaining compatible build dependencies.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Merged via the queue into Homebrew:main with commit 0f4d27d Oct 17, 2026
53 checks passed
@HaraldNordgren

Copy link
Copy Markdown
Contributor Author

Awesome!

HaraldNordgren added a commit to HaraldNordgren/brew that referenced this pull request Oct 17, 2026
- Check that `--build-bottle`, with or without `--build-from-source`,
  still upgrades outdated build dependencies under both keep rules: a
  matching `compatibility_version` (Homebrew#24273) and `:no_linkage`.
- Also check that a build dependency's own dependencies are upgraded
  when building a bottle, and that `:no_linkage` alone doesn't keep an
  outdated runtime dependency.
@HaraldNordgren
HaraldNordgren deleted the keep-compatible-build-dependencies branch October 18, 2026 09:28
HaraldNordgren added a commit to HaraldNordgren/brew that referenced this pull request Oct 18, 2026
- Check that `--build-bottle`, with or without `--build-from-source`,
  still upgrades outdated build dependencies under both keep rules: a
  matching `compatibility_version` (Homebrew#24273) and `:no_linkage`.
- Also check that a build dependency's own dependencies are upgraded
  when building a bottle, and that `:no_linkage` alone doesn't keep an
  outdated runtime dependency.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants