Feature request: incremental formulae for autobump

[lelegard]

Hi,

I open this discussion about autobump when a project needs a new dependency, or any non-automatic update in the content of the formula, in its future new version which will be autobumped.

The problem

Autobump is a great feature to deliver new versions of most projects in a timely manner. Within a couple of hours after a project is released in its initial repo (on GitHub for instance), the autobump action creates the PR for the new version in Homebrew.

However, this process is based on one limiting assumption: the new version of the project does not need any specific modification of the formula. Everything must remain unchanged. Autobump automatically updates the formula with the new url and sha256 of the new version. It cannot guess any specific required update in the formula, for instance new dependencies.

This is a common situation. A new version of a project often brings new features. A new feature may need a new library that was not previously listed in the formula of the project.

In that case, some human needs to update the formula with the new (or removed) dependency, or any other required change.

However, there is a race between the human maintainer and the autobump process:

• If the maintainer submits the PR before version N+1 is released, this will trigger a rebuild and update of version N. Just for a while, until version N+1 is released. In some cases, this can even result in a failure, when a dependency is removed for instance.
• If the maintainer submits the PR right after version N+1 is released, there is a race condition with the autobump process. The autobump PR may have already started building version N+1 with the previous formula, with missing dependencies, and it will fail.
• In all cases, there is a risk that 2 simultaneous PR are submitted for the same project, in some unpredictable order.

Suggested solution

There are several possible solutions. This section is just a proposition, any other solution which solves the above problem can be fine.

One solution could be to introduce a new section in the formula for "next things to do". However, it would be limited to a few features such as adding or removing dependencies. This may not suit all use cases.

I suggest an "incremental formula" or "next formula". Some kind of stage / commit version of the formula.

Instead of tweaking the existing formula file proj.rb in some complicated way, I propose that a manual PR creates a "next" version of the formula, named proj.rb.next or proj.rb.autobump. This file is the prepared content of the formula for the next version of the project, as processed by autobump.

The file proj.rb.autobump is mainly a copy of proj.rb with the required modifications for the next version of the project. The url, sha256, and bottle descriptions are not significant since they will be modified by autobump.

The PR which creates or updates proj.rb.autobump does not trigger anything because it is not an active formula. Triggering an analysis of the next formula is still recommended.

When the autobump process starts autobumping proj and proj.rb.autobump exists, it would first rename proj.rb.autobump as proj.rb, replacing the previous version. Then, autobump would proceed as usual, updating url, sha256 and the rest.

I think the problem is quite relevant (I already ran into it in the past and I will have it again in the next version of the project I maintain).

Do you think that the proposed solution, or some other solution, could add value to Homebrew?

[SMillerDev]

I think this is a way overcomplicated solution to a barely existing problem.

• If autobump fails to build, that's fine. Someone from homebrew might look what's needed to fix it, or someone from the project might submit a PR to update the formula.
• If upstream wants to test their changes beforehand, they can make a PR to test their release RC in a formula before the real release. This will block autobump from bumping because a PR for that version exists.

Either one works without additional files or configuration.

[lelegard]

OK, so, if I summarise:

• Providing new features in a new version of a project is a "barely existing problem".
• The solution: let it fail and manually fix later.

Okay...

[p-linnane]

Please engage with the answers you’ve been given. @SMillerDev described two existing workflows, including testing a release candidate before release. Summarizing that as “let it fail” ignores the preventive option he offered.

A second, staged formula introduces code, testing, and ongoing maintenance that we would need to support across thousands of packages. We have to weigh that cost against the problem it solves, even if the change looks straightforward for one project.

[SMillerDev]

The solution: let it fail and manually fix later.

We have a CI setup exactly for this reason. It checks if things work. The autobump is just a simple script that changes the version number and checksum. Sometimes it fails to build after, and that's fine. I'm not sure why you're so bothered by that concept. We also have humans contributing that regularly fail CI.