Skip to content

Add one more "late metadata"/"digest" file to rlib archives #138243

Description

@petrochenkov

The file should contain data about:

  • Native libraries bundled into the rlib, to avoid looking up native libraries during the regular "early" metadata construction, when it's unnecessary and cannot be done fully correctly (native_libs: More consistent native library search between rustc and linker #138170)
  • Object files bundled into the rlib, to support link time cfg on native libraries bundled as object files
  • Rust object files in the rlib to avoid hacks like looks_like_rust_object_file during LTO

Unlike regular metadata which is constructed early, this additional digest will be constructed and added to the archive in the very end, when all other members of the rlib archive are already added.
It is also going to be read only when rustc is going to link the rlib into something, not earlier.

The data could potentially use some text format like json, but it would probably be simpler to just use the same format as the regular metadata uses.
The implementation can also mirror what is done for regular metadata, in a simplified form.

Me and @belovdv wanted to do this when implementing +bundle,+whole-archive (#113301) and packed_bundled_libs (#108081), as a more principle solution, but ended up with just some workarounds instead, and never returned to this.

Activity

  1. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Mar 8, 2025
  2. added
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    A-linkageArea: linking into static, shared libraries and binaries
    E-mentorCall for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.
    E-help-wantedCall for participation: Help is requested to fix this issue.
    on Mar 8, 2025
  3. petrochenkov commented on Mar 8, 2025

    @petrochenkov
    ContributorAuthor

    I suggest first adding some data that is enough to avoid the file skipping hacks in fn link_staticlib.

  4. added
    C-enhancementCategory: An issue proposing an enhancement or a PR with one.
    and removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Mar 9, 2025
  5. bjorn3 commented on Mar 12, 2025

    @bjorn3
    Member

    Why put it at the end? That would mean we have to scan over each rlib twice during linking. Once to find the "digest" and a second time to copy the actual content. Ar archives don't allow random access. You need to iterate over them one entry at a time. Wouldn't it be possible to create this "digest" during rlib writing before adding object files, but after we already know which object files will be added?

  6. petrochenkov commented on Mar 12, 2025

    @petrochenkov
    ContributorAuthor

    fn link_rlib contains a number of archive_builder.add_file(...); steps.
    I assumed each of them would also add something into the digest, then the digest would become ready in the end.

    Of course it can be refactored to be a bit smarter as an optimization.

  7. davidtwco commented on Mar 18, 2025

    @davidtwco
    Member

    You might get more visibility on this as an MCP - it sounds reasonable to me.

  8. added a commit that references this issue on May 5, 2026
  9. added 2 commits that reference this issue on May 5, 2026
  10. 11 remaining items

  11. added 6 commits that reference this issue on Jul 10, 2026
  12. added a commit that references this issue on Jul 11, 2026
  13. added a commit that references this issue on Jul 11, 2026
  14. added 2 commits that reference this issue on Jul 31, 2026
  15. added a commit that references this issue on Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-linkageArea: linking into static, shared libraries and binariesC-enhancementCategory: An issue proposing an enhancement or a PR with one.E-help-wantedCall for participation: Help is requested to fix this issue.E-mentorCall for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions