Skip to content

trans/LLVM: Don't keep all LLVM modules in memory at the same time #39280

Closed

Description

@michaelwoerister

UPDATE: There are some mentoring instructions below if you're interested in looking into this issue.


NB: Part of the roadmap issue on incremental compilation.

Currently the compiler translates all codegen units into LLVM modules and then runs LLVM on all of them in parallel. In the context of incremental compilation, where there can be hundreds of codegen units for a single crate, but also for non-incremental builds with a high number of codegen units, this can mean a lot of memory pressure -- e.g. for Servo's script crate that's more than 10 GB peak memory usage (while otherwise it's around 4.5 GB).

There's no real need to keep more LLVM modules in memory than are currently being worked on. Two possible solutions:

  1. If the compiler uses N LLVM threads, start persisting LLVM modules to disk at the N+1st and reload them later for optimization and codegen. That's probably relatively easy to implement but involves more disk I/O than necessary.
  2. Translate only N codegen units to LLVM modules, translate them all the way to object files, then go back and translate the next codegen unit. This has the advantage that we would not need to temporarily store anything on disk, but we would need to keep the tcx in memory as long as there are still untranslated codegen units. It would also require a much bigger refactoring than the first approach.

Any other ideas?

cc @rust-lang/compiler @rust-lang/tools @nagisa

Activity

  1. added
    A-codegenArea: Code generation
    A-incr-compArea: Incremental compilation
    I-compiletimeIssue: Problems and improvements with respect to compile times.
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    on Jan 24, 2017
  2. nrc commented on Jan 24, 2017

    @nrc
    Member

    Will this actually cause problems? Unless there are too many threads (in which case that should be addressed), it feels like the OS will handle this better by paging to disk than we will by (effectively) doing so manually. Also, for many devs, 10GB is not too much memory usage and anything we do here (unless we're really smart) will slow down compilation for those users.

  3. alexcrichton commented on Jan 25, 2017

    @alexcrichton
    Member

    My first thought would be the latter, consider translation part of a "work unit" and don't actually start doing it until it's ready to go all the way into LLVM. That way we cap the number of active LLVM modules at any one point in time.

    I think @nrc's got a good point, though, in that using the disk as a cache may be better left to the OS rather than the compiler itself (until proven otherwise, of course)

  4. michaelwoerister commented on Jan 25, 2017

    @michaelwoerister
    MemberAuthor

    Some data points:

    Will this actually cause problems?

    It lead to my not being able to compile the script crate on Linux with 8 GB. Maybe if I had more swap space it would have worked. (EDIT: The VM had 4GB swap space)

    Unless there are too many threads (in which case that should be addressed)

    If everything fits into memory at once, having hundreds of parallel LLVM threads does not seem to have a detrimental effect (running 140 threads was a few percent faster than having 4 threads on a 4 core processor). OTOH, I would suspect that paging would completely destroy performance if many threads had to fight for RAM.

    Also, for many devs, 10GB is not too much memory usage and anything we do here (unless we're really smart) will slow down compilation for those users.

    That's certainly a valid concern!

  5. retep998 commented on Jan 25, 2017

    @retep998
    Contributor

    If you really want to allow the OS to do a good job of paging out memory you need to ensure that modules don't share pages with each other. If a given module is spread out across your memory with lots of fragmentation, then you'll have to have a lot more memory paged in than necessary, reducing the effectiveness of the OS paging stuff out. Also, there are ways to hint to the OS that you don't need some memory at the moment, so it can be more willing to page it out sooner in favor of keeping other memory in memory.

  6. michaelwoerister commented on Jan 25, 2017

    @michaelwoerister
    MemberAuthor

    It seems to me like letting the OS take care of this via paging is really the most efficient and most easily implemented solution. But I'm also a bit concerned about rustc getting the reputation of being a memory hog, even if unfairly. It also allows for fewer crates to be compiled by a 32-bit compiler (although the script crate, for example, is already above 2GB before trans, I think).

  7. hanna-kruppe commented on Jan 25, 2017

    @hanna-kruppe
    Contributor

    Would option 2 actually be any slower or otherwise less efficient? It doesn't seem to do any additional work. tcx would remain in memory longer, but since (in the case of huge crates with many modules) the proposed change saves a lot of memory on LLVM modules that shouldn't hurt much, if at all. Besides, in the easy case (enough threads to trans all CGUs in parallel), the tcx can still be dropped as soon as IR was generated and before the LLVM passes run.

    As @retep998 outlines, it is not guaranteed that the OS will actually do a perfect job of paging modules in and out. So ignoring option 1, which does a lot of unnecessary disk I/O, the choice seems to be between doing the same work and either:

    • Allocate all the memory up-front and pray the OS handles it gracefully, or
    • Allocate and free memory as needed, to hopefully avoid the need for paging in the first place.
  8. nikomatsakis commented on Jan 25, 2017

    @nikomatsakis
    Contributor

    I'm a bit confused. It seems to me that there are three things that need to be done:

    • Translation from HIR to LLVM IR.
    • Optimization and transformation of LLVM IR.
    • Generate .o file.

    Currently, we do the first one for everything. Then we start some threads to do the second one -- at what point do we do the third one? (I can't remember.) In particular, do we wait for all the threads to finish?

  9. michaelwoerister commented on Jan 25, 2017

    @michaelwoerister
    MemberAuthor

    at what point do we do the third one?

    That's done by the last LLVM pass, IIRC.

  10. nikomatsakis commented on Jan 25, 2017

    @nikomatsakis
    Contributor

    @michaelwoerister so, in that case, I disagree with the assessment of "Let the O/S do it". In particular, it seems likely that "optimization and transformation of the LLVM IR" takes a lot of (temporary) memory. If we start N threads (one per CGU) and those are all going simultaneously, that's going to lead to a larger peak than is necessary. But if we throttle starting at phase 2, this will reduce the peak, right?

    In particular, the O/S is good at paging memory that is no longer used, but if we start N threads, all that memory is in active use ("working set" is large). If we throttle, working set is smaller, and we can free as we finish each item (since .o file is generated), no?

    It might reduce the peak further to do the full pipeline one CGU at a time, but maybe not, because as it is we can free the HIR and tcx entirely once trans is done (and I believe we do so).

  11. 25 remaining items

  12. sophiajt commented on Jul 3, 2017

    @sophiajt
    Contributor

    @nikomatsakis - sounds like this one might be open for people jumping in. Would still be great to see this one go in

  13. nikomatsakis commented on Jul 17, 2017

    @nikomatsakis
    Contributor

    @michaelwoerister has done some initial refactoring, removing at least one obstacle here.

  14. nikomatsakis commented on Jul 17, 2017

    @nikomatsakis
    Contributor

    I think he plans to keep working on it.

  15. added
    I-compilememIssue: Problems and improvements with respect to memory usage during compilation.
    C-cleanupCategory: PRs that clean code up or issues documenting cleanup.
    on Jul 26, 2017
  16. alexcrichton commented on Aug 10, 2017

    @alexcrichton
    Member

    @michaelwoerister this is closed now right with #43506 landed?

  17. michaelwoerister commented on Aug 10, 2017

    @michaelwoerister
    MemberAuthor

    Yes 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-codegenArea: Code generationA-concurrencyArea: ConcurrencyA-incr-compArea: Incremental compilationC-cleanupCategory: PRs that clean code up or issues documenting cleanup.E-mentorCall for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.I-compilememIssue: Problems and improvements with respect to memory usage during compilation.I-compiletimeIssue: Problems and improvements with respect to compile times.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions