Repository navigation
Description
UPDATE: There are some mentoring instructions below if you're interested in looking into this issue.
NB: Part of the roadmap issue on incremental compilation.
Currently the compiler translates all codegen units into LLVM modules and then runs LLVM on all of them in parallel. In the context of incremental compilation, where there can be hundreds of codegen units for a single crate, but also for non-incremental builds with a high number of codegen units, this can mean a lot of memory pressure -- e.g. for Servo's script crate that's more than 10 GB peak memory usage (while otherwise it's around 4.5 GB).
There's no real need to keep more LLVM modules in memory than are currently being worked on. Two possible solutions:
- If the compiler uses
NLLVM threads, start persisting LLVM modules to disk at theN+1st and reload them later for optimization and codegen. That's probably relatively easy to implement but involves more disk I/O than necessary. - Translate only
Ncodegen units to LLVM modules, translate them all the way to object files, then go back and translate the next codegen unit. This has the advantage that we would not need to temporarily store anything on disk, but we would need to keep thetcxin memory as long as there are still untranslated codegen units. It would also require a much bigger refactoring than the first approach.
Any other ideas?
cc @rust-lang/compiler @rust-lang/tools @nagisa
Activity
nrc commentedon Jan 24, 2017
Will this actually cause problems? Unless there are too many threads (in which case that should be addressed), it feels like the OS will handle this better by paging to disk than we will by (effectively) doing so manually. Also, for many devs, 10GB is not too much memory usage and anything we do here (unless we're really smart) will slow down compilation for those users.
alexcrichton commentedon Jan 25, 2017
My first thought would be the latter, consider translation part of a "work unit" and don't actually start doing it until it's ready to go all the way into LLVM. That way we cap the number of active LLVM modules at any one point in time.
I think @nrc's got a good point, though, in that using the disk as a cache may be better left to the OS rather than the compiler itself (until proven otherwise, of course)
michaelwoerister commentedon Jan 25, 2017
Some data points:
It lead to my not being able to compile the
scriptcrate on Linux with 8 GB. Maybe if I had more swap space it would have worked. (EDIT: The VM had 4GB swap space)If everything fits into memory at once, having hundreds of parallel LLVM threads does not seem to have a detrimental effect (running 140 threads was a few percent faster than having 4 threads on a 4 core processor). OTOH, I would suspect that paging would completely destroy performance if many threads had to fight for RAM.
That's certainly a valid concern!
retep998 commentedon Jan 25, 2017
If you really want to allow the OS to do a good job of paging out memory you need to ensure that modules don't share pages with each other. If a given module is spread out across your memory with lots of fragmentation, then you'll have to have a lot more memory paged in than necessary, reducing the effectiveness of the OS paging stuff out. Also, there are ways to hint to the OS that you don't need some memory at the moment, so it can be more willing to page it out sooner in favor of keeping other memory in memory.
michaelwoerister commentedon Jan 25, 2017
It seems to me like letting the OS take care of this via paging is really the most efficient and most easily implemented solution. But I'm also a bit concerned about
rustcgetting the reputation of being a memory hog, even if unfairly. It also allows for fewer crates to be compiled by a 32-bit compiler (although thescriptcrate, for example, is already above 2GB before trans, I think).hanna-kruppe commentedon Jan 25, 2017
Would option 2 actually be any slower or otherwise less efficient? It doesn't seem to do any additional work.
tcxwould remain in memory longer, but since (in the case of huge crates with many modules) the proposed change saves a lot of memory on LLVM modules that shouldn't hurt much, if at all. Besides, in the easy case (enough threads to trans all CGUs in parallel), thetcxcan still be dropped as soon as IR was generated and before the LLVM passes run.As @retep998 outlines, it is not guaranteed that the OS will actually do a perfect job of paging modules in and out. So ignoring option 1, which does a lot of unnecessary disk I/O, the choice seems to be between doing the same work and either:
nikomatsakis commentedon Jan 25, 2017
I'm a bit confused. It seems to me that there are three things that need to be done:
.ofile.Currently, we do the first one for everything. Then we start some threads to do the second one -- at what point do we do the third one? (I can't remember.) In particular, do we wait for all the threads to finish?
michaelwoerister commentedon Jan 25, 2017
That's done by the last LLVM pass, IIRC.
nikomatsakis commentedon Jan 25, 2017
@michaelwoerister so, in that case, I disagree with the assessment of "Let the O/S do it". In particular, it seems likely that "optimization and transformation of the LLVM IR" takes a lot of (temporary) memory. If we start N threads (one per CGU) and those are all going simultaneously, that's going to lead to a larger peak than is necessary. But if we throttle starting at phase 2, this will reduce the peak, right?
In particular, the O/S is good at paging memory that is no longer used, but if we start N threads, all that memory is in active use ("working set" is large). If we throttle, working set is smaller, and we can free as we finish each item (since
.ofile is generated), no?It might reduce the peak further to do the full pipeline one CGU at a time, but maybe not, because as it is we can free the HIR and tcx entirely once trans is done (and I believe we do so).
25 remaining items
sophiajt commentedon Jul 3, 2017
@nikomatsakis - sounds like this one might be open for people jumping in. Would still be great to see this one go in
nikomatsakis commentedon Jul 17, 2017
@michaelwoerister has done some initial refactoring, removing at least one obstacle here.
nikomatsakis commentedon Jul 17, 2017
I think he plans to keep working on it.
alexcrichton commentedon Aug 10, 2017
@michaelwoerister this is closed now right with #43506 landed?
michaelwoerister commentedon Aug 10, 2017
Yes 🎉