Repository navigation
Stream destruction breaks async-iteration and ends nodejs event loop #23730
Description
- Version: v10.12.0
- Platform: Windows 10 64-bits
- Subsystem: streams
Async iteration over a readable stream breaks horribly if for any reason the stream is destroyed while the iteration is still running (or even if it is destroyed before starting the iteration). I would expect that the iteration runs normally or throws an exception. What happens instead is that the whole event loop is shut down.
Most notably this makes async iteration useless with pipeline.
Test code:
const {createReadStream} = require('fs');
const {pipeline} = require('stream');
const {createDeflate} = require('zlib');
async function iterate(name, stream, destroy = false) {
try {
console.log(`${name}: Starting loop`);
for await (const chunk of stream) {
console.log(`${name}: Got a chunk`);
if (destroy) {
console.log(`${name}: destroying stream`);
stream.destroy();
}
}
console.log(`${name}: loop ended`);
} catch (err) {
console.log(`${name}: loop ended with error: ${err}`);
}
}
async function main() {
// These work fine
await iterate('oneStream', createReadStream('test.js'));
await iterate('oneStreamError', createReadStream('doesnotexist'));
// Any of these just shuts down the node event loop...
await iterate('oneStreamDestroy', createReadStream('test.js'), true);
await iterate('pipeline', pipeline(createReadStream('doesnotexist'), createDeflate(), err => console.log(`pipeline ended: ${err}`)));
const stream = createReadStream('test.js');
stream.destroy();
await iterate('destroyedStream', stream);
}
main().catch(console.error);Output:
> node test.js
oneStream: Starting loop
(node:21664) ExperimentalWarning: Readable[Symbol.asyncIterator] is an experimental feature. This feature could change at any time
oneStream: Got a chunk
oneStream: loop ended
oneStreamError: Starting loop
oneStreamError: loop ended with error: Error: ENOENT: no such file or directory, open 'C:\Users\mfischer\src\tests\node-stream-async-iteration\doesnotexist'
oneStreamDestroy: Starting loop
oneStreamDestroy: Got a chunk
oneStreamDestroy: destroying stream
Activity
targos commentedon Oct 18, 2018
@nodejs/streams
Hakerh400 commentedon Oct 18, 2018
Simplified repro:
The
resolvefunction assigned toiterator[kLastResolve]never gets called:node/lib/internal/streams/async_iterator.js
Lines 143 to 152 in 05394d2
mcollina commentedon Oct 18, 2018
Definitely it's a bug, as @Hakerh400 showed.
@mika-fischer the event loop is shutting down because there is nothing asynchronous left to do. If you add the following:
The application will keep running
mcollina commentedon Oct 20, 2018
See #23785 for a fix.
mcollina commentedon Oct 24, 2018
Fixed in 3ec8cec.
stream: async iteration should work with destroyed stream
stream: async iteration should work with destroyed stream
stream: async iteration should work with destroyed stream
stream: async iteration should work with destroyed stream
stream: async iteration should work with destroyed stream