Question presented to candidate: "You need to compress a large file while reading it from disk and writing the result to a new file, without ever holding the whole file in memory. What Node.js concept is built exactly for this?"
What a strong answer should cover:
- A stream is an abstraction for working with data incrementally, in chunks, rather than loading an entire dataset into memory before processing it — the foundational idea behind file I/O, HTTP request/response bodies, and compression, all built on the same stream interfaces.
- There are four stream types: Readable (a source of data — a file read, an HTTP request body), Writable (a destination — a file write, an HTTP response), Duplex (both readable and writable, independently — a TCP socket), and Transform (a Duplex stream where the writable side's input is processed into the readable side's output — compression, encryption, parsing).
- 📌 Verified, not just described: a real custom
Transformstream (uppercasing each chunk) correctly produced"HELLO WORLD"from two separately-written chunks — confirming the transform genuinely processes data incrementally as it flows through, not as an all-at-once operation. - Streams exist specifically to solve the problem covered with directly measured proof in the dedicated readFile-vs-createReadStream and blocking-vs-non-blocking questions: bounding memory usage and starting output before an entire input has been fully read, regardless of total data size.
.pipe()(and its modern replacement,stream.pipeline(), both covered in their own dedicated questions) connects streams together —fs.createReadStream(input).pipe(zlib.createGzip()).pipe(fs.createWriteStream(output))is the canonical answer to the compression scenario in the prompt: a Readable, through a Transform, into a Writable, with no full-file buffer ever held in memory.- A precise answer names that streams are also EventEmitters underneath (
'data','end','error','finish'events) — the two question topics connect directly, covered in the dedicated EventEmitter question.
Clarifying questions expected:
- "Is the data source/destination large enough, or streamed from an external source, such that memory-bounding actually matters here?" — for a genuinely small, fully-available dataset, streams add complexity without a real benefit.
- "Does this need a Transform in the middle, or just a direct Readable-to-Writable pipe?" — decides how much of the four-type taxonomy is actually relevant.
Code / implementation expected: Yes — a real, working custom Transform stream, plus the canonical file-compression pipe chain, is the concrete deliverable here.