Skip to solution
mediumLow-Level Design

What are Node.js Streams and when would you use them?

615 views
01

Understand the problem

Question presented to candidate: "You need to compress a large file while reading it from disk and writing the result to a new file, without ever holding the whole file in memory. What Node.js concept is built exactly for this?"

What a strong answer should cover:

  • A stream is an abstraction for working with data incrementally, in chunks, rather than loading an entire dataset into memory before processing it — the foundational idea behind file I/O, HTTP request/response bodies, and compression, all built on the same stream interfaces.
  • There are four stream types: Readable (a source of data — a file read, an HTTP request body), Writable (a destination — a file write, an HTTP response), Duplex (both readable and writable, independently — a TCP socket), and Transform (a Duplex stream where the writable side's input is processed into the readable side's output — compression, encryption, parsing).
  • 📌 Verified, not just described: a real custom Transform stream (uppercasing each chunk) correctly produced "HELLO WORLD" from two separately-written chunks — confirming the transform genuinely processes data incrementally as it flows through, not as an all-at-once operation.
  • Streams exist specifically to solve the problem covered with directly measured proof in the dedicated readFile-vs-createReadStream and blocking-vs-non-blocking questions: bounding memory usage and starting output before an entire input has been fully read, regardless of total data size.
  • .pipe() (and its modern replacement, stream.pipeline(), both covered in their own dedicated questions) connects streams together — fs.createReadStream(input).pipe(zlib.createGzip()).pipe(fs.createWriteStream(output)) is the canonical answer to the compression scenario in the prompt: a Readable, through a Transform, into a Writable, with no full-file buffer ever held in memory.
  • A precise answer names that streams are also EventEmitters underneath ('data', 'end', 'error', 'finish' events) — the two question topics connect directly, covered in the dedicated EventEmitter question.

Clarifying questions expected:

  • "Is the data source/destination large enough, or streamed from an external source, such that memory-bounding actually matters here?" — for a genuinely small, fully-available dataset, streams add complexity without a real benefit.
  • "Does this need a Transform in the middle, or just a direct Readable-to-Writable pipe?" — decides how much of the four-type taxonomy is actually relevant.

Code / implementation expected: Yes — a real, working custom Transform stream, plus the canonical file-compression pipe chain, is the concrete deliverable here.

streamsi/operformancedata handling
02

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Nudge consolestandby

Stuck? Beam a request up — the console returns a conceptual nudge that guides your logic without spoiling the implementation.

03

Study the solution

Target Audience: Engineers preparing for Node.js interviews — assumes very basic file I/O familiarity, no prior stream-specific knowledge required. Difficulty: Medium

How to read this doc: Concepts are explained in plain language first, then tagged with 📌 Interview term:. The Transform stre

Solution ready — 2 min read

Classified // press E to declassify

04

Read the code

A real, working custom Transform stream, plus the canonical Readable-Transform-Writable compression chain
const { Transform } = require("stream");
const fs = require("fs");
const zlib = require("zlib");

// A real custom Transform stream:
const upper = new Transform({
  transform(chunk, enc, cb) {
    this.push(chunk.toString().toUpperCase());
    cb();
  },
});
let out = "";
upper.on("data", (c) => (out += c));
upper.on("end", () => console.log(out)); // HELLO WORLD
upper.write("hello ");
upper.write("world");
upper.end();

// The canonical use case — compressing a large file with bounded memory:
fs.createReadStream("input.txt")
  .pipe(zlib.createGzip())
  .pipe(fs.createWriteStream("input.txt.gz"));
// Readable -> Transform -> Writable; the whole file is never in memory at once.
05

Join the discussion

Discussion (0)

Sign in to join the discussion.

No responses yet. Be the first to share what you think.

Transmission complete // awaiting log

KEEP THE
STREAK ALIVE.

Dossier 82 of 152 decoded in the Node.js track. One more won't hurt.

Back to track