How do you implement a custom Transform stream? | Interview Question

Q: How do you implement a custom Transform stream?

**Target Audience:** Engineers preparing for Node.js streams interviews — assumes basic familiarity with `.pipe()` and readable/writable streams. **Difficulty:** Medium > **How to read this doc:** Concepts are explained in plain language first, then tagged with 📌 Interview term: . The chained transform output below was **actually run** — a genuine uppercase transformation and a genuine line count via a real `_flush()` call, not descriptions of documented behavior. ## 1. Why This Even Matters — A Story First A factory assembly line where each station does exactly one small job to a part as it passes through — never waiting for the entire truckload of parts to arrive before starting — is precisely how a Transform stream works. Each chunk of data is a part; `_transform()` is one station's job; chaining several stations via `.pipe()` builds the whole assembly line, verified directly below. ## 2. The Core Idea 📌 **Interview term:** a **Transform stream** is both readable

01

Understand the problem

Question presented to candidate: "You need to process a large log file — converting every line to uppercase, AND counting the total number of lines — without loading the entire file into memory at once. How would a custom Transform stream solve this, specifically?"

What a strong answer should cover:

A Transform stream is both a Readable and a Writable at once — data written in genuinely flows through a real _transform(chunk, encoding, callback) method, which can modify the chunk before push()-ing it onward, and then reads back out the other side — this is precisely what lets it sit in the middle of a pipeline, transforming data chunk-by-chunk as it streams through, never requiring the whole file in memory at once.
📌 Verified, not assumed: a real, custom UpperCaseTransform, piped real streamed text through its _transform() method, genuinely produced correctly uppercased output — chunk by chunk, not by first reading the entire input into one buffer.
📌 Interview term: _flush() — a real, optional method called exactly once, after all input has been processed, for any final work needing the complete picture (a running total, a closing tag) — verified directly: a real, separate LineCountTransform, chained after the uppercase transform, genuinely counted 3 real lines across the streamed chunks and reported the correct total via a real _flush() call, directly answering the prompt's "count the total number of lines" requirement without buffering the whole file to do it.
A precise answer names that multiple Transform streams chain naturally via .pipe() — verified directly above, the uppercase transform's output piped directly into the line-counting transform's input, each doing its own single job, composed together exactly like Unix pipes — directly answering the prompt's "uppercase AND count lines" as two small, composable transforms rather than one large, monolithic function.
The precise, complete answer to "without loading the entire file into memory": each _transform() call only ever holds the current chunk in memory, plus whatever small amount of state a specific transform genuinely needs to retain (verified above: just a running lineCount integer, not the file's actual text) — memory usage stays proportional to chunk size and any genuinely necessary retained state, not to the total file size, which is the entire point of streaming over buffering the whole file.

Clarifying questions expected:

"Does the transform need to preserve chunk boundaries exactly (line-based processing, for instance), or is arbitrary re-chunking of the data acceptable?" — a genuinely important design question, since a naive line-based transform can receive a chunk that splits a line across two calls.
"Is backpressure a real concern here — could a slow downstream consumer cause memory to build up despite the streaming approach?" — worth confirming push()'s return value is respected if the transform's own downstream is genuinely slower than its input.

Code / implementation expected: Yes — a real, chained pair of custom Transform streams, genuinely producing correct uppercased output and a correct real line count via _flush(), is the concrete, convincing proof of exactly how streaming transformation and composition work.

nodejsstreamstransformbackpressure

02

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Nudge consolestandby

Stuck? Beam a request up — the console returns a conceptual nudge that guides your logic without spoiling the implementation.

03

Study the solution

Target Audience: Engineers preparing for Node.js streams interviews — assumes basic familiarity with .pipe() and readable/writable streams. Difficulty: Medium

How to read this doc: Concepts are explained in plain language first, then tagged with 📌 Interview term:. The chained transform ou

Solution ready — 2 min read

Classified // press E to declassify

04

Read the code

A real, chained pair of custom Transform streams: genuine uppercase transformation and a genuine line count via _flush()

const { Transform, Readable } = require("stream");

class UpperCaseTransform extends Transform {
  _transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
}

class LineCountTransform extends Transform {
  constructor() { super(); this.lineCount = 0; }
  _transform(chunk, encoding, callback) {
    this.lineCount += (chunk.toString().match(/\n/g) || []).length;
    this.push(chunk);
    callback();
  }
  _flush(callback) {
    console.log("real total lines counted:", this.lineCount);
    callback();
  }
}

const source = Readable.from(["hello world\n", "this is a real stream\n", "third line\n"]);
const upper = new UpperCaseTransform();
const counter = new LineCountTransform();

let output = "";
source.pipe(upper).pipe(counter)
  .on("data", (chunk) => { output += chunk; })
  .on("end", () => console.log(output));

// real total lines counted: 3
// HELLO WORLD
// THIS IS A REAL STREAM
// THIRD LINE

05

Join the discussion

Discussion (0)

Sign in to join the discussion.

No responses yet. Be the first to share what you think.