Implementing a custom filtering Transform stream
Knowing the Transform stream contract.
Subclass Transform with objectMode, implement _transform to parse each chunk, push only matching objects, and call the callback; handle parse errors.
WHAT THIS TESTS: Whether you understand that a Transform stream is a Duplex that maps input chunks to output chunks, and whether you know the _transform/_flush callback contract and the realities of chunk boundaries.
A GOOD ANSWER COVERS: Subclass stream.Transform (or pass transform options to the constructor). Set objectMode true on the readable side so you can push parsed objects rather than buffers, and choose the writable side mode based on whether input arrives as strings or pre-split objects. Implement _transform(chunk, encoding, callback): parse the JSON for that record, test whether the resulting object has the target property, and if so call this.push(obj) to emit it downstream; objects without the property are simply not pushed, which filters them out. You must always call callback() exactly once when the chunk is processed, or callback(err) to surface an error; failing to call it stalls the entire pipeline because the stream waits for completion before requesting the next chunk. A subtlety: when reading raw bytes, a single chunk is not guaranteed to be one whole JSON record, so for newline-delimited JSON you buffer partial lines and split on newlines, handling any remainder in _flush(callback), which runs after the last chunk. Wrap JSON.parse in try/catch so a malformed line does not crash the process; decide whether to skip it or emit an error.
COMMON WRONG ANSWERS: Forgetting to call the callback, freezing the stream; assuming each chunk is exactly one complete JSON object; using push for filtered-out items anyway; not enabling objectMode and trying to push objects through a byte stream; ignoring backpressure by not respecting push's return value at high volume.
LIKELY FOLLOW-UPS: How do you handle JSON records split across chunk boundaries? What does _flush do? When do you signal an error vs skip a bad record? How does objectMode change buffering?
ONE CONCRETE EXAMPLE: A Transform in objectMode receives newline-delimited JSON, buffers partial lines, parses each complete line, and pushes only objects where obj.active is true; malformed lines are caught and skipped, and the trailing buffered line is parsed in _flush. Piped between a file read and a writer, it filters a huge log without loading it all into memory.
Read the original → nodejs.org
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.