Deviopath › Node.js

Chunked Transfer Decoder

Hard · HTTP

When a Node server does not know the response size in advance (for example it calls res.write() in a loop), it sends Transfer-Encoding: chunked: the body is a series of chunks, each prefixed with its size in hexadecimal. Implement a decoder that receives the raw body in pieces cut at arbitrary points.

The format (RFC 9112, every line ends with \r\n):

- A chunk is <size in hex>[;extensions]\r\n<exactly size bytes of data>\r\n. The size is one or more hex digits (any case). Everything from the first ; to the end of the line is a chunk extension: ignore it. Spaces or tabs right before the ; are allowed.

- The last chunk has size 0 (it may have extensions too) and no data.

- After it come optional trailer fields Name: value\r\n (like headers, but sent after the body), then an empty line \r\n that ends the message.

Implement class ChunkedDecoder with feed(text) and result():

- result() returns { body, trailers, done }. body joins the data of every chunk received completely so far (a chunk counts once its data and the \r\n after it arrived). trailers has lowercase names and trimmed values. done becomes true after the final empty line; anything after that is ignored.

- On invalid input throw an Error with the message "Invalid chunk": a size that is not valid hex, chunk data that is not followed by \r\n, or a trailer line without a :.

The data is ASCII (1 character = 1 byte) and may itself contain \r\n, so use the size, not line breaks, to find where the data ends. The driver runChunked(chunks) is already written: it feeds every piece and returns result(), or { error } if feed threw.

Next: Async Generator Pipeline