To log a response’s length, a proxy first calls await response.text() and then creates another Response. Data that could have been forwarded incrementally now waits for the whole body to arrive. A stream-shaped API is still present, but the application has turned the operation into whole-body buffering.
This guide uses a Worker forwarding a large public file as its example. Workflows requiring complete parsing, signing, or sanitization may legitimately buffer content. They still need a maximum size and a timeout. Decide whether early output is safe before optimizing for it; do not send unverified content simply to claim that every endpoint streams.
Forward a body without first collecting it
The source’s minimal public-file proxy preserves the upstream stream:
const upstream = await fetch(sourceURL);
return new Response(upstream.body, {
status: upstream.status,
statusText: upstream.statusText,
headers: approvedHeaders(upstream.headers),
});sourceURL and approvedHeaders belong to the application policy. The latter should remove inappropriate internal diagnostics and cookies. If a transformation changes bytes, it must also reconsider Content-Length, encoding and validators. Calling text before this return changes the pipeline into whole-body buffering even if a new Response is later constructed from the collected string.
Observe four stages instead of one API name
An upstream can flush regularly while the browser still displays everything at the end because the client calls text. Conversely, a ReadableStream implementation can feed previously buffered content. Observe upstream production, proxy forwarding, client reading and UI updates separately. UTF-8 decoding and message delimiters need finite carry-over state; a network chunk can split one scalar or contain several events. Full-content validation can legitimately delay output, but then the maximum size and accepted latency tradeoff must be stated.
Account for every branch retaining data
A tee feeding a fast client and a slow cache or logging branch can change buffering behavior. Do not clone large bodies for diagnostics when byte counts and time observations suffice. Cancellation should stop upstream reads and release handles under the actual adapter contract. After headers are sent, an unexpected midstream failure cannot become a fresh 500 page. Test finite slow-producer and slow-consumer fixtures, partial characters, oversized messages and disconnects; verify incomplete files fail integrity rather than appending normal-looking HTML to binary content.
A partial binary file cannot carry a JSON apology
Once half a binary download has been sent, appending an HTML or JSON error corrupts the file rather than returning a new error representation. Interrupt transmission and rely on the download protocol’s failure or integrity mechanism. A reliable length or digest may help the consumer detect incompleteness, but the delivery and verification contract must actually support it; an ignored custom header is not an integrity guarantee.
An event stream can define a typed failure and terminal state. Keep received partial data visibly incomplete and avoid exporting it as a finished result. Resume requires a defined offset or event identity, a retention window and duplicate handling; reconnecting alone does not make concatenation correct. Capacity also multiplies across connections. A memory-stable single stream can become costly with many simultaneous consumers, so account for per-stream buffers, upstream concurrency and cancellable admission waiting. First-byte time, complete transfer time and post-cancellation work answer different questions and should remain separate observations.
Transport chunks, application messages, and rendering have different boundaries
An upstream can flush a piece of text every second while the browser displays everything at the end. The client may be calling text() rather than reading incrementally. Conversely, code can construct a ReadableStream from a complete in-memory file and still look stream-oriented. Neither the API name nor one server-side timestamp proves end-to-end behavior.
Observe upstream production, proxy forwarding, browser reading, and interface updates separately. Record when data becomes available at each stage and when the user sees a usable result. Middleware can change the behavior between observations. Compression, logging, caching, content scanning, and error wrappers may all read or collect the body before the final client gets it.
Application messages have their own boundaries. A line-oriented log ends a message at a newline, an event protocol defines delimiters, and a JSON document may require a complete syntax tree. A network chunk can contain several messages or only part of a character. Never parse each chunk as one independent JSON object unless the application protocol explicitly supplies that guarantee.
For an unmodified file proxy, not parsing the body is often the simplest correct option. Forward the readable body and apply an explicit header policy. If verification must finish before download begins, accept the resulting latency and storage requirements. Streaming is a transport technique, not permission to bypass a security or integrity boundary that the product actually needs.
Header handling is part of the body contract
The minimal proxy example above uses application-defined sourceURL and approvedHeaders functions or values. It is not an unrestricted public URL proxy. The application must control the upstream target and decide which metadata may cross its boundary. Do not forward internal diagnostics or cookies solely because copying the entire header collection is convenient.
If a transformation changes the bytes, an old Content-Length may no longer describe the body. Representation validators and signatures may also be invalid. Review encoding and compression metadata against the bytes actually sent. A header that was correct at the upstream can become incorrect after decoding, rewriting, or recompression in the proxy.
Runtime handling of compressed bodies deserves particular care. Determine where decompression and recompression happen rather than assuming a text transform receives the same representation described by the original headers. Unsupported encoding can justify refusing transformation or forwarding unchanged under an explicit policy. Writing UTF-8 in a log line does not make arbitrary input bytes valid UTF-8.
Test the headers together with the consumer behavior. A download that begins quickly but later fails because the declared length is wrong is not a successful streaming optimization. Compare a byte-preserving path with a transformed path and verify which metadata remains meaningful. Keep examples narrow enough that omitted header policy cannot be mistaken for a production-safe default.
Backpressure is demand, not a synonym for chunking
Suppose an upstream produces ten megabytes per second while the consumer receives one. If application code keeps reading and appending chunks to an unbounded queue, the difference accumulates. Splitting output into small pieces does not change that arithmetic. The consumer must influence how much work the producer is allowed to continue doing.
Readable and writable streams can form a demand-aware pipeline. A manual loop still has responsibilities: respect writable readiness, avoid uncontrolled enqueue operations, and propagate terminal conditions. Putting chunks in an array while a separate asynchronous task sends them later moves the queue rather than bounding it. Backpressure cannot protect an unrelated producer that ignores the queue’s state.
The source uses a pass-through transformation to demonstrate the pipeline boundary:
const transformed = upstream.body.pipeThrough(new TransformStream({
transform(chunk, controller) {
controller.enqueue(chunk);
},
}));Here upstream is a response whose body has already been checked for availability. The transform performs no business conversion. If the body may be null, handle that case before calling pipeThrough. Real transforms also need bounds on retained input and output. A callback that collects every chunk in an external array defeats the memory objective even when the stream’s own queues behave correctly.
Observe actual memory and queue behavior under a deliberately slow consumer. The runtime and upstream may have additional buffers beyond the application’s visible queue. Standards-based interfaces do not justify promising constant memory across every adapter and network. State the tested conditions and distinguish a bounded application buffer from the total resource footprint of the connection.
Text decoding requires state and a deliberate final flush
A UTF-8 character can span byte chunks. Independently decoding each chunk can produce replacement characters and corrupt paths or diagnostic messages. Use a decoder with streaming state, and complete decoding at end of input. Decide whether invalid encoding is rejected or replaced according to the protocol; do not let the decoder’s default behavior silently define the business contract.
Text decoding alone does not assemble messages. For newline-delimited input, append decoded text to the unfinished remainder, emit only completed lines, and retain the final fragment. When the stream ends, the protocol must decide whether a last line without a newline is acceptable. A clean transport end does not automatically establish a complete application message.
Bound the unfinished message explicitly. An upstream can send many tiny chunks without ever sending a delimiter. Even though each chunk is small, the remainder grows without limit. Reaching the message-size ceiling should produce a protocol failure and cleanup, not silent truncation followed by normal processing. Truncation can alter identifiers, paths, or security decisions.
Test identical logical text under several chunk arrangements: one character split across chunks, a delimiter split across chunks, multiple messages in one chunk, and empty chunks. The emitted message sequence should be consistent. Add invalid encoding, overlong messages, and incomplete final messages. Check released locks and timers as well as the presence of an exception.
A stream can still hide a whole-body algorithm
Collecting a file into an array, hashing it after the final chunk, and returning it is not a low-memory streaming implementation. It may be a legitimate bounded-buffer implementation, but name the tradeoff accurately. Include auxiliary transformations and diagnostic branches in the inventory of retained bytes. A streaming core handler does not make every attached operation incremental.
Some hashing algorithms support incremental processing, but that does not prove the runtime API exposed to this project accepts incremental input. Use a mature implementation appropriate for the environment when required. If the available API needs a complete buffer, impose a file-size limit or move the operation to a component with suitable storage and execution capabilities.
Content validation has the same tension. If the whole document must be inspected before any byte may be trusted, early forwarding can violate the intended safety condition. Specify whether partial output is allowed and who can use it. Do not treat a lower first-byte measurement as a win when it comes from sending data before a required verification step.
Budgets should cover retained message fragments, transformed output, hash state, temporary files, and test logs where relevant. Logging the full body for troubleshooting can consume more memory or storage than the feature being measured. Byte counts, timing observations, and safe reason categories often provide enough evidence without retaining another content copy.
A slow tee branch changes the resource model
Teeing a body creates two consumers, not two free independent copies with an automatic constant-memory guarantee. A fast client branch can continue while data needed by a slower cache branch remains buffered. Response cloning can introduce similar multiple-consumer considerations. Behavior acceptable for a small HTML page may be unsuitable for a very large download.
Test a fast client with a slow cache, a slow client with a fast cache, and cancellation of either branch. Observe total retained bytes and actual completion, not only the fastest branch’s throughput. A logging branch that calls text() still reads and retains the whole representation. It changes the cost model even if the user-facing branch streams correctly.
If the cache branch fails, decide whether to cancel that branch while preserving a valid client transfer. If the client disconnects first, decide whether caching is allowed to continue and under which size and lifetime limits. It is an intentional background workload, not an accidental infinite extension of the original request.
One way to reduce risk is to cache only approved resource sizes or leave immutable file delivery to a suitable platform cache. Do not read the whole object merely to discover it is too large to cache: the buffering cost has already been paid. A trustworthy length header can inform admission, but observed byte counts still need an enforced budget when the upstream declaration is not sufficient.
Cancellation has to terminate real work
The user closes a page. Does the upstream keep downloading? A cancellation signal expresses intent; it does not by itself prove that the connection, parser, cache branch, or associated background task stopped. Propagate cancellation through the reader, upstream fetch, and pipes as supported by the runtime, then observe the actual resource lifecycle.
A controlled upstream can record progress and its stop event. Let a test client read a finite amount and disconnect, then inspect whether unnecessary work ends within the intended bound. Checking AbortController.signal.aborted alone proves only that a flag changed. A local emulator also cannot establish precisely how the deployed adapter handles a real socket disconnect.
Distinguish normal user cancellation from upstream timeout, transform failure, and total deadline expiration. Otherwise a popular long-running subscription may generate misleading server-error alerts whenever users navigate away. Cleanup belongs on normal completion and every failure path: release locks, cancel unused readers where appropriate, and clear timers that would otherwise fire after the task ended.
If work intentionally continues after disconnect, give that continuation a clear owner and budget. A public cache fill may be worth completing, but it still consumes bandwidth. Private content must not accidentally become an unrestricted background operation. Platform lifetime facilities such as waitUntil remain subject to platform limits; they are not permission to execute indefinitely.
Idle time, total time, and business progress are different clocks
A large valid file can take a long time to finish while making steady progress. A small task can remain stuck without sending useful data. Use an explicit idle policy and total lifetime where the business requires them rather than applying the same short completion timeout to every response size. Define which observations reset each timer.
Transport heartbeats prove connection activity, not progress toward a result. A task that emits only still-connected events can keep the socket busy without producing useful output. If every heartbeat renews the only timeout, an otherwise stalled task can occupy resources forever. Track transport activity and application progress separately when the protocol needs that distinction.
Test an upstream that sends only valid heartbeats, then one that sends a few results and continues with heartbeats alone. Verify when the task remains legitimately open and when its approved deadline ends it. A real-time subscription can intentionally stay open; a finite report generation task needs a different completion contract. Neither policy follows merely from the presence of incoming bytes.
Retries after output begins need special treatment. Repeating a request and appending its second response to partial earlier data can duplicate content. Safe resumption requires a defined byte offset or event identity, appropriate upstream support, and duplicate handling. Without those capabilities, explain that the user must restart rather than disguising concatenation as a reliable recovery mechanism.
Separate transport cancellation from business cancellation
Closing a browser page does not undo a database write or an already completed operation. When the product offers a stop control, define whether it closes the stream, stops generation, or requests cancellation of a separate business task. One ambiguous button should not promise all three if the backend can only honor the first.
If a report must continue and become downloadable later, model it as a background task with an identifier, access control, and lifecycle. Reconnection can query its actual state. Do not depend on a request context surviving without limits simply because the client once opened a stream. Completion and ownership must remain understandable after the original connection disappears.
The distinction also affects diagnostics. User-requested transport closure is not necessarily a failed report. A generation failure after transport closure may still need an internal alert or a stored task result. Record safe reason categories rather than dumping every data chunk into logs. Observability should explain what stopped and what continued without creating another full copy of the content.
Cancellation tests should therefore assert more than the end of the network response. Check the status of any intentionally independent work and the absence of work that should have stopped. Verify permission checks when a user reconnects to retrieve the result. A task identifier is useful for continuity, but must not become an unauthenticated download capability unless that is a deliberate design.
Clients need bounded consumption and honest completion states
A browser that adds every incoming event to an array forever can grow indefinitely even when the server uses little memory. A log viewer can retain a finite recent window and use suitable storage for history. A file download that must preserve every byte is a different workflow. Do not apply the same client-state policy to both.
Rendering on every tiny chunk can create unnecessary layout work and long tasks. Batch by application messages or a reasonable update schedule, while preserving error and terminal events. The interface should distinguish connecting, receiving, complete, stopped, and failed. Receiving one more piece of data is not the same thing as receiving the protocol’s successful terminal condition.
After a user stops a transfer, retained output is a partial result unless completeness was already established. Do not label an export as a finished report just because the screen contains plausible text. Event identifiers, replay windows, and duplicate rules belong in the protocol when reconnection is supported. Otherwise show a restart action instead of silently adding repeated output to the old view.
Observe active connection counts, durations, cancellation reasons, and retained client state as well as server memory. Long connections naturally exceed ordinary short-request durations. Alert on the documented abnormal condition, such as no useful progress or work continuing after cancellation, not merely on elapsed time. Keep production observations aggregated or appropriately sampled rather than logging the full stream.
Test capacity across concurrent streams, not just one file
A single stable transfer says little about many simultaneous consumers. Account for per-stream buffers, upstream concurrency, open connections, and platform deadlines together. Admission queues also need bounded waiting and cancellation. An unlimited queue before streaming begins simply moves the resource problem to an earlier phase of the request.
Long legitimate files hold capacity for longer than small ones. Decide fairness and limits according to the workload rather than treating every long connection as abuse. Test a mix of consumers, including slow ones, with a fixed controlled upstream. Record first-byte time, total transfer time, memory peaks, and work remaining after disconnect as separate measurements.
Streaming may improve first-byte latency or reduce application buffering without making the total transfer faster. A buffered implementation can be simpler for small bounded objects. State the conditions and limits of a comparison; do not invent a universal speedup or memory guarantee from the presence of a Stream API. The useful question is whether the chosen implementation meets the actual delivery and resource contract.
Finish with an end-to-end check through the deployed entry point, including compression, caching, logging, and error middleware. Test slow producers, slow consumers, large responses, split characters, oversized messages, and deliberate disconnects. Confirm that incomplete output is reported honestly and cleanup happens. A working pass-through component is only one link in the complete path from upstream work to a trustworthy user result.