hckrnws

> The symptoms were confusing: streaming worked perfectly with cURL and Postman, but failed completely with node-fetch and browser fetch.

It would have been helpful to mention what "failed completely" means. Did you get garbage data? Did the connection close abruptly? Did the connection hang and not deliver data? Did it deliver the data, just with a significant delay?

Paying attention to these things also tends to make it easier to debug.

I don't quite get it.

cURL works because it doesn't compression.

Browsers do not work because they're accepting compression and cloudflare silently enables compression to browser who advertises that they can accept compression.

So cloudflare's compression is just flawed?

I guess their compression buffers maybe a few KB of data before sending any output, and for streaming individual words from a chatbot, a few KB looks like no streaming at all

> One thing that stuck out was that our egress systems (ALB and Cloudflare) were stripping these headers:

  'Transfer-Encoding': 'chunked',
  'Connection': 'keep-alive',
  'Content-Encoding': 'none',

> Unlike Content-Encoding (Section 8.4.1 of [HTTP]), Transfer-Encoding is a property of the message, not of the representation. Any recipient along the request/response chain MAY decode the received transfer coding(s) or apply additional transfer coding(s) to the message body, assuming that corresponding changes are made to the Transfer-Encoding field value.

(https://www.rfc-editor.org/rfc/rfc9112#section-6.1)

> Intermediaries MUST parse a received Connection header field before a message is forwarded and, for each connection-option in this field, remove any header or trailer field(s) from the message with the same name as the connection-option, and then remove the Connection header field itself (or replace it with the intermediary's own control options for the forwarded message).

(https://datatracker.ietf.org/doc/html/rfc9110#section-7.6.1-...)

> Furthermore, intermediaries SHOULD remove or replace fields that are known to require removal before forwarding, whether or not they appear as a connection-option, after applying those fields' semantics. […] Transfer-Encoding

(https://www.rfc-editor.org/rfc/rfc9110#section-7.6.1-7)

I.e., it is spec-legal for an intermediary to remove these headers; it should be obvious that these are a property of the hop if you consider their purpose.

E.g., say your load-balancer is maintaining a keep-alive with the backend; a client sending Connection: close is not having it's header "stripped" by the LB proxying the request to the backend but without forwarding the header, it's a property of the client<->LB connection, and not the LB<->BE connection.

Same for Transfer-Encoding: consider an HTTP/1.1 connection hitting an intermediary that will upgrade it to HTTP/2; Transfer-Encoding: chunked makes no sense in h2 (its an innate property of h2), and the header will be removed from the proxied request.

Now, obviously, if an intermediary receives a "streaming" response, one hopes a "streaming" response goes out. (But I've written what amounts to an intermediary; it would de-stream responses sometimes b/c the logic it was implementing as an intermediary required it to. So … I also know "it depends", sometimes.)

> Compression breaks HTTP streaming - This is now permanently etched in my brain

It shouldn't.

But that leaves one more header:

  'Content-Encoding': 'none',

> Content codings are a property of the representation data, so a strong entity tag for a content-encoded representation has to be distinct from the entity tag of an unencoded representation to prevent potential conflicts during cache updates and range requests.

(I.e., if you changed the Content-Encoding header, either by removing it & decompressing the message, or by adding it & compressing the message, you would be corrupting the/sending a wrong ETag.)

But also … "none" (the string literal?) is not a valid Content-Encoding. (The header would be omitted, typically, if no content-coding is applied.)

This could just be CF idiosyncrasies, or bugs. I don't see why compression being supported by the client should de-stream the response. One could stream the compression on the fly (and inform the client of the coding used via a Transfer-Encoding to the downstream client; if the protocol doesn't support that, e.g., h2, then probably one should just forward the message without mucking with it…).

Given that the browsers are probably doing at least h2 with CF (i.e., a browser is not speaking HTTP/1.x) … there wouldn't be a Transfer-Encoding. (I don't know if QUIC changes the situation here any. Perhaps I'll assume QUIC works like h2, in that there is only Content-Encoding.) So that would mean that if CF is compressing the response, and there's no Transfer-Encoding header … then it would be doing so & setting the Content-Encoding header, which smells wrong to me. So with curl, setting & not setting the --compressed flag, how do the responses differ. (And perhaps also control/vary the HTTP version.)

I have a system at work that relies on HTTP long polling + chunked transfer encoding to do "streaming"

It took me a while to realise you can't observe this properly through "mitmproxy" during dev like you can with a regular request because the proxy by default effectively tries to wait until the entire response is in before forwarding it.

cURL at least does minimal to no buffering processing chunked responses.

"This isn't just about compression—it's about"

This isn't just about a technical issue—it's about...

...the effort one puts into their writing and how it affects perception of content, i.e. here's some extremely common LLM slop the writer couldn't be sussed to edit, what else did they miss? Does it affect anything I gleaned from the article?