How to write a middle-tier http API endpoint that can stream results as they arrive to the client?

The scenario is this - I have a frontend web-server that I'm writing in node.js. I have an as-yet-unwritten middle-tier internal-API layer written in, well, anything. The internal-API is the only thing allowed to talk to the data-store (which happens to be a relational database).

Disclaimer: I'm a node.js beginner.

node.js wants to do data-access asynchronously - that makes calls like Database.query.all inefficient, since the response callback wouldn't start until the whole list has been assembled. Documentation I've read suggests that instead, it'd be better to stream results one at a time to the client.

I would like to know how to write the frontend and middle-tier http internal-API such that I can take advantage of node.js' asynchronicity, here.

I guess the question is "how do I stream structured data over http"? I guess that's the feature of the internal API that I'm asking for support for.

Should I:

  • Get the frontend to ask for a list of IDs, then issue one request each to the backend? Sounds crude and chatty, plus I don't see a guarantee that the requests will return in the order that I want, so I'd have to wait 'til I had everything back at the frontend anyway..?
  • Get the frontend to make a series of requests against the internal API for pages of data, and treat each chunk as a stream-segment...?
  • Fetch only enough data for the first screen's worth, then request for subsequent chunks, writing each one to the end of the list as it arrives?
  • something cleverer!?

(Note: please don't say "get rid of the middle-tier so you can talk to the database directly" - that's not an option)

I am not sure what exactly you mean by "streaming"; from the ideas you give, it could be either interpreted as some HTTP server push or long polling technique, or simply making subsequent XHR requests.

Since you're using node, I recommend Socket.io, which allows you to really push data to the browser whenever you want.

If you chose to go with XHRs, simply tell the browser what to request next.

If that doesn't fit you, and you want to use server push or long polling, response.write() seems the way to go. But you will probably run into problems with request timeouts and such.