Node JS : send asynchronously produced data

I am Creating WebSite Crawling App, which will take site url and then starts crawling entire site asynchronously.

Now here i will send response of given url with its status code as in 200, 404 etc.

But then all the other url on that particular page will be going through the same process but as response is already sent so how the other urls response code will be sent.

please suggest how to send updated value in response again after sending it once ?

You pretty much can't modify a response after its sent. I think there are exceptions to this, but you probably don't want to use them. I think you have a couple of options

  • Catch all your callbacks and send one response
  • Open up some kind of tcp connection (socket.io?) and stream the data.

Can you provide a code sample?