Full source code available here.
In C# it is very easy to avoid doing something asynchronously, but I found out that this is not the case with JavaScript and Node.js.
I wanted to load a large amount of data in a remote service via and API, but had to do it one request at a time. There was no bulk option.
For a little while I’ve been learning JavaScript and some Node, but had not had a chance to use the language on anything useful. This blog post is the first I’ve written on JavaScript, so if you find something that is incorrect, leave a comment and I will update it.
There are many choices for making http requests with Node, for no particular reason I chose Axios.
Here is a very simple example of post method –
async function postDataToApi(index, dataToPost) { try { const response = await axios.post("http://httpbin.org/post", { dataToPost }); console.log(index, response.status) } catch (error) { console.log(error); } }
I am calling await
on the axios.post(...)
method.
For the sake of this example I am going to generate the data to post using the Casual library. This create a fake product with random data in it –
casual.define('product', function () { return { productId: casual.uuid, productName: casual.word, color: casual.color_name } });
Attempt 1
Now I needed call the remote service, sending it data in some sort of loop.
A simple for loop seemed the best candidate.
function simpleForLoop(start, end) { for (let index = start; index <= end; index++) { console.log("request:", index, "for loop calling postDataToApi"); postDataToApi(index, casual.product); } }
But this ran through the whole loop before it made a single outbound request and then attempted to make a very large number of requests simultaneously. This overwhelmed the remote service and got back a bunch of failures.
Here is what the output of the for loop looks like –
request: 1 simpleForLoop calling postDataToApi request: 2 simpleForLoop calling postDataToApi request: 3 simpleForLoop calling postDataToApi response: 2 200 postDataToApi called remote service response: 1 200 postDataToApi called remote service response: 3 200 postDataToApi called remote service
The three requests happen before any response is received. The responses don’t necessarily come in the order the requests were made.
All the requests are happening asynchronously. This is not what I want.
Attempt 2
Let’s try adding an async
and an await
to the for loop.
async function loopWithAwait() { for (let index = 1; index <= 3; index++) { console.log("request: ", index, "loopWithAwait calling post"); await postDataToApi(index, getProduct()); } }
Run the application again, and this time I get the output I want. A request is made, a response is received, then the next request is made and so on.
request: 1 loopWithAwait calling postDataToApi response: 1 200 postDataToApi called remote service request: 2 loopWithAwait calling postDataToApi response: 2 200 postDataToApi called remote service request: 3 loopWithAwait calling postDataToApi response: 3 200 postDataToApi called remote service
Full source code available here.