Synchronous For Loop in Node.js

Full source code available here.

In C# it is very easy to avoid doing something asynchronously, but I found out that this is not the case with JavaScript and Node.js.

I wanted to load a large amount of data in a remote service via and API, but had to do it one request at a time. There was no bulk option.

For a little while I’ve been learning JavaScript and some Node, but had not had a chance to use the language on anything useful. This blog post is the first I’ve written on JavaScript, so if you find something that is incorrect, leave a comment and I will update it.

There are many choices for making http requests with Node, for no particular reason I chose Axios.

Here is a very simple example of post method –

async function postDataToApi(index, dataToPost) {
    try {
        const response = await axios.post("http://httpbin.org/post", {
            dataToPost
        });
        console.log(index, response.status)
    } catch (error) {
        console.log(error);
    }
}

I am calling await on the axios.post(...) method.

For the sake of this example I am going to generate the data to post using the Casual library. This create a fake product with random data in it –

casual.define('product', function () {
    return {
        productId: casual.uuid,
        productName: casual.word,
        color: casual.color_name
    }
});

Attempt 1

Now I needed call the remote service, sending it data in some sort of loop.

A simple for loop seemed the best candidate.

function simpleForLoop(start, end) {
    for (let index = start; index <= end; index++) {
        console.log("request:", index, "for loop calling postDataToApi");
        postDataToApi(index, casual.product);
    }
}

But this ran through the whole loop before it made a single outbound request and then attempted to make a very large number of requests simultaneously. This overwhelmed the remote service and got back a bunch of failures.

Here is what the output of the for loop looks like –

request:  1 simpleForLoop calling postDataToApi
request:  2 simpleForLoop calling postDataToApi
request:  3 simpleForLoop calling postDataToApi
response: 2 200 postDataToApi called remote service
response: 1 200 postDataToApi called remote service
response: 3 200 postDataToApi called remote service

The three requests happen before any response is received. The responses don’t necessarily come in the order the requests were made.

All the requests are happening asynchronously. This is not what I want.

Attempt 2

Let’s try adding an async and an await to the for loop.

async function loopWithAwait() {
    for (let index = 1; index <= 3; index++) {
        console.log("request: ", index, "loopWithAwait calling post");
        await postDataToApi(index, getProduct());
    }
}

Run the application again, and this time I get the output I want. A request is made, a response is received, then the next request is made and so on.

request:  1 loopWithAwait calling postDataToApi
response: 1 200 postDataToApi called remote service
request:  2 loopWithAwait calling postDataToApi
response: 2 200 postDataToApi called remote service
request:  3 loopWithAwait calling postDataToApi
response: 3 200 postDataToApi called remote service

Full source code available here.

AutoMapper, ProjectTo() – Instance Version

Full source code available here.

In my previous post I showed how to use the wonderful AutoMapper ProjectTo() feature, the demo code shown worked with AutoMapper up to v8.1.1. It looked like this –

_salesContext.Products.OrderBy(p => p.ProductId).Take(count).ProjectTo<ProductModel>()

But this will not work with AutoMapper v9 and above. You will be promptly informed that No overload for the method ‘ProjectTo’ takes 0 arguments.

What happened? Well, AutoMapper moved away from a static API to an instance only API.
To get this working you need to make a few minor changes compared to previous post.
In ConfigureServices(…) replace this –

Mapper.Initialize(cfg => {
    cfg.AddProfile<MappingProfile>();
});

With this –

services.AddAutoMapper(typeof(Startup));

Then in the controller, change this

public class ProductsController : ControllerBase
{
    private readonly SalesContext _salesContext;  

    public ProductsController (SalesContext salesContext)
    {
        _salesContext = salesContext;
    }
    // snip…

To this –

 
private readonly SalesContext _salesContext;
private readonly IMapper _mapper;

public ProductsController (SalesContext salesContext, IMapper mapper)
{
    _salesContext = salesContext;
    _mapper = mapper;
}
// snip..

And finally, the ProjectTo call changes from this –

[HttpGet("projected/{count}")]
public ActionResult GetProjected(int count)
{
    return Ok(_salesContext.Products.OrderBy(p => p.ProductId).Take(count).ProjectTo<ProductModel>());
}

To this –

[HttpGet("projected/{count}")]
public ActionResult GetProjected(int count)
{
    return Ok(_mapper.ProjectTo<ProductModel>(_salesContext.Products.Take(count).OrderBy(p => p.ProductId).Take(count)));
}

The SQL produced by the instance ProjectTo() is the same as what was produced by the static version.

That’s it, easy once you figure it out.

Full source code available here.