Reading CSV Files into Objects with Node.js

Full source code available here.

As I am learning Node.Js I am constantly surprised by how easy it is to do some things, and how difficult to do others, and how poor the examples out there are.

Case in point, I want to read some data from as CSV file and then send it off to ElasticSearch for indexing. The ElasticSearch part I have figured out already, and in Node.js that is easy.
I expected it to be trivial to find an example of reading a file, creating objects based on the rows, processing the objects, and doing all that in chunks of say 1000 rows. This kind of stuff is easy in C# and there are plenty of examples. Not the case is Node.js.

Here two ways of doing it.

You can download a CSV with all of Shakespeare’s plays, but the example you can download from above has just the first 650 lines from that file.

I want to read the file 100 rows at a time, put the rows into objects and then process the rows.

The first module I came across was csv-parse.

Run this from the console –

npm install csv-parse

I have a Entry class that represents a row from the CSV file. Its constructor takes six parameters that represent each of the columns in the CSV file.

In this example processing the data in chunks is not necessary, but when I process the full file with over 130,000 rows, chunking becomes important.

var fs = require('fs');
var parse = require('csv-parse');

function readCSV() {
    let entries = [];
    let count = 0;

        .pipe(parse({ delimiter: ';', from_line: 2 }))
        .on('data', function (row) {
            entries.push(new Entry(row[0], row[1], row[2], row[3], row[4], row[5]))

            if (count % 100 == 0) {
                count = 0;
                entries = []; // clear the array
        .on('end', function () {

function processEntries(entries) {
    console.log(entries[0].Id + "  to " + entries[entries.length - 1].Id);

class Entry {
    constructor(id, play, characterLineNumber, actSceneLine, character, line) {
        this.Id = id;
        this.Play = play;
        this.CharacterLineNumber = characterLineNumber;
        this.ActSceneLine = actSceneLine;
        this.Character = character;
        this.Line = line;


Note how I have to make sure that the last chuck is also processed on line 22.

This approach seems fine, but then I found the csvtojson module.

This module makes what I’m trying to do a little easier by skipping over the need to explicitly construct an object with the data from the rows in the file.

First install the module –

npm install csvtojson

This is the code –

const csv=require('csvtojson');

function readCSV(){
    let entries = [];
    let count = 0;

            if(count % 100 == 0){
                count = 0;
                entries = []; // clear the array
    return entries;

function processEntries(entries){
    console.log(entries[0].Id + "  to " + entries[entries.length - 1].Id);


Again note how I process the chunks of 100, and then final chunk on line 20.

All of this is the aim of indexing all of Shakespeare’s works in ElasticSearch, and that I will show in another post.

Full source code available here.

The Simplest Hello World in Node.js

Full source code available here.

I am learning Node.js and have found it a bit of a struggle to locate good, simple documentation. It feels like most people writing in the space assume a lot of existing knowledge, like you know how plenty of JavaScript, or how to effectively use web browser debug tools, have a good understanding of HTML and CSS. If you go down the rabbit hole of asynchronous programming in Node.js the writers often assume you know about the previous Node.js approach of asynchronous, try reading about async/await and you’ll be pointed to examples with promises, promisify, microtasks and callbacks, again assuming a lot of knowledge on the part of the reader.

I even asked a friend who was using Node.js to build a complex web site to write a simple Hello World program for me that I could run from the command line, but he didn’t know. This was a very experienced developer working in the industry for almost thirty years, but he started with Node.js inside a an existing web site.

If you search for how to write a “Hello world” in Node.js, you’ll find examples that setup web servers, take requests, return response, and probably use callbacks.

Because of what I perceive as a lack of documentation, I’m going write some simple blog posts about things I learn but could not find a simple example of.

To kick off, here is Hello World in Node.js.

I’m going assume you have Node and the Node Package Manager installed, and that you can run node and npm from the command line. If you do not have them installed there is plenty of help out there for that!

Hello World
Create a new directory called SimpleHelloWorld.

Open a terminal/console inside that directory.

Run –

npm init -y

This creates a package.json file, you will edit this in a moment.

Create a new file called helloworld.js.

Add a single line to the file –

console.log("Hello World");

At this point you can run –

node helloworld.js

And you will see “Hello World” printed. There you a simple Hello World program.

Let’s do one more thing, open package.json.

Replace the scripts section with –

 "scripts": {
    "start": "node helloworld.js"

Now you can run –

npm start

Full source code available here.

Synchronous For Loop in Node.js

Full source code available here.

In C# it is very easy to avoid doing something asynchronously, but I found out that this is not the case with JavaScript and Node.js.

I wanted to load a large amount of data in a remote service via and API, but had to do it one request at a time. There was no bulk option.

For a little while I’ve been learning JavaScript and some Node, but had not had a chance to use the language on anything useful. This blog post is the first I’ve written on JavaScript, so if you find something that is incorrect, leave a comment and I will update it.

There are many choices for making http requests with Node, for no particular reason I chose Axios.

Here is a very simple example of post method –

async function postDataToApi(index, dataToPost) {
    try {
        const response = await"", {
        console.log(index, response.status)
    } catch (error) {

I am calling await on the method.

For the sake of this example I am going to generate the data to post using the Casual library. This create a fake product with random data in it –

casual.define('product', function () {
    return {
        productId: casual.uuid,
        productName: casual.word,
        color: casual.color_name

Attempt 1

Now I needed call the remote service, sending it data in some sort of loop.

A simple for loop seemed the best candidate.

function simpleForLoop(start, end) {
    for (let index = start; index <= end; index++) {
        console.log("request:", index, "for loop calling postDataToApi");
        postDataToApi(index, casual.product);

But this ran through the whole loop before it made a single outbound request and then attempted to make a very large number of requests simultaneously. This overwhelmed the remote service and got back a bunch of failures.

Here is what the output of the for loop looks like –

request:  1 simpleForLoop calling postDataToApi
request:  2 simpleForLoop calling postDataToApi
request:  3 simpleForLoop calling postDataToApi
response: 2 200 postDataToApi called remote service
response: 1 200 postDataToApi called remote service
response: 3 200 postDataToApi called remote service

The three requests happen before any response is received. The responses don’t necessarily come in the order the requests were made.

All the requests are happening asynchronously. This is not what I want.

Attempt 2

Let’s try adding an async and an await to the for loop.

async function loopWithAwait() {
    for (let index = 1; index <= 3; index++) {
        console.log("request: ", index, "loopWithAwait calling post");
        await postDataToApi(index, getProduct());

Run the application again, and this time I get the output I want. A request is made, a response is received, then the next request is made and so on.

request:  1 loopWithAwait calling postDataToApi
response: 1 200 postDataToApi called remote service
request:  2 loopWithAwait calling postDataToApi
response: 2 200 postDataToApi called remote service
request:  3 loopWithAwait calling postDataToApi
response: 3 200 postDataToApi called remote service

Full source code available here.