Indexing the Works of Shakespeare in Elasticsearch – Part 4, Searching via Web API in .NET 5
Full source code available here.
- Part 1 - Infrastructure as Code
- Part 2 - Bulk Indexing
- Part 3 - Sending the Lines to Kinesis
- Part 4 - Searching via Web API in .Net 5
This is part four of my four-part series on indexing the works of Shakespeare in Elasticsearch.
In this, I’ll show how to use the Elasticsearch “low level client” to perform the search. Previously, I wrote a blog showing how to use a HttpClient to perform the search using Json, and this works fine, but Steve Gordon suggested I try the Elastic client as it supports things like connection pooling and still lets me use Json directly with Elasticsearch.
To go along with the Elastic “low level client” there is a “high level client” called NEST, I have tried both and prefer to stick with Json, but you may find them more useful.
Because I develop on a few languages, Json is the natural choice for me. I use it when querying from Node.js, inside a HTTP client (Fiddler, Rest Client, etc) when figuring out my queries and I want to use it in .NET.
But Json and C# don’t go together very well, you have to jump through hoops to make it work with escaping. Or, as I have done, use a creative approach to deserializing via dynamic objects (I know some people won’t like this), I find this much more convenient than converting my Json queries to the Elastic client syntaxes.
This example shows how to use a Web API application to search for a piece of text in isolation or within a specific play.
There is very little to do here.
Startup.cs add the following to the
ConfigureServices(..) method -
services.AddSingleton<ElasticLowLevelClient>(new ElasticLowLevelClient(new ConnectionConfiguration(new Uri("http://localhost:9200"))));
In the SearchController add the following to pass the Elasticsearch client in via dependency injection -
I have two action methods, one to search for a play and line, and one to search for a line across all plays (I know they could be combined into a single action method, I want to keep things simple) -
All very straightforward so far, but now comes the “creative” approach to handling the Json problems.
I put my Elasticsearch queries into their own files. The first is
And the second is
These Json queries are loaded into dynamic objects and the relevant values are set in C#. See lines 5 and 14 & 15.
The strings the above methods return are the queries that will be sent to Elasticsearch.
The below method makes the request and deserializes the response into the
ESResponse class. That class was generated by https://json2csharp.com/.
You might have noticed that I use
Newtonsoft Json, this is because
System.Text.Json does not support dynamic deserialization, see this discussion - https://github.com/dotnet/runtime/issues/29690.
That’s it, searching, and parsing Elasticsearch results via a Web API application, feels a bit messy, but hope it helps.
Full source code available here.
- Using Step Functions and C# Lambdas to Orchestrate API Calls
- Executing an AWS .NET Lambda at an Exact Time with Step Functions
- GitHub Actions with .NET, Part 5 - Build a .NET Application and Upload to S3
- GitHub Actions with .NET, Part 4 - Building an S3 bucket with Pulumi
- C# and AWS Lambdas, Part 5 – Updating the Zip in S3 and Updating the Running Lambda, with Pulumi IaC