Indexing the Works of Shakespeare in Elasticsearch – Part 4, Searching via Web API in .NET 5

Want to learn more about AWS Lambda and .NET? Check out my A Cloud Guru course on ASP.NET Web API and Lambda.

Full source code available here.

• Part 1 - Infrastructure as Code
• Part 2 - Bulk Indexing
• Part 3 - Sending the Lines to Kinesis
• Part 4 - Searching via Web API in .Net 5

This is part four of my four-part series on indexing the works of Shakespeare in Elasticsearch.

In this, I’ll show how to use the Elasticsearch “low level client” to perform the search. Previously, I wrote a blog showing how to use a HttpClient to perform the search using Json, and this works fine, but Steve Gordon suggested I try the Elastic client as it supports things like connection pooling and still lets me use Json directly with Elasticsearch.

To go along with the Elastic “low level client” there is a “high level client” called NEST, I have tried both and prefer to stick with Json, but you may find them more useful.

Because I develop on a few languages, Json is the natural choice for me. I use it when querying from Node.js, inside a HTTP client (Fiddler, Rest Client, etc) when figuring out my queries and I want to use it in .NET.

But Json and C# don’t go together very well, you have to jump through hoops to make it work with escaping. Or, as I have done, use a creative approach to deserializing via dynamic objects (I know some people won’t like this), I find this much more convenient than converting my Json queries to the Elastic client syntaxes.

This example shows how to use a Web API application to search for a piece of text in isolation or within a specific play.

The setup

There is very little to do here.

In Startup.cs add the following to the ConfigureServices(..) method -

services.AddSingleton<ElasticLowLevelClient>(new ElasticLowLevelClient(new ConnectionConfiguration(new Uri("http://localhost:9200"))));

In the SearchController add the following to pass the Elasticsearch client in via dependency injection -

1public class SearchController : ControllerBase
2{
3    private readonly ElasticLowLevelClient _lowLevelClient;
4    public SearchController(ElasticLowLevelClient lowLevelClient)
5    {
6        _lowLevelClient = lowLevelClient;
7    }
8//snip ..

I have two action methods, one to search for a play and line, and one to search for a line across all plays (I know they could be combined into a single action method, I want to keep things simple) -

 1[HttpGet("Line")]
 2public ActionResult Line(string text)
 3{
 4    string queryWithParams = GetLineQuery(text);
 5    var lines = PerformQuery(queryWithParams);
 6    
 7    return Ok(lines);
 8}
 9
10[HttpGet("PlayAndLine")]
11public ActionResult PlayAndLine(string play, string text)
12{
13    string queryWithParams = GetPlayAndLineQuery(play, text);
14    var lines = PerformQuery(queryWithParams);
15
16    return Ok(lines);
17}

All very straightforward so far, but now comes the “creative” approach to handling the Json problems.

I put my Elasticsearch queries into their own files. The first is Line.Json -

1{
2    "query": {
3        "match_phrase_prefix" :{
4            "Line": ""
5        }
6    }
7} 

And the second is PlayAndLine.Json -

 1{
 2    "query":{
 3        "bool": {
 4            "must": [
 5                { "match": { "Play": "" } }
 6               ,{ "match_phrase_prefix": { "Line": "" } }
 7            ]
 8        }
 9    }
10}

These Json queries are loaded into dynamic objects and the relevant values are set in C#. See lines 5 and 14 & 15.

 1private string GetLineQuery(string text)
 2{
 3    string elasticSearchQuery = System.IO.File.ReadAllText($"Queries/Line.json");
 4    dynamic workableElasticSearchQuery = JsonConvert.DeserializeObject(elasticSearchQuery);
 5    workableElasticSearchQuery.query.match_phrase_prefix.Line = text;
 6
 7    return workableElasticSearchQuery.ToString();
 8}
 9
10private string GetPlayAndLineQuery(string play, string text)
11{
12    string elasticSearchQuery = System.IO.File.ReadAllText($"Queries/PlayAndLine.json");
13    dynamic workableElasticSearchQuery = JsonConvert.DeserializeObject(elasticSearchQuery);
14    workableElasticSearchQuery.query.@bool.must[0].match.Play = play;
15    workableElasticSearchQuery.query.@bool.must[1].match_phrase_prefix.Line = text;
16
17    return workableElasticSearchQuery.ToString();
18}

The strings the above methods return are the queries that will be sent to Elasticsearch.

The below method makes the request and deserializes the response into the ESResponse class. That class was generated by https://json2csharp.com/.

1private ESResponse PerformQuery(string queryWithParams)
2{
3    var response = _lowLevelClient.Search<StringResponse>("shakespeare", queryWithParams);
4    ESResponse esResponse = System.Text.Json.JsonSerializer.Deserialize<ESResponse>(response.Body);
5    return esResponse;
6}

You might have noticed that I use System.Text.Json and Newtonsoft Json, this is because System.Text.Json does not support dynamic deserialization, see this discussion - https://github.com/dotnet/runtime/issues/29690.

That’s it, searching, and parsing Elasticsearch results via a Web API application, feels a bit messy, but hope it helps.

Full source code available here.