Getting Started with Elasticsearch using .Net Core and NEST

Prerequisite: This demo requires Visual Studio Code, Visual Studio Express, and Docker for Windows. Please download and install the required software on your machine if needed.

Introduction

Elasticsearch is a distributed, scalable, open-source search and analysis engine built on Apache Lucene. It is a highly flexible tool
that enables you to analyze and visualize large volumes of data quickly and in real-time. Elasticsearch offers a variety of powerful search features, including searching, sorting, filtering, and text analysis.  Elasticsearch is the core component, along with open-source tools for data ingestion (Logstash, Beats) and visualization(Kibana).

The core principle is that users send JSON documents usingAPI or data ingestion tools (Logstash, Beats), and Elasticsearch stores the document and updates the index. Additionally, it integrates easily into your code, providing comprehensive client libraries for languages like Java, JavaScript, Ruby, Go, and .NET.

Setup ELK stack on docker

Demonstrating Elasticsearch in the .Net app would require setting up the ELK stack in the Docker environment. I‘ve seen many confusing, simple, or too complex examples for setting up the ELK stack in Docker. I devised a simple YAML file to deploy the ELK stack.

git clone https://github.com/vizzontech/ELK_Docker

The YAML file is pretty simple; it specifies which images to pull and how to map volumes and ports. The only exception is the Logstash service, where I have included an additional line to copy the sample data into a container. The Ingestion service subsequently uses this data to create the index and feed the data into it.

Open the VS code, click the terminal, and run the docker-compose command.

docker-compose up

The first time, pulling images and running services may take a few minutes. Let’s check if the services are running in the docker. 

docker ps –filter “name=elasticsearch” –filter “name=kibana” –filter “name=logstash”

 At this point, you should be able to view running services in the browser

Elasticsearch http://localhost:9200

Logstash http://localhost:9600

Kibana http://localhost:5601

Elasticsearch using NEST

Now that the ELK stack is running, we can clone our MVC web app. This app retrieves and displays “New York City Airbnb Open Data” in the table, allowing data filtering in Elasticsearch. The application interacts with Elasticsearch using the NEST NuGet package. NEST provides seamless integration by allowing direct mapping of .NET objects to Elasticsearch document types. The bootstrapping and implementation are designed to be simple, allowing you to use this sample app as a starter template and extend the code as needed. This provides clear and comprehensible code to help you grasp the core concepts.

git clone https://github.com/vizzontech/ElasticSearch_ELK_NEST

Run the application in VS studio by pressing F5, and you should be able to view the sample data.