Understanding ELK

Understanding ELK

E = Elasticsearch

  1. Elasticsearch is one of the most popular search engines.

  2. It is perfect for storing unstructured data, then retrieving data when needed with blazing speed via its search engine capabilities built on Apache Lucene.

  3. It is commonly used for:-

    • Log analytics,

    • Full-text search,

    • Security intelligence,

    • Business analytics,

    • Operational intelligence use cases.

  4. It automatically stores the original document and adds a searchable reference to the document in the cluster’s index.

  5. It offers the concept of the gateway, which allows for creating full backups easily.

L = Logstash

  1. Logstash is an open-source data ingestion tool that allows you to collect data from a variety of sources, transform it, and send it to your desired destination.

  2. It is lightweight.

  3. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine.

  4. It is a popular choice for loading data into Elasticsearch.

  5. It offers pre-built filters, readily transforms common data types & starts querying without having to build custom data transformation pipelines

K = Kibana

  1. Kibana lets users visualize data with charts and graphs in Elasticsearch.

  2. Users can create bar, line and scatter plots, or pie charts and maps on top of large volumes of data.

  3. It also provides a presentation tool, referred to as Canvas.

  4. It stores its objects as documents in the . kibana index in Elasticsearch.

  5. Elasticsearch acts as a database where the data is collected and Kibana uses the data from Elasticsearch to represent the data to the user in the form of bar graphs, pie charts, heat map

ELK Stack

  1. ELK is one of the popular log management platforms used worldwide for log analysis.

  2. A search engine at heart, users started using Elasticsearch for logs and wanted to easily ingest and visualize them. Enter Logstash, the powerful ingest pipeline, and Kibana, the flexible visualization tool.