capçalera

Case Study
Caso de Estudio
Cas d'Estudi

TMB speeds up its logs management based on Elastic Stack

Opentrends helps TMB design a platform based on Elastic Stack that allows the centralization of the process and the exploitation of different server logs and company operations.

CHALLENGE

The project consisted of identifying the different log sources, defining a process of intake, enrichment and design of different command tables for the exploitation and visualization of the logs. We designed an architecture capable of sustaining the load of logs to ingest, and to scale and mutate according to the needs that appear.

SOLUTION

A LOG PROCESSING PLATFORM MANAGED BY ELASTIC STACK (ELK)

Different tasks have been carried out in order to set up an ecosystem for the ingestion and exploitation of logs. It is a project in continuous evolution, where flexibility allows the solution to evolve and adapt. Additionally, other tools that may fit the ecosystem have been researched and evaluated.

COMPLETED TASKS
  • Logstash configurations for the different logs.
  • Use of nxLog and Filebeat for the ingestion of logs.
  • Use of Redis as broker. 
  • Definition of dashboards in Kibana and Grafana.
  • Implementation of services in NODE.JS for the enrichment of the logs
    (add info to the base logs to improve exploitation)
  • Lifecycle management of indexed data in Elasticsearch (index optimization, cleaning policies, backups, among others)
  • Alerts definition
Arquitectura

METHODOLOGY

We have worked hand in hand with the technical department of TMB. In weekly meetings, the completed work has been reviewed and the pending tasks have been defined following the Elastic best practices.

best practices Elastic

TECHNOLOGY

The following technologies have been used:

  • Elastic Stack (Elasticsearch, Logstash, Kibana, Filebeat, Metricbeat)
  • nxLog
  • NodeJs
  • Redis
  • Kafka
  • Grafana
  • Auth0

 

Tecnología
RESULTS

With the new log processing platform implanted by TMB and Opentrends, we have achieved an optimization of the infrastructure's performance and also a global economization of the system able to face very high levels of log processing.

Based on the results, the main characteristics include:

  • The fixed architecture service that provides flexibility and continuous improvement of the solution.
  • Governance and management of data's life cycle.
  • User-friendly display of the main customer indicators.