Project Details

Correlation Engine

enterprise

Project Overview

The Correlation Engine is a cybersecurity-focused solution built in Golang to process high-volume network logs and detect anomalies in near real-time. Its core objective was to strengthen Security Operations Center (SOC) capabilities by correlating logs against predefined rules and immediately flagging suspicious activity. By generating alerts with minimal latency and distributing them across multiple communication channels such as email, Rocket.Chat, and Slack, the engine ensures that stakeholders are notified promptly. This proactive approach improves the speed and accuracy of incident response, reducing organizational exposure to threats while providing a reliable foundation for rule-based anomaly detection.

Technical Insights

  • Designed a Pub/Sub architecture using Golang channels for concurrent log correlation.
  • Integrated Kafka to handle large-scale alert distribution across communication channels.
  • Implemented gRPC APIs to manage and update rules dynamically.
  • Stored alerts in Elasticsearch for centralized log management and visualization.

Tech Stack

Golang Kafka Elasticsearch gRPC

Tags

Anomaly Detection Pub/Sub SIEM

Relevant Articles