Project Details

Threat Intel

enterprise

Project Overview

Threat Intel is a dynamic cybersecurity project designed to enrich real-time threat detection by combining Golang and Python. The system ingests logs through Logstash, validates them against open-source threat intelligence platforms such as MISP, and further expands its coverage using Python crawlers that collect data from websites like blocklist.de and ellio.tech. The objective was to provide SOC teams with continuously updated, context-rich intelligence that enhances situational awareness. By integrating log processing, external lookups, and proactive data collection, Threat Intel creates a powerful pipeline that improves detection accuracy, reduces redundant queries through caching, and ensures threats are identified and acted upon faster.

Technical Insights

  • Implemented buffered channels in Golang to concurrently query MISP and other open-source intelligence sources.
  • Developed Python crawlers to periodically fetch data from blocklist.de and ellio.tech.
  • Used Redis for caching to prevent redundant lookups and speed up analysis.
  • Pushed enriched results to Elasticsearch for storage and visualization.

Tech Stack

Golang Elasticsearch Logstash Redis Python

Tags

Threat Intelligence Log Analysis Cybersecurity