Design and implementation of data ingestion extensions for threat intelligence platforms

Abstract

In this thesis, extensions were developed for two commonly used threat intelligence platforms. The extensions were created for ingestion of contextualized indicators of compromise (IoCs) from an external data source, to enable creation of actionable cyber threat intelligence (CTI). CTI is contextualized knowledge about an existing or potential cyber threat to relevant assets. It consists of the mechanisms, indicators, and actionable advice on the emerging or identified threat. The primary purpose of intelligence is for the subject to use it to make informed decisions in response to the identified threat, be that on the technical level of updating rulesets of intrusion detection systems or on the strategic, organisational level, e.g. choosing one software or hardware producer over another one.

Threat intelligence platforms (TIP) are tools that are used for collecting, analyzing, and handling threat intelligence data. The MISP and OpenCTI platforms are commonly used, free open-source TIPs. This thesis presents the design and implementation of extensions to ingest data from ThreatFox, an open-source database for malware IoCs, for both of these platforms. The implementations’ completeness and complexity with regard to the input and output data models are evaluated. In addition, the platforms’ data ingestion architectures and data models are compared based on the implementations. The designs are based on a study of the platforms, their capabilities and strategies of ingesting threat data, and how such data can be converted into contextualized information using the data models employed by the platforms, MISP core format and STIX2.