Publications

Conferences

2024

1. SIGCSE

   Cybersecurity as a Crosscutting Concept Across an Undergrad Computer
   Science Curriculum: An Experience Report

   Nadeem, Azqa

   In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education, 2024

   Abstract PDF

   Although many Computer Science (CS) programs offer cybersecurity courses, they are typically optional and placed at the periphery of the program. We advocate to integrate cybersecurity as a crosscutting concept in CS curricula, which is also consistent with latest cybersecurity curricular guidelines, e.g., CSEC2017. We describe our experience of implementing this crosscutting intervention across three undergraduate core CS courses at a leading technical university in Europe between 2018 and 2023, collectively educating over 2200 students. The security education was incorporated within CS courses using a partnership between the responsible course instructor and a security expert, i.e., the security expert (after consultation with course instructors) developed and taught lectures covering multiple CSEC2017 knowledge areas. This created a complex dynamic between three stakeholders: the course instructor, the security expert, and the students. We reflect on our intervention from the perspective of the three stakeholders – we conducted a post-course survey to collect student perceptions, and semi-supervised interviews with responsible course instructors and the security expert to gauge their experience. We found that while the students were extremely enthusiastic about the security content and retained its impact several years later, the misaligned incentives for the instructors and the security expert made it difficult to sustain this intervention without organizational support. By identifying limitations in our intervention, we suggest ideas for sustaining it.

2023

1. EuroS&P

   SoK: Explainable Machine Learning for Computer Security Applications

   Nadeem, Azqa, Vos, Daniel, Cao, Clinton, Pajola, Luca, Dieck, Simon, Baumgartner, Robert, and Verwer, Sicco

   In IEEE European Symposium on Security and Privacy (Euro S&P), 2023

   Abstract PDF Code

   Explainable Artificial Intelligence (XAI) is a promising solution to improve the transparency of machine learning (ML) pipelines. We systematize the increasingly growing (but fragmented) microcosm of studies that develop and utilize XAI methods for defensive and offensive cybersecurity tasks. We identify 3 cybersecurity stakeholders, i.e., model users, designers, and adversaries, that utilize XAI for 5 different objectives within an ML pipeline, namely 1) XAI-enabled decision support, 2) applied XAI for security tasks, 3) model verification via XAI, 4) explanation verification & robustness, and 5) offensive use of explanations. We further classify the literature w.r.t. the targeted security domain. Our analysis of the literature indicates that many of the XAI applications are designed with little understanding of how they might be integrated into analyst workflows – user studies for explanation evaluation are conducted in only 14% of the cases. The literature also rarely disentangles the role of the various stakeholders. Particularly, the role of the model designer is minimized within the security literature. To this end, we present an illustrative use case accentuating the role of model designers. We demonstrate cases where XAI can help in model verification and cases where it may lead to erroneous conclusions instead. The systematization and use case enable us to challenge several assumptions and present open problems that can help shape the future of XAI within cybersecurity.

2022

1. ECML

   SECLEDS: Sequence Clustering in Evolving Data Streams via Multiple Medoids
   and Medoid Voting

   Nadeem, Azqa, and Verwer, Sicco

   In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD), 2022

   Abstract PDF Code Slides

   Sequence clustering in a streaming environment is challenging because it is computationally expensive, and the sequences may evolve over time. K-medoids or Partitioning Around Medoids (PAM) is commonly used to cluster sequences since it supports alignment-based distances, and the k-centers being actual data items helps with cluster interpretability. However, offline k-medoids has no support for concept drift, while also being prohibitively expensive for clustering data streams. We therefore propose SECLEDS, a streaming variant of the k-medoids algorithm with constant memory footprint. SECLEDS has two unique properties: i) it uses multiple medoids per cluster, producing stable highquality clusters, and ii) it handles concept drift using an intuitive Medoid Voting scheme for approximating cluster distances. Unlike existing adaptive algorithms that create new clusters for new concepts, SECLEDS follows a fundamentally different approach, where the clusters themselves evolve with an evolving stream. Using real and synthetic datasets, we empirically demonstrate that SECLEDS produces high-quality clusters regardless of drift, stream size, data dimensionality, and number of clusters. We compare against three popular stream and batch clustering algorithms. The state-of-the-art BanditPAM is used as an offline benchmark. SECLEDS achieves comparable F1 score to BanditPAM while reducing the number of required distance computations by 83.7%. Importantly, SECLEDS outperforms all baselines by 138.7% when the stream contains drift. We also cluster real network traffic, and provide evidence that SECLEDS can support network bandwidths of up to 1.08 Gbps while using the (expensive) dynamic time warping distance.

2021

1. VizSec

   SAGE: Intrusion Alert-driven Attack Graph Extractor

   Nadeem, Azqa, Verwer, Sicco, and Yang, Shanchieh J

   In Proceedings of the 18th IEEE Symposium on Visualization for Cyber Security (VizSec), 2021

   Abstract PDF Code Slides Video

   Attack graphs (AG) are used to assess pathways availed by cyber adversaries to penetrate a network. State-of-the-art approaches for AG generation focus mostly on deriving dependencies between system vulnerabilities based on network scans and expert knowledge. In real-world operations however, it is costly and ineffective to rely on constant vulnerability scanning and expert-crafted AGs. We propose to automatically learn AGs based on actions observed through intrusion alerts, without prior expert knowledge. Specifically, we develop an unsupervised sequence learning system, SAGE, that leverages the temporal and probabilistic dependence between alerts in a suffix-based probabilistic deterministic finite automaton (S-PDFA) – a model that accentuates infrequent severe alerts and summarizes paths leading to them. AGs are then derived from the S-PDFA on a per-objective, per-victim basis. Tested with intrusion alerts collected through Collegiate Penetration Testing Competition, SAGE compresses over 330k alerts into 93 AGs. These AGs reflect the strategies used by the participating teams. The AGs are succinct, interpretable, and capture behavioral dynamics, e.g., that attackers will often follow shorter paths to re-exploit objectives.

Workshops

2022

1. LearnAut

   Robust Attack Graph Generation

   Mouwen, Dennis, Verwer, Sicco, and Nadeem, Azqa

   In Learning and Automata workshop (LearnAut), 2022

   Abstract PDF

   We present a method to learn automaton models that are more robust to input modifications. It iteratively aligns sequences to a learned model, modifies the sequences to their aligned versions, and re-learns the model. Automaton learning algorithms are typically very good at modeling the frequent behavior of a software system. Our solution can be used to also learn the behavior present in infrequent sequences, as these will be aligned to the frequent ones represented by the model. We apply our method to the SAGE tool for modeling attacker behavior from intrusion alerts. In experiments, we demonstrate that our algorithm learns models that can handle noise such as added and removed symbols from sequences. Furthermore, it learns more concise models that fit better to the training data.

2020

1. AISec

   The Robust Malware Detection Challenge and Greedy Random Accelerated
   Multi-Bit Search

   Verwer, Sicco, Nadeem, Azqa, Hammerschmidt, Christian, Bliek, Laurens, Al-Dujaili, Abdullah, and O’Reilly, Una-May

   In Proceedings of the 13th Workshop on Artificial Intelligence and Security (AISec), 2020

   Abstract PDF Code

   Training classifiers that are robust against adversarially modified examples is becoming increasingly important in practice. In the field of malware detection, adversaries modify malicious binary files to seem benign while preserving their malicious behavior. We report on the results of a recently held robust malware detection challenge. There were two tracks in which teams could participate: the attack track asked for adversarially modified malware samples and the defend track asked for trained neural network classifiers that are robust to such modifications. The teams were unaware of the attacks/defenses they had to detect/evade. Although only 9 teams participated, this unique setting allowed us to make several interesting observations.We also present the challenge winner: GRAMS, a family of novel techniques to train adversarially robust networks that preserve the intended (malicious) functionality and yield high-quality adversarial samples. These samples are used to iteratively train a robust classifier. We show that our techniques, based on discrete optimization techniques, beat purely gradient-based methods. GRAMS obtained first place in both the attack and defend tracks of the competition.

2. MLCS

   Hybrid connection and host clustering for community detection in spatial-temporal network data

   Roeling, Mark P, Nadeem, Azqa, and Verwer, Sicco

   In Proceedings of the 2nd Workshop on Machine Learning for Cybersecurity (MLCS), 2020

   Abstract PDF Video

   Network data clustering and sequential data mining are large fields of research, but how to combine them to analyze spatial-temporal network data remains a technical challenge. This study investigates a novel combination of two sequential similarity methods (Dynamic Time Warping and N-grams with Cosine distances), with two state-of-the-art unsupervised network clustering algorithms (Hierarchical Density-based Clustering and Stochastic Block Models). A popular way to combine such methods is to first cluster the sequential network data, resulting in connection types. The hosts in the network can then be clustered conditioned on these types. In contrast, our approach clusters nodes and edges in one go, i.e., without giving the output of a first clustering step as input for a second step. We achieve this by implementing sequential distances as covariates for host clustering. While being fully unsupervised, our method outperforms many existing approaches. To the best of our knowledge, the only approaches with comparable performance require manual filtering of connections and feature engineering steps. In contrast, our method is applied to raw network traffic. We apply our pipeline to the problem of detecting infected hosts (network nodes) from logs of unlabelled network traffic (sequential data). On data from the Stratosphere IPS project (CTU-Malware-Capture-Botnet-91), which includes malicious (Conficker botnet) as well as benign hosts, we show that our method perfectly detects peripheral, benign, and malicious hosts in different clusters. We replicate our results in the well-known ISOT dataset (Storm, Waledac, Zeus botnets) with comparable performance: conjointly, 99.97% of nodes were categorized correctly.

Journals

2021

1. TDSC

   Alert-driven Attack Graph Generation using S-PDFA

   Nadeem, Azqa, Verwer, Sicco, Moskal, Stephen, and Yang, Shanchieh J

   In IEEE Transcations on Dependable and Secure Computing, 2021

   Abstract PDF Code Slides

   Ideal cyber threat intelligence (CTI) includes insights into attacker strategies that are specific to a network under observation. Such CTI currently requires extensive expert input for obtaining, assessing, and correlating system vulnerabilities into a graphical representation, often referred to as an attack graph (AG). Instead of deriving AGs based on system vulnerabilities, this work advocates the direct use of intrusion alerts. We propose SAGE, an explainable sequence learning pipeline that automatically constructs AGs from intrusion alerts without a priori expert knowledge. SAGE exploits the temporal and probabilistic dependence between alerts in a suffix-based probabilistic deterministic finite automaton (S-PDFA) — a model that brings infrequent severe alerts into the spotlight and summarizes paths leading to them. Attack graphs are extracted from the model on a per-victim, per-objective basis. SAGE is thoroughly evaluated on three open-source intrusion alert datasets collected through security testing competitions in order to analyze distributed multi-stage attacks. SAGE compresses over 330k alerts into 93 AGs that show how specific attacks transpired. The AGs are succinct, interpretable, and provide directly relevant insights into strategic differences and fingerprintable paths. They even show that attackers tend to follow shorter paths after they have discovered a longer one in 84.5% of the cases.

Book Chapters

2023

1. AICA

   Learning about the adversary

   Nadeem, Azqa, Verwer, Sicco, and Yang, Shanchieh J

   In Autonomous Intelligent Agents for Cyber Defense, 2023

   Abstract PDF

   The evolving nature of the tactics, techniques, and procedures used by cyber adversaries have made signature and template based methods of modeling adversary behavior almost infeasible. We are moving into an era of data-driven autonomous cyber defense agents that learn contextually meaningful adversary behaviors from observables. In this chapter, we explore what can be learnt about cyber adversaries from observable data, such as intrusion alerts, network traffic, and threat intelligence feeds. We describe the challenges of building autonomous cyber defense agents, such as learning from noisy observables with no ground truth, and the brittle nature of deep learning based agents that can be easily evaded by adversaries. We illustrate three state-of-the-art autonomous cyber defense agents that model adversary behavior from traffic induced observables without a priori expert knowledge or ground truth labels. We close with recommendations and directions for future work.

2022

1. Sec+AI

   Intelligent Malware Defenses: A survey

   Nadeem, Azqa, Rimmer, Vera, Joosen, Wouter, and Verwer, Sicco

   In Security and Artificial Intelligence, 2022

   Abstract PDF

   With rapidly evolving threat landscape surrounding malware, intelligent defenses based on machine learning are paramount. In this chapter, we review the literature proposed in the past decade and identify the state-of-the-art in various related research directions — malware detection, malware analysis, adversarial malware, and malware author attribution. We discuss challenges that emerge when machine learning is applied to malware. We also identify the key issues that need to be addressed by the research community in order to further deepen and systematize research in the malware domain.

2. Sec+AI

   Open-World Network Intrusion Detection

   Rimmer, Vera, Nadeem, Azqa, Verwer, Sicco, Preuveneers, Davy, and Joosen, Wouter

   In Security and Artificial Intelligence, 2022

   Abstract PDF

   This chapter contributes to the ongoing discussion of strengthening security by applying AI techniques in the scope of intrusion detection. The focus is set on open-world detection of attacks through data-driven network traffic analysis. This research topic is complementary to the earlier chapter on intelligent malware detection. In this chapter, we revisit the foundations of machine learning-based solutions for network security, which aim to make network defense tools more autonomous, adaptive, proactive and responsive. Specifically, we give a comprehensive introduction to the research on anomaly detection for network intrusion detection – that is, defensive schemes that do not assume complete prior knowledge of malicious patterns and instead learn the notion of normality from benign traffic. Along with outlining the recent advances in the field, we provide insights and reflect on the current limitations and research challenges. Therefore, this chapter presents compelling research opportunities to advance machine learning techniques in network security and push the boundaries of open-world network intrusion detection.

2021

1. MAAIDL

   Beyond Labeling: Using Clustering to Build Network Behavioral Profiles of
   Malware Families

   Nadeem, Azqa, Hammerschmidt, Christian, Ganan, Carlos H, and Verwer, Sicco

   In Malware Analysis Using Artificial Intelligence and Deep Learning, 2021

   Abstract PDF Code

   Malware family labels are known to be inconsistent. They are also black-box since they do not represent the capabilities of malware. The current state of the art in malware capability assessment includes mostly manual approaches, which are infeasible due to the ever-increasing volume of discovered malware samples. We propose a novel unsupervised machine learning-based method called MalPaCA, which automates capability assessment by clustering the temporal behavior in malware’s network traces. MalPaCA provides meaningful behavioral clusters using only 20 packet headers. Behavioral profiles are generated based on the cluster membership of malware’s network traces. A Directed Acyclic Graph shows the relationship between malwares according to their overlapping behaviors. The behavioral profiles together with the DAG provide more insightful characterization of malware than current family designations. We also propose a visualization-based evaluation method for the obtained clusters to assist practitioners in understanding the clustering results. We apply MalPaCA on a financial malware dataset collected in the wild that comprises 1.1 k malware samples resulting in 3.6 M packets. Our experiments show that (i) MalPaCA successfully identifies capabilities, such as port scans and reuse of Command and Control servers; (ii) It uncovers multiple discrepancies between behavioral clusters and malware family labels; and (iii) It demonstrates the effectiveness of clustering traces using temporal features by producing an error rate of 8.3%, compared to 57.5% obtained from statistical features.

Posters

2022

1. VizSec

   Critical Path Exploration Dashboard for Alert-driven Attack Graphs

   Nadeem, Azqa, Leal Diaz, Sonia, and Verwer, Sicco

   In VizSec, 2022

   Poster

2021

1. CCS

   Enabling Visual Analytics via Alert-driven Attack Graphs

   Nadeem, Azqa, Verwer, Sicco, Moskal, Stephen, and Yang, Shanchieh J

   In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021

   Abstract Poster

   Attack graphs (AG) are a popular area of research that display all the paths an attacker can exploit to penetrate a network. Existing techniques for AG generation rely heavily on expert input regarding vulnerabilities and network topology. In this work, we advocate the use of AGs that are built directly using the actions observed through intrusion alerts, without prior expert input.We have developed an unsupervised visual analytics system, called SAGE, to learn alert-driven attack graphs.We show how these AGs (i) enable forensic analysis of prior attacks, and (ii) enable proactive defense by providing relevant threat intelligence regarding attacker strategies. We believe that alert-driven AGs can play a key role in AI-enabled cyber threat intelligence as they open up new avenues for attacker strategy analysis whilst reducing analyst workload.

2019

1. Cybersec&AI

   Clustering Connections and Hosts using a Hybrid Sequential Approach

   Nadeem, Azqa, Roeling, Mark P, and Verwer, Sicco

   In Cybersec & AI connected conference, 2019

   Poster

2018

1. Nordsec

   Clustering Malware’s Network Behavior using Simple Sequential
   Features

   Nadeem, Azqa, Ganan, Carlos H, and Verwer, Sicco

   In Nordic Conference on Secure IT Systems, 2018

   Poster

Theses

2024

1. PhD. thesis

   Understanding Adversary Behavior via XAI: Leveraging Sequence Clustering
   to Extract Threat Intelligence

   Nadeem, Azqa

   TU Delft, 2024

   Abstract Thesis

   Understanding the behavior of cyber adversaries provides threat intelligence to security practitioners, and improves the cyber readiness of an organization. With the rapidly evolving threat landscape, data-driven solutions are becoming essential for automatically extracting behavioral patterns from data that are otherwise too time-consuming to discover manually. This dissertation advocates the use of machine learning (ML) to obtain insights into adversary behavior for creating AI-assisted practitioners. However, developing adversary behavior models is challenging since cyber data is often unlabeled, noisy, infrequent, and contains intricate patterns that evolve over time. We demonstrate that sequential features are effective at addressing these challenges. Yet, they have limited interpretability and algorithmic support. This dissertation starts by defining the notion of explainability as it is currently used within cybersecurity by systematizing available literature in Chapter 2. We find that the literature frequently relies on black-box models that use off-the-shelf explanation methods without considering the explanation stakeholders. In contrast, literature on sequence learning models that are interpretable by design is severely limited. We address these challenges by developing special algorithms that learn sequential patterns from infrequent events, and evolving data in an unsupervised setting. We utilize these algorithms to create interpretable tool-chains for understanding the behavior of various types of adversaries. We show that it is possible to learn interpretable models (even for complex sequential data in an unsupervised setting) that providemore insights than just prediction probabilities, while achieving competitive performance. In doing so, we encourage the security community to look beyond accuracy scores, and focus on extracting actionable insights fromMLmodels. We make our tool-chains open-source. The first part of this thesis models the strategies employed by human threat actors. Chapters 3 and 4 develop a novel paradigmof attack graphs (AG) that are learned directly fromintrusion alerts for capturing attacker strategies. The attacker strategies are learned using our S-PDFA model, which is interpretable, fast, and effective. We learn alert-driven AGs from 3 open-source datasets, and show their ability to compress over 1.4 million alerts in 401 AGs in under 5 minutes. The AGs provide actionable intelligence regarding strategic differences and fingerprintable paths. They also reduce analyst alert fatigue by triaging critical attacks. The second part of the thesis models the capabilities exhibited by automated threat actors (malware). Chapters 5 and 6 develop an explainable sequence clustering toolchain to automatically characterize the network behavior of malware samples. We use this tool-chain to create behavioral profiles of 1196 real-world malware samples that explain their capabilities. We also develop a streaming sequence clustering algorithm for real-time behavior profiling, which is evaluated on 5 datasets and against 4 clustering algorithms. By automatically creating behavioral profiles of bot-infected hosts in realtime, we distinguish benign and malicious hosts with 100% accuracy.

2018

1. MSc. thesis

   Clustering Malware’s Network Behavior using Simple Sequential Features

   Nadeem, Azqa

   TU Delft, 2018

   Abstract Thesis

   Developing malware variants is extremely cheap for attackers because of the availability of various obfuscation tools. These variants can be grouped in malware families, based on information retrieved from their static and dynamic analysis. Dynamic, network-level analysis of malware shows its core behavior since it captures the interaction with its developer. On the other hand, increasingly more emphasis is given to using Deep Packet Inspection (DPI) in order to cluster malware’s network behavior. However, DPI has severe privacy implications, as it involves inspecting payloads of the network traffic. This thesis presents an exploratory study, the aim of which is to characterize and cluster malware behavior using high-level, non-privacy-invasive, sequential features extracted from its network activity. The key intuition behind the proposed solution is that if the underlying infrastructure of distinct malware samples is similar, the order in which they perform certain actions should also be similar. The results of this research show that sequence clustering allows flexible and robust clusters, as opposed to using non-sequential features. The clusters themselves reveal interesting attacking capabilities, such as port scans, and the same Command and Control server responding to different malware families. Lastly, a comparison with clusters obtained from static analysis reveals that network-based clustering is far more qualified to determine the many behaviors exhibited by a single malware family, as well as behaviors common across multiple malware families.