Explainable Incident Response

PhD position on cybersecurity, explanability, and incident response

For details and application submission: Apply here

Application deadline: 16 February 2024

Description:

Are you interested in interpretable machine learning for cybersecurity, and want to empower security practitioners by improving the current state of cyber incident response? The Semantics, Cybersecurity, and Services (SCS) group at the University of Twente and the Twente University Centre for Cybersecurity Research (TUCCR) invites applications for a fully funded PhD Position in Explainable Incident Response.

In recent years, machine learning (ML) solutions are increasingly being deployed in Security Operations Centers (SOCs) to enhance security coverage, and to reduce the number of missed attacks. Not only do these ML systems create many false positives, it is often very difficult to understand how they work in the first place (Nadeem et al., 2023). Moreover, the forensic analysis of incidents and incident response are largely manual procedures, leading to analyst burnout and ‘alert fatigue’.

The objective of this PhD project is to create ‘AI-assisted practitioners’ for incident response by developing novel ML algorithms that reduce analyst workload and provide decision-making assistance (Nadeem et al., 2021). We propose to develop explainable ML algorithms that summarize large volumes of observable data (intrusion alerts, network & system logs) in order to discover contextually meaningful patterns from them. The student will explore multi-modal learning and generative AI to produce actionable explanations from these discovered patterns that are tailored to the operator’s expertise.

The evaluation of these algorithms will be done under closed-world and open-world settings. For the closed-world setting, a major challenge is the lack of suitable datasets to evaluate ML models (Rimmer et al., 2022). The student will set up a testbed together with our industry collaborators for the collection of intrusion alert datasets. For the open-world setting, the student will deploy these algorithms in real SOC environments in order to measure the extent of workload reduction experienced by security analysts. In doing so, we aim to develop technologies that are not only novel but also have real-world applications.

The PhD student will be embedded within the Semantics, Cybersecurity, and Services (SCS) group at University of Twente. The student will have the opportunity to participate in internships and/or collaboration with industry partners under the TUCCR initiative. The SCS group offers a stimulating, supportive, and diverse research environment, as well as plenty of opportunities for personal and professional growth.

About the organization:

The mission of the Semantics, Cybersecurity, and Services (SCS) group within the faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) is to advance the development of innovative online services with improved quality through context-alignment, and with reduced security and privacy threats. SCS is part of the Twente University Centre for Cybersecurity Research (TUCCR), a public-private partnership where experts, professionals, entrepreneurs, researchers, and students from industry and knowledge partners collaborate to deliver talents, innovations, and know-how in the domain of cybersecurity. The mission of TUCCR is to strengthen the security and digital sovereignty of our society by performing top-level research on real-world data, systems, and network security challenges. TUCCR’s founding partners are Betaalvereniging Nederland, BetterBe, Cisco, NCSC, NDIX, Northwave, SIDN, SURF, Thales, TNO, and the University of Twente.

The University of Twente connects technology, science, and engineering with social sciences. Established in 1961, it has since evolved into a leading hub for multidisciplinary research and education, hosting over 16,500 UT staff – students and academics. At UT, we work together to find groundbreaking solutions, at the forefront of technology and the digital revolution. Our research and education reaches beyond the campus. We push boundaries and join forces with relevant partners. We do this, for example, within the cooperation of the four technical universities in the Netherlands (4TU.Federation) and with the Vrije Universiteit in Amsterdam, with whom we offer a growing number of bachelor’s programmes. This attracts ambitious people from all over the world. With students from 85 different countries studying in one of our 16 bachelor’s or 30 master’s programmes, our community is diverse and international. Our campus, designed to encourage interaction, spans over 146 hectares and boasts modern amenities, sports facilities, and cultural events.

References

EuroS&P

SoK: Explainable Machine Learning for Computer Security Applications

Nadeem, Azqa, Vos, Daniel, Cao, Clinton, Pajola, Luca, Dieck, Simon, Baumgartner, Robert, and Verwer, Sicco

In IEEE European Symposium on Security and Privacy (Euro S&P), 2023

Abstract PDF Code

Explainable Artificial Intelligence (XAI) is a promising solution to improve the transparency of machine learning (ML) pipelines. We systematize the increasingly growing (but fragmented) microcosm of studies that develop and utilize XAI methods for defensive and offensive cybersecurity tasks. We identify 3 cybersecurity stakeholders, i.e., model users, designers, and adversaries, that utilize XAI for 5 different objectives within an ML pipeline, namely 1) XAI-enabled decision support, 2) applied XAI for security tasks, 3) model verification via XAI, 4) explanation verification & robustness, and 5) offensive use of explanations. We further classify the literature w.r.t. the targeted security domain. Our analysis of the literature indicates that many of the XAI applications are designed with little understanding of how they might be integrated into analyst workflows – user studies for explanation evaluation are conducted in only 14% of the cases. The literature also rarely disentangles the role of the various stakeholders. Particularly, the role of the model designer is minimized within the security literature. To this end, we present an illustrative use case accentuating the role of model designers. We demonstrate cases where XAI can help in model verification and cases where it may lead to erroneous conclusions instead. The systematization and use case enable us to challenge several assumptions and present open problems that can help shape the future of XAI within cybersecurity.
TDSC

Alert-driven Attack Graph Generation using S-PDFA

Nadeem, Azqa, Verwer, Sicco, Moskal, Stephen, and Yang, Shanchieh J

In IEEE Transcations on Dependable and Secure Computing, 2021

Abstract PDF Code Slides

Ideal cyber threat intelligence (CTI) includes insights into attacker strategies that are specific to a network under observation. Such CTI currently requires extensive expert input for obtaining, assessing, and correlating system vulnerabilities into a graphical representation, often referred to as an attack graph (AG). Instead of deriving AGs based on system vulnerabilities, this work advocates the direct use of intrusion alerts. We propose SAGE, an explainable sequence learning pipeline that automatically constructs AGs from intrusion alerts without a priori expert knowledge. SAGE exploits the temporal and probabilistic dependence between alerts in a suffix-based probabilistic deterministic finite automaton (S-PDFA) — a model that brings infrequent severe alerts into the spotlight and summarizes paths leading to them. Attack graphs are extracted from the model on a per-victim, per-objective basis. SAGE is thoroughly evaluated on three open-source intrusion alert datasets collected through security testing competitions in order to analyze distributed multi-stage attacks. SAGE compresses over 330k alerts into 93 AGs that show how specific attacks transpired. The AGs are succinct, interpretable, and provide directly relevant insights into strategic differences and fingerprintable paths. They even show that attackers tend to follow shorter paths after they have discovered a longer one in 84.5% of the cases.
Sec+AI

Open-World Network Intrusion Detection

Rimmer, Vera, Nadeem, Azqa, Verwer, Sicco, Preuveneers, Davy, and Joosen, Wouter

In Security and Artificial Intelligence, 2022

Abstract PDF

This chapter contributes to the ongoing discussion of strengthening security by applying AI techniques in the scope of intrusion detection. The focus is set on open-world detection of attacks through data-driven network traffic analysis. This research topic is complementary to the earlier chapter on intelligent malware detection. In this chapter, we revisit the foundations of machine learning-based solutions for network security, which aim to make network defense tools more autonomous, adaptive, proactive and responsive. Specifically, we give a comprehensive introduction to the research on anomaly detection for network intrusion detection – that is, defensive schemes that do not assume complete prior knowledge of malicious patterns and instead learn the notion of normality from benign traffic. Along with outlining the recent advances in the field, we provide insights and reflect on the current limitations and research challenges. Therefore, this chapter presents compelling research opportunities to advance machine learning techniques in network security and push the boundaries of open-world network intrusion detection.