Phishing Detection Using Knowledge Distilled from Large Language Models

Abstract

This thesis focuses on privacy-preserving knowledge distillation applied to email phishing detection. Email phishing remains a critical cybersecurity threat, and although large language models, such as GPT-4o, demonstrate strong classification capabilities, their deployment in production introduces high costs, latency, and privacy concerns. The thesis describes a streaming- based design utilizing knowledge distillation to approximate the classification behavior of GPT-4o, which serves as the teacher model, within Cisco's email anti-phishing system. The proposed solution operates entirely on local in- frastructure to fulfill rigorous operational constraints, specifically regarding temporal email data retention and zero external data transfer. Moreover, a fully automated pipeline is implemented where student models are retrained weekly on borderline cases labeled by the teacher model. Through dynamic torch.compile optimization, the student model's inference latency is re- duced to meet the millisecond-scale requirements of the production pipeline. Experimental results identify a bidirectional language model as the optimal solution which provides an ideal balance between predictive and runtime performance. According to the experiment outcomes, this model replicates the teacher model's behavior on borderline...

This thesis focuses on privacy-preserving knowledge distillation applied to email phishing detection. Email phishing remains a critical cybersecurity threat, and although large language models, such as GPT-4o, demonstrate strong classification capabilities, their deployment in production introduces high costs, latency, and privacy concerns. The thesis describes a streaming- based design utilizing knowledge distillation to approximate the classification behavior of GPT-4o, which serves as the teacher model, within Cisco's email anti-phishing system. The proposed solution operates entirely on local in- frastructure to fulfill rigorous operational constraints, specifically regarding temporal email data retention and zero external data transfer. Moreover, a fully automated pipeline is implemented where student models are retrained weekly on borderline cases labeled by the teacher model. Through dynamic torch.compile optimization, the student model's inference latency is redu- ced to meet the millisecond-scale requirements of the production pipeline. Experimental results identify a bidirectional language model as the optimal solution which provides an ideal balance between predictive and runtime per- formance. According to the experiment outcomes, this model replicates the teacher model's behavior on borderline...