Adversarial Robustness of Phishing Email Detection: A Comparative Study of TF-IDF + Logistic Regression and Fine-Tuned DistilBERT

Most machine-learning classifiers used to detect phishing emails appear nearly perfect when tested on standard, clean datasets. However, this perfection is often an illusion. It is created by evaluating models on data that looks exactly like what they were trained on. When faced with emails that have been subtly altered, these highly accurate models tend to collapse. Such alterations might include swapping a few characters or replacing key words with synonyms.

The relationship between attackers and defenders is often described as a "cat-and-mouse" contest. As defenders deploy new detection methods, attackers evolve new evasion techniques. In the context of phishing, this means a model might boast 99% accuracy in a laboratory setting. That number tells us very little about performance when an attacker deliberately tries to bypass it. This paper investigates whether moving from simple, frequency-based models to sophisticated, context-aware transformers provides any meaningful protection against these deliberate manipulations.

The Illusion of High Accuracy

The status quo in phishing detection research relies heavily on "in-distribution" testing. This involves taking a dataset and splitting it into training and testing sets. Researchers then measure how well the model performs on the held-out test set. Because the test set is drawn from the same distribution as the training data, the results typically show extremely high accuracy. Many studies report figures exceeding 99%. As the authors note, these figures describe performance on held-out data. They say little about what happens once an attacker rewords a message to evade detection.

This creates a significant blind spot. If a defender selects a model based solely on these "clean" benchmarks, they are optimizing for pattern recognition within a narrow, static environment. An adversary, however, operates in a black-box evasion setting. This is a scenario where an attacker can modify the input text to move it across the model's decision boundary. Crucially, they can do this without ever needing to know the model's internal weights or parameters. The gap between laboratory success and real-world utility is precisely where the danger lies.

Comparing Lexical and Contextual Approaches

To investigate this gap, the authors conducted a controlled, pairwise comparison of two fundamentally different architectures. They ensured both models were trained on the same unified corpus of 82,255 emails. They also evaluated them under identical conditions.

1. The Lexical Baseline (TF-IDF + Logistic Regression): This approach uses Term Frequency-Inverse Document Frequency (TF-IDF). This is a method that converts text into a numerical vector based on word frequency. It captures the "bag-of-words" essence of an email. It looks for specific, high-signal tokens like "account," "click," or "http." This is paired with Logistic Regression. This is a linear model that efficiently handles the large, sparse feature spaces produced by TF-IDF.
2. The Contextual Transformer (Fine-tuned DistilBERT): Unlike the lexical model, DistilBERT is a transformer-based model. It learns contextual token representations. Instead of just counting words, it understands how the meaning of a word changes based on surrounding words. The authors fine-tuned this model from a pre-trained checkpoint using the Hugging Face Trainer API. This allowed the model to capture deeper semantic nuances.

By testing these side-by-side, the researchers aimed to see if the "intelligence" of the transformer would translate into robustness. They wanted to see if it could withstand adversarial perturbations. These are small, targeted changes to the text designed to fool the classifier.

Robustness Collapse Across Architectures

The results of the study are striking in their uniformity. The authors report that while both models achieved exceptional performance on normal, in-distribution data, they both suffered a catastrophic decline when faced with adversarial emails. Specifically, both models achieved 98.59% (TF-IDF + LR) and 99.04% (DistilBERT) accuracy on clean data.

Under adversarial testing, both models fell to approximately 64% accuracy. The TF-IDF + LR model saw a 34.59 percentage-point drop. DistilBERT saw a 35.40 percentage-point drop. The difference between the two models' adversarial accuracy was a mere 0.36 percentage points. This is equivalent to a single email in the 275-sample adversarial test set.

While DistilBERT maintained a slight edge in phishing recall (detecting 90.80% of phishing emails compared to 87.12% for the baseline), it suffered more heavily in legitimate recall. As shown in, the confusion matrices reveal a major issue.

Under attack, both models struggle significantly to identify legitimate emails. DistilBERT's legitimate recall dropped to 24.11%, compared to 30.36% for the lexical model. This suggests a massive operational risk. The primary danger is not just missing phishing attempts, but being overwhelmed by a flood of false positives (incorrectly flagging legitimate emails as threats).

Statistical Constraints and Generalization

While the findings are compelling, several caveats limit the conclusions. First, the adversarial test set is quite small, containing only 275 samples. As the authors acknowledge, this limits the statistical power of their comparison. A single misclassification accounts for roughly 0.36 percentage points of accuracy. Therefore, the negligible difference between the two models might simply be noise.

Second, the study is limited by its focus on the English language. The training corpus is predominantly English. It is unclear if these robustness trends would hold in multilingual contexts. Finally, the DistilBERT model was fine-tuned on a 50,000-example subset of the training data. This was due to hardware constraints. This limitation might introduce subtle biases in its performance profile.

Given these limitations, we must reconsider how we evaluate model quality. The data suggests that the perceived advantage of transformer-based models vanishes under adversarial pressure. The intuition that contextual understanding would provide a "buffer" against word-level perturbations does not hold here. Instead, both lexical and contextual signals appear to degrade simultaneously.

Consequently, clean-data accuracy should not be used as a proxy for security efficacy. If you are building or procuring email security tools, you should demand adversarial robustness metrics. You should also ask for false-positive rates under stress. Relying on high F1-scores on standard benchmarks may be misleading. The authors suggest exploring ensemble approaches. Combining the unique failure modes of lexical and transformer models might be a promising research direction. For practitioners, the immediate takeaway is a warning. Your 99% accurate model may actually be a 64% accurate model in the wild.

Code and data for the underlying datasets are reportedly available via the links provided in the paper (see the IWSPA GitHub repository).