Cyber Aggression Detection

The study explores how bystanders influence online aggression and how machine learning models can capture these complex dynamics. While traditional cyberbullying detection focuses on standalone posts, bystanders play a critical role in shaping and mitigating online harm. To address this, the project is structured across three progressive phases, evolving from initial pilot data to advanced, automated labeling using large language models.   

Phase One  The project began with the CYBY23 pilot dataset, which introduced thread-level analysis by capturing both main posts and bystander replies across 639 filtered tweets. This foundation was subsequently expanded into the CYBY24 dataset, encompassing approximately 13,309 tweets across 2,789 threads.   

The early findings from this phase are particularly interesting. Expert annotators altered their severity classification of a main post 60 percent of the time after observing the bystander replies, demonstrating that defenders, instigators, and impartial observers directly shape the perception of online harm. A step-wise machine learning model utilizing BERT and BiLSTM showed that incorporating these bystander roles improves cyberbullying detection. However, experiments revealed that initial high performance was partly due to the model memorizing reply identifiers. Without these identifiers, the model achieved a weighted F1 score of 0.70.  

Phase Two started with a methodological audit and analysis of CYBY24. The findings showed that the data collection and annotation practices significantly impact model performance. The audit also uncovered structural issues, such as integer overflow errors that duplicated identifiers and cross-contaminated unrelated conversational threads. Furthermore, the legacy annotation guidelines relied on subjective, four-class taxonomy definitions that lacked mutually exclusive criteria.

This points to a necessary theoretical and structural shift. The project transitioned to a broader “cyber-aggression” framework, simplifying the main post label into a strict, objective binary classification of Aggression versus No Aggression. Bystander roles were redesigned into an agreement-based taxonomy consisting of Agree, Disagree, Impartial, and Other, removing preemptive assumptions about the hostility of the main post. By deploying a custom three-stage chronological annotation workflow, the project created the highly refined CABY26 corpus.

Phase Three is now underway, exploring the practical role of Large Language Models (LLMs) to reduce the massive manual labeling burden. Because manual bystander-role annotation is highly costly and yields varying human agreement, this phase tested whether LLMs could replace human annotators as “DirectLabelers”. The research evaluated multiple LLM architectures on the CABY26 corpus, testing zero-shot prompting, few-shot prompting, and parameter-efficient fine-tuning techniques like LoRA. The results demonstrate that few-shot Llama-3.3-70B produced the best prompted role labels, while RoBERTa proved to be the best fine-tuned model.

Importantly, utilizing LLM auto-labeled roles instead of human annotations resulted in no statistically significant difference in downstream aggression detection performance. This statistical equivalence proves that automated models can reliably match human-level input for this task. Using a dialogue-level detector without identifier leakage, the new framework achieved a weighted F1 score of 0.78 with human roles, 0.79 using roles from Llama-3.3-70B, and 0.81 using roles from RoBERTa. These results are noticeably better than the 0.70 baseline of the previous CYBY24 model. Ultimately, this demonstrates that LLMs offer a highly capable and cost-effective alternative to human labeling, allowing context-aware risk detection systems to scale efficiently.