Question and dataset
Can language models identify controversial Wikipedia topics more effectively than edit-history and lexical signals? We manually annotated a balanced dataset of 142 topics (71 controversial and 71 non-controversial), using group discussion to establish labels. Preprocessing extracted article titles, edit counts, article content, summaries, controversy scores, and labels.
Classical and revision-history baselines
Evaluated logistic regression on edit counts, Naive Bayes on article text, logistic regression combining CountVectorizer features with edit counts, and a controversy measure based on mutual reverts. The strongest baseline combined text and edit counts, reaching 93.1% accuracy; the other reported results were 75.9%, 86.2%, and 77.4%, respectively.
Language-model and prompt experiments
Compared BART NLI zero-shot classification (65%) and a RoBERTa-base few-shot configuration (44%). Adding edit counts reached 80%; Pegasus summaries reached 81%. Grouping articles into nine thematic subtopics and supplying related examples reached 87%, the best reported prompting result. BERT sentence embeddings and cosine similarity were also used to select in-context examples, reaching 84%.
Evaluation and interpretation
The report evaluates on just 29 examples. Classical text-plus-edit features outperform the reported language-model configurations; this study does not establish an LLM advantage or broad generalization. Dataset expansion and more diverse evaluation were identified as future work.
Research record
Co-authored the project report with Rudhar Pratap Singh and Ananya Aakriti. Conducted at IIT Delhi under Prof. Abhijnan Chakraborty, Oct-Nov 2023.