Stop Removing Stopwords: How an Inherited Preprocessing Default Distorts Legal Text-as-Data

Summary

This paper argues that stopword removal — a preprocessing default inherited from mid-century information retrieval — yields no measurable benefit for sparse, interpretable legal text classification. Matching Supreme Court Database labels (decisionDirection, lawType) to Caselaw Access Project opinion texts, the author builds two binary tasks bracketing F1 headroom: ideological direction (no-removal holdout F1 0.6819, N=7,668) and constitutional versus non-constitutional law type (0.9185, N=7,001). The core method is an exhaustive single-word ablation: each of roughly 18,500 candidate unigrams is deleted in turn, the full TF-IDF (1,3) plus L2 logistic regression pipeline is refit, and ΔF1 is recorded, at a total cost of some 400 compute hours. Three generic stoplists (scikit-learn, NLTK, spaCy) fall below the no-removal baseline in all twelve tests (deltas −0.0013 to −0.0168, none individually significant); the optimized lists (78 and 95 words) shift holdout F1 by +0.0023 and +0.0001 with confidence intervals spanning zero; and XGBoost meta-models over nine word-level statistics fail to predict beneficial removals (R² −0.007 and 0.038, AUC 0.504 and 0.537). A Longformer baseline ties on law type and trails on ideology. The author acknowledges that joint removals go untested, holdouts are small, only one pipeline is examined, and both tasks are binary.