Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

Summary

This paper is a technical manual for an open-source toolkit designed to test whether transformer language models separate the senses of a single word form according to context. The core construct, the 'bridge form,' is one written word (e.g., 'current') recurring unchanged across two or more subject domains with a different sense in each. Because all occurrences share one embedding-layer vector, any domain separation arising in later layers is attributable to context. The documented pipeline spans six stages: declarative bridge-form specification over specific, non-overlapping Wikipedia categories; corpus acquisition via batched MediaWiki API calls returning article introductions; whole-word occurrence localization; offset-based layer-wise representation extraction (first occurrence per document only, dropping truncation-window misses); a domain-pairwise silhouette coefficient in cosine distance computed at full hidden width for every layer; and a shared-basis PCA visualization paired with per-pair silhouette curves. Each design choice is justified by the failure mode it prevents, including the multi-group bias of pooled silhouette scores and axis-comparability artifacts in projections. The toolkit supports six public models spanning BERT-family encoders and GPT-2/Pythia causal decoders. The manuscript deliberately reports no empirical results, positioning itself as an instrument reference; code and corpora are archived on Zenodo (CC BY 4.0). Main limitations, acknowledged in Section 8: no significance testing or confidence intervals, no minimum sample-size enforcement, domain labels serving as unverified sense proxies, and no control condition distinguishing sense-specific individuation from generic topical context effects.