GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs
Ahmed Sohair Khan, Estrid He, Chenglong Ma, Monica Wachowicz, Elham Naghizade
cs.CL · cs.CR
Quick verdict
Claims auditable LLM attribute inference via source-linked PKGs at near-baseline ASR; evidence validation strong, but graph-vs-RAG ablation absent, stats thin.
Summary
The paper introduces GraphProfiler, an LLM-based personal-attribute inference system that indexes each user's post history into a personal knowledge graph (GraphRAG v2.7.0 with GPT-4o) whose entities, relations, and community reports retain links to source posts. Attribute-specific graph queries return predictions that are resolved, via cited graph records, to a small set of originating source files, yielding an auditable evidence object intended to support targeted privacy mitigation. On SynthPAI the system reaches 86.7% attack success rate (ASR) across eight attributes, slightly below text-only baselines FTI (88.7%) and AutoProfiler (88.3%); on PANDORA it reaches 84.6% with 100% prediction coverage, versus 69.2% coverage for AutoProfiler. Provenance is validated three ways: a GPT-4o judge finds cited evidence more supportive than matched random evidence (69.4% vs 16.9% on PANDORA), blinded human annotators agree (85% vs 33%, 100 cases, two in-team annotators), and evidence-only inference recovers attributes at 51.9% vs 7.5% for random controls. A removal ablation on 30 users per dataset shows cited-source deletion lowers ASR more than matched random deletion (PANDORA: 100% to 73.3% vs 88.3%). Main limitations, partly acknowledged by the authors: no chunk-level RAG baseline isolating the graph's contribution, no statistical testing (n=30 ablations), GPT-4o only, and PANDORA's flair-derived proxy labels.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- GraphProfiler resolves over 98% of predictions (99.1% SynthPAI, 98.8% PANDORA) to cited graph records and source files, with cited evidence covering a median 2.6-3.1% of a user's posts on PANDORA.
- It stays within two ASR points of text-only baselines on SynthPAI (86.7% vs 88.7%/88.3%) while achieving 100% prediction coverage on PANDORA, where AutoProfiler reaches only 69.2%.
- Cited evidence is functionally important: on 30 users per dataset, deleting cited sources and re-indexing drops ASR far more than matched random deletion (PANDORA 100%→73.3% vs 88.3%; SynthPAI →66.4% vs 80.9%).
- Support audits and blinded human annotation show cited evidence is far more supportive than random same-count evidence (69.4% vs 16.9% judge on PANDORA; 85% vs 33% human).
Strengths
Functional validation of provenance, not just plausibility
Section 6.3-6.4 and Figures 3-5 triangulate citation quality: support audit vs matched random evidence (69.4% vs 16.9% PANDORA judge; 88.3% vs 17.5% SynthPAI), evidence-only inference (51.9% vs 7.5%; 58.2% vs 4.9%), and matched cited-vs-random deletion with full graph re-indexing (stricter than post-hoc node deletion, Appendix E).
Coverage robustness on long real-world histories
Table 1: GraphProfiler is the only method completing all PANDORA users (100% coverage vs 69.2% for AutoProfiler, which fails on users with large comment counts; FTI fails on 7 users). This is a concrete, defensible practical advantage of per-user graph indexing over whole-history prompting.
Blinded human validation with cited-vs-random controls
Candid limitations and full prompt disclosure
Citations recovered structurally, not free-form
Weaknesses
The decisive baseline is missing
No statistical inference anywhere
Lenient matching combined with forced guessing inflates absolute ASR
Same-model judging and in-team annotators
Removal results are conditional, not mitigation estimates
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
GraphProfiler (arXiv 2609.12448, RMIT) is a dual-use privacy study: an LLM attribute-inference attack that also emits auditable provenance. The claimed contribution is not raw accuracy but an *evidence object* — each prediction resolved through cited GraphRAG records to originating post files — enabling targeted mitigation (edit the ~2.6–3.1% of posts that carry the signal, not the whole history; Section 6.2). Four central claims: (1) ASR within two points of text-only baselines (Table 1); (2) >98% of predictions carry resolved citations; (3) cited evidence is supportive by judge and human audits; (4) cited sources are functionally causal by removal ablation.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is GraphProfiler?
An LLM-based system that infers sensitive attributes (age, income, occupation, location, etc.) from a user's social-media history while producing auditable evidence: it builds a personal knowledge graph whose entities and relations link back to the originating posts, and each prediction cites the graph records and source files that support it, enabling defenders to edit only the few leaking posts.
How accurate is GraphProfiler?
On the synthetic SynthPAI benchmark it reaches an 86.7% attack success rate across eight attributes, about two points below text-only baselines FTI (88.7%) and AutoProfiler (88.3%). On a real Reddit-derived PANDORA subset it reaches 84.6% (age 80.8%, sex 88.5%) with 100% prediction coverage, versus 69.2% coverage for AutoProfiler.
How does the paper show the cited evidence is genuine rather than post-hoc rationalization?
Three checks: blinded human annotators and an LLM judge rate cited evidence supportive far more often than matched random evidence (85% vs 33% human); inferring the attribute from cited texts alone succeeds far above the random control (51.9% vs 7.5% on PANDORA); and deleting cited posts then rebuilding the graph lowers attack success much more than deleting random posts.
What are the main limitations of the study?
The authors acknowledge no chunk-level RAG baseline was run, so the graph's marginal benefit over plain retrieval-based citation is untested; ablations cover only 30 users per dataset with no statistical tests; everything runs on GPT-4o; PANDORA labels are flair-derived proxies; and removal experiments are conditioned on already-successful inferences, so they do not estimate population-level mitigation effectiveness.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The sampled excerpt omits the Section 5 baseline descriptions (FTI/AutoProfiler details), part of Appendix A's matching policy, and portions of the Appendix J-L prompts (sampling gaps); judgments touching those parts rely on the visible text only.
- Figure content was extracted as flattened label/number runs (Figures 3-6, 9-14); per-attribute figure values cited in this review are taken from prose or Table 1, not parsed from figure axes, so individual figure-cell attributions were avoided.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.