On the security and privacy of LLMs in Mobility
Mauro Conti, Lorenzo Perinello, Umberto Salviati
cs.CR · cs.AI
Quick verdict
Claims first security survey of 35 LLM-mobility works (AI Act rubric); timely, but count contradictions and an untested 'significant' claim weaken it.
Summary
This GoodIT '26 survey examines the security, privacy, and reliability posture of LLM deployments in mobility, claiming to be the first security-focused survey of this intersection. The authors searched Web of Science (mobility AND generative-AI terms), screened 125 hits down to 35 papers, and coded each for model family, functional role, and treatment of security/privacy/reliability; separately they translate EU AI Act articles into technical criteria scored on a 5-point Likert rubric. Findings: GPT appears in 19/35 works and Llama in 11; decision facilitation dominates (25/35). Only 2 works implement security measures, 1 addresses privacy, 6 address reliability, and 13/35 release code. Compliance scoring (35×8=280 points) yields near-floor means for adversarial testing (1.11), cybersecurity protection (1.17), pipeline attack testing (1.09), watermarking (1.03), and training-data transparency (1.00), versus 3.03 for accuracy metrics. The authors conclude the gap reflects research priorities, not technical infeasibility. The survey is timely and the rubric is usable, but the version reviewed contains internal inconsistencies: nine criteria claimed versus eight evaluated; contradictory counts for security/privacy discussion; an impossible statistic (μ=1.00, σ=0.00, max=5); non-LLM works in the corpus; and an unexplained claim of 'statistical significance' despite purely descriptive reporting. Reproducibility is moderate: the corpus and rubric are documented, but no annotation data or inter-rater statistics are released.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 3
● low 0
5 strengths · 5 weaknesses · 0 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLM research in mobility concentrates on GPT (19/35 works) and Llama (11/35) and on traffic tasks while largely neglecting security (2/35 implement), privacy (1/35) and reliability (6/35).
- Mapped against AI Act-derived criteria, the literature scores near floor on vulnerability, risk-management and transparency requirements (1.00-1.17) versus defined accuracy metrics (3.03).
- Among 35 reviewed works, only one contains a partial vulnerability assessment and one a partial risk-management treatment, so regulatory adherence is effectively absent.
- The compliance gap reflects research priorities rather than technical infeasibility, since isolated works reach the rubric ceiling (max = 5).
- No prior survey systematically addresses security of LLMs in mobility, and because the AI Act classifies transportation AI as high-risk (Art. 6(2), Annex III), security-by-design is a legal requirement, not an option.
Strengths
Novel, well-motivated intersection
§2 shows existing LLM-in-ITS surveys (e.g., Mahmud et al. [45], Hassija et al. [32], Kaur et al. [37]) omit security entirely; this is the first survey to make security/privacy/reliability the coding axis for LLM mobility research (35 papers).
Actionable regulatory rubric
§4.2 maps specific AI Act articles (Art. 55(1)(a-d), 9(2), 17(1), 50(1-4), 53(1), 14, 15) to concrete technical criteria with an explicit 5-point Likert rubric (§4.3.2) and a defensible scope cut (§4.3.1) separating researcher-controllable duties from commercial-deployment obligations.
Transparent per-paper evidence table
Honest, quantified negative findings
Timely policy relevance
Weaknesses
Unsupported inferential claim
Impossible descriptive statistic
Systematic internal count inconsistencies
Survey methodology opacity
Corpus contamination and weak model-selection rationale
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review — 'On the security and privacy of LLMs in Mobility' (GoodIT '26, DOI 10.1145/3794786.3830783)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 0 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What did the survey find about security of LLMs in mobility research?
Across 35 reviewed papers, only 2 implemented any security mechanism and 4-5 merely discussed it; only 1 addressed privacy concretely, 6 included reliability mechanisms, and 13 of 35 released code. Aggregate AI Act-style compliance scores sit near the scale floor (about 1.0-1.2 out of 5) for vulnerability, risk-management and transparency criteria, versus 3.03 for accuracy metrics.
Why does the EU AI Act matter for LLM-based traffic systems?
The Act classifies AI safety components in road traffic management and transport operations as high-risk (Article 6(2), Annex III), with full applicability from August 2026 and deadlines for embedded high-risk mobility systems extending to August 2027. The paper translates the relevant articles into technical criteria scored 1-5 and finds academic LLM-mobility research near the minimum on most of them.
Which LLM families dominate mobility research according to the paper?
GPT-family models appear in 19 of the 35 reviewed works (over half) and Llama in 11; BERT variants appear in 4, Mistral in 3, with Gemini, Qwen, Gemma and others marginal. Functionally, most works use LLMs as decision facilitators (25/35) or information processors (18/35), and traffic management/prediction tasks dominate.
Can the survey's results be reproduced or verified?
Partially. The 35-paper corpus is identifiable from the reference list and the 5-point scoring rubric is defined in the paper, so an independent re-coding study is possible. But the authors release no annotation dataset, no search log or dates, and report no inter-rater reliability; several internal counts (nine vs eight criteria; an impossible μ=1.00/σ=0.00 with max=5) need correction before exact numbers can be cited.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (model-family distribution) is referenced but only its caption is extractable; family counts were taken from the §3.3.1 narrative.
- Some §3.3.1/§3.3.3 sentences appear garbled (e.g., Vicuna counted as both two and one); a few count mismatches could be extraction artifacts, though most are corroborated against Table 1.
- Per-paper AI Act sub-scores in Table 1 (rightmost four columns) carry some column-shift risk in extraction; aggregate Table 2 values were preferred where both existed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.