PUBG Ally: A Conversational Embodied Agent as an AI Teammate
Beomsoo Kim, Byeongju Kim, Dohyun Kim, Dongwon Kim, Eunchong Kim, Hongmin Kim, Hyeojung Im, Hyeonbin Hwang, et al.
cs.AI · cs.CL · cs.HC
Quick verdict
Impressive report of a live on-device AI teammate in PUBG trained on 39k real-player sessions; strong engineering, but proprietary data limits verification.
Summary
This industrial technical report from KRAFTON presents PUBG Ally, a voice-enabled embodied agent deployed as an AI duo teammate in PUBG: BATTLEGROUNDS. The system pairs an event-driven language-model agent (System 2) operating through a bounded tool interface with a deterministic behavior-tree controller (System 1) for latency-critical movement and combat. Training data comes from 28 days at a Korean PC bang where 1,046 participants played 38,956 sessions; a Gemma 4 31B teacher, prompt-optimized with GEPA, generated demonstrations and DAgger-style corrections of student rollouts, which were distilled into 2B per-locale students (4-bit, ~1.6–2.1 GB VRAM) running on-device alongside STT and TTS on RTX 4060-class GPUs (median LM latency 0.80s vs 2.54s cloud; end-to-end exchange ~1.6s). Safety combines specifications, mixed training data, keyword guardrails, and memory redaction; final models reach 98.3–99.4% harmless rates on broad benchmarks but 84.1–99.0% on production-representative dialogues, with boundary-breached cases at 63.6% and over-refusal rising to 2.2–6.1%. In a live beta survey across 141 countries, positive minus negative responses to recommending Ally was +25.1 percentage points among respondents with confirmed play. Main limitations: proprietary data and code limit reproducibility; evaluation relies on LLM judges near score ceiling; player-satisfaction figures come from self-selected survey respondents without reported response rates.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A layered System 1/System 2 architecture enables a 2B on-device SLM to serve as a real-time voice teammate in live PUBG duo matches, with ~1.6s end-to-end spoken exchanges vs 3.4s for a cloud configuration.
- Nearly 39k real-player sessions support DAgger-style teacher correction that closes the gap between teacher demonstrations and the contexts a deployed student actually visits.
- Iteratively refining the offline evaluation suite against player A/B preferences aligns internal model selection with what players value in a teammate (§6.4).
- Contextual safety training plus runtime guardrails achieve 98.3–99.4% harmless rates on broad benchmarks while keeping over-refusal at 2.2–6.1%, with boundary-breached dialogues (63.6%) as the main residual failure.
- In the live beta survey across 141 countries, respondents with confirmed play net-recommended Ally by +25.1 percentage points, with 50.0% choosing teammate or companion framings.
Strengths
Deployment evidence at genuine scale
38,956 sessions with 1,046 real players over 28 days (§4.1), a blind randomized A/B with 797 responses from 433 players (§6.3), and a live-service beta survey spanning 141 countries (§8) — far beyond the simulation-only evidence typical of game-agent papers.
Best-practice safety evaluation protocol
§6.2/§7.2 curate locale-specific sets from 15+ public benchmarks plus production-representative multi-turn dialogues stratified by boundary state (held/weakened/breached), report rates before and after the runtime guardrail, and calibrate the Gemini 3 Flash judge on n=266 expert labels (sensitivity 0.91, specificity 0.79, κ=0.70).
Candid negative reporting
Engineering detail unusually thorough for an industry report
DAgger-style correction adapted to LM agents
Weaknesses
No code, data, model, or harness release
Headline survey result lacks denominators and baselines
Near-ceiling LLM-judge evaluation with structural circularity
Weakest locale closed by lexical filter, not model behavior
Cost-driven teacher with acknowledged frontier gap, no ablation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an industrial technical report from KRAFTON describing PUBG Ally, a voice-enabled AI teammate (co-playable character) deployed in a two-week live beta of PUBG: BATTLEGROUNDS — duo mode, Sanhok, 64-player matches, English/Korean/Chinese. The paper claims five contributions: a layered architecture pairing an event-driven LM agent (System 2) with a behavior-tree controller (System 1); large-scale training data from 38,956 real-player sessions; a player-preference-grounded evaluation framework; contextual safety for player-facing deployment; and on-device production engineering. Headline quantitative claims: ~1.6s end-to-end spoken exchange on-device (vs 3.4s cloud), 98.3–99.4% harmless rates on broad safety benchmarks, and +25.1pp net recommendation in a 141-country beta survey.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PUBG Ally?
PUBG Ally is a voice-enabled AI teammate (co-playable character) deployed by KRAFTON in PUBG: BATTLEGROUNDS. It plays duo matches alongside one human player, communicates by voice, and acts autonomously — reviving downed players, looting on request, calling out enemies — with all language and speech models running on the player's own GPU rather than in the cloud.
How was PUBG Ally trained?
KRAFTON collected 38,956 gameplay sessions from 1,046 real players over 28 days at a Korean PC bang. A Gemma 4 31B teacher model, prompt-optimized with GEPA, generated demonstrations and DAgger-style corrections of student play; this data was distilled into small 2B per-language student models quantized to 4-bit so they fit alongside the game on 8GB consumer GPUs.
How fast is PUBG Ally, and does it run in the cloud?
It runs on-device. On RTX 4060-class GPUs the median language-model step takes 0.80s versus 2.54s for a cloud configuration, and a complete spoken exchange takes about 1.6s versus 3.4s in the cloud; each language model occupies roughly 1.6–2.1GB of VRAM including its working cache.
Did players actually like PUBG Ally?
In KRAFTON's live beta survey reaching players in 141 countries, among respondents whose play was confirmed in game records, positive responses exceeded negative ones by 25.1 percentage points on whether they would recommend Ally; 50.0% chose teammate or companion framings. This is self-selected survey feedback without a reported response rate, not a controlled study.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extracted text omits portions of Sections 3.1, 5, 6.4, the start of 7.1, most of 8, and all appendices; Tables 1, 3, 4, and 6 are partially or fully missing, so some A/B and survey details could not be verified.
- Figures (interaction storyboards, survey charts, safety curves) are referenced but not analyzable from the extracted text.
- arXiv v1 industry report without peer review; the contributor/acknowledgment section is truncated in the extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.