LLM-agent reliability researcher: diagnosability of agent traces and construct validity of guardrail evaluations
2
papers here
1
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
—
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Insufficient data: the output axis measures publication volume — paper counts, venue types, and output cadence — but this snapshot contains only two arXiv preprints with no aggregate bibliometric counts, so no output score can be computed.
Impact—
No impact score is available: this axis would measure citation-based influence such as citation counts, h-index, and downstream uptake, but the snapshot provides no citation records for Sidi Chang, so the dimension is left unassessed rather than estimated.
Focus100/100
The focus dimension scores 100, derived from the radar's field distribution, which records a single field entry — AI reliability / fault diagnosis for autonomous agents. Both catalogued 2026 papers orbit one question — whether evidence about LLM-agent behaviour is diagnosable and whether evaluation constructs hold up — indicating a tightly concentrated rather than scattered research programme.
Collaboration22/100
Collaboration scores 22 from a small raw network: only 2 distinct coauthors appear in the snapshot, and the recurring partner is Peiying Zhu. Both 2026 papers place Sidi Chang in the last author position without corresponding-author status, consistent with a compact, stable two-person working pattern rather than a broad distributed collaboration network.
Activity—
Insufficient data: the activity axis would track recent publishing cadence — papers per year and ongoing projects — but only two 2026 preprints are in scope with no multi-year record, so the current activity level cannot be scored.
Reproducibility55/100
Reproducibility scores 55, the mean over a single scored paper (reproPapers = 1), and both catalogued preprints carry a medium reproducibility-readiness rating. The diagnosability paper's pre-registered two-gate test and the evaluation audit's C1–C4 contract signal methodological care, while Evolve notes that external validity resting on one toy simulator limits confidence.
Research summary
Sidi Chang is an early-stage researcher working at the intersection of AI reliability and evaluation methodology for LLM-based agents, currently represented in the Evolve catalog by two 2026 arXiv preprints. The first, "When Are Aggregate Agent Traces Diagnosable?", proposes a pre-registered two-gate diagnosability test for aggregate agent traces, combining traffic-governed interpretation with calibrated abstention; Evolve rates its simulation evidence as rigorous while cautioning that external validity rests on a single toy simulator. The second, "When Guardrails Look Effective?", is a self-audit of LLM agent-market simulations showing that a guardrail's apparent welfare gain of +87.4 collapses to +7.2 once a schema confound is removed, and introducing a C1–C4 contract for gating policy claims. Both papers list Sidi Chang in the last author position (non-corresponding), with Peiying Zhu as the recurring coauthor in a two-person collaboration network. The radar profile scores focus at 100 — the field distribution records a single field entry — reproducibility at 55 (mean over one scored paper), and collaboration at 22; output, impact, and activity lack sufficient data and are not scored. Together the work signals a coherent programme: deciding when evidence about autonomous agents is trustworthy at all.
Collaboration pattern
The collaboration footprint is compact: only 2 distinct coauthors appear in the bibliometric snapshot, yielding a collaboration score of 22. Peiying Zhu is the recurring partner across both 2026 preprints, and on each paper Sidi Chang holds the last author position without corresponding-author status. This points to a stable two-person working unit centred on agent reliability and evaluation auditing rather than a broad, distributed collaboration network.
Research keywords
Highlights
Pre-registered two-gate diagnosability test for aggregate agent traces
Construct validity collapse: guardrail gains fall from +87.4 to +7.2
Calibrated abstention as a first-class output of fault diagnosis
Timeline
- 2026First catalogued preprint, "When Are Aggregate Agent Traces Diagnosable?", introduces a pre-registered two-gate diagnosability test combining traffic-governed interpretation with calibrated abstention.
- 2026Second preprint, "When Guardrails Look Effective?", audits LLM agent-commerce evaluation, showing guardrail welfare gains of +87.4 collapse to +7.2 once a schema confound is removed.
- 2026Both papers list Sidi Chang as last author (non-corresponding), with Peiying Zhu as the recurring coauthor, consolidating a two-person research line on the trustworthiness of agent evidence.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Radar scores come from a minimal bibliometric snapshot; the output, impact, and activity axes are unscored because the underlying data is insufficient.
- Institution, country, and ORCID are not provided; this profile is name-resolved from public bibliographic data, and the name may match multiple researchers.
- The evidence base is two 2026 arXiv preprints reviewed by Evolve; no peer-reviewed venue records or citation counts are in scope.
- The reproducibility score of 55 is a mean over a single scored paper (reproPapers = 1), so it carries high uncertainty.