What LLM-based judges reward besides answer quality, and how to build evaluation protocols that control for artifacts such as response length.
- Explaining Length Bias in LLM-Based Preference Evaluations · Findings of EMNLP 2025
Zheyuan Xiao
- M.S. Candidate in Computer Science, The University of Texas at Austin
- Software Engineer II, Resideo · Auckland, New Zealand
I am an M.S. candidate in Computer Science at The University of Texas at Austin, working on large language models. My research asks how LLMs should be evaluated, how post-training data shapes what they learn, and how they behave as agents and in simulation. I am currently collaborating with researchers at the Allen Institute for AI (Ai2).
I also work as a Software Engineer II at Resideo in Auckland, New Zealand, where I build automated verification and validation systems for firmware and cloud-connected products. Working on reliability, testing, and end-to-end behavior in production has shaped my interest in systems that are not only capable, but also measurable, dependable, and useful in practice.
Before UT Austin, I received a First Class Honours B.E. in Software Engineering from The University of Auckland.
News
Our paper Population-Aligned Persona Generation for LLM-based Social Simulation has been accepted to the NeurIPS 2026 Main Track. See you in Sydney this December! 
Our paper When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection has been accepted to the EMNLP 2026 Main Conference.
Our work explaining length bias in LLM-based preference evaluations appears in Findings of EMNLP 2025.
Our paper Unveiling the Learning Mind of Language Models has been accepted to the NeurIPS 2025 Main Conference.
Research
How the choice of supervision shapes what a model learns: selecting teacher answers by a student model's learning cost rather than by teacher strength, and studying how models learn from instructors, concepts, and experience.
- Student-Centric Answer Selection · EMNLP 2026
- Unveiling the Learning Mind of Language Models · NeurIPS 2025
LLM-based social simulation with population-aligned personas, and autonomous web agents that collect and verify evidence for academic talent discovery, with Microsoft Research Asia.
- Population-Aligned Persona Generation · NeurIPS 2026
Publications
- Advances in Neural Information Processing Systems 39, 2026. Forthcoming.
Builds representative persona sets by aligning LLM-generated profiles with real population-level psychometric distributions.
- Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, 2026. * Equal contribution.
Selects verified teacher answers by student-specific learning cost instead of assuming the strongest teacher always teaches best.
- Findings of the Association for Computational Linguistics: EMNLP 2025, 2025.
Explains length bias through desirability and information mass, then introduces a length-aligned evaluation protocol.
- Advances in Neural Information Processing Systems 38, 2025.
Introduces a cognitive framework for testing how language models learn from instructors, concepts, and experience.
Experience
Research
Allen Institute for AI (Ai2) · Research Collaborator
Microsoft Research Asia · Research Collaborator, TalentScope Copilot
LLM-driven autonomous web agents for evidence-grounded academic talent discovery.
Microsoft Research Asia · Research Collaborator, LLM Knowledge Inheritance
Studied how domain knowledge is retained and transferred during language-model fine-tuning.
HKUST(GZ) · Research Assistant
LLM preference evaluation, post-training data selection, and social simulation.
Industry
Resideo (Honeywell Home) · Software Engineer II
Built an end-to-end automated test system spanning GitHub Actions, a cloud camera platform, and MQTT-based device validation.
Fisher & Paykel Appliances · Intermediate Software Engineer
Led development of an automated testing platform integrating hardware, firmware, and higher-level software.
The University of Auckland · Teaching Assistant
Supported courses in advanced databases, object-oriented software development, functional programming, and distributed services.
Intellifusion · Machine Learning Intern
Developed and optimized face-recognition models for security and surveillance products.
Education
The University of Texas at Austin · M.S. in Computer Science
GPA: 4.0/4.0
The University of Auckland · B.E. (Hons.) in Software Engineering
First Class Honours · GPA: 7.029/9
Contact
Email is the best way to reach me for research and collaboration.