Research vision / Jingwei Ni
Alignment beyond the verifiable.
People instruct agents in natural language, which leaves a fuzzy and unverifiable part. I study how to make LLMs and agents trustworthy on that part.
Research vision
The fuzzy part is the hardest part.
People instruct agents and write rubrics in natural language, which leaves a fuzzy and unverifiable part whose ground truth is a distribution of human judgments rather than a point answer. I believe that getting this part right, so that a model matches the distribution of human judgments and knows how well it knows that distribution, is key to teaching AI taste, preventing reward hacking, and aligning AI beyond the verifiable.
My PhD work covers two foundations: judgment with fuzzy criteria, evaluating and improving how LLMs handle human disagreement and building human-AI collaborative annotation tools, and uncertainty quantification that reads a model’s beliefs about its own answers from its hidden states.
Fuzzy criteria · Disagreement
Evaluating and improving how LLMs handle fuzzy criteria.
Whatever a rubric leaves open, an LLM judge decides from its own prior. So I test models against distributions of human judgments rather than a single majority label. In Can Reasoning Help LLMs Capture Human Annotator Disagreement?, we find that LLMs collapse legitimate disagreement into one overconfident answer, and that RLVR-style reasoning training makes this worse.
DIRAS shows the constructive side: under nuanced natural-language relevance criteria, an LLM can learn calibrated judgments that distill into small models, so that calibrated judges run at deployment cost.
Fuzzy criteria · Tools
Human-AI collaborative annotation tools.
Judgment under fuzzy criteria needs people in the loop, and the tool decides where their judgment goes. Co-DETECT lets an expert refine a rough codebook: it uses annotation uncertainty to surface edge cases the codebook does not cover, induces generalizable rules for them, and leaves the expert in charge of what enters the codebook.
AFaCTA and DIRAS calibrate LLM annotators on a small human set and route items between people and models, so that validated labels scale into benchmarks. Our disagreement work adds a reason to take each person’s judgment seriously: disagreement is mostly systematic rather than noise.
Uncertainty quantification
Reading a model’s beliefs from its hidden states.
The spread of human judgments is irreducible. A model’s uncertainty about that spread is not, and it is what a clarification question can resolve. ReProbe reads a frozen LLM’s internal states to estimate the credibility of each reasoning step, enabling efficient verification and test-time search without a large process reward model. BELT reads a model’s beliefs about its current and future answer from the same hidden states and fits a small stopping policy on them, so one probe serves different accuracy-cost trade-offs and transfers from math to coding and agentic tasks.
Our disagreement work adds a caution: reasoning training degrades a model’s ability to capture how annotators disagree, so calibration has to be measured against distributions explicitly.