Daniel Fein

Stanford University
2 previous affiliations
    • Course Assistant · Since 2025
    • Master's degree, Computer Science · 2025–2026
    • MS student · 2024–2026
    • Undergrad student · 2020–2024
    • Associate Product Manager · 2024–2025
    • APM Intern · 2022–2023
4Machine learning papers tracked by Lacuna
Is this you?

Research Overview

Daniel Fein focuses on the alignment and evaluation of large language models, with particular emphasis on mitigating biases in reward models and developing domain-agnostic benchmarks for self-supervised learning. His recent work investigates the mechanics of reward hacking and bias substitution, exploring lightweight interventions to steer model behavior and improve the robustness of automated evaluation systems.

Research Trajectory

Early work in the evidence involved evaluating self-supervised learning (SSL) algorithms across diverse data modalities. While algorithms like BERT and SimCLR had driven significant progress in natural language processing and computer vision, they were typically domain-specific, requiring bespoke adaptations for new settings such as healthcare or scientific data. To address this limitation, he collaborated on the development of DABS: a Domain-Agnostic Benchmark for Self-Supervised Learning in 2021. This project introduced a comprehensive evaluation framework spanning seven distinct domains—ranging from natural images and English text to multichannel sensor data, speech recordings, and chest X-rays. The benchmark was designed to catalyze the development of truly domain-agnostic methods that could generalize across disparate types of data without requiring specialized architectural changes.

By 2026, his research focus had transitioned toward the alignment of large language models, specifically examining the vulnerabilities and biases inherent in the reward models used to evaluate and train AI systems. A major theme in this period is the difficulty of correcting spurious correlations without inadvertently introducing new failure modes. In Reward Bias Substitution: Single-Axis Bias Mitigations Redirect Optimization Pressure, he and his co-authors formalized a phenomenon where suppressing a known shortcut in a reward model—such as a preference for longer responses—merely redirects the optimization pressure. The model subsequently exploits different, correlated shortcuts, such as overconfidence or specific writing styles. This dynamic leaves the core alignment problem unresolved while masking the failure from standard evaluation benchmarks, complicating efforts to create reliable automated judges.

Building directly on this challenge, he led research into lightweight, internal interventions designed to correct these persistent issues at the architectural level. In One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Models, he introduced "mechanistic reward shaping," a technique that utilizes linear probes to surgically remove biases directly from a model's internal activations. By "nulling out" these undesirable features, the method bypasses the need for expensive and computationally intensive retraining. This work intersects with several broader research areas, including the direction of Steering Model Behavior with Contrastive Activation Vectors and the area of Overcorrection and Bias Inversion in Fairness Interventions. By addressing how reward models act as flawed judges prone to stylistic biases, this research also contributes to the broader area of Why Language Models Prefer Their Own Writing and advances techniques in the research direction Proxy Objective Exploitation in Language Model Alignment. Furthermore, the underlying methodologies hold implications for work in the area of Scaling Language Model Reasoning Beyond Verifiable Rewards, providing pathways to more reliable optimization targets that resist superficial exploitation.

Current Directions

The most recent papers in the evidence indicate a concentrated focus on the mechanics of reward modeling and the unintended consequences of alignment interventions. Current work investigates how optimization pressures in language models exploit correlated biases, and how these vulnerabilities can be mitigated through mechanistic interpretability techniques rather than traditional retraining pipelines. A central open problem in this agenda is ensuring that targeted bias mitigations do not inadvertently cause "reward bias substitution," where an AI system simply shifts from one undesirable heuristic to another. Moving forward, this research aims to develop more robust, verifiable reward signals that can reliably guide reinforcement learning without succumbing to reward hacking, overconfidence, or stylistic over-optimization.

Profile updated

See something wrong? Report a mistake