Hang Le

Harvard University
3 previous affiliations
4Machine learning papers tracked by Lacuna
Is this you?

Research Overview

Hang Le focuses on the representational geometry of artificial neural networks, specifically investigating how deep learning models organize information into structured manifolds. This work operates at the intersection of machine learning and computational neuroscience, utilizing geometric techniques to analyze how networks learn task-relevant features, achieve perceptual robustness, and mirror biological sensory processing. Recent research focuses on moving beyond traditional theoretical frameworks of neural network training by using manifold capacity and untangling metrics to characterize the transition from random initializations to structured feature representations.

Research Trajectory

The early stage of Le’s research, beginning around 2020, centered on the geometric properties of representations within deep language models and perceptual systems. In Emergence of Separable Manifolds in Deep Language Representations, Le contributed to an analysis of how deep neural networks (DNNs) solve perceptual tasks by organizing data into separable manifolds. This work drew direct parallels between the "untangling" of manifolds in artificial models and the biological processes observed in the brain, suggesting that while DNNs are only loosely inspired by biology, they converge on similar geometric solutions for categorization and representation.

Following this, Le participated in research exploring the relationship between internal geometry and model robustness. In Neural Population Geometry Reveals the Role of Stochasticity in Robust Perception, the investigation turned to adversarial vulnerability—a key area where machine learning models diverge from biological systems. By applying geometric techniques from computational neuroscience, the study examined how response stochasticity, a characteristic of biological neurons, influences internal representations. The findings suggested that biologically-inspired stochasticity can improve adversarial robustness by altering the geometry of neural population responses, providing a bridge between neuroscientific observations and machine learning security.

By 2025, Le’s focus shifted toward the fundamental dynamics of feature learning, specifically challenging the "lazy versus rich" dichotomy that has dominated recent theoretical machine learning. This later work moves away from analyzing static representations or weight matrices toward a dynamic understanding of how task-relevant information is integrated during training. By employing "manifold capacity" as a metric, Le has contributed to a more nuanced view of how networks develop features that allow for generalization, identifying distinct learning stages that traditional linear or kernel-based models fail to capture.

Current Directions

Le’s current research is primarily concerned with the mechanics of feature learning through the lens of task-relevant manifold untangling. This direction seeks to provide a representation-based alternative to the Neural Tangent Kernel (NTK) and weight-matrix analyses, which often overlook the geometric evolution of the data itself during training.

A significant component of this work involves defining the boundaries between different training regimes. While the "lazy" regime (where networks behave like linear models of initial features) and the "rich" regime (where features are actively learned) are well-documented, Le’s recent contributions suggest these categories are insufficient. By analyzing the representational geometry of data clusters, this research identifies how networks "untangle" manifolds to isolate task-relevant information. This approach is particularly relevant for neuroscience, as it offers a method to study feature learning that is grounded in the population-level representations observable in both artificial and biological systems.

Furthermore, this research explores why certain learning strategies lead to better generalization than others. By measuring how well a network untangles data manifolds at different stages of training, Le aims to explain the failure modes of deep networks and the geometric conditions necessary for robust, task-relevant feature extraction.

Representative Work

In Emergence of Separable Manifolds in Deep Language Representations, Le and collaborators investigated the geometric structure of representations in deep language models. The study utilized manifold capacity—a measure of how many objects can be linearly separated given a specific representation—to show that as data passes through the layers of a deep network, the underlying manifolds become increasingly untangled and separable. This provided empirical evidence for the "manifold untangling" hypothesis in the context of natural language processing, aligning artificial model behavior with theories of biological sensory hierarchies.

Le participated in the research presented in Neural Population Geometry Reveals the Role of Stochasticity in Robust Perception, which addressed the divergence between human perception and machine vulnerability to adversarial attacks. The work demonstrated that adding stochasticity to neural responses—mimicking the variability found in biological brains—can lead to more robust internal representations. The study used geometric analysis to visualize how adversarial perturbations affect these representations, showing that stochasticity helps maintain the integrity of the data manifolds under attack.

In Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry, Le contributed to a framework that moves beyond the simplistic binary of lazy versus rich training. The paper introduces representational geometry as a tool to identify distinct learning strategies that networks employ. By analyzing how "manifold capacity" evolves, the authors demonstrate that networks can exhibit complex feature-learning behaviors that are not captured by existing theoretical models, providing a more granular understanding of how task-relevant features emerge during the training process.

The research in Beyond the Lazy versus Rich Dichotomy: Geometry Insights in Feature Learning from Task-Relevant Manifold Untangling further develops these geometric insights. Le and the research team argue that focusing on weight matrices or kernels limits the applicability of machine learning theory to neuroscience. Instead, they propose a representation-based method to study feature learning. The paper details how the ability of a network to integrate task-relevant information is directly tied to the geometric untangling of data manifolds, offering a new perspective on the transition from random features to structured intelligence.

Profile updated

See something wrong? Report a mistake