Robert D Nowak

Robert D Nowak

Rice University · Toyota Technological Institute at Chicago · University of Wisconsin–Madison
3 previous affiliations
185Machine learning papers tracked by Lacuna
Is this you?

Research Overview

Robert D Nowak focuses on the mathematical foundations of deep learning, algorithms for data-efficient machine learning, and signal processing. His trajectory spans early work in wavelet-based noise removal and compressed sensing to foundational contributions in multi-armed bandits and active learning. Recently, his research investigates the theoretical properties of neural networks and the intersection of artificial intelligence with human cognition.

Research Trajectory

The early arc of Nowak's research focused heavily on signal processing, inverse problems, and statistical learning theory. In 1999, he explored techniques for medical imaging in Wavelet-based Rician noise removal for magnetic resonance imaging. Moving into classification and learning theory, he investigated Minimax-optimal classification with dyadic decision trees (2006). As the field of compressed sensing emerged, he contributed to optimization methods for sparse recovery, detailing these approaches in Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems (2007). This foundational period also saw him establish theoretical limits for data-efficient learning paradigms, as seen in Minimax Bounds for Active Learning (2008).

In the following years, his focus expanded to adaptive sampling strategies and multi-armed bandit problems. He introduced a novel adaptive sampling strategy in Distilled Sensing: Adaptive Sampling for Sparse Detection and Estimation (2011), which intelligently allocates measurement resources to detect weak sparse signals, contributing to the broader area of Adaptive Resource Allocation for Sparse Signal Recovery. His work in sequential decision-making includes participation in developing lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits (2014), an upper confidence bound procedure for identifying the optimal arm in fixed confidence settings. This advanced the research direction of Anytime Valid Inference with the Iterated Logarithm. Concurrently, he addressed matrix completion and subspace identification in Deterministic Conditions for Subspace Identifiability from Incomplete Sampling (2014), providing necessary and sufficient conditions for identifying subspaces from incomplete observations and shaping the area of Deterministic Conditions for Subspace Identifiability.

More recently, Nowak's trajectory has heavily emphasized the mathematical foundations of deep learning and the practical challenges of active learning in complex environments. He explored the theoretical properties of network architectures in Banach Space Representer Theorems for Neural Networks and Ridge Splines (2021), aligning with the research direction of Neural Networks as Variational Splines. In the domain of data-efficient learning, he and his collaborators developed frameworks to handle domain shifts and novel classes with minimal human labeling. This includes AHA: Human-Assisted Out-of-Distribution Generalization and Detection (2024), which targets "maximum disambiguation regions" to improve generalization. Continuing this thread, Deep Active Learning in the Open World (2025) introduced the ALOE algorithm for discovering new classes in the wild using diversity-based clustering and gradient-based out-of-distribution detection. He also addressed extreme class imbalance and noisy labels in Improved Algorithm for Deep Active Learning under Imbalance via Optimal Separation (2025), presenting the DIRECT algorithm to reduce annotation costs by framing the task as a robust one-dimensional separation problem.

His latest work continues to push the boundaries of bandit learning and theoretical optimization. In Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning (2025), he contributed to methods that enable learning near-optimal policies on unseen tasks without privileged information. He also investigated Multi-task Representation Learning for Fixed Budget Pure-Exploration in Linear and Bilinear Bandits (2025), leveraging shared low-dimensional structures to expedite learning and identify optimal arms within a pre-specified budget, contributing to areas such as Scaling Contextual Bandits via Hidden Low Dimensional Structure and Fixed Budget Best Arm Identification. Concluding the recent trajectory, his theoretical optimization work includes Global Minimizers of ℓp-Regularized Objectives Yield the Sparsest ReLU Neural Networks (2025), which proves that minimizing the path norm finds the sparsest network fitting a dataset, and Compositional Function Spaces for Deep Learning (2026), further formalizing the mathematical properties of deep learning architectures.

Current Directions

The most recent papers in the evidence indicate a dual focus on the theoretical underpinnings of deep neural networks and the development of highly data-efficient machine learning algorithms. A major ongoing theme is active learning in complex, open-world environments, particularly addressing challenges like out-of-distribution detection, extreme class imbalance, and noisy labels to minimize human annotation costs. Additionally, current research explores multi-task structured bandit problems, leveraging representation learning and decision transformers to enable effective in-context learning on unseen tasks. Theoretical efforts continue to formalize the mathematical properties of neural networks, including the study of compositional function spaces and the optimization of network sparsity through rigorous mathematical frameworks.

Profile updated

See something wrong? Report a mistake