Export
Ngai Wong
· 1 previous affiliation
- University of Hong KongSince 2003
- Associate Professor · Since 2011
- Current
- Assistant Professor · 2003–2010
Research Overview
Ngai Wong focuses on tensor-based computational methods, electronic design automation, and hardware-efficient machine learning. His early work addressed nonlinear circuit modeling and model order reduction before expanding into high-dimensional tensor decompositions for complex systems. Recently, his research investigates ultra-low-parameter adaptation for large language models, compute-in-memory architectures, and the computational utility of nanoscale hardware noise.
Research Trajectory
Wong’s early research concentrated on the stability and simulation of complex electronic circuits. He developed methods for analyzing nonlinear dynamical behavior, such as identifying unstable embedded limit cycles in Fast detection of instability in sigma-delta modulators based on unstable embedded limit cycles. This focus on circuit dynamics naturally extended to the computational challenges of simulating large-scale systems. He introduced efficient techniques for solving continuous-time algebraic Riccati equations in Two Algorithms for Fast and Accurate Passivity-Preserving Model Order Reduction, addressing bottlenecks in passivity-preserving balanced stochastic truncation.
Recognizing that traditional matrix-based methods struggled with the curse of dimensionality in electronic design automation (EDA), Wong’s trajectory shifted toward tensor networks. He outlined this paradigm in Tensor Computation: A New Framework for High-Dimensional Problems in EDA, demonstrating how higher-order generalizations could manage the exploding parameter spaces of 3D field solvers and multirate circuit simulations. To operationalize this, he contributed to the broader area of Generalizing Matrix Operations to Higher Order Tensors with A Constructive Algorithm for Decomposing a Tensor into a Finite Sum of Orthonormal Rank-1 Terms, which provided a stable, interpretable decomposition analogous to the matrix singular value decomposition. This mathematical foundation enabled new approaches to circuit simulation, such as the TNMOR algorithm presented in Model Reduction and Simulation of Nonlinear Circuits via Tensor Decomposition, which captured high-order nonlinearities without relying on dense reduced-order system matrices.
Building on these decomposition techniques, Wong applied tensor networks to large-scale data approximation and machine learning. He developed an implementation of the randomized SVD algorithm in the matrix product operator format in Computing Low-Rank Approximations of Large-Scale Matrices with the Tensor Network Randomized SVD, achieving significant computational speedups. His work also explored how tensor structures could enhance predictive models. By replacing the rank-one tensor in a support tensor machine with a tensor train, he introduced A Support Tensor Train Machine to mitigate overfitting in high-dimensional spaces. This research aligns with his contributions to the research direction of Low-Rank Tensor Decompositions via Alternating Optimization. He further extended tensor trains to image and video inpainting in Fast and Accurate Tensor Completion With Total Variation Regularized Tensor Trains, reformulating the completion task as a regression problem to handle extreme data sparsity.
In recent years, Wong’s focus has broadened to encompass the intersection of neural networks, physical hardware, and model efficiency. He investigated the physical reliability of circuits using deep learning in A Space-Time Neural Network for Analysis of Stress Evolution Under DC Current Stressing. Concurrently, he leveraged his expertise in the area of Tensor Factorization for Parameter Efficient Model Adaptation to address the massive computational costs of modern AI. With collaborators, he developed LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models, a method that drastically reduces trainable parameters while maintaining performance in multi-task learning. Continuing this work on large language model efficiency, he contributed to ParallelComp: Parallel Long-Context Compressor for Length Extrapolation, a training-free framework that accelerates the prefill stage and enables significant length extrapolation through parallel chunk processing and key-value cache eviction. Pushing the limits of model compression, he also explored ultra-low-bit quantization in BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models, tackling the performance degradation that typically occurs when reducing models to two or three bits.
Alongside algorithmic efficiency, Wong has explored the physical substrate of AI computation and its security implications. To overcome the energy bottlenecks of analog-to-digital converters, he contributed to Memristor-based adaptive analog-to-digital conversion for efficient and accurate compute-in-memory, which utilizes programmable analog memory cells for distribution-aware quantization. Rather than treating hardware non-idealities purely as defects, his review Transforming memristor noises into computational innovations reframes inherent stochasticity as a resource for probabilistic AI and secure hardware. In the domain of model security, he has investigated the vulnerabilities of machine learning systems, demonstrating how adversarial perturbations can force arbitrary label predictions in Poisoning-based Backdoor Attacks for Arbitrary Target Label with Positive Triggers.
Current Directions
The most recent papers in the evidence indicate a strong focus on the extreme optimization and physical deployment of large-scale artificial intelligence. A major theme is the algorithmic compression of large language models, focusing on techniques that allow massive architectures to operate under severe memory and compute constraints. This includes developing ultra-low-bit quantization strategies that preserve reasoning capabilities and designing parallelized context compressors that manage key-value cache eviction for long-sequence extrapolation.
Concurrently, research in this portfolio explores the hardware-software co-design of AI accelerators, particularly leveraging memristor-based compute-in-memory systems. By adapting quantization to specific neural network data distributions and harnessing nanoscale hardware noise as a computational resource, this work aims to build highly efficient, noise-resilient, and secure physical substrates for next-generation machine learning.
Profile updated
Loading new papers…
Other researchers whose papers cluster into the same research directions — a proxy for overlapping technical interests and shared intellectual neighborhoods.
See something wrong? Report a mistake