Neural Value Iteration
Lists
Only you see this
Abstract
The value function of a POMDP exhibits the piecewise-linear-convex (PWLC) property and can be represented as a finite set of hyperplanes, known as $α$-vectors. Most state-of-the-art POMDP solvers (offline planners) follow the point-based value iteration scheme, which performs Bellman backups on $α$-vectors at reachable belief points until convergence. However, since each $α$-vector is $|S|$-dimensional, these methods quickly become intractable for large-scale problems due to the prohibitive computational cost of Bellman backups. In this work, we demonstrate that the PWLC property allows a POMDP's value function to be alternatively represented as a finite set of neural networks. This insight enables a novel POMDP planning algorithm called \emph{Neural Value Iteration}, which combines the generalization capability of neural networks with the classical value iteration framework. Our approach achieves near-optimal solutions even in extremely large POMDPs that are intractable for existing offline solvers.
BibTeX
@misc{you2025neural,
author = {Yang You and Ufuk Çakır and Alex Schutz and Nick Hawes},
title = {{Neural Value Iteration}},
year = {2025},
eprint = {2511.08825},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2511.08825},
}Loading related content...