Some neural networks can be compiled straight into FPGA hardware. In these models each neuron is a small Boolean function stored in one of the chip's look-up tables (LUTs, the tiny programmable truth tables that make up an FPGA), so the trained network becomes a circuit with nanosecond latency. The practical problem is fit. A trained network often needs more LUTs than the target chip has, and today it is shrunk by a greedy, layer-by-layer procedure that clamps chosen gates to constant 0 or 1 so the hardware compiler deletes the dead logic. We propose to treat this shrinking step as one constrained optimization over the whole network: minimize the loss of accuracy subject to a gate budget, solved by a proximal method that keeps every intermediate network within budget.
The key ingredient is a proximal term, a quadratic penalty that keeps each step close to the previous solution. It does two jobs. First, the surviving gates may only recalibrate slightly, so the compiled circuit changes mainly where gates were clamped. Second, because the circuit changes only locally, the real post-synthesis LUT count measured between outer steps remains a valid correction to the cheap software gate count. In two sentences: we replace greedy layer-wise gate clamping with a global, budget-constrained proximal optimization whose outer loop is corrected by actual synthesis results. One run should then produce a full curve of models trading accuracy for area, each one guaranteed to fit its stated budget.
Core Claim
Two-Stage Unit Tying shows that clamping ("tying") gate outputs to constants in differentiable logic gate networks triggers compiler logic elimination. It reports up to 48.3% fewer post-synthesis LUTs on CIFAR-10 at a 50% tied ratio, and 0.35% accuracy loss on MNIST at that ratio. The same paper shows why ordinary pruning fails here: switching a Boolean function takes a weight change orders of magnitude larger than in a ResNet, so Taylor-style importance scores misjudge gates. Its own procedure screens gates with a Gauss-Newton logit-distortion score and then refines with binary splits, layer by layer. It never decides how to spread the budget across layers, and it never revises the surviving gates. Our claim is that a global constrained formulation, solved with the feasible-level proximal machinery of LCPP, will reach a given post-synthesis LUT count with less accuracy loss than greedy tying. We also claim the proximal anchor is what makes the software-to-hardware calibration work. Lee et al. report that soft gate count does not track real LUT usage linearly; the anchor is our answer to that mismatch.
This idea is not done in the file's literature. The file finds no work that uses proximal stabilization inside a synthesis pipeline. The closest work, Lee et al., is greedy and has no constrained formulation. BitLogic, which trains LUT networks end to end and exports Verilog, does no budget fitting at all.
Formulation
Let $\theta_0$ be the trained gate-choice logits. For each gate $i$ we introduce a keep variable $s_i \in [0,1]$; a gate with $s_i = 0$ outputs its majority constant on calibration data. Let $L(\theta, s)$ be the distillation loss of the modified network against the original network's outputs on a calibration set. At outer step $k$ we solve
$$\min_{\theta, s}\; L(\theta,s) + \frac{\rho}{2}\big\|(\theta,s)-(\theta_k,s_k)\big\|^2 \quad \text{s.t.}\quad \sum_i c_i\,\phi(s_i) \le \eta_k ,$$
Here $\phi$ is a nonconvex, sparsity-promoting surrogate for "gate is kept," such as capped-$\ell_1$, of the kind LCPP was built to handle. $c_i$ is a per-gate cost weight, $\rho$ is the proximal strength, and $\eta_k$ is the current budget level. LCPP supplies the method in three ways: it majorizes the nonconvex constraint into a convex one, it projects exactly in $O(d\log d)$ time, and it gives an $O(1/\epsilon)$ guarantee to reach stationary points satisfying the constraint. With the proximal term, each subproblem is strongly convex in a neighborhood of the current iterate.
The step specific to hardware happens between outer iterations. We round $s$, export Verilog through a BitLogic-style flow, run synthesis, and compare measured LUTs with the surrogate. We then rescale $\eta_{k+1}$ and refit the costs $c_i$ by regression on the gates tied so far. This captures the cascade effects that make gate count a poor predictor. A full synthesis run is expensive, so it happens only once per outer step, and the inner solves are left inexact, in the spirit of Catalyst's inexact proximal outer loop.
One theoretical caveat needs stating. LCPP's guarantee assumes a feasible starting point and fixed or increasing levels. Our budget decreases from the trained size toward the target, and the levels are adjusted by measurement. Exact projection keeps every iterate feasible with respect to the surrogate. Whether the convergence rate carries over to this shrinking, data-corrected schedule is an analysis question we will work out, not something we assume.
Research Program
We first reproduce Lee et al.'s setup: LogicTreeNet-style differentiable logic gate networks on MNIST and CIFAR-10, synthesized for Xilinx Virtex-7 2000T and Kintex-7 410T parts. We then add the BitLogic CIFAR-10 models, so the method is tested on a second network family and export flow. We build the constrained solver on the projected-gradient inner loop with LCPP projection, and add the synthesis-in-the-loop recalibration. The analysis has three parts. The first is how the global solver spreads tied gates across layers compared with greedy tying. The second is how often surviving gates flip their Boolean function, as a function of $\rho$. The third is how closely the corrected surrogate predicts measured LUTs over the run.
Baselines
The naive reference points are the untouched network and random tying at the same tied ratio. Together they bound what any careful selection should achieve.
The strongest existing method is Two-Stage Unit Tying from Lee et al., which reports a 48.3% post-synthesis LUT reduction at 50% tied ratio on CIFAR-10, 42.9% on MNIST, and 0.35% MNIST accuracy loss at 50% tying. It is the only method in the file that targets this exact problem, and it already beats Taylor pruning. We will run it with the same trained networks, the same calibration set, the same synthesis tool version, and the same number of synthesis calls. The last point matters because HLSFactory documents significant resource-report differences between Vitis HLS versions, so tool versions will be pinned. We also run a stronger variant: Lee et al. followed by distillation fine-tuning of the surviving gates.
The third baseline is a penalized sparsity method that replaces our hard budget with a tuned penalty weight. We give it the same tuning budget as our $\rho$ and level schedule.
Our ablations each remove one ingredient. Setting $\rho=0$ isolates the proximal anchor. Removing synthesis recalibration isolates the hardware feedback. Restricting the solver to one layer at a time isolates global allocation.
Beating the strongest baseline means two things. At matched measured LUT count, our method must show lower accuracy loss than Lee et al. plus fine-tuning, consistently across seeds on CIFAR-10 at 30%, 50%, and 70% LUT reduction. And every reported model must fit its stated budget.
Why not just…?
Why not just run Lee et al.'s tying and then fine-tune? This is the most direct competitor, and it is a real threat. If the accuracy loss mostly comes from failing to retrain surviving gates, rather than from poor budget allocation, then tying plus fine-tuning will match us. The layer-restricted ablation settles this. If our layer-restricted solver matches our global one, then allocation is not the gain, and the contribution shrinks to the calibration loop.
Why not just add a sparsity penalty and tune its weight? A penalty does not guarantee that any given iterate fits a target chip. Each new budget needs a new search over the penalty weight, and Lee et al. show that soft gate counts mislead about real LUTs. C-PG also shows that unregularized Lagrangian schemes oscillate and guarantee feasibility only on average over iterates. Even so, a well-tuned penalty may reach similar points. We compare the full accuracy-area curves under equal tuning compute.
Why not just train a smaller network from scratch at the target size? This is a serious alternative. BitLogic reports 72.3% on CIFAR-10 with under 0.3M gates, so small networks can be strong. However, BitLogic also reports that training deep logic networks is fragile, and every new budget would need a full training run. We include matched-LUT from-scratch training as a baseline. If it wins, we will report that tying-based simplification is not worth it at that scale.
Why not just use projected gradient without the proximal term? That is the $\rho=0$ ablation. Our prediction is that without the anchor, surviving gates flip their Boolean function more often. That would make each synthesis measurement stale and the calibrated surrogate inaccurate. We measure flip rate and surrogate-versus-measured LUT error directly. If neither degrades at $\rho=0$, the proximal term is not earning its place.
Decisive Checks
The first check is the matched-LUT accuracy comparison against Lee et al. plus fine-tuning on CIFAR-10. The second is whether a single decreasing-budget run produces models that synthesize under their stated budgets at every level. In particular, we test whether it yields a model that moves from the Virtex-7 2000T to the Kintex-7 410T with smaller accuracy loss than greedy tying, which Lee et al. already achieve. The third is whether the recalibrated surrogate's LUT prediction error falls over outer steps when $\rho>0$ but not when $\rho=0$. If the first check fails while the third passes, the useful contribution is the synthesis-calibrated surrogate on its own, and we would report it that way.
References
- Two-Stage Unit Tying for Simplifying Differentiable Logic Gate Networks
- A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization
- BitLogic: Training Framework for Gradient-Based FPGA-Native Neural Networks
- A Universal Catalyst for First-Order Optimization
- HLSFactory: A Framework Empowering High-Level Synthesis Datasets for Machine Learning and Beyond
- Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning