Correction to “Nonparametric regression using deep neural networks with ReLU activation function”

Johannes Schmidt-Hieber, Don VuThe Annals of Statistics 2024 · DOI

Abstract

Correction: Condition (ii) of Theorem 1 in [1] should be changed to (Equation presented). Moreover, the constants C, C<sup>'</sup> in Theorem 1 also depend on the implicit constants that appear in conditions (ii)-(iv). There are large regimes where the new condition (ii') is weaker than (ii). Explanation: Rather than choosing m and N in the proof of Theorem 1 globally, one should instead apply Theorem 5 individually to each i with (Equation presented) and (Equation presented), where 0 &lt; c ≤ 1/2 is a sufficiently small constant. As mentioned at the beginning of the proof of Theorem 1, it is sufficient to prove the result for sufficiently large n. Therefore, we can assume that m<sub>i</sub> ≥ 1 for all i = 0,..., q and (Equation presented). The latter implies (Equation presented). If we now define L'<sub>i</sub> := 8 + (m<sub>i</sub> + 5)(1 + ⌈log<sub>2</sub>(t<sub>i</sub> ∨ β<sub>i</sub>)⌉), then there exists a network (Equation presented)<sub>ij</sub> ∈ F(L'<sub>i</sub>, (t<sub>i</sub>, 6(t<sub>i</sub> + ⌈β<sub>i</sub>⌉)N<sub>i</sub>,..., 6(t<sub>i</sub> + ⌈β<sub>i</sub>⌉)N<sub>i</sub>, 1), s<sub>i</sub>) with s<sub>i</sub> ≤ 141(t<sub>i</sub> + β<sub>i</sub> + 1)<sup>3+t<sup>i</sup></sup>N<sub>i</sub>(m<sub>i</sub> + 6), such that (Equation presented), where Q<sub>i</sub> is any upper bound of the Hölder norms of h<sub>ij</sub>, j = 1,..., d<sub>i</sub>+1. We can now argue as in the original proof to show that the composite network f<sup>∗</sup> is in the class F(E, (d, 6r<sub>i</sub> max<sub>i</sub> N<sub>i</sub>,..., 6r<sub>i</sub> max<sub>i</sub> N<sub>i</sub>, 1), (Equation presented)), with (Equation presented). Using the definition of L'<sub>i</sub> above, it can be shown as in the original proof that (Equation presented) (log<sub>2</sub>(4) + log<sub>2</sub>(t<sub>i</sub> ∨ β<sub>i</sub>))log<sub>2</sub>(n) for all sufficiently large n. All remaining steps are the same as in the original proof of Theorem 1. The constant c in the definition of N<sub>i</sub> will also depend on the implicit constant in the conditions L ≾ nφ<sub>n</sub>, nφ<sub>n</sub> ≾ min<sub>i=1,..., L</sub> p<sub>i</sub> and s nφ<sub>n</sub> log n. Further comments: - Lemma 1 requires that the constant K is large enough such that Theorem 3 is applicable. - First display on page 1886: The value t<sub>2</sub> is N not Nd. - Equation (18) also requires that the inputs are nonnegative. - In Lemma 3, the L<sup>∞</sup>-norms should be replaced by the supremum, that is, f <sub>L</sub>∞<sub>(A)</sub> should be changed to sup<sub>x∈A</sub> |f (x)|. - In the proof of Theorem 1, r<sub>i</sub> does not depend on i and should be named r. Three lines after equation (26), C should be replaced by C<sup>'</sup>. - In the proof of Theorem 3, β<sup>∗</sup> in the first line on page 1893 should be β<sup>∗∗</sup>. It is sufficient to check that the Hölder constant of φ<sub>w</sub> is bounded by (β<sup>∗</sup> + 1)<sup>t∗</sup> (t<sup>∗</sup> + 1) as all later arguments of the proof carry over. Moreover, g<sub>i</sub>(x) = (x<sub>1</sub>,..., x<sub>di</sub> ) should be g<sub>i</sub>(x) = (x<sub>1</sub>,..., x<sub>di</sub>+<sub>1</sub> ) if d<sub>i</sub> ≥ d<sub>i</sub>+<sub>1</sub> and g<sub>i</sub>(x) = (x<sub>1</sub>,..., x<sub>di</sub>, 0,..., 0) if d<sub>i</sub> &lt; d<sub>i</sub>+1. Finally, ψ<sub>u</sub><sup>2</sup><sub>2</sub> should be replaced by ψ<sub>u</sub><sup>B2</sup><sub>2</sub>. - In the proof of Lemma 2, one can simply take the constant function h<sub>j, α</sub> = K if μ<sub>0</sub>/= 0. This immediately gives d<sub>j, k</sub> = Kμ<sup>d</sup><sub>0</sub> 2<sup>−jd/2</sup> for all wavelet coefficients d<sub>j, k</sub>. For the case μ<sub>0</sub> = 0, one should replace the binomial coefficient (Equation presented) by the multinomial coefficient (Equation presented) = (dr)!/(r!)<sup>d</sup>. - To verify the last inequality in (B.8) of the Supplementary Material, one can replace ≤ 1/M by &lt; 1/M. - In the proof of Lemma 4, some F are missing. In particular, it should be (Equation presented). Also, on page 12 of the Supplementary Material it should be P (T ≥ t) ≤ 1 ∧ 2N<sub>n</sub> max<sub>j</sub> (Equation presented). Since r<sub>j</sub> ≥ F n<sup>−1</sup> log N<sub>n</sub>, we can argue as before and obtain P (T ≥ t) ≤ 2N<sub>n</sub> exp(−3t√log N<sub>n</sub>/(16√n)) for all t ≥ 6√n log N<sub>n</sub>. The conclusion of (I) is still valid. - In the proof of Lemma 5, we always work with the | · |<sub>∞</sub>-norm for vectors. The grid size of an individual parameter should be taken as δ/((L + 1)V).
BibTeX
@article{schmidthieber2024correction,
  author = {Johannes Schmidt-Hieber and Don Vu},
  title = {{Correction to “Nonparametric regression using deep neural networks with ReLU activation function”}},
  journal = {The Annals of Statistics},
  year = {2024},
  doi = {10.1214/24-aos2351},
}

Loading related content...

Figure