In this blog post, we will provide a rough sketch of how the de la Peña inequality works. Some technical details are missing, but the architecture of the argument is morally correct. This was a really important result for my thesis, and I wrote this to better understand how the inequality works.

We will not provide a proof of this result, even in the simplified form, as this is a complex technical tool with a corresponding degree of detail needed to prove in full. We will attempt a proof sketch in the hopes of painting the broad strokes of the theory. The goal of this section is to illustrate the components of the decoupling inequality provided in Theorem 1.7.5, not to provide a rigorous proof on the level of [DlPG12].

First consider the following: let \(\boldsymbol{X}_i\) and \(\boldsymbol{X}_j\) be independent random variables, then set \(\boldsymbol{X}_i'\) and \(\boldsymbol{X}_j'\) be independent copies as well. Now consider, for i.i.d. symmetric Rademacher random variables

\[\epsilon_i = \begin{cases} 1 & \frac{1}{2}\\ -1 & \frac{1}{2}, \end{cases}\]

the following random variables

\[ \boldsymbol{Z}_i = \begin{cases} \boldsymbol{X}_i & \epsilon_i = 1\\ \boldsymbol{X}_i'&\epsilon_i = -1 \end{cases} \text{ and } \boldsymbol{Z}_i' = \begin{cases} \boldsymbol{X}_i' & \epsilon_i = 1\\ \boldsymbol{X}_i&\epsilon_i = -1. \end{cases} \]

For simplicity, we will consider \(h_{ij} = h\). The goal is, for \(S = \sum_{i\neq j} h(\boldsymbol{X}_i,\boldsymbol{X}_j)\) and \(S' = \sum_{i\neq j} h(\boldsymbol{X}_i',\boldsymbol{X}_j)\), to show an inequality of the form

\[ \mathbb{P}\left( \left\Vert S\right\Vert\geq t\right) \leq C \mathbb{P}\left( \left\Vert S'\right\Vert\geq \frac{t}{C}\right). \]

The proof proceeds by first showing that the decoupling allows for a bound on the expectation of convex functions. First, notice that \(\text{Law}(\boldsymbol{X}_1,...\boldsymbol{X}_n,\boldsymbol{X}_1',...,\boldsymbol{X}_n') = \text{Law}(\boldsymbol{Z}_1,...\boldsymbol{Z}_n,\boldsymbol{Z}_1',...\boldsymbol{Z}_n)\). As such,

\[ \text{Law}(\boldsymbol{Z}_1,...\boldsymbol{Z}_n) = \text{Law}(\boldsymbol{X}_1,...\boldsymbol{X}_n). \]

Letting \(\mathcal{Z} = \sigma(\boldsymbol{X}_i,\boldsymbol{X}_i': i=1,...,n)\) be the \(\sigma\)-algebra generated by the variables \(\boldsymbol{X}\), we can note that

\[\begin{align*} \mathbb{E}(h(\boldsymbol{Z}_i,\boldsymbol{Z}_j)\vert\mathcal{Z}) &= \mathbb{E}(h(\boldsymbol{Z}_i,\boldsymbol{Z}_j')\vert\mathcal{Z}) =\mathbb{E}(h(\boldsymbol{Z}_i',\boldsymbol{Z}_j)\vert\mathcal{Z}) =\mathbb{E}(h(\boldsymbol{Z}_i',\boldsymbol{Z}_j')\vert\mathcal{Z})\\ & = \frac{1}{4}\left(h(\boldsymbol{X}_i,\boldsymbol{X}_j) + h(\boldsymbol{X}_i,\boldsymbol{X}_j')+h(\boldsymbol{X}_i',\boldsymbol{X}_j)+h(\boldsymbol{X}_i',\boldsymbol{X}_j')\right). \end{align*}\]

As such, we can see that for a kernel \(h\), we have that for any non-negative convex function \(\Phi\) that, for \(\mathfrak{X} = \sigma(\boldsymbol{X}_i\mid i=1,...,n) \),

\[\begin{align*} &\mathbb{E} \Phi \Big( \big\| \sum_{i \neq j} h(\mathbf{X}_i, \mathbf{X}_j) \big\| \Big) \\ &\quad \le \frac{1}{2} \mathbb{E} \Phi \Big( 2 \big\| \sum_{i \neq j} \mathbb{E} [ h(\mathbf{X}_i, \mathbf{X}_j) + h(\mathbf{X}_i', \mathbf{X}_j) + h(\mathbf{X}_i, \mathbf{X}_j') + h(\mathbf{X}_i', \mathbf{X}_j') | \mathfrak{X} ] \big\| \Big) \\ &\quad\quad + \frac{1}{2} \mathbb{E} \Phi \Big( 2 \big\| \sum_{i \neq j} \mathbb{E} [ h(\mathbf{X}_i, \mathbf{X}_j') + h(\mathbf{X}_i', \mathbf{X}_j) + h(\mathbf{X}_i', \mathbf{X}_j') | \mathfrak{X} ] \big\| \Big) \\ &\quad \le \frac{1}{2} \mathbb{E} \Phi \Big( 2 \big\| \sum_{i \neq j} [ h(\mathbf{X}_i, \mathbf{X}_j) + h(\mathbf{X}_i', \mathbf{X}_j) + h(\mathbf{X}_i, \mathbf{X}_j') + h(\mathbf{X}_i', \mathbf{X}_j') ] \big\| \Big) \\ &\quad\quad + \frac{1}{6} \mathbb{E} \Phi \Big( 6 \big\| \sum_{i \neq j} \mathbb{E} (h(\mathbf{X}_i', \mathbf{X}_j) | \mathfrak{X}) \big\| \Big) + \frac{1}{6} \mathbb{E} \Phi \Big( 6 \big\| \sum_{i \neq j} \mathbb{E} (h(\mathbf{X}_i, \mathbf{X}_j') | \mathfrak{X}) \big\| \Big) \\ &\quad\quad + \frac{1}{6} \mathbb{E} \Phi \Big( 6 \big\| \sum_{i \neq j} \mathbb{E} (h(\mathbf{X}_i', \mathbf{X}_j') | \mathfrak{X}) \big\| \Big) \\ &\quad \le \frac{1}{2} \mathbb{E} \Phi \Big( 8 \big\| \sum_{i \neq j} \mathbb{E} (h(\mathbf{Z}_i, \mathbf{Z}_j') | \mathcal{Z}) \big\| \Big) + \frac{1}{3} \mathbb{E} \Phi \Big( 6 \big\| \sum_{i \neq j} h(\mathbf{X}_i, \mathbf{X}_j') \big\| \Big) \\ &\quad\quad + \frac{1}{6} \Phi \Big( 6 \big\| \sum_{i \neq j} \mathbb{E} h(\mathbf{X}_i', \mathbf{X}_j') \big\| \Big) \\ &\quad \le \frac{1}{2} \mathbb{E} \Phi \Big( 8 \big\| \sum_{i \neq j} h(\mathbf{Z}_i, \mathbf{Z}_j') \big\| \Big) + \frac{1}{3} \mathbb{E} \Phi \Big( 6 \big\| \sum_{i \neq j} h(\mathbf{X}_i, \mathbf{X}_j') \big\| \Big) \\ &\quad\quad + \frac{1}{6} \Phi \Big( 6 \big\| \sum_{i \neq j} \mathbb{E} h(\mathbf{X}_i', \mathbf{X}_j') \big\| \Big) \\ &\quad = \frac{1}{2} \mathbb{E} \Phi \Big( 8 \big\| \sum_{i \neq j} h(\mathbf{X}_i, \mathbf{X}_j') \big\| \Big) + \frac{1}{3} \mathbb{E} \Phi \Big( 6 \big\| \sum_{i \neq j} h(\mathbf{X}_i, \mathbf{X}_j') \big\| \Big) \\ &\quad\quad + \frac{1}{6} \Phi \Big( 6 \big\| \sum_{i \neq j} \mathbb{E} h(\mathbf{X}_i, \mathbf{X}_j') \big\| \Big) \\ &\quad \le \mathbb{E} \Phi \Big( 8 \big\| \sum_{i \neq j} h(\mathbf{X}_i, \mathbf{X}_j') \big\| \Big) \tag{3.1.8} \end{align*}\]

where the first inequality follows from the triangle inequality and convexity, the second inequality follows similarly, the third inequality follows from the identity of \(\mathbb{E}(h(\boldsymbol{Z}_i,\boldsymbol{Z}_j)\mid\mathcal{Z})\) and properties of conditional expectation, the next inequality follows from Jensen's and distributional equivalence of \(\boldsymbol{X}_i\), \(\boldsymbol{X}_i'\), \(\boldsymbol{Z}_i\), and \(\boldsymbol{Z}_i'\), and the last inequality follows from Jensen's. Now, this is just a proof on the expectation bound, and all of the results stated in Section 3.2 are stated in terms of tail bounds. To bridge this gap, we will need moment inequalities, and specifically moment inequalities that are generated by \(\epsilon_i\). Rademacher random variables satisfy a Khintchine inequality, which establishes the equivalence of \(L^p\) and \(L^q\) norms for polynomials of Rademacher variables. For elements \(\boldsymbol{x}_i\) in a normed linear space, this means

\[ \left(\mathbb{E}\left\Vert \sum_i \boldsymbol{x}_i\epsilon_i\right\Vert^q\right)^{1/q}\leq C_{q,p}\left(\mathbb{E}\left\Vert \sum_i \boldsymbol{x}_i\epsilon_i\right\Vert^p\right)^{1/p}, \quad 0<p<q<\infty, \]

where \(C_{q,p}\) is a constant depending only on the values \(p\) and \(q\). In words, this inequality states that one can always upper bound higher-order moments of a Rademacher chaos by lower-order moments, indicating a high degree of concentration. This property can actually be strengthened further via hypercontractivity to hold for multiple sums (chaoses of degree \(d \ge 2\)), and is of a similar flavor as the Khintchine inequality, if more technical. The critical idea is that of higher-order moment control: the strict equivalence of higher-order moments with lower-order moments forms the bedrock for bridging expectation bounds and tail probabilities. Without this higher-order moment control providing sharp concentration of the Rademacher chaos, higher-order moments might diverge and not allow for the derivation of sharp tail inequalities.

The last missing piece is that of a Paley-Zygmund inequality. This allows for the development of tail bounds, which then allows the proof of statements like Theorem 1.7.5 and more general statements than the one provided. The Paley-Zygmund inequality provides a lower bound on the probability that a positive random variable exceeds a certain fraction of its expectation. For a non-negative random variable \(Z\) with finite variance and any \(\theta \in (0, 1)\), the inequality states:

\[ \mathbb{P}(Z \geq \theta \mathbb{E}[Z]) \geq (1-\theta)^2 \frac{(\mathbb{E}[Z])^2}{\mathbb{E}[Z^2]}. \]

In the context of bounding higher-order moments, we can generalize this to state the following inequality: let \(\xi\) be a non-negative random variable with bounded \(q\)-th moment for some \(q>0\). Then for any \(t\geq 0\) and \(0<p<q\), we have that

\[ \mathbb{P}(\xi>t) \geq \left[ \frac{\min\{\mathbb{E}\xi^p - t^p,0\}}{\left(\mathbb{E}\xi^q\right)^{p/q}}\right]^{\frac{q-p}{p}}. \]

In the context of decoupling, we apply this generalized inequality to establish a bridge between moments and tail probabilities. The complete logic forms a chain of inequalities: first, Markov's inequality bounds the tail of the coupled sum \(S\) from above using its moments. Second, the expectation inequality derived earlier (via Jensen's inequality) bounds the moments of \(S\) from above using the moments of the decoupled sum \(S'\). Finally, we must bound the moments of \(S'\) from above using its own tail probability.

This is where the generalized Paley-Zygmund inequality acts as the final link. By applying it to \(\xi = \Vert S' \Vert\), we obtain a lower bound on the tail probability in terms of its moments. Because hypercontractivity guarantees that the higher moments \(\mathbb{E}[\xi^q]\) are bounded by a constant multiple of the lower moments \(\left(\mathbb{E}[\xi^p]\right)^{q/p}\), the ratio in this lower bound is strictly bounded away from zero by an absolute constant. As such, the tail probability \(\mathbb{P}(\Vert S' \Vert \geq t/C)\) effectively acts as an upper bound on the moments of \(S'\). Taken together, Markov's inequality, hypercontractivity, and the generalized Paley-Zygmund inequality sandwich the probabilities, ensuring that the tail of \(S\) is tightly bounded by the tail of \(S'\) up to universal constants, yielding Theorem 1.7.5. This is a rough sketch of the argument provided in [DlPG12], so interested readers should see Chapter 3 of this book for a very detailed treatment in full generality.