Cauchy–Schwarz inequality

In mathematics, the Cauchy–Schwarz inequality (also known as the Bunyakovsky inequality, the Schwarz inequality, or the Cauchy–Bunyakovsky–Schwarz inequality), is a useful inequality encountered in many different settings, such as linear algebra, analysis, probability theory, and other areas. It is considered to be one of the most important inequalities in all of mathematics.[1] It has a number of generalizations, among them Hölder's inequality.

The inequality for sums was published by Augustin-Louis Cauchy (1821), while the corresponding inequality for integrals was first stated by Viktor Bunyakovsky (1859) and rediscovered by Hermann Amandus Schwarz (1888).

Statement of the inequality

The Cauchy–Schwarz inequality states that for all vectors x and y of an inner product space it is true that

\( |\langle x,y\rangle| ^2 \leq \langle x,x\rangle \cdot \langle y,y\rangle, \)

where \langle\cdot,\cdot\rangle is the inner product. Equivalently, by taking the square root of both sides, and referring to the norms of the vectors, the inequality is written as

\( |\langle x,y\rangle| \leq \|x\| \cdot \|y\|.\, \)

Moreover, the two sides are equal if and only if x and y are linearly dependent (or, in a geometrical sense, they are parallel or one of the vectors is equal to zero).

If x_1,\ldots, x_n\in\mathbb C and y_1,\ldots, y_n\in\mathbb C are any complex numbers and the inner product is the standard inner product then the inequality may be restated in a more explicit way as follows:

\( |x_1 \bar{y}_1 + \cdots + x_n \bar{y}_n|^2 \leq (|x_1|^2 + \cdots + |x_n|^2) (|y_1|^2 + \cdots + |y_n|^2). \)

When viewed in this way the numbers x1, ..., xn, and y1, ..., yn are the components of x and y with respect to an orthonormal basis of V.

Even more compactly written:

\( \left|\sum_{i=1}^n x_i \bar{y}_i\right|^2 \leq \sum_{j=1}^n |x_j|^2 \sum_{k=1}^n |y_k|^2 . \)

Equality holds if and only if x and y are linearly dependent, that is, one is a scalar multiple of the other (which includes the case when one or both are zero).

The finite-dimensional case of this inequality for real vectors was proved by Cauchy in 1821, and in 1859 Cauchy's student Bunyakovsky noted that by taking limits one can obtain an integral form of Cauchy's inequality. The general result for an inner product space was obtained by Schwarz in the year 1885.
Proof

Let u, v be arbitrary vectors in a vector space V over F with an inner product, where F is the field of real or complex numbers. We prove the inequality

\( \big| \langle u,v \rangle \big| \leq \left\|u\right\| \left\|v\right\|. \, \)

This inequality is trivial in the case v = 0, so we may assume from here on that v is nonzero. Let

\( z= u-\frac {\langle u, v \rangle} {\langle v, v \rangle} v. \)

Then

\( \langle z, v \rangle = \left\langle u -\frac {\langle u, v \rangle} {\langle v, v \rangle} v, v\right\rangle = \langle u, v \rangle - \frac {\langle u, v \rangle} {\langle v, v \rangle} \langle v, v \rangle = 0, \)

i.e., z is a vector orthogonal to the vector v (Indeed, z is the projection of u onto the plane orthogonal to v.) We can thus apply the Pythagorean theorem to

\( u= \frac {\langle u, v \rangle} {\langle v, v \rangle} v+z, \)

which gives

\( \left\|u\right\|^2 = \frac{|\langle u, v \rangle|^2}{|\langle v, v \rangle|^2} \left\|v\right\|^2 + \left\|z\right\|^2 = \frac{|\langle u, v \rangle|^2}{\left\|v\right\|^2} + \left\|z\right\|^2 \geq \frac{|\langle u, v \rangle|^2}{\left\|v\right\|^2}, \)

with equality if and only if z = 0 (i.e., in the case where u is a multiple of v). This establishes the theorem.
Notable special cases
Rn

In Euclidean space Rn with the standard inner product, the Cauchy–Schwarz inequality is

\( \left(\sum_{i=1}^n x_i y_i\right)^2\leq \left(\sum_{i=1}^n x_i^2\right) \left(\sum_{i=1}^n y_i^2\right). \)

To prove this form of the inequality, consider the following quadratic polynomial in z.

\( (x_1 z + y_1)^2 + \cdots + (x_n z + y_n)^2. \)

Since it is nonnegative it has at most one real root in z, whence its discriminant is less than or equal to zero, that is,

\( \left(\sum ( x_i \cdot y_i ) \right)^2 - \sum {x_i^2} \cdot \sum {y_i^2} \le 0, \)

which yields the Cauchy–Schwarz inequality.

An equivalent proof for Rn starts with the summation below.

Expanding the brackets we have:

\( \sum_{i=1}^n \sum_{j=1}^n \left( x_i y_j - x_j y_i \right)^2 = \sum_{i=1}^n x_i^2 \sum_{j=1}^n y_j^2 + \sum_{j=1}^n x_j^2 \sum_{i=1}^n y_i^2 - 2 \sum_{i=1}^n x_i y_i \sum_{j=1}^n x_j y_j , \)

collecting together identical terms (albeit with different summation indices) we find:

\( \frac{1}{2} \sum_{i=1}^n \sum_{j=1}^n \left( x_i y_j - x_j y_i \right)^2 = \sum_{i=1}^n x_i^2 \sum_{i=1}^n y_i^2 - \left( \sum_{i=1}^n x_i y_i \right)^2 . \)

Because the left-hand side of the equation is a sum of the squares of real numbers it is greater than or equal to zero, thus:

\( \sum_{i=1}^n x_i^2 \sum_{i=1}^n y_i^2 - \left( \sum_{i=1}^n x_i y_i \right)^2 \geq 0. \)

This form is used usually when solving school math problems.

Yet another approach when n ≥ 2 (n = 1 is trivial) is to consider the plane containing x and y. More precisely, recoordinatize Rn with any orthonormal basis whose first two vectors span a subspace containing x and y. In this basis only x_1,~x_2,~y_1 and y_2~ are nonzero, and the inequality reduces to the algebra of dot product in the plane, which is related to the angle between two vectors, from which we obtain the inequality:

\( |x \cdot y| = \|x\| \|y\| | \cos \theta | \le \|x\| \|y\|. \)

When n = 3 the Cauchy–Schwarz inequality can also be deduced from Lagrange's identity, which takes the form

\( \langle x,x\rangle \cdot \langle y,y\rangle = |\langle x,y\rangle|^2 + |x \times y|^2 \)

from which readily follows the Cauchy–Schwarz inequality.
L2

For the inner product space of square-integrable complex-valued functions, one has

\( \left|\int f(x) \overline{g(x)}\,dx\right|^2\leq\int \left|f(x)\right|^2\,dx \cdot \int\left|g(x)\right|^2\,dx. \)

A generalization of this is the Hölder inequality.
Use

The triangle inequality for the inner product is often shown as a consequence of the Cauchy–Schwarz inequality, as follows: given vectors x and y:

\( \begin{align} \|x + y\|^2 & = \langle x + y, x + y \rangle \\ & = \|x\|^2 + \langle x, y \rangle + \langle y, x \rangle + \|y\|^2 \\ & = \|x\|^2 + 2 \text{ Re} \langle x, y \rangle + \|y\|^2\\ & \le \|x\|^2 + 2|\langle x, y \rangle| + \|y\|^2 \\ & \le \|x\|^2 + 2\|x\|\|y\| + \|y\|^2 \\ & = \left (\|x\| + \|y\|\right)^2. \end{align} \)

Taking square roots gives the triangle inequality.

The Cauchy–Schwarz inequality allows one to extend the notion of "angle between two vectors" to any real inner product space, by defining:

\( \cos\theta_{xy}=\frac{\langle x,y\rangle}{\|x\| \|y\|}. \)

The Cauchy–Schwarz inequality proves that this definition is sensible, by showing that the right hand side lies in the interval [−1, 1], and justifies the notion that (real) Hilbert spaces are simply generalizations of the Euclidean space.

It can also be used to define an angle in complex inner product spaces, by taking the absolute value of the right hand side, as is done when extracting a metric from quantum fidelity.

The Cauchy–Schwarz is used to prove that the inner product is a continuous function with respect to the topology induced by the inner product itself.

The Cauchy–Schwarz inequality is usually used to show Bessel's inequality.
Probability theory

For the multivariate case, \( \text{Var}\left(Y\right)\ge\text{Cov}\left(Y,X\right)\text{Var}^{-1}\left(X\right)\text{Cov}\left(X^{T},Y^{T}\right) . \)

For the univariate case, \( \text{Var}\left(Y\right)\ge\frac{\text{Cov}\left(Y,X\right)\text{Cov}\left(Y,X\right)}{\text{Var}\left(X\right)} \). Indeed, for random variables X and Y, the expectation of their product is an inner product. That is,

\( \langle X, Y \rangle \triangleq \operatorname{E}(X Y), \)

and so, by the Cauchy–Schwarz inequality,

\( |\operatorname{E}(XY)|^2 \leq \operatorname{E}(X^2) \operatorname{E}(Y^2). \)

Moreover, if μ = E(X) and ν = E(Y), then

\( \begin{align} |\operatorname{Cov}(X,Y)|^2 &= |\operatorname{E}( (X - \mu)(Y - \nu) )|^2 = | \langle X - \mu, Y - \nu \rangle |^2\\ &\leq \langle X - \mu, X - \mu \rangle \langle Y - \nu, Y - \nu \rangle \\ & = \operatorname{E}( (X-\mu)^2 ) \operatorname{E}( (Y-\nu)^2 ) \\ & = \operatorname{Var}(X) \operatorname{Var}(Y). \end{align} \)

where Var denotes variance and Cov denotes covariance.
Generalizations

Various generalizations of the Cauchy–Schwarz inequality exist in the context of operator theory, e.g. for operator-convex functions, and operator algebras, where the domain and/or range of φ are replaced by a C*-algebra or W*-algebra.

This section lists a few of such inequalities from the operator algebra setting, to give a flavor of results of this type.
Positive functionals on C*- and W*-algebras

One can discuss inner products as positive functionals. Given a Hilbert space L2(m), m being a finite measure, the inner product < · , · > gives rise to a positive functional φ by

\( \phi (g) = \langle g, 1 \rangle. \)

Since < ƒ, ƒ > ≥ 0, φ(f*f) ≥ 0 for all ƒ in L2(m), where ƒ* is pointwise conjugate of ƒ. So φ is positive. Conversely every positive functional φ gives a corresponding inner product < ƒ, g >φ = φ(g*ƒ). In this language, the Cauchy–Schwarz inequality becomes

\( | \phi(g^*f) |^2 \leq \phi(f^*f) \phi(g^*g), \, \)

which extends verbatim to positive functionals on C*-algebras.

We now give an operator theoretic proof for the Cauchy–Schwarz inequality which passes to the C*-algebra setting. One can see from the proof that the Cauchy–Schwarz inequality is a consequence of the positivity and anti-symmetry inner-product axioms.

Consider the positive matrix

\( M = \begin{bmatrix} f^*\\ g^* \end{bmatrix} \begin{bmatrix} f & g \end{bmatrix} = \begin{bmatrix} f^*f & f^* g \\ g^*f & g^*g \end{bmatrix}. \)

Since φ is a positive linear map whose range, the complex numbers C, is a commutative C*-algebra, φ is completely positive. Therefore

\( M' = (I_2 \otimes \phi)(M) = \begin{bmatrix} \phi(f^*f) & \phi(f^* g) \\ \phi(g^*f) & \phi(g^*g) \end{bmatrix} \)

is a positive 2 × 2 scalar matrix, which implies it has positive determinant:

\( \phi(f^*f) \phi(g^*g) - | \phi(g^*f) |^2 \geq 0 \quad \text{i.e.} \quad \phi(f^*f) \phi(g^*g) \geq | \phi(g^*f) |^2. \, \)

This is precisely the Cauchy–Schwarz inequality. If ƒ and g are elements of a C*-algebra, f* and g* denote their respective adjoints.

We can also deduce from above that every positive linear functional is bounded, corresponding to the fact that the inner product is jointly continuous.
Positive maps

Positive functionals are special cases of positive maps. A linear map Φ between C*-algebras is said to be a positive map if a ≥ 0 implies Φ(a) ≥ 0. It is natural to ask whether inequalities of Schwarz-type exist for positive maps. In this more general setting, usually additional assumptions are needed to obtain such results.
Kadison–Schwarz inequality

The following theorem is named after Richard Kadison.

Theorem. If Φ is a unital positive map, then for every normal element a in its domain, we have Φ(a*a) ≥ Φ(a*)Φ(a) and Φ(a*a) ≥ Φ(a)Φ(a*).

This extends the fact φ(a*a) · 1 ≥ φ(a)*φ(a) = |φ(a)|2, when φ is a linear functional.

The case when a is self-adjoint, i.e. a = a*, is sometimes known as Kadison's inequality.
2-positive maps

When Φ is 2-positive, a stronger assumption than merely positive, one has something that looks very similar to the original Cauchy–Schwarz inequality:

Theorem (Modified Schwarz inequality for 2-positive maps)[2] For a 2-positive map Φ between C*-algebras, for all a, b in its domain,

\( \Phi(a)^*\Phi(a) \leq \Vert\Phi(1)\Vert\Phi(a^*a) \)
\( \Vert\Phi(a^*b)\Vert^2 \leq \Vert\Phi(a^*a)\Vert \cdot \Vert\Phi(b^*b)\Vert \)

A simple argument for (2) is as follows. Consider the positive matrix

\( M= \begin{bmatrix} a^* & 0 \\ b^* & 0 \end{bmatrix} \begin{bmatrix} a & b \\ 0 & 0 \end{bmatrix} = \begin{bmatrix} a^*a & a^* b \\ b^*a & b^*b \end{bmatrix}. \)

By 2-positivity of Φ,

\( (I_2 \otimes \Phi) M = \begin{bmatrix} \Phi(a^*a) & \Phi(a^* b) \\ \Phi(b^*a) & \Phi(b^*b) \end{bmatrix} \)

is positive. The desired inequality then follows from the properties of positive 2 × 2 (operator) matrices.

Part (1) is analogous. One can replace the matrix \( \begin{bmatrix} a & b \\ 0 & 0 \end{bmatrix} \) by \( \begin{bmatrix} 1 & a \\ 0 & 0 \end{bmatrix}. \)

Physics

The general formulation of the Heisenberg uncertainty principle is derived using the Cauchy–Schwarz inequality in the Hilbert space of quantum observables.
See also

Hölder's inequality
Minkowski inequality
Jensen's inequality
Notes

^ The Cauchy–Schwarz Master Class: an Introduction to the Art of Mathematical Inequalities, Ch. 1 by J. Michael Steele.
^ Paulsen, p. 40.

References

Bityutskov, V.I. (2001), "Bunyakovskii inequality", in Hazewinkel, Michiel, Encyclopedia of Mathematics, Springer, ISBN 978-1556080104
Bouniakowsky, V. (1859), "Sur quelques inegalités concernant les intégrales aux différences finies" (PDF), Mem. Acad. Sci. St. Petersbourg 7 (1): 9
Cauchy, A. (1821), Oeuvres 2, III, p. 373
Dragomir, S. S. (2003), "A survey on Cauchy–Bunyakovsky–Schwarz type discrete inequalities", JIPAM. J. Inequal. Pure Appl. Math. 4 (3): 142 pp
Kadison, R.V. (1952), "A generalized Schwarz inequality and algebraic invariants for operator algebras", Ann. Math. 56 (3): 494–503, doi:10.2307/1969657, JSTOR 1969657.
Lohwater, Arthur (1982), Introduction to Inequalities, Online e-book in PDF fomat
Paulsen, V. (2003), Completely Bounded Maps and Operator Algebras, Cambridge University Press.
Schwarz, H. A. (1888), "Über ein Flächen kleinsten Flächeninhalts betreffendes Problem der Variationsrechnung" (PDF), Acta Societatis scientiarum Fennicae XV: 318
Solomentsev, E.D. (2001), "Cauchy inequality", in Hazewinkel, Michiel, Encyclopedia of Mathematics, Springer, ISBN 978-1556080104
Steele, J.M. (2004), The Cauchy–Schwarz Master Class, Cambridge University Press, ISBN 0-521-54677-X

External links

Earliest Uses: The entry on the Cauchy–Schwarz inequality has some historical information.
Example of application of Cauchy–Schwarz inequality to determine Linearly Independent Vectors Tutorial and Interactive program.

Mathematics Encyclopedia