20 Introduction tion (or equivalently,summing the logarithms of the individual Fourier transforms),the "free convolution"is obtained through the sum of the so-called R-transforms introduced by Voiculescu [285].Examples of asymptotically free random matrices include independent Gaussian random matrices,and A and UBU*where A and B are Hermitian and U is uniformly distributed on the manifold of unitary matrices and independent of A and B. In free probability,the role of the Gaussian distribution in classical probability is taken by the semicircle law (1.18)in the sense of the free analog of the central limit theorem [284]:the spectrum of the normal- ized sum of free random matrices (with given spectrum)converges to the semicircle law (1.18).Analogously,the spectrum of the normalized sum of free random matrices with unit rank converges to the Marcenko- Pastur law (1.10),which then emerges as the free counterpart of the Poisson distribution [239,295].In the general context of free random variables,Voiculescu has found an elegant definition of free-entropy [288,289,291,292,293].A number of structural properties have been shown for free-entropy in the context of non-commutative probabil- ity theory(including the counterpart of the entropy-power inequality [248]).The free counterpart to Fisher's information has been investi- gated in [289].However,a free counterpart to the divergence between two distributions is yet to be discovered. A connection between random matrices and information theory was made by Balian [17]in 1968 considering the inverse problem in which the distribution of the entries of the matrix must be determined while being consistent with certain constraints.Taking a maximum entropy method,the ensemble of Gaussian matrices is the solution to the prob- lem where only a constraint on the energy of the singular values is placed
20 Introduction tion (or equivalently, summing the logarithms of the individual Fourier transforms), the “free convolution” is obtained through the sum of the so-called R-transforms introduced by Voiculescu [285]. Examples of asymptotically free random matrices include independent Gaussian random matrices, and A and UBU∗ where A and B are Hermitian and U is uniformly distributed on the manifold of unitary matrices and independent of A and B. In free probability, the role of the Gaussian distribution in classical probability is taken by the semicircle law (1.18) in the sense of the free analog of the central limit theorem [284]: the spectrum of the normalized sum of free random matrices (with given spectrum) converges to the semicircle law (1.18). Analogously, the spectrum of the normalized sum of free random matrices with unit rank converges to the Mar˘cenkoPastur law (1.10), which then emerges as the free counterpart of the Poisson distribution [239, 295]. In the general context of free random variables, Voiculescu has found an elegant definition of free-entropy [288, 289, 291, 292, 293]. A number of structural properties have been shown for free-entropy in the context of non-commutative probability theory (including the counterpart of the entropy-power inequality [248]). The free counterpart to Fisher’s information has been investigated in [289]. However, a free counterpart to the divergence between two distributions is yet to be discovered. A connection between random matrices and information theory was made by Balian [17] in 1968 considering the inverse problem in which the distribution of the entries of the matrix must be determined while being consistent with certain constraints. Taking a maximum entropy method, the ensemble of Gaussian matrices is the solution to the problem where only a constraint on the energy of the singular values is placed
2 Random Matrix Theory In this section,we review a wide range of existing mathematical results that are relevant to the analysis of the statistics of random matrices arising in wireless communications.We also include some new results on random matrices that were inspired by problems of engineering interest. Throughout the monograph,complex Gaussian random variables are always circularly symmetric,i.e.,with uncorrelated real and imagi- nary parts,and complex Gaussian vectors are always proper complex.I 2.1 Types of Matrices and Non-Asymptotic Results We start by providing definitions for the most important classes of random matrices:Gaussian,Wigner,Wishart and Haar matrices.We also collect a number of results that hold for arbitrary (non-asymptotic) matrix sizes. 1In the terminology introduced in [188),a random vector with real and imaginary compo- nents x and y,respectively,is proper compler if E (x-Ex])(y-Ely])=0. 21
2 Random Matrix Theory In this section, we review a wide range of existing mathematical results that are relevant to the analysis of the statistics of random matrices arising in wireless communications. We also include some new results on random matrices that were inspired by problems of engineering interest. Throughout the monograph, complex Gaussian random variables are always circularly symmetric, i.e., with uncorrelated real and imaginary parts, and complex Gaussian vectors are always proper complex.1 2.1 Types of Matrices and Non-Asymptotic Results We start by providing definitions for the most important classes of random matrices: Gaussian, Wigner, Wishart and Haar matrices. We also collect a number of results that hold for arbitrary (non-asymptotic) matrix sizes. 1 In the terminology introduced in [188], a random vector with real and imaginary components x and y, respectively, is proper complex if E h (x − E[x]) (y − E[y])T i = 0 . 21
22 Random Matrix Theory 2.1.1 Gaussian Matrices Definition 2.1.A standard real/complex Gaussian m x n matrix H has i.i.d.real/complex zero-mean Gaussian entries with identical vari- ance o2 The p.d.f.of a complex Gaussian matrix with i.i.d. zero-mean Gaussian entries with variance o2 is (πo2)-mn exp tr(HH 2 (2.1) The following result is the complex counterpart of those given in [18, 78,27,245]and[182,Thm.3.2.14: Lemma 2.1.[104]Let H be an m x n standard complex Gaussian matrix with n m.Denote its QR-decomposition by H=QR.The upper triangular matrix R is independent of Q,which is uniformly distributed over the manifold2 of complex m x n matrices such that QQt=I.The entries of R are independent and its diagonal entries, Rii for i{1,...,m},are such that 2mR are x2 random variables with 2(n-i+1)degrees of freedom while the off-diagonal entries,Ri.j for i<j,are independent zero-mean complex Gaussian with variance m The proof of Lemma 2.1 uses the expression of the p.d.f.of H given in (2.1)and [67,Theorem 3.1]. The p.d.f.of the eigenvalues of standard Gaussian matrices is stud- ied in [32,68].If the nxn matrix coefficients are real,[69]gives an exact expression for the expected number of real eigenvalues which grows as v2n/π. 2.1.2 Wigner Matrices Definition 2.2.An nxn Hermitian matrix W is a Wigner matrix if its upper-triangular entries are independent zero-mean random variables with identical variance.If the variance is,then W is a standard Wigner matrix. 2This is called the Stiefel manifold and it is a subspace of dimension 2mn-m2 with total olme2mrmn-含m(m-Ⅱg1
22 Random Matrix Theory 2.1.1 Gaussian Matrices Definition 2.1. A standard real/complex Gaussian m × n matrix H has i.i.d. real/complex zero-mean Gaussian entries with identical variance σ2 = 1 m. The p.d.f. of a complex Gaussian matrix with i.i.d. zero-mean Gaussian entries with variance σ2 is (πσ2) −mn exp −tr{HH†} σ2 . (2.1) The following result is the complex counterpart of those given in [18, 78, 27, 245] and [182, Thm. 3.2.14]: Lemma 2.1. [104] Let H be an m × n standard complex Gaussian matrix with n ≥ m. Denote its QR-decomposition by H = QR. The upper triangular matrix R is independent of Q, which is uniformly distributed over the manifold2 of complex m × n matrices such that QQ† = I. The entries of R are independent and its diagonal entries, Ri,i for i ∈ {1,...,m}, are such that 2mR2 i,i are χ2 random variables with 2(n − i + 1) degrees of freedom while the off-diagonal entries, Ri,j for i<j, are independent zero-mean complex Gaussian with variance 1 m . The proof of Lemma 2.1 uses the expression of the p.d.f. of H given in (2.1) and [67, Theorem 3.1]. The p.d.f. of the eigenvalues of standard Gaussian matrices is studied in [32, 68]. If the n×n matrix coefficients are real, [69] gives an exact expression for the expected number of real eigenvalues which grows as 2n/π. 2.1.2 Wigner Matrices Definition 2.2. An n×n Hermitian matrix W is a Wigner matrix if its upper-triangular entries are independent zero-mean random variables with identical variance. If the variance is 1 n, then W is a standard Wigner matrix. 2 This is called the Stiefel manifold and it is a subspace of dimension 2mn − m2 with total volume 2mπmn− 1 2 m(m−1) Qm i=1 1 (n−i)!
2.1.Types of Matrices and Non-Asymptotic Results 23 Theorem 2.2.Let W be an n x n complex Wigner matrix whose (diagonal and upper-triangle)entries are i.i.d.zero-mean Gaussian with unit variance.3 Then,its p.d.f.is 2-n/2r-n2/2 exp tr(w2) (2.2) 2 while the joint p.d.f.of its ordered eigenvalues入i≥..≥入nis n-I 2m2e31 1 员Ⅱa-为2 (2.3) =1 Theorem 2.3.[307]Let W be an n x n complex Gaussian Wigner matrix defined as in Theorem 2.2.The marginal p.d.f.of the unordered eigenvalues is 1 n名2V2 （e号H,(】 (2.4) with Hi()the ith Hermite polynomial [1]. As shown in [304,172,81,175],the spacing between adjacent eigen- values of a Wigner matrix exhibits an interesting behavior.With the eigenvalues of a Gaussian Wigner matrix sorted in ascending order,de- note by L the spacing between adjacent eigenvalues relative to the mean eigenvalue spacing.The density of L in the large-dimensional limit is accurately approximated by4 九(o)≈5e4 (2.5) For small values of s,(2.5)approaches zero implying that very small spacings are unlikely and that the eigenvalues somehow repel each other. 3Such matrices are often referred to as simply Gaussian Wigner matrices. 4 Wigner postulated (2.5)in [304]by assuming that the energy levels of a nucleus behave like a modified Poisson process.Starting from the joint p.d.f.of the eigenvalues of a Gaussian Wigner matrix,(2.5)has been proved in [81,175 where its exact expression has been derived.Later,Dyson conjectured that (2.5)may also hold for more general random matrices [65,66].This conjecture has been proved by [129]for a certain subclass of not necessarily Gaussian Wigner matrices
2.1. Types of Matrices and Non-Asymptotic Results 23 Theorem 2.2. Let W be an n × n complex Wigner matrix whose (diagonal and upper-triangle) entries are i.i.d. zero-mean Gaussian with unit variance.3 Then, its p.d.f. is 2−n/2π−n2/2 exp −tr{W2} 2 (2.2) while the joint p.d.f. of its ordered eigenvalues λ1 ≥ ... ≥ λn is 1 (2π)n/2 e− 1 2 Pn i=1 λ2 i n −1 i=1 1 i! n i<j (λi − λj ) 2. (2.3) Theorem 2.3. [307] Let W be an n × n complex Gaussian Wigner matrix defined as in Theorem 2.2. The marginal p.d.f. of the unordered eigenvalues is 1 n n −1 i=0 1 2i i! √2π e− x2 4 Hi(x) 2 (2.4) with Hi(·) the ith Hermite polynomial [1]. As shown in [304, 172, 81, 175], the spacing between adjacent eigenvalues of a Wigner matrix exhibits an interesting behavior. With the eigenvalues of a Gaussian Wigner matrix sorted in ascending order, denote by L the spacing between adjacent eigenvalues relative to the mean eigenvalue spacing. The density of L in the large-dimensional limit is accurately approximated by4 fL(s) ≈ π 2 s e− π 4 s2 (2.5) For small values of s, (2.5) approaches zero implying that very small spacings are unlikely and that the eigenvalues somehow repel each other. 3 Such matrices are often referred to as simply Gaussian Wigner matrices. 4Wigner postulated (2.5) in [304] by assuming that the energy levels of a nucleus behave like a modified Poisson process. Starting from the joint p.d.f. of the eigenvalues of a Gaussian Wigner matrix, (2.5) has been proved in [81, 175] where its exact expression has been derived. Later, Dyson conjectured that (2.5) may also hold for more general random matrices [65, 66]. This conjecture has been proved by [129] for a certain subclass of not necessarily Gaussian Wigner matrices
24 Random Matrix Theory 2.1.3 Wishart Matrices Definition 2.3.The m x m random matrix A HHt is a (central) real/compler Wishart matrix with n degrees of freedom and covariance matrix∑，(A~Wm(n,∑)，if the columns of the m×n matrix H are zero-mean independent real/complex Gaussian vectors with covariance matrix X.5 The p.d.f.of a complex Wishart matrix A ~Wm(n,>for n≥mis[244,p.84,[182,125]6 元-m(m-1)/2 fnB)=det(-ep[-tr{②-'B月detB"-m.(2.0 2.1.4 Haar Matrices Definition 2.4.A square matrix U is unitary if UU=UU=I. Definition 2.5.[107]An n x n random matrix U is a Haar matrix7 if it is uniformly distributed on the set,u(n),of n x n unitary matrices.8 Its density function on u(n)is given by [107,67] 2-nxn(n+1) m- (2.7) =1 Lemma 2.4.[107]The eigenvalues,Ci for i {1,...,n},of an n x n Haar matrix lie on the unit circle,i.e.,Ci=e,and their joint p.d.f.is 是亚- (2.8) i<l Lemma2.5.(e.g.[1l0])If1≤i,j,k,l≤n,i≠k,j≠(，and U is an 5If the entries of H have nonzero mean,HHt is a non-central Wishart matrix. 6 The case n<m is studied in [267]. 7Also called isotropic in the multi-antenna literature [171]. 8A real Haar matrix is uniformly distributed on the set of real orthogonal matrices
24 Random Matrix Theory 2.1.3 Wishart Matrices Definition 2.3. The m × m random matrix A = HH† is a (central) real/complex Wishart matrix with n degrees of freedom and covariance matrix Σ, (A ∼ Wm(n, Σ)), if the columns of the m × n matrix H are zero-mean independent real/complex Gaussian vectors with covariance matrix Σ. 5 The p.d.f. of a complex Wishart matrix A ∼ Wm(n, Σ) for n ≥ m is [244, p. 84], [182, 125]6 fA(B) = π−m(m−1)/2 detΣn m i=1(n − i)! exp −tr Σ−1B detBn−m. (2.6) 2.1.4 Haar Matrices Definition 2.4. A square matrix U is unitary if UU† = U† U = I. Definition 2.5. [107] An n×n random matrix U is a Haar matrix7 if it is uniformly distributed on the set, U(n), of n × n unitary matrices.8 Its density function on U(n) is given by [107, 67] 2−nπ− 1 2n(n+1) n i=1 (n − i)! (2.7) Lemma 2.4. [107] The eigenvalues, ζi for i ∈ {1,...,n}, of an n × n Haar matrix lie on the unit circle, i.e., ζi = ejθi , and their joint p.d.f. is 1 n! i< |ζi − ζ| 2. (2.8) Lemma 2.5. (e.g. [110]) If 1 ≤ i, j, k, ≤ n, i = k, j = , and U is an 5 If the entries of H have nonzero mean, HH† is a non-central Wishart matrix. 6 The case n<m is studied in [267]. 7 Also called isotropic in the multi-antenna literature [171]. 8 A real Haar matrix is uniformly distributed on the set of real orthogonal matrices