694 Vol.11,No.3/September 2019/Advances in Optics and Photonics Tutorial "cross-terms"disappear.Explicitly,for a wave that is a sum of a set of (non-zero) waves{中g}, 1)=∑1pg, 20 where those waves are all orthogonal,which we can write as (φplpg)=0 if and only if p≠q, 21) then P=(p) (区（②）-三刻--四 where Pg=(中gl中g)》 23) is the power in the wave Later,as we generalize the mathematics,we may formally define inner products that explicitly give the power or energy for electromagnetic waves;these will not just be the simple Cartesian products of wave functions,though they will still satisfy the more basic mathematical properties required of inner products. A second important property is that the inner product of a(non-zero)vector with itself is always positive;this is easy to see for the Cartesian inner product of a vector such as l中R）as in Eq.(9):explicitly, (中Rl中R)=[f作f度… fNR】 f* f2>0 (24) because it is a sum of positive (or at least non-negative)quantities f2,at least one of which must be greater than zero for a non-zero vector. 3.5.Orthonormal Functions and Vectors Now,returning to Eq.(18),we presume we want to find the choices of source func- tions or vectors s)that give the largest total powers in the set of receivers (or microphones).To make comparisons easier,we will presume that we normalize all the source functions of interest to us.Normalization means that we adjust the func- tion or vector by some multiplicative(normalizing)factor so that the inner product of the function or vector with itself is unity,i.e., (usjlwsj〉=1. (25) A particularly convenient set of functions is one that is normalized and in which all the different elements are orthogonal;this is called an orthonormal set of functions,and its elements would therefore satisfy (vsplWsq〉=δpg: (26) where the Kronecker delta is
“cross-terms” disappear. Explicitly, for a wave jϕi that is a sum of a set of (non-zero) waves fjϕqig, jϕi X q jϕqi, (20) where those waves are all orthogonal, which we can write as hϕpjϕqi 0 if and only if p ≠ q, (21) then P hϕjϕi X p hϕpj ! X q jϕqi ! X p, q hϕpjϕqi X q hϕqjϕqi X q Pq, (22) where Pq hϕqjϕqi (23) is the power in the wave jϕqi. Later, as we generalize the mathematics, we may formally define inner products that explicitly give the power or energy for electromagnetic waves; these will not just be the simple Cartesian products of wave functions, though they will still satisfy the more basic mathematical properties required of inner products. A second important property is that the inner product of a (non-zero) vector with itself is always positive; this is easy to see for the Cartesian inner product of a vector such as jϕRi as in Eq. (9); explicitly, hϕRjϕRi f 1 f 2 f NR 2 6 6 6 4 f1 f2 . . . f NR 3 7 7 7 5 X NR j1 f j fj X NR j1 j fjj 2 > 0, (24) because it is a sum of positive (or at least non-negative) quantities j f jj 2, at least one of which must be greater than zero for a non-zero vector. 3.5. Orthonormal Functions and Vectors Now, returning to Eq. (18), we presume we want to find the choices of source functions or vectors fjψSjig that give the largest total powers in the set of receivers (or microphones). To make comparisons easier, we will presume that we normalize all the source functions of interest to us. Normalization means that we adjust the function or vector by some multiplicative (normalizing) factor so that the inner product of the function or vector with itself is unity, i.e., hψSjjψSji 1: (25) A particularly convenient set of functions is one that is normalized and in which all the different elements are orthogonal; this is called an orthonormal set of functions, and its elements would therefore satisfy hψSpjψSqi δpq, (26) where the Kronecker delta is 694 Vol. 11, No. 3 / September 2019 / Advances in Optics and Photonics Tutorial
Tutorial Vol.11,No.3/September 2019/Advances in Optics and Photonics 695 if p=q 0 ifp丰q 27) 3.6.Vector Spaces,Operators,and Hilbert Spaces The mathematics here gives some very powerful tools and concepts.We are not going to prove these properties now for two reasons:first,for finite matrices,these properties are discussed in standard matrix algebra texts [130,131].Second,we will give these results for the more general (and difficult)cases of continuous functions and infinite matrices in Section 6(and with proofs in [122]);finite matrices are then a simple special case. An operator's properties can only be completely described if we are specific about the mathematical"space"(often called a vector space)in which it operates.For example,the ordinary (geometrical)vector dot product is an operator that operates in the mathematical space based on ordinary three-dimensional geometric space.This mathematical space contains all vectors that can exist in a geometrical space that is three-dimensional,with the algebraic property of having a vector dot product-the"inner product"for this space. The operators of interest to us act on vectors or functions in a Hilbert space (formally defined in Subsection 6.4).Any given Hilbert space will have a specific dimension- ality,which may be finite or infinite,and it must have an inner product.We can think of this mathematical Hilbert space as being analogous to the mathematical space of ordinary geometrical vectors,but allowing arbitrary dimensionality and complex coefficients (geometrical vectors can only be associated with real amplitudes or coefficients). The possible source functions exist in one Hilbert space Hs,associated with the source volume Vs.In our example here,this space Hs contains all possible Ns-dimensional mathematical vectors with finite complex elements that are the am- plitudes of specific point sources.The possible wave functions exist in another Hilbert space HR,associated with the receiving volume Ve.This space Hg also contains all possible NR-dimensional mathematical vectors with finite complex elements that are the possible amplitudes of specific waves at the point"microphones"(or the corre- sponding signals from those microphones).Each of these spaces Hs and Hg has a Cartesian inner product,though later we may use different"underlying"inner prod- ucts in different spaces. 3.7.Eigenproblems and Singular-Value Decomposition Now we see that the operator Gsg is something that maps between these two spaces. Specifically,as in Eq.(10),it operates on the vector ls),which is in space Hs,to generate the vector l),which is in space Hg.Now,we want to find some "best" choices of such source vectors ls)that will give us the"best"resulting waves l). For such best choices,our instinct might be to try to find eigenvectors of some oper- ator.However,we cannot just find eigenvectors of Gsg;we might be able mathemati- cally to find eigenvectors of the matrix Gsg,but these may have dubious physical meaning in our problem,because Gsk is an operator mapping between one space and another,not an operator within a space.So,Gsg cannot map a function back onto a multiple of itself in the same space. We could,of course,define a Green's function operating within a space,and we might do so for a resonator problem;we could even base that on the exactly the same kind of mathematical expression as in Eq.(4)for G(r;r),with r and r'being positions within the same volume.Here,however,we are effectively basing our operator
δpq 1 if p q 0 if p ≠ q : (27) 3.6. Vector Spaces, Operators, and Hilbert Spaces The mathematics here gives some very powerful tools and concepts. We are not going to prove these properties now for two reasons: first, for finite matrices, these properties are discussed in standard matrix algebra texts [130,131]. Second, we will give these results for the more general (and difficult) cases of continuous functions and infinite matrices in Section 6 (and with proofs in [122]); finite matrices are then a simple special case. An operator’s properties can only be completely described if we are specific about the mathematical “space” (often called a vector space) in which it operates. For example, the ordinary (geometrical) vector dot product is an operator that operates in the mathematical space based on ordinary three-dimensional geometric space. This mathematical space contains all vectors that can exist in a geometrical space that is three-dimensional, with the algebraic property of having a vector dot product—the “inner product” for this space. The operators of interest to us act on vectors or functions in a Hilbert space (formally defined in Subsection 6.4). Any given Hilbert space will have a specific dimensionality, which may be finite or infinite, and it must have an inner product. We can think of this mathematical Hilbert space as being analogous to the mathematical space of ordinary geometrical vectors, but allowing arbitrary dimensionality and complex coefficients (geometrical vectors can only be associated with real amplitudes or coefficients). The possible source functions exist in one Hilbert space HS, associated with the source volume VS. In our example here, this space HS contains all possible NS-dimensional mathematical vectors with finite complex elements that are the amplitudes of specific point sources. The possible wave functions exist in another Hilbert space HR, associated with the receiving volume VR. This space HR also contains all possible NR-dimensional mathematical vectors with finite complex elements that are the possible amplitudes of specific waves at the point “microphones” (or the corresponding signals from those microphones). Each of these spaces HS and HR has a Cartesian inner product, though later we may use different “underlying” inner products in different spaces. 3.7. Eigenproblems and Singular-Value Decomposition Now we see that the operator GSR is something that maps between these two spaces. Specifically, as in Eq. (10), it operates on the vector jψSi, which is in space HS, to generate the vector jϕRi, which is in space HR. Now, we want to find some “best” choices of such source vectors jψSi that will give us the “best” resulting waves jϕRi. For such best choices, our instinct might be to try to find eigenvectors of some operator. However, we cannot just find eigenvectors of GSR; we might be able mathematically to find eigenvectors of the matrix GSR, but these may have dubious physical meaning in our problem, because GSR is an operator mapping between one space and another, not an operator within a space. So, GSR cannot map a function back onto a multiple of itself in the same space. We could, of course, define a Green’s function operating within a space, and we might do so for a resonator problem; we could even base that on the exactly the same kind of mathematical expression as in Eq. (4) for Gωr; r0 , with r and r0 being positions within the same volume. Here, however, we are effectively basing our operator Tutorial Vol. 11, No. 3 / September 2019 / Advances in Optics and Photonics 695
696 Vol.11,No.3/September 2019/Advances in Optics and Photonics Tutorial GsR on the mathematical operator (or kernel in the language of integral equations) G(rR;rs),where rg and rs are definitely in different volumes. The key to constructing the right eigenproblems here is to look at those associated with the operator GGsk and with the complementary operator GsGAs we said, the operator Gsg maps a vector in Hs to a vector in HThe operator G however, maps a vector in Hg to a vector in Hs.So,overall,GGsk maps a vector in Hs to a vector in Hs.Similarly,the operator GsRGsg maps a vector in Hg to a vector in Hg. Hence,it is physically meaningful to consider eigenproblems for each of these oper- ators GsGsR and GsRGSR.It is the mathematics of such eigenproblems that is at the core of SVD. Here we can usefully introduce several more definitions and results(mostly without proofs for the moment).First,we note that the operators GseGsk and GsRGsg are Hermitian-that is,each is its own Hermitian adjoint.Explicitly, (GSRGSR)=GsR(GSR)=GsRGsR, (28) where we have used Eqs.(13)and (15),and similarly for GskG An operator such as GGsk is also a positive operator,which means that an expres- sion such as sGG)is always greater than or equal to zero.(Similarly,the operator GsRG is also Hermitian and positive.) Now,so-called "compact"[132]Hermitian operators (defined formally in Section 6) have several properties (and all finite Hermitian matrices are compact Hermitian operators): (1)Their eigenvalues are real. (2)Their eigenfunctions are orthogonal (or,at least formally,the ones corresponding to different eigenvalues are orthogonal,and different ones corresponding to the same eigenvalue can always be chosen to be orthogonal). (3)Their eigenfunctions form complete sets for the Hilbert spaces in which they op- erate [133]-in other words,we can write any function in the space as some linear combination of these eigenfunctions, and if those operators are positive (4)Their eigenvalues are greater than or equal to zero. (5)Their eigenfunctions and their corresponding eigenvalues satisfy maximization properties-specifically,if we set out to find the normalized "input"vector or function that led to the largest "output"vector (in terms of its inner product), then that is the eigenfunction with the largest eigenvalue,and we could find the eigenfunction with the second largest eigenvalue and corresponding eigen- vector by repeating the maximization process to find a function orthogonal to the first one,and so on. The formal proofs of all of these properties are given in [122]for finite and infinite- dimensional spaces,and we also discuss these topics further in Section 6. Furthermore,any operator that can be approximated to any sufficient degree by a finite matrix is also effectively compact;indeed,this idea is beginning to get close to the idea of what a compact operator really is.So,certainly our finite matrix problem with positive operators GGsR or GsRG here has all of the properties (1)to (5)above, and it will retain these properties no matter how large we make the (finite)matrix
GSR on the mathematical operator (or kernel in the language of integral equations) GωrR; rS, where rR and rS are definitely in different volumes. The key to constructing the right eigenproblems here is to look at those associated with the operator G† SRGSR and with the complementary operator GSRG† SR. As we said, the operator GSR maps a vector in HS to a vector in HR. The operator G† SR, however, maps a vector in HR to a vector in HS. So, overall, G† SRGSR maps a vector in HS to a vector in HS. Similarly, the operator GSRG† SR maps a vector in HR to a vector in HR. Hence, it is physically meaningful to consider eigenproblems for each of these operators G† SRGSR and GSRG† SR. It is the mathematics of such eigenproblems that is at the core of SVD. Here we can usefully introduce several more definitions and results (mostly without proofs for the moment). First, we note that the operators G† SRGSR and GSRG† SR are Hermitian—that is, each is its own Hermitian adjoint. Explicitly, G† SRGSR † G† SRG† SR † G† SRGSR, (28) where we have used Eqs. (13) and (15), and similarly for GSRG† SR. An operator such as G† SRGSR is also a positive operator, which means that an expression such as hψSjG† SRGSRjψSi is always greater than or equal to zero. (Similarly, the operator GSRG† SR is also Hermitian and positive.) Now, so-called “compact” [132] Hermitian operators (defined formally in Section 6) have several properties (and all finite Hermitian matrices are compact Hermitian operators): (1) Their eigenvalues are real. (2) Their eigenfunctions are orthogonal (or, at least formally, the ones corresponding to different eigenvalues are orthogonal, and different ones corresponding to the same eigenvalue can always be chosen to be orthogonal). (3) Their eigenfunctions form complete sets for the Hilbert spaces in which they operate [133]—in other words, we can write any function in the space as some linear combination of these eigenfunctions, and if those operators are positive (4) Their eigenvalues are greater than or equal to zero. (5) Their eigenfunctions and their corresponding eigenvalues satisfy maximization properties—specifically, if we set out to find the normalized “input” vector or function that led to the largest “output” vector (in terms of its inner product), then that is the eigenfunction with the largest eigenvalue, and we could find the eigenfunction with the second largest eigenvalue and corresponding eigenvector by repeating the maximization process to find a function orthogonal to the first one, and so on. The formal proofs of all of these properties are given in [122] for finite and infinitedimensional spaces, and we also discuss these topics further in Section 6. Furthermore, any operator that can be approximated to any sufficient degree by a finite matrix is also effectively compact; indeed, this idea is beginning to get close to the idea of what a compact operator really is. So, certainly our finite matrix problem with positive operators G† SRGSR or GSRG† SR here has all of the properties (1) to (5) above, and it will retain these properties no matter how large we make the (finite) matrix. 696 Vol. 11, No. 3 / September 2019 / Advances in Optics and Photonics Tutorial
Tutorial Vol.11,No.3/September 2019/Advances in Optics and Photonics 697 Now,for specific choices of the numbers and positions of the point sources and receiv- ers,we can simply write down the Ng x Ns matrix Gsg as in Eq.(9),using the for- mula Eq.(7)to work out the necessary matrix elements.So,we are ready to turn any such specific problem into a simple numerical problem to find the eigenvectors and eigenvalues.So,therefore,for any such problem,we can solve for the (orthonormal) eigenvectors lws)of the Ns x Ns matrix GsRGsR.The eigenvalues are necessarily positive (because GseGsk is a positive operator),and so we can write them in the form s2.That is,explicitly, GSRGSRluΨs〉=Is P2wsi). (29) Similarly,we can solve for the (orthonormal)eigenvectors logi)of the Ng x NR ma- trix GsRGs.It is not too surprising that these have the same [134]eigenvalues s,2. That is, GSRGSRlO中R〉=IsP1. (30) In fact,we can show (Appendix E)that GsRlWsi〉=Sl中R 31) and G5RlpR〉=lws》 32) Hence by solving two eigenvalue problems,one for GGs and a second for GskG we have established two sets of eigenfunctions,one,s),for the source vectors or functions in Hs,and a second set {gi)for the wave vectors or functions in Hg. Note,too,that these vectors or functions are paired:a source vector or function lysi)in Hs leads to the corresponding wave vector or function li)in Hg, with an amplitude s.The numbers s;are called the singular values of the operator or matrix GsR. Note that,in practice,we only actually have to solve one eigenvalue problem-that is, either Eq.(29)or Eq.(30).If we know the eigenfunctions {si)from solving Eq.(29),then we can deduce the eigenfunctions g)from Eq.(31),and similarly if we know the eigenfunctions {o)from Eq.(30),we can deduce the {s))from Eg.(32),at least for all the cases in which the singular value is not zero [134].In practice,one of these eigenproblems may be simpler or effectively "smaller"than the other,and we can conveniently choose that one if we prefer. The fact that these two sets of functions {ls)and {are each eigenfunctions of a Hermitian operator guarantees that each of these sets is orthogonal and complete [135]for its Hilbert space (Hs or Hg,respectively). From Eqs.(31)and (32),we can see that we can rewrite Gsg as GSR= ∑sl中)sl, (33) =1 where N is the smaller of Ns and Ng [134].This expression Eq.(33)is called the singular-value decomposition of the operator Gsg.We can also similarly write
Now, for specific choices of the numbers and positions of the point sources and receivers, we can simply write down the NR × NS matrix GSR as in Eq. (9), using the formula Eq. (7) to work out the necessary matrix elements. So, we are ready to turn any such specific problem into a simple numerical problem to find the eigenvectors and eigenvalues. So, therefore, for any such problem, we can solve for the (orthonormal) eigenvectors jψSji of the NS × NS matrix G† SRGSR. The eigenvalues are necessarily positive (because G† SRGSR is a positive operator), and so we can write them in the form jsjj 2. That is, explicitly, G† SRGSRjψSjijsjj 2jψSji: (29) Similarly, we can solve for the (orthonormal) eigenvectors jϕRji of the NR × NR matrix GSRG† SR. It is not too surprising that these have the same [134] eigenvalues jsjj 2. That is, GSRG† SRjϕRjijsjj 2jϕRji: (30) In fact, we can show (Appendix E) that GSRjψSji sjjϕRji (31) and G† SRjϕRji s j jψSji: (32) Hence by solving two eigenvalue problems, one for G† SRGSR and a second for GSRG† SR, we have established two sets of eigenfunctions, one, fjψSjig, for the source vectors or functions in HS, and a second set fjϕRjig for the wave vectors or functions in HR. Note, too, that these vectors or functions are paired: a source vector or function jψSji in HS leads to the corresponding wave vector or function jϕRji in HR, with an amplitude sj. The numbers sj are called the singular values of the operator or matrix GSR. Note that, in practice, we only actually have to solve one eigenvalue problem—that is, either Eq. (29) or Eq. (30). If we know the eigenfunctions fjψSjig from solving Eq. (29), then we can deduce the eigenfunctions fjϕRjig from Eq. (31), and similarly if we know the eigenfunctions fjϕRjig from Eq. (30), we can deduce the fjψSjig from Eq. (32), at least for all the cases in which the singular value is not zero [134]. In practice, one of these eigenproblems may be simpler or effectively “smaller” than the other, and we can conveniently choose that one if we prefer. The fact that these two sets of functions fjψSjig and fjϕRjig are each eigenfunctions of a Hermitian operator guarantees that each of these sets is orthogonal and complete [135] for its Hilbert space (HS or HR, respectively). From Eqs. (31) and (32), we can see that we can rewrite GSR as GSR X N m j1 sjjϕRjihψSjj, (33) where N m is the smaller of NS and NR [134]. This expression Eq. (33) is called the singular-value decomposition of the operator GSR. We can also similarly write Tutorial Vol. 11, No. 3 / September 2019 / Advances in Optics and Photonics 697
698 Vol.11,No.3/September 2019/Advances in Optics and Photonics Tutorial 34) Incidentally,a product of the form (sil,which has a column vector on the left and a row vector on the right,is sometimes called an outer product.Standard matrix manipulations show that an outer product of two N-element vectors is an N x N ma- trix.So, this process of singular-value decomposition,performed by solving the two re- lated eigenproblems,one for the matrix or operator GsGsk and the second for the matrix or operator GsRGs.leads to our desired two sets of orthogonal vectors or functions. These are source vectors or functions in Hs and wave vectors or functions in Hr,and these are"paired up,"with each source eigenvector in Hs giving rise to its correspond- ing wave eigenvector in Hg (with amplitude given by the corresponding singu- lar value). Furthermore,because these are eigenvectors or eigenfunctions of a positive Hermitian operator,by property (5)above,they are the"best"possible choices.Specifically,if we choose to order the eigenvectors by decreasing size of s 2,then [neglecting degen- eracy of eigenvalues(i.e.,more than one eigenvector for a given eigenvalue)for sim- plicity here at the moment], the source vector lys)in Hs gives rise to the largest possible magnitude of wave vector in HR,and has the form R); the source vector ls2)is the source vector in Hs that is orthogonal to ys)and that gives rise to the second largest possible magnitude of wave vector in Hg,which has the form中2）and is orthogonal to中ri: the source vector ls3)is the source vector in Hs that is orthogonal to lus)and ls2)and that gives rise to the third largest possible magnitude of wave vector in HR,which has the form loR3)and is orthogonal to l)and log2); and so on. We have therefore established the best set of possible orthogonal(and therefore zero “crosstalk")“channels”between the two volumes(at least“best”as given by the mag- nitude of the inner product).Note explicitly that these channels are orthogonal to one another,both at the sources and at the receivers. Incidentally,in matrix terms,the SVD as in Eq.(33)can also be written in the form GSR VDdiagUT, (35) where Ddiag is a diagonal matrix with the singular values s;as the diagonal elements,V is a matrix whose columns are the vectors l),and U'is a matrix whose rows are the vectors (wsil (or equivalently U is a matrix whose columns are the vectors ls)). Technically,the matrices V and Uf (and also U)are "unitary."(See Appendix E.) 3.8.Sum Rule on Coupling Strengths One very important point emerges from this algebra,which is a"sum rule"on the s;2. Specifically,we can show that,quite generally for finite numbers Ns and Ng of sources and receiver points
G† SR X N m j1 s j jψSjihϕRjj: (34) Incidentally, a product of the form jϕRjihψSjj, which has a column vector on the left and a row vector on the right, is sometimes called an outer product. Standard matrix manipulations show that an outer product of two N-element vectors is an N × N matrix. So, this process of singular-value decomposition, performed by solving the two related eigenproblems, one for the matrix or operator G† SRGSR and the second for the matrix or operator GSRG† SR, leads to our desired two sets of orthogonal vectors or functions. These are source vectors or functions in HS and wave vectors or functions in HR, and these are “paired up,” with each source eigenvector in HS giving rise to its corresponding wave eigenvector in HR (with amplitude given by the corresponding singular value). Furthermore, because these are eigenvectors or eigenfunctions of a positive Hermitian operator, by property (5) above, they are the “best” possible choices. Specifically, if we choose to order the eigenvectors by decreasing size of jsjj 2, then [neglecting degeneracy of eigenvalues (i.e., more than one eigenvector for a given eigenvalue) for simplicity here at the moment], the source vector jψS1i in HS gives rise to the largest possible magnitude of wave vector in HR, and has the form jϕR1i; the source vector jψS2i is the source vector in HS that is orthogonal to jψS1i and that gives rise to the second largest possible magnitude of wave vector in HR, which has the form jϕR2i and is orthogonal to jϕR1i; the source vector jψS3i is the source vector in HS that is orthogonal to jψS1i and jψS2i and that gives rise to the third largest possible magnitude of wave vector in HR, which has the form jϕR3i and is orthogonal to jϕR1i and jϕR2i; and so on. We have therefore established the best set of possible orthogonal (and therefore zero “crosstalk”) “channels” between the two volumes (at least “best” as given by the magnitude of the inner product). Note explicitly that these channels are orthogonal to one another, both at the sources and at the receivers. Incidentally, in matrix terms, the SVD as in Eq. (33) can also be written in the form GSR VDdiagU†, (35) where Ddiag is a diagonal matrix with the singular values sj as the diagonal elements, V is a matrix whose columns are the vectors jϕRji, and U† is a matrix whose rows are the vectors hψSjj (or equivalently U is a matrix whose columns are the vectors jψSji). Technically, the matrices V and U† (and also U) are “unitary.” (See Appendix E.) 3.8. Sum Rule on Coupling Strengths One very important point emerges from this algebra, which is a “sum rule” on the jsjj 2. Specifically, we can show that, quite generally for finite numbers NS and NR of sources and receiver points, 698 Vol. 11, No. 3 / September 2019 / Advances in Optics and Photonics Tutorial