2Complexityand Entropysegments ofthe molecule (with lengths of order 50 nm) whose elasticity,for smalldeviations from equilibrium, is well described by the FJC model described above.For these short segments in DNA, the force constant associated with Hook's lawisfound tobek=0.1pN [11].2.7Multiplicity and Entropy of an Einstein SolidEinstein developed a very simplemodel for mechanical vibrations on a latticeThismodel is called theEinstein solid and consists of a three dimensional latticewhich contains N/3 lattice sites,with one atom attached to each lattice site.Eachatom can oscillate about its lattice site in three independent spatial directions,(x,y,z).Thus, each lattice site contains three independent oscillators.The entirelattice contains a total of N oscillators, which are assumed to be harmonic os-cillators, all having the same radial frequency o.The vibrations of the solid aredueto these N harmonic oscillators.A singleharmonic oscillator has an ener-gyE=(1/2)ho+qho, whereh is Planck'sconstant, (1/2)ho is the zero-pointenergy of the harmonic oscillator, and q= O,1,2, ..., co is an integer. A harmon-ic oscillator has zero point energy because of the Heisenberg uncertainty rela-tion px≥h, which arises from thewave nature of particles.The oscillatorcan never come to restbecause that would cause Ax-O and px-→O, whichcannotbe satisfiedquantummechanically.For a lattice with N harmonic oscillators, the total vibrational energy can bewrittenINha+qha,E(N,q) =(2.35)2where q=O,1,2,., co is again an integer.The oscillators are independent ofoneanotherand canbeindifferentstatesofmotion.Ifthelatticehasatotalen-ergyE(N,q),theqquantaofenergycanbedistributed among theNharmonicoscillators in many different ways.2.7.1Multiplicity of an Einstein SolidLet us assume that"q quanta onthe lattice"is a macroscopic state,and let us deter-minethemultiplicityofthismacroscopicstate[190].Weneedtodeterminehowmany waysq quanta can bedistributed among N distinct pots.This is straightfor-ward if we draw a picture. Representa quantum of energyby an"x"and N pots byN-1 vertical lines.For example, if q =9 and N = 6, then one way to distributethe quanta is represented by the picture (xx|xxx|/x|xxx). We can determine allpossible ways to distributeq=9quanta among N=6pots byfinding the numberpermutations of nine"x's and five vertical lines.More generally, the number ofways todistributeqquanta among Npots is thetotal number of permutations
18 2 Complexity and Entropy segments of the molecule (with lengths of order 50 nm) whose elasticity, for small deviations from equilibrium, is well described by the FJC model described above. For these short segments in DNA, the force constant associated with Hook’s law is found to be k = 0.1 pN [11]. 2.7 Multiplicity and Entropy of an Einstein Solid Einstein developed a very simple model for mechanical vibrations on a lattice. This model is called the Einstein solid and consists of a three dimensional lattice which contains N∕3 lattice sites, with one atom attached to each lattice site. Each atom can oscillate about its lattice site in three independent spatial directions, (x, y, z). Thus, each lattice site contains three independent oscillators. The entire lattice contains a total of N oscillators, which are assumed to be harmonic oscillators, all having the same radial frequency ω. The vibrations of the solid are due to these N harmonic oscillators. A single harmonic oscillator has an energy E = (1∕2)ℏω + qℏω, where ℏ is Planck’s constant, (1∕2)ℏω is the zero-point energy of the harmonic oscillator, and q = 0, 1, 2, . , ∞ is an integer. A harmonic oscillator has zero point energy because of the Heisenberg uncertainty relation ΔpxΔx ≥ ℏ, which arises from the wave nature of particles. The oscillator can never come to rest because that would cause Δx → 0 and Δpx → 0, which can not be satisfied quantum mechanically. For a lattice with N harmonic oscillators, the total vibrational energy can be written E(N, q) = 1 2 Nℏω + qℏω , (2.35) where q = 0, 1, 2, . , ∞ is again an integer. The oscillators are independent of one another and can be in different states of motion. If the lattice has a total energy E(N, q), the q quanta of energy can be distributed among the N harmonic oscillators in many different ways. 2.7.1 Multiplicity of an Einstein Solid Let us assume that “q quanta on the lattice” is a macroscopic state, and let us determine the multiplicity of this macroscopic state [190]. We need to determine how many ways q quanta can be distributed among N distinct pots. This is straightforward if we draw a picture. Represent a quantum of energy by an “x” and N pots by N − 1 vertical lines. For example, if q = 9 and N = 6, then one way to distribute the quanta is represented by the picture {xx|xxx||x|xx|x}. We can determine all possible ways to distribute q = 9 quanta among N = 6 pots by finding the number permutations of nine “x”s and five vertical lines. More generally, the number of ways to distribute q quanta among N pots is the total number of permutations
2.7MultiplicityandEntropy ofan Einstein Solidof q"xs with N-1 vertical lines.This number is the multiplicity N(q) of themacrostate"qquanta onthelattice"and isgivenby(N+q-1)!NN(q)=(2.36)q!(N- 1)!If there is only one quantum of vibrational energy on the lattice and the latticehas N=12 oscillators (fourlattice sites),then it can beplaced in theN=12 os-cillators in Ni2(1) =12 different ways.However, if there are q =12 quanta onthe lattice of N=12oscillators, they can be distributed among theoscillatorsinNi2(12)=1352078different ways.Generally,weare interested in apieceofa crystal whichtypicallyhas N=1023 oscillators.Then,ifthere areenough quantaonthelatticetoexciteevenasmallfractionoftheoscillators,thenumberofmi-croscopicstatesavailableisamazinglylarge.It isclearthatweneedawaytomakecontact with measurablequantities,and thermodynamics will give us that.Noteagain that ourabilityto count thenumber of vibrational states availableto thelat-tice is a consequence of quantum mechanics and the quantization of vibrationalenergy2.7.2Entropyof the Einstein SolidFor the Einstein solid, we have one less macroscopic parameter than for the spinsystem.In the spin system, in the absence ofa magneticfield,all microscopic configurations have the same energy.However,for the spin system we have anotherparameter, n, (due to spin conservation)which is the number of spins-up on thelatticeandwecanuseitto constructdifferentmacroscopicstatesdependingonthevalue of n.Forthe Einstein solid,fora fixed number ofharmonic oscillatorsN,we onlyhave the parameter q which is the number of quanta of oscillation energyand is proportional to theenergyE.Therefore,fortheEinstein solid,wehave onlyonemacroscopicstate,determinedbythevalueofg,andmanymicroscopicstatesgiven by the number of ways todistribute theq quanta among the N harmonicoscillators.If we use Eqs. (2.8) and (2.36), the entropy of the Einstein solid is given by[(N+q-1)![(N + q)N+S(N,q) = kg lnkIr(2.37)q!(N-1)!NNq!where we have assumed that Nand qarelarge and we have used Stirling's approx-imation Eq (2.15).We can now use thermodynamics to relate the energyto thetemperature via the relation(as)(崇) =六(g)).=1(2.38)TaElwhere, in the middle term, we have used the relation between E and q given inEq. (2.35).If we take the derivative of the entropy, as indicated in Eq. (2.38), and
2.7 Multiplicity and Entropy of an Einstein Solid 19 of q “x”s with N − 1 vertical lines. This number is the multiplicity (q) of the macrostate “q quanta on the lattice” and is given by N (q) = (N + q − 1)! q!(N − 1)! . (2.36) If there is only one quantum of vibrational energy on the lattice and the lattice has N = 12 oscillators (four lattice sites), then it can be placed in the N = 12 oscillators in 12(1) = 12 different ways. However, if there are q = 12 quanta on the lattice of N = 12 oscillators, they can be distributed among the oscillators in 12(12) = 1 352 078 different ways. Generally, we are interested in a piece of a crystal which typically has N = 1023 oscillators. Then, if there are enough quanta on the lattice to excite even a small fraction of the oscillators, the number of microscopic states available is amazingly large. It is clear that we need a way to make contact with measurable quantities, and thermodynamics will give us that. Note again that our ability to count the number of vibrational states available to the lattice is a consequence of quantum mechanics and the quantization of vibrational energy. 2.7.2 Entropy of the Einstein Solid For the Einstein solid, we have one less macroscopic parameter than for the spin system. In the spin system, in the absence of a magnetic field, all microscopic con- figurations have the same energy. However, for the spin system we have another parameter, n, (due to spin conservation) which is the number of spins-up on the lattice and we can use it to construct different macroscopic states depending on the value of n. For the Einstein solid, for a fixed number of harmonic oscillators N, we only have the parameter q which is the number of quanta of oscillation energy and is proportional to the energy E. Therefore, for the Einstein solid, we have only one macroscopic state, determined by the value of q, and many microscopic states given by the number of ways to distribute the q quanta among the N harmonic oscillators. If we use Eqs. (2.8) and (2.36), the entropy of the Einstein solid is given by S(N, q) = kB ln [ (N + q − 1)! q!(N − 1)! ] ≈ kB ln [ (N + q) N+q NN qq ] , (2.37) where we have assumed that N and q are large and we have used Stirling’s approximation Eq. (2.15). We can now use thermodynamics to relate the energy to the temperature via the relation ( 𝜕S 𝜕E ) N = 1 ℏω (𝜕S 𝜕q ) N = 1 T , (2.38) where, in the middle term, we have used the relation between E and q given in Eq. (2.35). If we take the derivative of the entropy, as indicated in Eq. (2.38), and
2 Complexity and Entropysolvefor the energy, we obtainNhoe-Bha1(2.39)E(N,T)=Nhw+(1 - e-βha)2where β = 1/(kg T). As we shall see in the chapter on thermodynamics, the heatcapacityof theEinstein solid canbewrittenN(ho)e-BhaECN =(2.40)kg T2(1 - e-βha)2aIn the limitT-→O, the heatcapacity of the Einstein solidgoes to zero exponen-tially as a function of temperature T.This result provided an important break-through in our understanding of the effect ofquantummechanics on the thermalproperties ofsolids atlowtemperature.Classical physics couldnotexplain the ex-perimentally observedfact that the heat capacity of solids do tend to zero as thetemperature tends to O K.The heat capacitygiven by the Einstein solid is not quitecorrect. The heat capacity of real solids goes to zero as T3, not exponentially.Thereasonfor this will become clear when weconsider a Debye solid,which allowslatticesitestobecoupled2.8MultiplicityandEntropyofan IdealGasWe now consider an ideal gas of N particles in a box of volume V =L3.State-countinginthissystemisdifferentfromthespinsystemandEinsteinsolid be-cause the particles in the gas movefreelythroughthebox. If the gas particles haveno distinguishing internal characteristics such as different mass or different spin,theyareindistinguishable.For thespinsystem and the Einstein solid,particles areattachedtospecific latticesites and remainthere,sotheyaredistinguishableanddistinct.2.8.1Multiplicityofan ldealGasWe wantto determinethe multiplicityof states available to an ideal gas ofNparticles. By ideal gas we mean the real gas density is low enough that the energy ofinteraction betweentheparticles is negligible.Thekeytoobtaininga countableset ofmicroscopic states is to remember that thissystem is intrinsicallyquantummechanical and that the quantum states available to each particle occupy a finitesizeregion of phase space whose size is determined by Planck's constant,h. Sincethe particles move in three dimensional configuration space, the number of de-grees offreedomofthe system is D =3N and the momentum of each particle has
20 2 Complexity and Entropy solve for the energy, we obtain E(N, T ) = 1 2 Nℏω + Nℏωe−βℏω (1 − e−βℏω) , (2.39) where β = 1∕(kBT). As we shall see in the chapter on thermodynamics, the heat capacity of the Einstein solid can be written CN = ( 𝜕E 𝜕T ) N = N(ℏω) 2e−βℏω kBT2(1 − e−βℏω)2 . (2.40) In the limit T → 0, the heat capacity of the Einstein solid goes to zero exponentially as a function of temperature T. This result provided an important breakthrough in our understanding of the effect of quantum mechanics on the thermal properties of solids at low temperature. Classical physics could not explain the experimentally observed fact that the heat capacity of solids do tend to zero as the temperature tends to 0 K. The heat capacity given by the Einstein solid is not quite correct. The heat capacity of real solids goes to zero as T3, not exponentially. The reason for this will become clear when we consider a Debye solid, which allows lattice sites to be coupled. 2.8 Multiplicity and Entropy of an Ideal Gas We now consider an ideal gas of N particles in a box of volume V = L3. Statecounting in this system is different from the spin system and Einstein solid because the particles in the gas move freely through the box. If the gas particles have no distinguishing internal characteristics such as different mass or different spin, they are indistinguishable. For the spin system and the Einstein solid, particles are attached to specific lattice sites and remain there, so they are distinguishable and distinct. 2.8.1 Multiplicity of an Ideal Gas We want to determine the multiplicity of states available to an ideal gas of N particles. By ideal gas we mean the real gas density is low enough that the energy of interaction between the particles is negligible. The key to obtaining a countable set of microscopic states is to remember that this system is intrinsically quantum mechanical and that the quantum states available to each particle occupy a finite size region of phase space whose size is determined by Planck’s constant, h. Since the particles move in three dimensional configuration space, the number of degrees of freedom of the system is D = 3N and the momentum of each particle has
2.8 Multiplicity and Entropy of an ldeal Gasthree independent components. The total energy of the gas can be written3WP(2.41)E=22miwherethesum isoverall 3N components ofmomentum,p,isoneofthe3N com-ponentsofmomentum,and m is themass ofeachparticle.The volume of phase space available to a single quantum state of the gas isApiAxi×-.. ×p3nx3n=h3N,whereh is Planck's constant.Thevolumeof phase space Q(D)(E) that has energy less than E can be written Q(D)(E) =vN(D'(E),where we have explicitly separated the spatial volume V from thevolume of the momentum coordinates.This is possible because there are noexternal fields present and interactions between the particles are neglected.It is useful to begin witha very simple case.Let us first obtain the phase spacevolumeavailabletooneparticle (N=1)in twodimensional configuration spaceandD=2degrees offreedom.TheenergyoftheparticleisE=1/(2m)(p+p2)Therefore,we canwritep?+p?=2mE=R2,whereRisthe radius of theallowedcircle in momentum space.The area (volume) within the circle in momentumspace is Q(2(E) = (V2mE)2 = /2mE. Thus, the volume of phase space withenergylessthanorequaltoEisQ(2)(E)=L2mE.In a similarmanner, we candetermine the volume of phase space Q(D)(E)=vN o(D)(E)that has energyless than or equal to Efor the case of Nparticles inabox ofvolumeV.ForD=3N degrees offreedom,Eq.(2.41)can bewritten intheform p + ..+ pp = 2mE = R2. The volume in momentum space with energy lessthan orequal toEcanbewritten in theform(E)=ARD=Ap(R2)P/2,whereA, is an unknown constant. It is useful to write (d(D)(E)/(dR2)=ApD/2RD-2and,therefore,2dop(E)-R=Ab号r()dR?(2.42)dR2wherer(x)istheGammafunction.To determine the constant Ap,compute p(E) in another way.Note thatQ(D)(E)dppo(R? - p -.- p),(2.43)where (R?-p-.- p) is the Heaviside function. (Heaviside functions havethepropertythat(x)=0ifx<0,(x)=1ifx>0,and @(x)=1/2ifx=0.)Becausethederivative of a Heaviside function isa Dirac delta function, (x)=d(x)/dx,we canwritedo(D)(E)dpi".dppo(R2-p-...-p)(2.44)dR2
2.8 Multiplicity and Entropy of an Ideal Gas 21 three independent components. The total energy of the gas can be written E = ∑3N j=1 p2 j 2m , (2.41) where the sum is over all 3N components of momentum, pj is one of the 3N components of momentum, and m is the mass of each particle. The volume of phase space available to a single quantum state of the gas is Δp1Δx1 × ⋯ × Δp3N Δx3N = h3N , where h is Planck’s constant. The volume of phase space Ω(D) (E) that has energy less than E can be written Ω(D) (E) = V N Ω(D) p (E), where we have explicitly separated the spatial volume V from the volume of the momentum coordinates. This is possible because there are no external fields present and interactions between the particles are neglected. It is useful to begin with a very simple case. Let us first obtain the phase space volume available to one particle (N = 1) in two dimensional configuration space and D = 2 degrees of freedom. The energy of the particle is E = 1∕(2m)(p2 x + p2 y ). Therefore, we can write p2 x + p2 y = 2mE = R2, where R is the radius of the allowed circle in momentum space. The area (volume) within the circle in momentum space is Ω(2) p (E) = π( √ 2mE) 2 = π2mE. Thus, the volume of phase space with energy less than or equal to E is Ω(2) (E) = L2π2mE. In a similar manner, we can determine the volume of phase space Ω(D) (E) = V N Ω(D) p (E) that has energy less than or equal to E for the case of N particles in a box of volume V. For D = 3N degrees of freedom, Eq. (2.41) can be written in the form p2 1 +⋯+ p2 D = 2mE = R2. The volume in momentum space with energy less than or equal to E can be written in the form ΩD p (E) = ADRD = AD(R2) D∕2, where AD is an unknown constant. It is useful to write (dΩ(D) p (E))∕(dR2) = ADD∕2RD−2 and, therefore, ∞ ∫ 0 dR2 dΩ(D) p (E) dR2 e−R2 = AD D 2 Γ ( D 2 ) , (2.42) where Γ(x) is the Gamma function. To determine the constant AD, compute ΩD p (E) in another way. Note that Ω(D) p (E) = ∞ ∫ −∞ d p1 ⋯ ∞ ∫ −∞ d pDΘ ( R2 − p2 1 − ⋯ − p2 D ) , (2.43) where Θ(R2 − p2 1 − ⋯ − p2 D) is the Heaviside function. (Heaviside functions have the property that Θ(x) = 0 if x < 0, Θ(x) = 1 if x > 0, and Θ(x) = 1∕2 if x = 0.) Because the derivative of a Heaviside function is a Dirac delta function, δ(x) = dΘ(x)∕dx, we can write dΩ(D) p (E) dR2 = ∞ ∫ −∞ d p1 ⋯ ∞ ∫ −∞ d pDδ ( R2 − p2 1 − ⋯ − p2 D ) . (2.44)
2Complexityand EntropyNow note thatdOD(Edpe-(p++P++P)=元D/2(2.45)dR2If we equateEqs. (2.42)and (2.45),we obtain Ap=2D/2 /(Dr(D/2)Using the above results we find that,for an ideal gas with D=3N degrees offreedom,thetotalvolume ofphasespace withenergyequalto orlessthanenergyEisQ(D)(E)= VNo(P)(E) = VNAb(R3)P/ _ (2mE)W/(2.46)(3N/2)!where we have assumed that 3N is an even integer and the Gamma function canbe written I(n +1)=n!.WenowcandeterminethemultiplicityNr(E)ofthemicrostatesoftheideal gaswith energy less than or equal to E. We divide the phase space volume Q(D)(E) bythevolume h3N ofa single quantum state ofthe Nparticle gas.Since theparticlesare indistinguishable, wemust also divideby N!to avoid over counting states.Wethen obtainVN(2mE)3N/2NN(E)=(2.47)N!h3N(3N/2)!As we shall seelater, thefactor N! is essential to obtain the correct equation ofstate foran ideal classical gas of indistinguishable particles.2.8.2Entropyof an IdealGasThe entropy of an ideal gas is determined by the number of microscopic stateswith energyEand not the number of microscopic states with energyless thanor equal to E, which was obtained in Eq.(2.47).However,as wewill now show,in the limit N -→+ co, these two numbers give values of the entropy that are thesame, to good approximation. Let us first divide phase space into a sequence ofenergy shells, each of width E.The phase space volume ofthe shell at energy Eis e(E).We can then writea sequence of inequalities between the sizes ofthesevariousvolumessuchthatQAE(E)<Q(D)(E)< (E/AE)QAE(E)(2.48)Next notethat In(AE(E)~D, In(Q(D)(E)~D and InE~InD.Therefore,foraverylargenumberofdegreesoffreedom(1023),wecanassumethatIn(Ae(E)In(Q(D)(E) and the multiplicity N(E), derived in Eq. (2.47), can be used to ob-tain the entropy ofan ideal gas.Theentropyof an ideal gas cannowbewrittenVN(2mE)3N/(2.49)S = kg In(NN(E)) = kg InN!h3N(3N/2)!
22 2 Complexity and Entropy Now note that ∞ ∫ 0 dR2 dΩ(D) p (E) dR2 e−R2 = ∞ ∫ −∞ d p1 ⋯ ∞ ∫ −∞ d pDe−(p2 1+p2 2+⋯+p2 D) = πD∕2 . (2.45) If we equate Eqs. (2.42) and (2.45), we obtain AD = 2πD∕2∕(DΓ(D∕2)). Using the above results we find that, for an ideal gas with D = 3N degrees of freedom, the total volume of phase space with energy equal to or less than energy E is Ω(D) (E) = V N Ω(D) p (E) = V N AD(R2) D∕2 = V N (2πmE) 3N∕2 (3N∕2)! , (2.46) where we have assumed that 3N is an even integer and the Gamma function can be written Γ(n + 1) = n!. We now can determine the multiplicity N(E) of the microstates of the ideal gas with energy less than or equal to E. We divide the phase space volume Ω(D) (E) by the volume h3N of a single quantum state of the N particle gas. Since the particles are indistinguishable, we must also divide by N! to avoid over counting states. We then obtain N (E) = V N (2πmE) 3N∕2 N!h3N (3N∕2)! . (2.47) As we shall see later, the factor N! is essential to obtain the correct equation of state for an ideal classical gas of indistinguishable particles. 2.8.2 Entropy of an Ideal Gas The entropy of an ideal gas is determined by the number of microscopic states with energy E and not the number of microscopic states with energy less than or equal to E, which was obtained in Eq. (2.47). However, as we will now show, in the limit N → ∞, these two numbers give values of the entropy that are the same, to good approximation. Let us first divide phase space into a sequence of energy shells, each of width ΔE. The phase space volume of the shell at energy E is ΩΔE(E). We can then write a sequence of inequalities between the sizes of these various volumes such that ΩΔE(E) < Ω(D) (E) < (E∕ΔE)ΩΔE (E) . (2.48) Next note that ln(ΩΔE(E)) ∼ D, ln(Ω(D) (E)) ∼ D and ln E ∼ ln D. Therefore, for a very large number of degrees of freedom (1023), we can assume that ln(ΩΔE(E)) ≈ ln(Ω(D) (E)) and the multiplicity N (E), derived in Eq. (2.47), can be used to obtain the entropy of an ideal gas. The entropy of an ideal gas can now be written S = kB ln(N (E)) = kB ln [ V N (2πmE) 3N∕2 N!h3N (3N∕2)! ] . (2.49)