H. Barrow and J. Tenenbaum, "Computational vision, Proc. IEEE, vol. 69, Pp. 572-595, May 1981 A Gersho and R. M. Gray, vector Quantization and Signal Compression, Norwell, Mass. Kluwer Academic Publishers, 1991 R C. Gonzalez and P. Wintz, Digital Image Processing, Reading, Mass. Addison-Wesley, 1991 G T. Herman, Image Reconstruction from Projections, New York: Springer-Verlag, 1979 uence Analysis, New York ringer-Verlag, 1981 A. K Jain, Fundamentals of Digital Image Processing, Englewood Cliffs, N.J. Prentice-Hall, 1989. A Kak and M. Slaney, Principles of Computerized Tomographic Imaging, New York: IEEE Press, 1988 A. Macovski, Medical Imaging Systems, Englewood Cliffs, N]: Prentice-Hall, 1983 M. D. McFarlane,"Digital pictures fifty years ago, " Proc. IEEE, Pp 768-770, July 1972. w. K. Pratt, Digital Image Processing, New York: Wiley, 1991 A. Rosenfeld and A. Kak, Digital Picture Processing, vols. I and 2, San Diego: Academic Press, 1982. J. Serra, Image Analysis and Mathematical Morphology, vols. I and 2, San Diego: Academic Press, 1982 and 1988 Further Information A number of textbooks are available that cover the broad area of image processing and several that focus on more specialized topics within this field. The texts by gonzalez and wintz[ 1991], Jain [1989], Pratt[1991], and Rosenfeld and Kak(vol. 1)[1982] are quite broad in their scope. Gonzalez and wintz's treatment is written t a somewhat lower level than that of the other texts. For a more detailed treatment of computed tomography and other medical imaging modalities, the reader may consult the texts by Herman [1979], Macovski [1983] and Kak and Slaney [1988]. To explore the field of computer vision, the reader is advised to consult the text by Ballard and Brown [1982]. Current research and applic of journals. Of particular note are the IEEE Transactions on Image Processing the IEEE Transactions on Pattern talysis and Machine Intelligence; the IEEE Transactions on Geoscience and Remote Sensing; the IEEE Transactions on Medical Imaging; the Journal of the Optical Society of America, A: Optical Engineering; the Journal of Electronic Imaging; and Computer Vision, Graphics, and Image Processing 17.2 Video Signal Processing Sarah a. rajala Video signal processing is the area of specialization concerned with the processing of time sequences of image data, i. e, video. Because of the significant es in computing power and increases in available transmission bandwidth, there has been a proliferation of potential applications in the area of video signal processing. Applications such as high-definition television, digital video, multimedia, video phone, interactive video, medical imaging, and information processing are the driving forces in the field today. As diverse as the applications may seem, it is possible to specify a set of fundamental principles and methods that can be used to develop the applications. Considerable understanding of a video signal processing system can be gained by representing the system with the block diagram given in Fig. 17.10. Light from a real-world scene is captured by a scanning system and causes an image frame f(x,) to) to be formed on a focal plane. A video signal is a sequence of image frames that are created when a scanning system captures a new image frame at periodic intervals in time. In general, each frame of the video sequence is a function of two spatial variables x and y and one temporal variable t An integral part of the scanning system is the process of converting the original analog signal into an appropriate digital representation. The conversion process includes the operations of sampling and quantization Sampling Processing Display FIGURE 17 10 Video signal processing system block diagram c2000 by CRC Press LLC
© 2000 by CRC Press LLC H. Barrow and J. Tenenbaum, “Computational vision,’’ Proc. IEEE, vol. 69, pp. 572–595, May 1981. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Norwell, Mass.: Kluwer Academic Publishers, 1991. R. C. Gonzalez and P. Wintz, Digital Image Processing, Reading, Mass.: Addison-Wesley, 1991. G.T. Herman, Image Reconstruction from Projections, New York: Springer-Verlag, 1979. T. S. Huang, Image Sequence Analysis, New York: Springer-Verlag, 1981. A. K. Jain, Fundamentals of Digital Image Processing, Englewood Cliffs, N.J.: Prentice-Hall, 1989. A. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, New York: IEEE Press, 1988. A. Macovski, Medical Imaging Systems, Englewood Cliffs, N.J.: Prentice-Hall, 1983. M. D. McFarlane, “Digital pictures fifty years ago,’’ Proc. IEEE, pp. 768–770, July 1972. W. K. Pratt, Digital Image Processing, New York: Wiley, 1991. A. Rosenfeld and A. Kak, Digital Picture Processing, vols. 1 and 2, San Diego: Academic Press, 1982. J. Serra, Image Analysis and Mathematical Morphology, vols. 1 and 2, San Diego: Academic Press, 1982 and 1988. Further Information A number of textbooks are available that cover the broad area of image processing and several that focus on more specialized topics within this field. The texts by Gonzalez and Wintz [1991], Jain [1989], Pratt [1991], and Rosenfeld and Kak (Vol. 1) [1982] are quite broad in their scope. Gonzalez and Wintz’s treatment is written at a somewhat lower level than that of the other texts. For a more detailed treatment of computed tomography and other medical imaging modalities, the reader may consult the texts by Herman [1979], Macovski [1983], and Kak and Slaney [1988]. To explore the field of computer vision, the reader is advised to consult the text by Ballard and Brown [1982]. Current research and applications of image processing are reported in a number of journals. Of particular note are the IEEE Transactions on Image Processing; the IEEE Transactions on Pattern Analysis and Machine Intelligence; the IEEE Transactions on Geoscience and Remote Sensing; the IEEE Transactions on Medical Imaging; the Journal of the Optical Society of America, A: Optical Engineering; the Journal of Electronic Imaging; and Computer Vision, Graphics, and Image Processing. 17.2 Video Signal Processing Sarah A. Rajala Video signal processing is the area of specialization concerned with the processing of time sequences of image data, i.e., video. Because of the significant advances in computing power and increases in available transmission bandwidth, there has been a proliferation of potential applications in the area of video signal processing. Applications such as high-definition television, digital video, multimedia, video phone, interactive video, medical imaging, and information processing are the driving forces in the field today. As diverse as the applications may seem, it is possible to specify a set of fundamental principles and methods that can be used to develop the applications. Considerable understanding of a video signal processing system can be gained by representing the system with the block diagram given in Fig. 17.10. Light from a real-world scene is captured by a scanning system and causes an image frame f (x,y,t0) to be formed on a focal plane. A video signal is a sequence of image frames that are created when a scanning system captures a new image frame at periodic intervals in time. In general, each frame of the video sequence is a function of two spatial variables x and y and one temporal variable t. An integral part of the scanning system is the process of converting the original analog signal into an appropriate digital representation. The conversion process includes the operations of sampling and quantization. Sampling FIGURE 17.10 Video signal processing system block diagram
is the process of converting a continuous-time/space signal into a discrete-time/space signal. Quantization is the process of converting a continuous-valued signal into a discrete-valued signal Once the video signal has been sampled and quantized, it can be processed digitally. Processing can be performed on special-purpose hardware or general-purpose computers. The type of processing performed depends on the particular application. For example, if the objective is to generate high-definition television, the processing would typically include compression and motion estimation. In fact, in most of the applications sted above these are the fundamental operations. Compression is the process of compactly representing the information contained in an image or video signal. Motion estimation is the process of estimating the dis- placement of the moving objects in a video sequence. The displacement information can then be used to interpolate missing frame data or to improve the performance of compression algorithms fter the processing is complete, a video signal is ready for transmission over some channel or storage on ome medium. If the signal is transmitted, the type of channel will vary depending on the application. For example, today analog television signals are transmitted one of three ways: via satellite, terrestrially, or by cable. All three channels have limited transmission bandwidths and can adversely affect the signals because of the imperfect frequency responses of the channels. Alternatively, with a digital channel, the primary limitation will be the bandwidth The final stage of the block diagram shown in Fig. 17.10 is the display. Of critical importance at this stage is the human observer. Understanding how humans respond to visual stimuli, i.e., the psychophysics of vision, will not only allow for better evaluation of the processed video signals but will also permit the design of better systems. Sampling If a continuous-time video signal satisfies certain conditions, it can be exactly represented by and be recon- structed from its sample values. The conditions which must be satisfied are specified in the sampling theorem The sampling theorem can be stated as follows: Sampling Theoren Let f(x,yr) be a bandlimited signal with Ro,, o, )=0 for o_>om,o, >o, m, and o>o f(x,rn) is uniquely determined by its samples f(jXs, kYs, ITS )=f(jk, D), where j, k, 1=0,+1, #2, uii.Then Osx>20M,Os> 20,, and Os>20M Osx=2/Xs, O5= 2T/Ys, and @t=2T/Ts Xs is the sampling period along the x direction, @=2/Xs is the spatial sampling frequency along the x direction, Ys is the sampling period along the y direction, @, =2/Ys is the spatial sampling frequency along the y direction, Ts is the sampling period along the temporal direction, and o,= 2m/Ts is the temporal Given these samples, f(x,)t) can be reconstructed by generating a periodic impulse train in which suc- cessive impulses have amplitudes that are successive sample values. This impulse train is then processed through an ideal low-pass filter with appropriate gain and cut-off frequencies. The resulting output signal will be exactly equal to f(x,y n. Source: Oppenheim et al., 1983, p. 519.) If the sampling theorem is not satisfied, aliasing will occur. Aliasing occurs when the signal is undersampled and therefore no longer recoverable by low-pass filtering. Figure 17. 11(a)shows the frequency spectrum of a sampled bandlimited signal with no aliasing. Figure 17. 11(b)shows the frequency response of the same signal with aliasing. The aliasing occurs at the points where there is overlap in the diamond-shaped regions. For video signals aliasing in the temporal direction will give rise to flicker on the display. For television systems, the standard temporal sampling rate is 30 frames per second in the United States and Japan and 25 frames per cond in Europe. However, these rates would be insufficient without the use of interlacing If the sampling rate(spatial and/or temporal)of a system is fixed, a standard approach for minimizing the effects of aliasing for signals that do not satisfy the sampling theorem is to use a presampling filter Presampling e 2000 by CRC Press LLC
© 2000 by CRC Press LLC is the process of converting a continuous-time/space signal into a discrete-time/space signal. Quantization is the process of converting a continuous-valued signal into a discrete-valued signal. Once the video signal has been sampled and quantized, it can be processed digitally. Processing can be performed on special-purpose hardware or general-purpose computers. The type of processing performed depends on the particular application. For example, if the objective is to generate high-definition television, the processing would typically include compression and motion estimation. In fact, in most of the applications listed above these are the fundamental operations. Compression is the process of compactly representing the information contained in an image or video signal. Motion estimation is the process of estimating the displacement of the moving objects in a video sequence. The displacement information can then be used to interpolate missing frame data or to improve the performance of compression algorithms. After the processing is complete, a video signal is ready for transmission over some channel or storage on some medium. If the signal is transmitted, the type of channel will vary depending on the application. For example, today analog television signals are transmitted one of three ways: via satellite, terrestrially, or by cable. All three channels have limited transmission bandwidths and can adversely affect the signals because of the imperfect frequency responses of the channels. Alternatively, with a digital channel, the primary limitation will be the bandwidth. The final stage of the block diagram shown in Fig. 17.10 is the display. Of critical importance at this stage is the human observer. Understanding how humans respond to visual stimuli, i.e., the psychophysics of vision, will not only allow for better evaluation of the processed video signals but will also permit the design of better systems. Sampling If a continuous-time video signal satisfies certain conditions, it can be exactly represented by and be reconstructed from its sample values. The conditions which must be satisfied are specified in the sampling theorem. The sampling theorem can be stated as follows: Sampling Theorem: Let f(x,y,t) be a bandlimited signal with F(wx,wy,wt) = 0 for *wx * > wxM, *wy * > wyM, and *wt* > wtM. Then f(x,y,t) is uniquely determined by its samples f( jXS,kYS ,lTS ) = f(j,k,l), where j,k,l = 0, ±1, ±2, ... if wsx > 2wx M , wsy > 2wy M , and wst > 2wtM and wsx = 2p/XS , wsy = 2p/YS , and wst = 2p/TS XS is the sampling period along the x direction, wx = 2p/XS is the spatial sampling frequency along the x direction, YS is the sampling period along the y direction, wy = 2p/YS is the spatial sampling frequency along the y direction, TS is the sampling period along the temporal direction, and wt = 2p/TS is the temporal sampling frequency. Given these samples, f(x,y,t) can be reconstructed by generating a periodic impulse train in which successive impulses have amplitudes that are successive sample values. This impulse train is then processed through an ideal low-pass filter with appropriate gain and cut-off frequencies. The resulting output signal will be exactly equal to f(x,y,t). (Source: Oppenheim et al., 1983, p. 519.) If the sampling theorem is not satisfied, aliasing will occur. Aliasing occurs when the signal is undersampled and therefore no longer recoverable by low-pass filtering. Figure 17.11(a) shows the frequency spectrum of a sampled bandlimited signal with no aliasing. Figure 17.11(b) shows the frequency response of the same signal with aliasing. The aliasing occurs at the points where there is overlap in the diamond-shaped regions. For video signals aliasing in the temporal direction will give rise to flicker on the display. For television systems, the standard temporal sampling rate is 30 frames per second in the United States and Japan and 25 frames per second in Europe. However, these rates would be insufficient without the use of interlacing. If the sampling rate (spatial and/or temporal) of a system is fixed, a standard approach for minimizing the effects of aliasing for signals that do not satisfy the sampling theorem is to use a presampling filter. Presampling