Introduction to Nonparametric Analysisin Time Series EconometricsYongmiao Hong2020一
Introduction to Nonparametric Analysis in Time Series Econometrics Yongmiao Hong 2020 1
This is Chapter 6 of a manuscript entitled as Modern Time Series Analysis: Theoryand Applications written by theauthor.We will introduce some popular nonparametricmethods,particularly thekernel smoothing method and the local polynomial smoothingmethod, to estimate functions of interest in time series contexts, such as probabilitydensity functions, autoregression functions, spectral density functions, and generalizedspectral density functions. Empirical applications of these functions crucially depend onthe consistent estimation of these functions. We will discuss the large sample statisticalproperties of nonparametric estimators in various contexts.Key words:Asymptotic normality, bias, boundary problem, consistency, curse of di-mensionality,density function,generalized spectral density,global smoothing, integratedmean squared error, law of large numbers, local polynomial smoothing, local smoothing,locally stationary time series model, mean squared error, kernel method, regression func-tion, series approximation, smoothing, spectral density function, Taylor series expansion,variance.Reading Materials and ReferencesThis lecture note is self-contained. However, the following references will be usefulforlearningnonparametricanalysis.(1) Nonparametric Analysis in Time Domain. Silverman, B. (1986): Nonparametric Density Estimation and Data Analysis. Chap-man and Hall: London.·Hardle, W. (1990): Applied Nonparametric Regression. Cambridge UniversityPress: Cambridge. Fan, J. and Q. Yao (2003), Nonlinear Time Series: Parametric and NonparametricMethods, Springer: New York.(2) Nonparametric Methods in Frequency Domain.Priestley,M.(1981),Spectral Analysis and Time Series.Academic Press:NewYork. Hannan, E. (1970), Multiple Time Series, John Wiley: New York.2
This is Chapter 6 of a manuscript entitled as Modern Time Series Analysis: Theory and Applications written by the author. We will introduce some popular nonparametric methods, particularly the kernel smoothing method and the local polynomial smoothing method, to estimate functions of interest in time series contexts, such as probability density functions, autoregression functions, spectral density functions, and generalized spectral density functions. Empirical applications of these functions crucially depend on the consistent estimation of these functions. We will discuss the large sample statistical properties of nonparametric estimators in various contexts. Key words: Asymptotic normality, bias, boundary problem, consistency, curse of dimensionality, density function, generalized spectral density, global smoothing, integrated mean squared error, law of large numbers, local polynomial smoothing, local smoothing, locally stationary time series model, mean squared error, kernel method, regression function, series approximation, smoothing, spectral density function, Taylor series expansion, variance. Reading Materials and References This lecture note is self-contained. However, the following references will be useful for learning nonparametric analysis. (1) Nonparametric Analysis in Time Domain Silverman, B. (1986): Nonparametric Density Estimation and Data Analysis. Chapman and Hall: London. H‰rdle, W. (1990): Applied Nonparametric Regression. Cambridge University Press: Cambridge. Fan, J. and Q. Yao (2003), Nonlinear Time Series: Parametric and Nonparametric Methods, Springer: New York. (2) Nonparametric Methods in Frequency Domain Priestley, M. (1981), Spectral Analysis and Time Series. Academic Press: New York. Hannan, E. (1970), Multiple Time Series, John Wiley: New York. 2
1 MotivationSuppose [xt] is a strictly stationary process with marginal probability density func-tion g(r) and pairwise joint probability density function f;(r, y), and a random sample[Xt]T-, of size T is observed. Then,. How to estimate the marginal pdf g(r) of [Xt)?. How to estimate the pairwise joint pdf fi(r, y) of (Xt, Xt-j)?. How to estimate the autoregression function r;(r) = E(Xt/Xt-j = r)?. How to estimate the spectral density h(w) of [Xt]?. How to estimate the generalized spectral density f(w, u, ) of [xt]?. How to estimate the bispectral density b(wi, w2)?.Howto estimateanonlinearautoregressiveconditional heteroskedasticmodelXt = μ(Xt-1, ., Xt-p) + o(Xt-1,., Xt-g)et,[et) ~ i.i.d.(0, 1),where μ() and o() are unknown functions of the past information. Under certainregularity conditions, μ() is the conditional mean of Xt given It-1 = {Xt-1, Xt-2, ..and o2() is the conditional variance of Xt given It-1..How to estimate a semi-nonparametric functional coefficient autoregressive processPXt=Ea;(Xt-d)Xt-j+Et,E(et/It-1) = 0 a.s.,j=1where a;()is unknown,and d>0 is a time lag parameter?. How to estimate a nonparametric additive autoregressive processPXt = Zu;(Xt-i)+et,E(et/It-1) = 0 a.s.,j=1where the μ,() functions are unknown?· How to estimate a locally linear time-varying regression modelYt=XB(t/T)+Etwhereβ()isan unknown smooth deterministicfunction of time?3
1 Motivation Suppose fXtg is a strictly stationary process with marginal probability density function g(x) and pairwise joint probability density function fj (x; y); and a random sample fXtg T t=1 of size T is observed. Then, How to estimate the marginal pdf g(x) of fXtg? How to estimate the pairwise joint pdf fj (x; y) of (Xt ; Xtj )? How to estimate the autoregression function rj (x) = E(Xt jXtj = x)? How to estimate the spectral density h(!) of fXtg? How to estimate the generalized spectral density f(!; u; v) of fXtg? How to estimate the bispectral density b(!1; !2)? How to estimate a nonlinear autoregressive conditional heteroskedastic model Xt = (Xt1; :::; Xtp) + (Xt1; :::; Xtq)"t ; f"tg i:i:d:(0; 1); where () and () are unknown functions of the past information. Under certain regularity conditions, () is the conditional mean of Xt given It1 = fXt1; Xt2; :::g and 2 () is the conditional variance of Xt given It1. How to estimate a semi-nonparametric functional coe¢ cient autoregressive process Xt = X p j=1 j (Xtd)Xtj + "t ; E("t jIt1) = 0 a.s., where j () is unknown, and d > 0 is a time lag parameter? How to estimate a nonparametric additive autoregressive process Xt = X p j=1 j (Xtj ) + "t ; E("t jIt1) = 0 a.s., where the j () functions are unknown? How to estimate a locally linear time-varying regression model Yt = X 0 t (t=T) + "t ; where () is an unknown smooth deterministic function of time? 3
How to use these estimators in economic and financial applications?Nonparametric estimation is often called nonparametric smoothing, since a keyparameter called smoothing parameter is used to control the degree of the estimatedcurve. Nonparametric smoothing first arose from spectral density estimation in timeseries analysis. In a discussion of the seminal paper by Bartlett (1946), Henry Danielssuggested that a possible improvement on spectral density estimation could be madeby smoothing the periodogram (see Chapter 3), which is the squared discrete Fouriertransform of the random sample [X,iT-,.The theory and techniques were then system-atically developed by Bartlett (1948,1950).Thus, smoothing techniques were alreadyprominently featured in time series analysis more than 70 years ago.In the earlier stage of nonlinear time series analysis (see Tong (1990)), the focus wason various nonlinear parametric forms, such as threshold autoregressive models, smoothtransition autoregressive models, and Regime-switch Markov chain autoregressive mod-els (see Chapter 8 for details). Recent interest has been mainly in nonparametric curveestimation, which does not require the knowledge of the functional form beyond certainsmoothness conditions on the underlying function of interest.Question: Why is nonparametric smoothing popular in statistics and econometrics?There are several reasons for the popularity of nonparametric analysis. In particular.three main reasons are:. Demands for nonlinear approaches;.Availability of large data sets;.Advance in computer technologyIndeed, as Granger (1999) points out, the speed in computing technology increasesmuch faster than the speed at which data grows.To obtain basic ideas about nonparametric smoothing methods, we now consider twoexamples, one is the estimation of a regression function, and the other is the estimationof a probability density function.4
How to use these estimators in economic and Önancial applications? Nonparametric estimation is often called nonparametric smoothing, since a key parameter called smoothing parameter is used to control the degree of the estimated curve. Nonparametric smoothing Örst arose from spectral density estimation in time series analysis. In a discussion of the seminal paper by Bartlett (1946), Henry Daniels suggested that a possible improvement on spectral density estimation could be made by smoothing the periodogram (see Chapter 3), which is the squared discrete Fourier transform of the random sample fXtg T t=1. The theory and techniques were then systematically developed by Bartlett (1948,1950). Thus, smoothing techniques were already prominently featured in time series analysis more than 70 years ago. In the earlier stage of nonlinear time series analysis (see Tong (1990)), the focus was on various nonlinear parametric forms, such as threshold autoregressive models, smooth transition autoregressive models, and Regime-switch Markov chain autoregressive models (see Chapter 8 for details). Recent interest has been mainly in nonparametric curve estimation, which does not require the knowledge of the functional form beyond certain smoothness conditions on the underlying function of interest. Question: Why is nonparametric smoothing popular in statistics and econometrics? There are several reasons for the popularity of nonparametric analysis. In particular, three main reasons are: Demands for nonlinear approaches; Availability of large data sets; Advance in computer technology. Indeed, as Granger (1999) points out, the speed in computing technology increases much faster than the speed at which data grows. To obtain basic ideas about nonparametric smoothing methods, we now consider two examples, one is the estimation of a regression function, and the other is the estimation of a probability density function. 4
Example 1 [Regression Function]: Consider the first order autoregression functionri(r)= E(XtXt-1 = r).We can writeXt= ri(Xt-1) +Et,where E(et/Xt-1) = 0 by construction. We assume E(X?) < oo.Suppose a sequence of bases [b;(r)) constitutes a complete orthonormal basis forthe space of square-integrable functions. Then we can always decompose the functionri(a) =aj;(r),j=0wheretheFouriercoefficientri(r)p,;(r)dr,which is the projection of ri(r) on the base ;(r)Suppose there is a quadratic function ri() = r2 for r E [-元, ]. Then元2cos(2r)cos(3r)cos(r) -ri(c)3223280元2(-1)-1 cos(jr)4 >3j2j=1For another example, suppose the regression function is a step function, namely-1if-π<<0,ri(c) =o ifr=0,1if0<<元.Then we can still expand it as an infinite sum of periodic series.sin(3c)4sin(5r)sin(r) + ri(r) =35T4sin[(2j + 1)a]>(2j + 1)元j=05
Example 1 [Regression Function]: Consider the Örst order autoregression function r1(x) = E(Xt jXt1 = x): We can write Xt = r1(Xt1) + "t ; where E("t jXt1) = 0 by construction. We assume E(X2 t ) < 1: Suppose a sequence of bases f j (x)g constitutes a complete orthonormal basis for the space of square-integrable functions. Then we can always decompose the function r1(x) = X1 j=0 j j (x); where the Fourier coe¢ cient j = Z 1 1 r1(x) j (x)dx; which is the projection of r1(x) on the base j (x): Suppose there is a quadratic function r1(x) = x 2 for x 2 [; ]: Then r1(x) = 2 3 4 cos(x) cos(2x) 2 2 + cos(3x) 3 2 = 2 3 4 X1 j=1 (1)j1 cos(jx) j 2 : For another example, suppose the regression function is a step function, namely r1(x) = 8 >>< >>: 1 if < x < 0; 0 if x = 0; 1 if 0 < x < : Then we can still expand it as an inÖnite sum of periodic series, r1(x) = 4 sin(x) + sin(3x) 3 + sin(5x) 5 + = 4 X1 j=0 sin[(2j + 1)x] (2j + 1) : 5