Week 1 1 1
M: max ∑ 𝑞𝑡 . Z: ∑ 𝛻 𝑞𝑡 = 0. ▪ Requires ONLY: E&M (1.1) + Uniform
𝑇 𝑇
Likelihood: 𝑝 Convergence (1.2).
OLS: 𝜃̂ → 𝜃0 consistent ⇔ 𝐸(𝜀𝑡 |𝑥𝑡 ) = 0. Valid IV 𝑧𝑡 : 𝐶𝑜𝑣(𝑥𝑡 , 𝑧𝑡 ) ≠ 0
𝜕𝜖𝑡 ▪ NO need for Identifiable Uniqueness (1.3)!
The process: 𝑥𝑡 = ⋯ − 𝜖𝑡 → 𝜖𝑡 = 𝑥𝑡 − ⋯ → 𝐽 = | | = ⋯ → 𝑓(𝜖) =
𝜕𝑥𝑡 (Relevant) AND 𝐸(𝜀𝑡 |𝑧𝑡 ) = 0 (Exogenous), If not OLS 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = ▪ Result: Estimator converges to the set of all
𝜕𝑞(𝜀)
⋯ → 𝑓(𝑥𝑡 ) = |
𝜕𝜖𝑡
|∙𝑓( ⏟
1
𝜖 ) , 𝐿 𝑇 (𝜃) = ∑𝑇𝑡=2 ln 𝑓 (𝑥𝑡 |ℱ𝑡−1 )
0(𝑞(𝑥) → 𝑞(𝜀) →
𝜕𝜀
= 𝛻𝑞(𝜖𝑡 ) → 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = 0) maximizers 𝛩0 = arg max 𝑄∞ (𝜃).
𝜕𝑥𝑡
𝑥𝑡 −⋯
𝑇
• Measure error: OLS lowered good for predict. 2. Invertibility of Hessian 𝛻 2 𝑄∞ (𝜃0 ) = 𝔼[𝛻 2 𝑞𝑡 (𝜃0 )] is invertible
𝑝
Nesting models: • Predict: Endogenutiy not a problem. (Pre-condition: 𝜃ˆ𝑇 → 𝜃0 ∈ int(Θ) from 1). Holds automatically since
Model A nests B / B ⊂ A(Eq A → Eq B with fixation of params = 0 ), The • Hausman: Endogeneity different values, not the same. 𝜃0 is the unique maximizer of 𝑄∞ (𝜃) AND 𝛻 2 𝑄∞ (𝜃0 ) is regular
error class of A wider or equal B (Error class A ⊃ Error class B), WN ⊃ Week 4-5 (constant rank in neighborhood of 𝜃0 — always given/assumed).
MDS ⊃ i.i.d. with moments ⊃ TID ⊃ NID. UID ⊄ TID, but UID ⊂ WN. B well- 𝑑 𝑑
Asymptotic Normality: √𝑇(𝜃̂ − 𝜃0 ) → 𝒩(0, 𝑉) 3. Asymptotic normality of score √𝑇∇𝑄𝑇 (𝑥𝑇 , 𝜃0 ) → 𝒩(0, Σ) Proved
spec → A well-spec. A misspec → B misspec.
√𝑇(𝜃ˆ𝑇 − 𝜃0 ) ≈ ⏟
−1
−(𝛻 2 𝑄𝑇 (𝜃0 )) ⋅ (√𝑇𝛻𝑄 by Billingsley's CLT well-specified or NED CLT for miss-specified.
𝑁(0, 𝜎 2 ), 𝑈(𝑎, 𝑏),Exp(𝜆) ⟹ 𝔼|𝜖|𝑛 < ∞ all moments exist. TID(𝜈) ⏟ 𝑇 (𝜃0 ))
3.1. SE: {𝑥𝑡 } is SE and 𝑞𝑡 (𝜃0 ) is cont. differentiable ⟹ ∇𝑞𝑡 (𝜃0 )
𝔼|𝜖|𝑛 < ∞ moment exist on when 𝐧 < 𝛎 Ω(LLN) 𝒩(0,Σ)(CLT)
is continuous in (𝑥𝑡 , 𝑥𝑡−1 ) ⟹ {∇𝑞𝑡 (𝜃0 )} is SE by Krengel.
When Model A nests Model B 𝔼|𝜖|𝑛𝐵 ≥ 𝔼|𝜖|𝑛𝐴 Ω ∙ 𝒩(0, Σ)~𝒩(0, ΩΣΩ⊤ )
3.2. Bounded 2nd moment: 𝔼|∇𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1 , 𝜃0 )|2 < ∞. Holds
Parametric/non parametric For any M-estirmator (MLE,NLS, Robust) (and 𝜃0 ∈ int(Θ)) 𝑉 = ΩΣΩ⊤:
−1 since 𝑞𝑡 (𝜃0 ) is WB of order 2 + 𝔼|𝑞𝑡 (𝜃)|2 < ∞ (prove
• Parametric: finite 𝜃 + exaсt 𝜖 dist. (NID, TID, Exp). Ω = (𝛻 2 𝑄∞ (𝜃0 )) = (𝔼[𝛻 2 𝑞(𝜃0 )])−1 , 𝔼|𝑞𝑡 |2 < ∞ via 𝑐𝑛 -inequality!).
• Semi-parametric: finite 𝜃 + 𝜖 dist not given (WN/MDS/i.i.d.). 𝑇
−1
In LS/NLS 𝑞𝑡 (𝜃0 ) = −𝜀 2 𝑡 , if 𝜀𝑡 dist. Has finite moments
• Non-parametric: infinite 𝜃(no formula). Semi-nonparametric: 1
̂ 𝑇 = ( ∑ 𝛻 2 𝑞𝑡 (𝜃ˆ𝑇 ))
Ω 3.3. Martingale Difference Sequence: 𝔼(∇𝑞𝑡 ∣ ℱ𝑡−1 ) = 0
amount 𝜃 growth 𝑇. 𝑇
𝑡=2 [Well-specified]: {𝛻𝑞𝑡 (𝜃0 )} is an MDS (𝔼(∇𝑞𝑡 (𝜃0 ) ∣ ℱ𝑡−1 )
Parameter-driven vs Observation-driven. ⟹ apply MDS CLT.
Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ],
• Observation-driven: hid. param 𝜎𝑡2 or 𝜆𝑡 is driven by past 𝑇 [Misspecified]: Not MDS! (the residuals will generally be
values of 𝑥𝑡−1 and his previous values 𝜎𝑡−1 2 1 ⊤
Σ̂ 𝑇 = ∑ 𝛻 𝑞𝑡 (𝜃ˆ𝑇 )𝛻𝑞𝑡 (𝜃ˆ𝑇 ) autocorrelated, model cannot capture true mds)
• Parameter-driven: 𝜆𝑡 does not depend on 𝑥𝑡−1 and has own 𝑇
𝑡=2 ▪ Assume {𝑥𝑡 } is NED of size 1 on an α-mixing
shocks 𝑣𝑡 • If MDS Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ] process
Time Series Hierarchy: ▪ Score 𝛻𝑞𝑡 (𝜃0 )is Lipschitz continuous in data
• If not MDS, Misspecified, errors/scores autocorrelated Σ
i.i.d. (+𝔼 = 0) ⟹ MDS (+𝑉𝑎𝑟 <∞) ⟹ WN ⟹ WS. (i.i.d. ⟹ SS always). (also ned of size 1)
includes autocovariances.
• SS (Strictly Stationary): Joint dist. Does not change through Well-specified MLE (IME holds): Ω = Σ −1 ▪ Moment is slightly > 2: 𝐸‖𝛻𝑞𝑡 (𝜃0 )‖𝑟 < ∞
time. If Var < ∞, then SS ⟹ WS. 𝑉 = ΩΣΩ⊤ = Σ −1 ΣΣ−1 = Σ −1 = −(𝔼[𝛻 2 𝑞𝑡 (𝜃0 )])−1 (for some 𝑟 > 2). ⟹ apply NED CLT and we
• MDS: 𝔼(𝑥𝑡 |ℱ𝑡−1) = 0. 1 moment Consequently 𝔼[𝑥𝑡 ] = 0, Hypothesis testing 𝑉 = Σ −1 get asymptotic normality of score
Cov = 0. 4. Uniform convergence of Hessian sup𝜃 ‖∇2 𝑄𝑇 (𝑥𝑇 , 𝜃) −
√𝑇 ⋅ (Est − Hypo)
• WN (White Noise): 𝔼[𝑥𝑡 ] = 0, Var(𝑥𝑡 ) = const < ∞, 2 𝑡= ∇2 𝑄∞ (𝜃)‖ → 0
𝑝
moments Cov(𝑥𝑡 , 𝑥𝑡−𝑘 ) = 0. (Not obligated i.i.d., does NOT √var from matrix
1. If Correctly Specified Proved by Simple moments for asymptotic normality
imply SE/SS) 4.1. Bounded 2nd moment: 𝔼|𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1, 𝜃)|2 < ∞: already
→ 𝜃0 is the TRUE parameter of the DGP.
• NID, UID (Uniform), Exp(𝜆), Gamma: exist all moments proved in 3.2 via 𝑐𝑛 -inequality. for some 𝜃 ∈ Θ (Cheat 50/50
→ Rejecting H0 means this effect TRULY EXISTS in the real-world DGP.
• WS (Weakly Stationary): 𝔼[𝑥𝑡 ] = const, Var(𝑥𝑡 ) = const < 𝜃0 )
2. If Misspecified:
∞, Cov depend only on lag 𝑘. 4.2. Well-behavedness (for 𝜽 ∈ 𝚯): 𝑞𝑡 (𝜃) is WB of order 2, and
→ 𝜃0 is only a PSEUDO-TRUE parameter (the model is still wrong!)
• MA(1) TRAP: 𝑥𝑡 = 𝜖𝑡 + 𝛽𝜖𝑡−1 (where 𝜖𝑡 ∼ i.i.d.). The row is ∇𝑞𝑡 (𝜃), ∇2 𝑞𝑡 (𝜃) are WB of order 1: holds since 𝑞𝑡 is 3
→ Rejecting H0 does NOT prove the real world has this feature.
SS (Krengel) and WS (if 𝜎𝜖2 < ∞). But not WN (т.к. → It only means that adding this parameter provides a BETTER times cont. differentiable / built from standard smooth blocks
Cov(𝑥𝑡 , 𝑥𝑡−1 ) ≠ 0) and not i.i.d. APPROXIMATION to the DGP. on compact Θ.
Ergodicity – forgets start point, manages to experience every possible state. Boundary Trap (Asymp. Normality): Requires 𝜃0 ∈ int(Θ). If the true 𝑉
Week 2 √𝑇(𝜃̂ − 𝜃0 ) ∼ 𝒩(0, 𝑉) ⟹ 𝜃̂ ∼ 𝒩 (𝜃0 , 𝑇 )
parameter is on the boundary (e.g., 𝛿 ∈ [0,10] but true 𝛿 = 0), a symmetric
Bougerol’s Theorem (SE + Invertibility) Discontinuous Indicator Trap:
normal bell curve is impossible ⟹ Asymptotic Normality FAILS.
• A1: {𝜖𝑡 } is SE (Krengel or i.i.d) 𝑝 Continuous in 𝜃 on compact Θ ⟹ Maximum exist (𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃))
1. Consistency 𝜃̂ → 𝜃0 with 𝜃0 ∈ int(Θ) Proved by Consistency Continuous in data 𝑥 ⟹ the estimator is a random variable (Measurable).
• A2: 𝔼 log + |𝜙(𝑥1 , 𝜖𝑡 )| < ∞ (log +|𝑦| ≤ |𝑦|, True if shocks
Theorem If 1(𝑥𝑡−1 ≥ 0) ⟹not cont. in 𝑥 ⟹there is a max, but 𝜃̂ may not be a
finite first moment)
𝜕𝜙(𝑥 ,𝜖𝑡 )∣ 1.1. E&M: Θ is compact and 𝑄𝑇 [if TVP: both 𝑞(⋅) and filter random variable.
• A3: 𝔼 log sup ∣∣∣ ∣∣ < 0 (𝔼 log | 𝑌| ≤ log(𝔼|𝑌|) < 𝜆ˆ𝑡 (𝜃, 𝜆ˆ1 )] is continuous in data and 𝜃 ⟹ 𝜃ˆ𝑇 exists and is
𝑥 𝜕𝑥 If 1(𝛽 ≥ 0) ⟹ maximum may not exist(discontinuous in parameters) and not
0 ⟺ 𝔼|𝑌| < 1) measurable. measurable.
𝑝
UCT (SE + Bounded Moment 𝔼|𝑥|𝑛 < ∞) 1.2. Uniform convergence: sup𝜃 |𝑄𝑇 (𝑥𝑇 , 𝜃) − 𝑄∞ (𝜃)| → 0 If Θ is not compact ⟹ maximum is also not exist.
Use when shock disappeared after derivative or bounded in function, Proved by Stochastic Equicontinuity Theorem If Θ = [. . . , . . . ] it holds 𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃), if Θ = (. . . , . . . ) we cannot
otherwise Power-𝒏 1.2.1. Pointwise convergence: for all 𝜃 ∈ Θ guarantee.
2 𝑝
𝔼|𝑥𝑡 |4 < ∞, 𝑥𝑡2 = 𝜎𝑡2 → 𝑥𝑡4 = 𝜎𝑡2 , 𝑛 = 2 𝑄𝑇 (𝜃) → 𝑄∞ (𝜃) Proved by Ergodic theorem (LLN) Week 6
• A1: {𝜖𝑡 } is SE ▪ SE: Since {𝑥𝑡 } [and {𝜆𝑡 (𝜃)}] is SE and 𝑞(⋅) is Pseudo-true 𝜃0 = arg max 𝑄∞ (𝜃): MLE minimizes KL distance
• A2: 𝔼|𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ (𝔼|𝑎𝐴 + 𝐵 + с|𝑛 ≤ continuous, {𝑞𝑡 (𝜃)} is SE by Krengel. 𝔼 log 𝑓0 − 𝔼 log 𝑓 (𝜃) (asymmetric, best DGP approx if misspecified; if well-
𝑐𝑛 (|𝑎|𝑛 𝔼|𝐴|𝑛 + 𝔼|𝐵| 𝑛 + |с|𝑛 )) ▪ Bounded 1st moment: 𝔼|𝑞𝑡 |2 < ∞ ⟹ specified, attains CRLB = ℐ(𝜃0 )−1 = efficient). LS/NLS minimizes weighted
𝜕𝜙(𝑥 ,𝜖 )∣ 𝐿2 -norm.
• A3: sup ∣∣ <1 𝔼|𝑞𝑡 | < ∞ by Jensen. Or use 𝔼|𝐴𝐵| ≤
𝜕𝑥 ∣
𝑥,𝜖
√𝔼𝐴2 𝔼𝐵2 , 𝐸|𝐴 ± 𝐵|𝑛 ≤ 𝑐𝑛 (𝐸|𝐴|𝑛 + Hausman Test (correct spec / exogeneity): Compares consistent 𝜃ˆ𝑇1
Power-𝒏 Contraction Theorem (SE + Bounded Moment) (NLS/OLS) vs efficient 𝜃ˆ𝑇2 (MLE/IV). If misspecified/endogenous ⟹ 𝜃01 ≠ 𝜃02 .
𝐸|𝐵|𝑛 )
• A1: {𝜖𝑡 } are strictly i.i.d. 𝜕𝑄𝑇 𝑑
• A2: 𝔼 |𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ 1.2.2. Stochastic equicontinuity: sup 𝑇 𝔼sup𝜃 ‖ ‖ < Under 𝐻0 : √𝑇(𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 ) → 𝒩(0, Σ1 − Σ2 ) or 𝐻𝑇 = 𝑇(𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 )′(Σ̂1 −
𝜕𝜃
𝑛 −1 𝑑
𝜕𝜙(𝑥 ,𝜖𝑡 )∣ ∞ Proved by Simple Stochastic Equicontinuity Σ̂2 ) (𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 ) → 𝜒 2 (𝑘).
• A3: 𝔼 [ sup ∣∣∣ ∣∣ ] < 1
𝑥 𝜕𝑥 Theorem Diebold-Mariano (predictive accuracy on Test 𝑁): 𝐻0 : 𝔼[𝑑𝑡 ] = 0, 𝑑𝑡 =
Krengel theorem – if you take SE parts in measuarble function, the result also ▪ Norm Model: 1st moment 𝔼|𝑞𝑡 | < ∞ + 𝑞𝑡 (𝜃) 𝑑̅ 𝑑
will be SE., does not work for 𝑥𝑡 = 𝑔(𝜃)𝑥𝑡−1 + ⋯ 𝑔(𝑒𝑡1 ) − 𝑔(𝑒𝑡2 ) ⟹ 𝐷𝑀 = √𝑁 → 𝒩(0,1).
is WB of order 1 on compact for some 𝜃 ∈ Θ 𝜎ˆ𝑑
Generalized Hölder Inequality softer then Cauchy (𝔼|𝐴𝐵| ≤ √𝔼𝐴2 𝔼𝐵2 ) (Cheat allowed 𝔼|𝑞𝑡 | = 𝔼|𝑒𝑡2 |). AIC = 2𝑘 − 2𝐿 𝑇 (lowest wins) & 𝑅̅ 2: Penalize 𝑘 to prevent in-sample
𝑛 𝑛 𝑝𝑞 ▪ TVP Model: 2nd moment 𝔼|𝑞𝑡 |2 < ∞ and overfitting. If equal 𝑘 (e.g., all have 3 params), rank directly by in-sample 𝐿 𝑇 /
𝔼|𝑧𝑡 ⋅ 𝑤𝑡 |n ≤ (𝔼|𝑧𝑡 |𝑝 )𝑝 (𝔼|𝑤𝑡 |𝑞 )𝑞 , (𝑛 = )
𝑝+𝑞 𝔼|𝑓𝑡 |2 < ∞ + both 𝑞, 𝜙 are WB of order 2 on 𝑅2 . AIC pros: Works for misspecified & non-nested models; preferred when
Is to prove 𝑥𝑡 SE? ⟹ Krengel compact Θ + True process is uniformly data is scarce (no split needed).
Is to prove 𝑥𝑡 (start) convergence to SE limit 𝑥𝑡 (|𝑥𝑡 (start) − 𝑥𝑡 | →
𝑒.𝑎.𝑠.
0)? 𝜕𝑓 2 Relative comparison by Likelihood is not prefered(overfitting)
contracting: sup | 𝑡+1 | < 1
𝜕𝑓𝑡 Cross-Validation / Sub-sample on Time Series: Possible via buffer ℎ:
⟹ MVT + Product e.a.s. 𝒑
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠. 1.2.3. Filter Convergence [Only when TVP in 𝜽 ̂ → 𝜽𝟎 Train(𝑇) → Drop(ℎ) → Test(𝑁) (or drop ℎ on both sides in CV). Pros:
|𝑥𝑡 (𝜃, 𝜇1 ) − 𝑥𝑡 (𝜃)| → 0 (Product e.a.s. + MVT): If 𝑧𝑡 → 0 & {𝜖𝑡 } SE Flexible custom loss 𝑔(𝑒𝑡 ). Cons: Wastes data (bad if scarce); cannot
𝑒.𝑎.𝑠. questions] 𝐿̂ 𝑇 (𝜆̂𝑡 ) → 𝐿 𝑇 (𝜆𝑡 )
with 𝔼 log | 𝜖𝑡 | < ∞ ⟹ |𝜖𝑡 | ⋅ 𝑧𝑡 →
+
0. (For GARCH 𝑥𝑡 = 𝜎𝑡 𝜖𝑡 , apply 𝑒.𝑎.𝑠. distinguish two well-specified models as 𝑇, 𝑁 → ∞.
1 𝑒.𝑎.𝑠. ▪ Since filter is 𝜆ˆ𝑡 → 𝜆𝑡 and 𝑞 is WB1 with
MVT to √⋅ first: |𝑥ˆ𝑡 − 𝑥𝑡 | ≤ |𝜖𝑡 | |𝜎ˆ𝑡2 − 𝜎𝑡2 | → 0). 𝑝 Small Samples & Outliers: Small 𝑇 (asymptotics fail) ⟹ Monte Carlo for
2√𝜔 ˆ
𝔼|𝑞𝑡 | < ∞, then sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0 finite-sample bias/skewness. Outliers/fat tails ⟹ Robust bounded 𝑞 or LARE
Week 3 𝜃∈Θ 𝑡
by triangle inequality sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − (−|𝑢𝑡 |).
Univariate Uniform Invertibility for any given 𝜃 sup|𝛼̂𝑡 (𝜃, 𝛼̂1 ) −
𝜃∈Θ 𝜃∈Θ VaR Backtesting (Kupiec): Hit ℎ𝑡 = 𝕀(𝑟𝑡 < VaR𝑡 (𝛼)) ∼ Bern(𝛼).
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠.
𝛼𝑡 (𝜃)| → 0, 𝛼̂𝑡 → 𝛼𝑡 : 𝑄∞ (𝜃)| ≤ sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − 𝑄𝑇 (𝜆 𝑇 )| + √𝑇(ℎ ̅−𝛼) 𝑑
𝑒.𝑎.𝑠. ⏟
𝜃∈Θ 𝐻0 : 𝔼[ℎ𝑡 ] = 𝛼 ⟹ 𝑍 = → 𝒩(0,1). Var = 𝛼(1 − 𝛼)
√𝛼(1−𝛼)
• A1: {𝑥𝑡 } SE (if 𝑥𝑡 generated |𝑥ˆ𝑡 − 𝑥𝑡 | → 0, perturbed 𝑝
→0 (Filter Inver) 𝜆
equation below) 𝑝 Penalized (arg max 𝐿 𝑇 (𝜃) − 𝛼 ‖𝜃‖2 , 𝜆 > 0):
sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0 𝑇
• A2: 𝔼 log + sup|𝜙(𝛼ˆ1 , 𝑥1 , 𝜃)| < ∞ ( 𝛼ˆ1 ∈ 𝛩) ⏟
𝜃∈Θ • Shrinks |𝜃ˆ(𝜆)| < |𝜃ˆ(0)| (prior: params are small). Good for
θ∈Θ 𝑝
→0 (by 1.2) prediction (lower finite-𝑇 var), bad for structural (biased).
• A3: 𝔼 log sup ∣∣ ∣𝜕𝛼̂𝑡+1 ∣
̂
𝜃,𝛼 ̂𝑡
𝜕𝛼 ∣∣ < 0 1.3. Identifiable uniqueness: sup𝜃∈𝑆 𝑐 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 ) LASSO (𝜆 ∑ | 𝜃𝑖 |) sets params to exact 0.
Multivariate Uniform Invertibility : Proved by Identifiable uniqueness theorem • Consistency: Needs 𝛼 > 0 (penalty vanishes); if 𝛼 = 0, 𝜆 >
𝑒.𝑎.𝑠. 1.3.1. Continuity: 𝑄∞ (𝜃) continuous on Θ. 0 ⟹ inconsistent! Does not reduce asymptotic variance.
Perturbed Dynamic Equation Theorem sup|𝑓ˆ𝑡 − 𝑓𝑡 | → 0:
𝜃∈Θ Automatically proved when 1.2 Uniform • Bayesian MAP ⊂ Penalized (not vice versa: requires 𝐿 𝑇 , prior
𝑒.𝑎.𝑠.
• A1: 𝛼̂𝑡 → 𝛼𝑡 (SE) proved above. Convergence of 𝑄𝑇 proved and continuity of 𝑞 density 𝑝(𝜃), no 𝑇 𝛼 control).
• A2: 𝔼 log + |𝜙(𝑓1 , 𝛼𝑡 , 𝜃)| < ∞ 1.3.2. Unique maximizer: 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 ) for all 𝜃 Ensembles (𝑥ˆ𝑡 = ∑ 𝜓𝑖 𝑥ˆ𝑡𝑖 ): Stacking: linear comb. on full train set. Random
Holds since Θ is compact + Continuity (𝑄∞ (𝜃) Forest (Bagging): average of trees on bootstrap samples. Boosting
A3: 𝔼 log sup ∣∣∣ 𝑡+1 ∣∣∣ < 0
𝜕𝑓
• continuous by 1.1) + Unique Maximizer(Given):
𝜃,𝑓 𝜕𝑓𝑡 (AdaBoost/XGBoost): sequential fit; low-fit obs re-sampled with higher prob.
• A4: 𝔼 log + |𝑓𝑡 | < ∞. (True process finite, assumed, regularity) ▪ MLE: well-specified + 𝜃0 is identified, unique
by Information inequality
A5: Uniformly Lipschitz / Bounded derivatives, sup ∣∣∣ 𝑡+1 ∣∣∣ <
𝜕𝑓
•
𝜕𝛼𝑡 ▪ LS/NLS: well-specified + 𝜃0 is identified +
∣ 𝜕2 𝑓𝑡+1 ∣
∞, sup ∣∣ ∣<∞ Exogeneity 𝐸(𝜀𝑡 |𝑥𝑡−1 ) = 𝐸(𝜀𝑡 ) =
𝜕𝛼𝑡 𝜕𝑓𝑡 ∣
0(since 𝜀𝑡 ⊥ 𝑥𝑡−1 ), MSE uniquely minimized.
DGP (SE) vs FILTER (Invertibility) – The Universal Rule:
DGP SE (Nature, no hats): SUBSTITUTE 𝑥𝑡 formula into dynamic eq). 1.4. Set Consistency: 𝑑(𝜃̂𝑇 , 𝛩0 ) → 0:
Filter Invertible (Algorithm, hats): KEEP 𝑥𝑡 fixed (don't substitute!). 1.4.1. (Use when model is MISSPECIFIED and parameter
Measurable function – any cont math(exp, log) is NOT IDENTIFIED / lacks unique maximizer).
M: max ∑ 𝑞𝑡 . Z: ∑ 𝛻 𝑞𝑡 = 0. ▪ Requires ONLY: E&M (1.1) + Uniform
𝑇 𝑇
Likelihood: 𝑝 Convergence (1.2).
OLS: 𝜃̂ → 𝜃0 consistent ⇔ 𝐸(𝜀𝑡 |𝑥𝑡 ) = 0. Valid IV 𝑧𝑡 : 𝐶𝑜𝑣(𝑥𝑡 , 𝑧𝑡 ) ≠ 0
𝜕𝜖𝑡 ▪ NO need for Identifiable Uniqueness (1.3)!
The process: 𝑥𝑡 = ⋯ − 𝜖𝑡 → 𝜖𝑡 = 𝑥𝑡 − ⋯ → 𝐽 = | | = ⋯ → 𝑓(𝜖) =
𝜕𝑥𝑡 (Relevant) AND 𝐸(𝜀𝑡 |𝑧𝑡 ) = 0 (Exogenous), If not OLS 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = ▪ Result: Estimator converges to the set of all
𝜕𝑞(𝜀)
⋯ → 𝑓(𝑥𝑡 ) = |
𝜕𝜖𝑡
|∙𝑓( ⏟
1
𝜖 ) , 𝐿 𝑇 (𝜃) = ∑𝑇𝑡=2 ln 𝑓 (𝑥𝑡 |ℱ𝑡−1 )
0(𝑞(𝑥) → 𝑞(𝜀) →
𝜕𝜀
= 𝛻𝑞(𝜖𝑡 ) → 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = 0) maximizers 𝛩0 = arg max 𝑄∞ (𝜃).
𝜕𝑥𝑡
𝑥𝑡 −⋯
𝑇
• Measure error: OLS lowered good for predict. 2. Invertibility of Hessian 𝛻 2 𝑄∞ (𝜃0 ) = 𝔼[𝛻 2 𝑞𝑡 (𝜃0 )] is invertible
𝑝
Nesting models: • Predict: Endogenutiy not a problem. (Pre-condition: 𝜃ˆ𝑇 → 𝜃0 ∈ int(Θ) from 1). Holds automatically since
Model A nests B / B ⊂ A(Eq A → Eq B with fixation of params = 0 ), The • Hausman: Endogeneity different values, not the same. 𝜃0 is the unique maximizer of 𝑄∞ (𝜃) AND 𝛻 2 𝑄∞ (𝜃0 ) is regular
error class of A wider or equal B (Error class A ⊃ Error class B), WN ⊃ Week 4-5 (constant rank in neighborhood of 𝜃0 — always given/assumed).
MDS ⊃ i.i.d. with moments ⊃ TID ⊃ NID. UID ⊄ TID, but UID ⊂ WN. B well- 𝑑 𝑑
Asymptotic Normality: √𝑇(𝜃̂ − 𝜃0 ) → 𝒩(0, 𝑉) 3. Asymptotic normality of score √𝑇∇𝑄𝑇 (𝑥𝑇 , 𝜃0 ) → 𝒩(0, Σ) Proved
spec → A well-spec. A misspec → B misspec.
√𝑇(𝜃ˆ𝑇 − 𝜃0 ) ≈ ⏟
−1
−(𝛻 2 𝑄𝑇 (𝜃0 )) ⋅ (√𝑇𝛻𝑄 by Billingsley's CLT well-specified or NED CLT for miss-specified.
𝑁(0, 𝜎 2 ), 𝑈(𝑎, 𝑏),Exp(𝜆) ⟹ 𝔼|𝜖|𝑛 < ∞ all moments exist. TID(𝜈) ⏟ 𝑇 (𝜃0 ))
3.1. SE: {𝑥𝑡 } is SE and 𝑞𝑡 (𝜃0 ) is cont. differentiable ⟹ ∇𝑞𝑡 (𝜃0 )
𝔼|𝜖|𝑛 < ∞ moment exist on when 𝐧 < 𝛎 Ω(LLN) 𝒩(0,Σ)(CLT)
is continuous in (𝑥𝑡 , 𝑥𝑡−1 ) ⟹ {∇𝑞𝑡 (𝜃0 )} is SE by Krengel.
When Model A nests Model B 𝔼|𝜖|𝑛𝐵 ≥ 𝔼|𝜖|𝑛𝐴 Ω ∙ 𝒩(0, Σ)~𝒩(0, ΩΣΩ⊤ )
3.2. Bounded 2nd moment: 𝔼|∇𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1 , 𝜃0 )|2 < ∞. Holds
Parametric/non parametric For any M-estirmator (MLE,NLS, Robust) (and 𝜃0 ∈ int(Θ)) 𝑉 = ΩΣΩ⊤:
−1 since 𝑞𝑡 (𝜃0 ) is WB of order 2 + 𝔼|𝑞𝑡 (𝜃)|2 < ∞ (prove
• Parametric: finite 𝜃 + exaсt 𝜖 dist. (NID, TID, Exp). Ω = (𝛻 2 𝑄∞ (𝜃0 )) = (𝔼[𝛻 2 𝑞(𝜃0 )])−1 , 𝔼|𝑞𝑡 |2 < ∞ via 𝑐𝑛 -inequality!).
• Semi-parametric: finite 𝜃 + 𝜖 dist not given (WN/MDS/i.i.d.). 𝑇
−1
In LS/NLS 𝑞𝑡 (𝜃0 ) = −𝜀 2 𝑡 , if 𝜀𝑡 dist. Has finite moments
• Non-parametric: infinite 𝜃(no formula). Semi-nonparametric: 1
̂ 𝑇 = ( ∑ 𝛻 2 𝑞𝑡 (𝜃ˆ𝑇 ))
Ω 3.3. Martingale Difference Sequence: 𝔼(∇𝑞𝑡 ∣ ℱ𝑡−1 ) = 0
amount 𝜃 growth 𝑇. 𝑇
𝑡=2 [Well-specified]: {𝛻𝑞𝑡 (𝜃0 )} is an MDS (𝔼(∇𝑞𝑡 (𝜃0 ) ∣ ℱ𝑡−1 )
Parameter-driven vs Observation-driven. ⟹ apply MDS CLT.
Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ],
• Observation-driven: hid. param 𝜎𝑡2 or 𝜆𝑡 is driven by past 𝑇 [Misspecified]: Not MDS! (the residuals will generally be
values of 𝑥𝑡−1 and his previous values 𝜎𝑡−1 2 1 ⊤
Σ̂ 𝑇 = ∑ 𝛻 𝑞𝑡 (𝜃ˆ𝑇 )𝛻𝑞𝑡 (𝜃ˆ𝑇 ) autocorrelated, model cannot capture true mds)
• Parameter-driven: 𝜆𝑡 does not depend on 𝑥𝑡−1 and has own 𝑇
𝑡=2 ▪ Assume {𝑥𝑡 } is NED of size 1 on an α-mixing
shocks 𝑣𝑡 • If MDS Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ] process
Time Series Hierarchy: ▪ Score 𝛻𝑞𝑡 (𝜃0 )is Lipschitz continuous in data
• If not MDS, Misspecified, errors/scores autocorrelated Σ
i.i.d. (+𝔼 = 0) ⟹ MDS (+𝑉𝑎𝑟 <∞) ⟹ WN ⟹ WS. (i.i.d. ⟹ SS always). (also ned of size 1)
includes autocovariances.
• SS (Strictly Stationary): Joint dist. Does not change through Well-specified MLE (IME holds): Ω = Σ −1 ▪ Moment is slightly > 2: 𝐸‖𝛻𝑞𝑡 (𝜃0 )‖𝑟 < ∞
time. If Var < ∞, then SS ⟹ WS. 𝑉 = ΩΣΩ⊤ = Σ −1 ΣΣ−1 = Σ −1 = −(𝔼[𝛻 2 𝑞𝑡 (𝜃0 )])−1 (for some 𝑟 > 2). ⟹ apply NED CLT and we
• MDS: 𝔼(𝑥𝑡 |ℱ𝑡−1) = 0. 1 moment Consequently 𝔼[𝑥𝑡 ] = 0, Hypothesis testing 𝑉 = Σ −1 get asymptotic normality of score
Cov = 0. 4. Uniform convergence of Hessian sup𝜃 ‖∇2 𝑄𝑇 (𝑥𝑇 , 𝜃) −
√𝑇 ⋅ (Est − Hypo)
• WN (White Noise): 𝔼[𝑥𝑡 ] = 0, Var(𝑥𝑡 ) = const < ∞, 2 𝑡= ∇2 𝑄∞ (𝜃)‖ → 0
𝑝
moments Cov(𝑥𝑡 , 𝑥𝑡−𝑘 ) = 0. (Not obligated i.i.d., does NOT √var from matrix
1. If Correctly Specified Proved by Simple moments for asymptotic normality
imply SE/SS) 4.1. Bounded 2nd moment: 𝔼|𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1, 𝜃)|2 < ∞: already
→ 𝜃0 is the TRUE parameter of the DGP.
• NID, UID (Uniform), Exp(𝜆), Gamma: exist all moments proved in 3.2 via 𝑐𝑛 -inequality. for some 𝜃 ∈ Θ (Cheat 50/50
→ Rejecting H0 means this effect TRULY EXISTS in the real-world DGP.
• WS (Weakly Stationary): 𝔼[𝑥𝑡 ] = const, Var(𝑥𝑡 ) = const < 𝜃0 )
2. If Misspecified:
∞, Cov depend only on lag 𝑘. 4.2. Well-behavedness (for 𝜽 ∈ 𝚯): 𝑞𝑡 (𝜃) is WB of order 2, and
→ 𝜃0 is only a PSEUDO-TRUE parameter (the model is still wrong!)
• MA(1) TRAP: 𝑥𝑡 = 𝜖𝑡 + 𝛽𝜖𝑡−1 (where 𝜖𝑡 ∼ i.i.d.). The row is ∇𝑞𝑡 (𝜃), ∇2 𝑞𝑡 (𝜃) are WB of order 1: holds since 𝑞𝑡 is 3
→ Rejecting H0 does NOT prove the real world has this feature.
SS (Krengel) and WS (if 𝜎𝜖2 < ∞). But not WN (т.к. → It only means that adding this parameter provides a BETTER times cont. differentiable / built from standard smooth blocks
Cov(𝑥𝑡 , 𝑥𝑡−1 ) ≠ 0) and not i.i.d. APPROXIMATION to the DGP. on compact Θ.
Ergodicity – forgets start point, manages to experience every possible state. Boundary Trap (Asymp. Normality): Requires 𝜃0 ∈ int(Θ). If the true 𝑉
Week 2 √𝑇(𝜃̂ − 𝜃0 ) ∼ 𝒩(0, 𝑉) ⟹ 𝜃̂ ∼ 𝒩 (𝜃0 , 𝑇 )
parameter is on the boundary (e.g., 𝛿 ∈ [0,10] but true 𝛿 = 0), a symmetric
Bougerol’s Theorem (SE + Invertibility) Discontinuous Indicator Trap:
normal bell curve is impossible ⟹ Asymptotic Normality FAILS.
• A1: {𝜖𝑡 } is SE (Krengel or i.i.d) 𝑝 Continuous in 𝜃 on compact Θ ⟹ Maximum exist (𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃))
1. Consistency 𝜃̂ → 𝜃0 with 𝜃0 ∈ int(Θ) Proved by Consistency Continuous in data 𝑥 ⟹ the estimator is a random variable (Measurable).
• A2: 𝔼 log + |𝜙(𝑥1 , 𝜖𝑡 )| < ∞ (log +|𝑦| ≤ |𝑦|, True if shocks
Theorem If 1(𝑥𝑡−1 ≥ 0) ⟹not cont. in 𝑥 ⟹there is a max, but 𝜃̂ may not be a
finite first moment)
𝜕𝜙(𝑥 ,𝜖𝑡 )∣ 1.1. E&M: Θ is compact and 𝑄𝑇 [if TVP: both 𝑞(⋅) and filter random variable.
• A3: 𝔼 log sup ∣∣∣ ∣∣ < 0 (𝔼 log | 𝑌| ≤ log(𝔼|𝑌|) < 𝜆ˆ𝑡 (𝜃, 𝜆ˆ1 )] is continuous in data and 𝜃 ⟹ 𝜃ˆ𝑇 exists and is
𝑥 𝜕𝑥 If 1(𝛽 ≥ 0) ⟹ maximum may not exist(discontinuous in parameters) and not
0 ⟺ 𝔼|𝑌| < 1) measurable. measurable.
𝑝
UCT (SE + Bounded Moment 𝔼|𝑥|𝑛 < ∞) 1.2. Uniform convergence: sup𝜃 |𝑄𝑇 (𝑥𝑇 , 𝜃) − 𝑄∞ (𝜃)| → 0 If Θ is not compact ⟹ maximum is also not exist.
Use when shock disappeared after derivative or bounded in function, Proved by Stochastic Equicontinuity Theorem If Θ = [. . . , . . . ] it holds 𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃), if Θ = (. . . , . . . ) we cannot
otherwise Power-𝒏 1.2.1. Pointwise convergence: for all 𝜃 ∈ Θ guarantee.
2 𝑝
𝔼|𝑥𝑡 |4 < ∞, 𝑥𝑡2 = 𝜎𝑡2 → 𝑥𝑡4 = 𝜎𝑡2 , 𝑛 = 2 𝑄𝑇 (𝜃) → 𝑄∞ (𝜃) Proved by Ergodic theorem (LLN) Week 6
• A1: {𝜖𝑡 } is SE ▪ SE: Since {𝑥𝑡 } [and {𝜆𝑡 (𝜃)}] is SE and 𝑞(⋅) is Pseudo-true 𝜃0 = arg max 𝑄∞ (𝜃): MLE minimizes KL distance
• A2: 𝔼|𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ (𝔼|𝑎𝐴 + 𝐵 + с|𝑛 ≤ continuous, {𝑞𝑡 (𝜃)} is SE by Krengel. 𝔼 log 𝑓0 − 𝔼 log 𝑓 (𝜃) (asymmetric, best DGP approx if misspecified; if well-
𝑐𝑛 (|𝑎|𝑛 𝔼|𝐴|𝑛 + 𝔼|𝐵| 𝑛 + |с|𝑛 )) ▪ Bounded 1st moment: 𝔼|𝑞𝑡 |2 < ∞ ⟹ specified, attains CRLB = ℐ(𝜃0 )−1 = efficient). LS/NLS minimizes weighted
𝜕𝜙(𝑥 ,𝜖 )∣ 𝐿2 -norm.
• A3: sup ∣∣ <1 𝔼|𝑞𝑡 | < ∞ by Jensen. Or use 𝔼|𝐴𝐵| ≤
𝜕𝑥 ∣
𝑥,𝜖
√𝔼𝐴2 𝔼𝐵2 , 𝐸|𝐴 ± 𝐵|𝑛 ≤ 𝑐𝑛 (𝐸|𝐴|𝑛 + Hausman Test (correct spec / exogeneity): Compares consistent 𝜃ˆ𝑇1
Power-𝒏 Contraction Theorem (SE + Bounded Moment) (NLS/OLS) vs efficient 𝜃ˆ𝑇2 (MLE/IV). If misspecified/endogenous ⟹ 𝜃01 ≠ 𝜃02 .
𝐸|𝐵|𝑛 )
• A1: {𝜖𝑡 } are strictly i.i.d. 𝜕𝑄𝑇 𝑑
• A2: 𝔼 |𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ 1.2.2. Stochastic equicontinuity: sup 𝑇 𝔼sup𝜃 ‖ ‖ < Under 𝐻0 : √𝑇(𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 ) → 𝒩(0, Σ1 − Σ2 ) or 𝐻𝑇 = 𝑇(𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 )′(Σ̂1 −
𝜕𝜃
𝑛 −1 𝑑
𝜕𝜙(𝑥 ,𝜖𝑡 )∣ ∞ Proved by Simple Stochastic Equicontinuity Σ̂2 ) (𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 ) → 𝜒 2 (𝑘).
• A3: 𝔼 [ sup ∣∣∣ ∣∣ ] < 1
𝑥 𝜕𝑥 Theorem Diebold-Mariano (predictive accuracy on Test 𝑁): 𝐻0 : 𝔼[𝑑𝑡 ] = 0, 𝑑𝑡 =
Krengel theorem – if you take SE parts in measuarble function, the result also ▪ Norm Model: 1st moment 𝔼|𝑞𝑡 | < ∞ + 𝑞𝑡 (𝜃) 𝑑̅ 𝑑
will be SE., does not work for 𝑥𝑡 = 𝑔(𝜃)𝑥𝑡−1 + ⋯ 𝑔(𝑒𝑡1 ) − 𝑔(𝑒𝑡2 ) ⟹ 𝐷𝑀 = √𝑁 → 𝒩(0,1).
is WB of order 1 on compact for some 𝜃 ∈ Θ 𝜎ˆ𝑑
Generalized Hölder Inequality softer then Cauchy (𝔼|𝐴𝐵| ≤ √𝔼𝐴2 𝔼𝐵2 ) (Cheat allowed 𝔼|𝑞𝑡 | = 𝔼|𝑒𝑡2 |). AIC = 2𝑘 − 2𝐿 𝑇 (lowest wins) & 𝑅̅ 2: Penalize 𝑘 to prevent in-sample
𝑛 𝑛 𝑝𝑞 ▪ TVP Model: 2nd moment 𝔼|𝑞𝑡 |2 < ∞ and overfitting. If equal 𝑘 (e.g., all have 3 params), rank directly by in-sample 𝐿 𝑇 /
𝔼|𝑧𝑡 ⋅ 𝑤𝑡 |n ≤ (𝔼|𝑧𝑡 |𝑝 )𝑝 (𝔼|𝑤𝑡 |𝑞 )𝑞 , (𝑛 = )
𝑝+𝑞 𝔼|𝑓𝑡 |2 < ∞ + both 𝑞, 𝜙 are WB of order 2 on 𝑅2 . AIC pros: Works for misspecified & non-nested models; preferred when
Is to prove 𝑥𝑡 SE? ⟹ Krengel compact Θ + True process is uniformly data is scarce (no split needed).
Is to prove 𝑥𝑡 (start) convergence to SE limit 𝑥𝑡 (|𝑥𝑡 (start) − 𝑥𝑡 | →
𝑒.𝑎.𝑠.
0)? 𝜕𝑓 2 Relative comparison by Likelihood is not prefered(overfitting)
contracting: sup | 𝑡+1 | < 1
𝜕𝑓𝑡 Cross-Validation / Sub-sample on Time Series: Possible via buffer ℎ:
⟹ MVT + Product e.a.s. 𝒑
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠. 1.2.3. Filter Convergence [Only when TVP in 𝜽 ̂ → 𝜽𝟎 Train(𝑇) → Drop(ℎ) → Test(𝑁) (or drop ℎ on both sides in CV). Pros:
|𝑥𝑡 (𝜃, 𝜇1 ) − 𝑥𝑡 (𝜃)| → 0 (Product e.a.s. + MVT): If 𝑧𝑡 → 0 & {𝜖𝑡 } SE Flexible custom loss 𝑔(𝑒𝑡 ). Cons: Wastes data (bad if scarce); cannot
𝑒.𝑎.𝑠. questions] 𝐿̂ 𝑇 (𝜆̂𝑡 ) → 𝐿 𝑇 (𝜆𝑡 )
with 𝔼 log | 𝜖𝑡 | < ∞ ⟹ |𝜖𝑡 | ⋅ 𝑧𝑡 →
+
0. (For GARCH 𝑥𝑡 = 𝜎𝑡 𝜖𝑡 , apply 𝑒.𝑎.𝑠. distinguish two well-specified models as 𝑇, 𝑁 → ∞.
1 𝑒.𝑎.𝑠. ▪ Since filter is 𝜆ˆ𝑡 → 𝜆𝑡 and 𝑞 is WB1 with
MVT to √⋅ first: |𝑥ˆ𝑡 − 𝑥𝑡 | ≤ |𝜖𝑡 | |𝜎ˆ𝑡2 − 𝜎𝑡2 | → 0). 𝑝 Small Samples & Outliers: Small 𝑇 (asymptotics fail) ⟹ Monte Carlo for
2√𝜔 ˆ
𝔼|𝑞𝑡 | < ∞, then sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0 finite-sample bias/skewness. Outliers/fat tails ⟹ Robust bounded 𝑞 or LARE
Week 3 𝜃∈Θ 𝑡
by triangle inequality sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − (−|𝑢𝑡 |).
Univariate Uniform Invertibility for any given 𝜃 sup|𝛼̂𝑡 (𝜃, 𝛼̂1 ) −
𝜃∈Θ 𝜃∈Θ VaR Backtesting (Kupiec): Hit ℎ𝑡 = 𝕀(𝑟𝑡 < VaR𝑡 (𝛼)) ∼ Bern(𝛼).
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠.
𝛼𝑡 (𝜃)| → 0, 𝛼̂𝑡 → 𝛼𝑡 : 𝑄∞ (𝜃)| ≤ sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − 𝑄𝑇 (𝜆 𝑇 )| + √𝑇(ℎ ̅−𝛼) 𝑑
𝑒.𝑎.𝑠. ⏟
𝜃∈Θ 𝐻0 : 𝔼[ℎ𝑡 ] = 𝛼 ⟹ 𝑍 = → 𝒩(0,1). Var = 𝛼(1 − 𝛼)
√𝛼(1−𝛼)
• A1: {𝑥𝑡 } SE (if 𝑥𝑡 generated |𝑥ˆ𝑡 − 𝑥𝑡 | → 0, perturbed 𝑝
→0 (Filter Inver) 𝜆
equation below) 𝑝 Penalized (arg max 𝐿 𝑇 (𝜃) − 𝛼 ‖𝜃‖2 , 𝜆 > 0):
sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0 𝑇
• A2: 𝔼 log + sup|𝜙(𝛼ˆ1 , 𝑥1 , 𝜃)| < ∞ ( 𝛼ˆ1 ∈ 𝛩) ⏟
𝜃∈Θ • Shrinks |𝜃ˆ(𝜆)| < |𝜃ˆ(0)| (prior: params are small). Good for
θ∈Θ 𝑝
→0 (by 1.2) prediction (lower finite-𝑇 var), bad for structural (biased).
• A3: 𝔼 log sup ∣∣ ∣𝜕𝛼̂𝑡+1 ∣
̂
𝜃,𝛼 ̂𝑡
𝜕𝛼 ∣∣ < 0 1.3. Identifiable uniqueness: sup𝜃∈𝑆 𝑐 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 ) LASSO (𝜆 ∑ | 𝜃𝑖 |) sets params to exact 0.
Multivariate Uniform Invertibility : Proved by Identifiable uniqueness theorem • Consistency: Needs 𝛼 > 0 (penalty vanishes); if 𝛼 = 0, 𝜆 >
𝑒.𝑎.𝑠. 1.3.1. Continuity: 𝑄∞ (𝜃) continuous on Θ. 0 ⟹ inconsistent! Does not reduce asymptotic variance.
Perturbed Dynamic Equation Theorem sup|𝑓ˆ𝑡 − 𝑓𝑡 | → 0:
𝜃∈Θ Automatically proved when 1.2 Uniform • Bayesian MAP ⊂ Penalized (not vice versa: requires 𝐿 𝑇 , prior
𝑒.𝑎.𝑠.
• A1: 𝛼̂𝑡 → 𝛼𝑡 (SE) proved above. Convergence of 𝑄𝑇 proved and continuity of 𝑞 density 𝑝(𝜃), no 𝑇 𝛼 control).
• A2: 𝔼 log + |𝜙(𝑓1 , 𝛼𝑡 , 𝜃)| < ∞ 1.3.2. Unique maximizer: 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 ) for all 𝜃 Ensembles (𝑥ˆ𝑡 = ∑ 𝜓𝑖 𝑥ˆ𝑡𝑖 ): Stacking: linear comb. on full train set. Random
Holds since Θ is compact + Continuity (𝑄∞ (𝜃) Forest (Bagging): average of trees on bootstrap samples. Boosting
A3: 𝔼 log sup ∣∣∣ 𝑡+1 ∣∣∣ < 0
𝜕𝑓
• continuous by 1.1) + Unique Maximizer(Given):
𝜃,𝑓 𝜕𝑓𝑡 (AdaBoost/XGBoost): sequential fit; low-fit obs re-sampled with higher prob.
• A4: 𝔼 log + |𝑓𝑡 | < ∞. (True process finite, assumed, regularity) ▪ MLE: well-specified + 𝜃0 is identified, unique
by Information inequality
A5: Uniformly Lipschitz / Bounded derivatives, sup ∣∣∣ 𝑡+1 ∣∣∣ <
𝜕𝑓
•
𝜕𝛼𝑡 ▪ LS/NLS: well-specified + 𝜃0 is identified +
∣ 𝜕2 𝑓𝑡+1 ∣
∞, sup ∣∣ ∣<∞ Exogeneity 𝐸(𝜀𝑡 |𝑥𝑡−1 ) = 𝐸(𝜀𝑡 ) =
𝜕𝛼𝑡 𝜕𝑓𝑡 ∣
0(since 𝜀𝑡 ⊥ 𝑥𝑡−1 ), MSE uniquely minimized.
DGP (SE) vs FILTER (Invertibility) – The Universal Rule:
DGP SE (Nature, no hats): SUBSTITUTE 𝑥𝑡 formula into dynamic eq). 1.4. Set Consistency: 𝑑(𝜃̂𝑇 , 𝛩0 ) → 0:
Filter Invertible (Algorithm, hats): KEEP 𝑥𝑡 fixed (don't substitute!). 1.4.1. (Use when model is MISSPECIFIED and parameter
Measurable function – any cont math(exp, log) is NOT IDENTIFIED / lacks unique maximizer).
