Week 1 DGP SE (Nature, no hats): SUBSTITUTE 𝑥𝑡 formula into dynamic eq). ▪ LS/NLS: well-specified + 𝜃0 is identified +
Likelihood: Filter Invertible (Algorithm, hats): KEEP 𝑥𝑡 fixed (don't substitute!). Exogeneity 𝐸(𝜀𝑡 |𝑥𝑡−1 ) = 𝐸(𝜀𝑡 ) =
The process: 𝑥𝑡 = ⋯ − 𝜖𝑡 → 𝜖𝑡 = 𝑥𝑡 − ⋯ → 𝐽 = | | = ⋯ → 𝑓(𝜖) = Measurable function – any cont math(exp, log) 0(since 𝜀𝑡 ⊥ 𝑥𝑡−1 ), MSE uniquely minimized.
𝜕𝜖𝑡
𝜕𝑥𝑡 1 1
M: max ∑ 𝑞𝑡 . Z: ∑ 𝛻 𝑞𝑡 = 0. 1.4. Set Consistency: 𝑑(𝜃̂𝑇 , 𝛩0 ) → 0:
𝑇 𝑇
𝜕𝜖
⋯ → 𝑓(𝑥𝑡 ) = | 𝑡 | ∙ 𝑓 ( ⏟
1
𝜖 ) , 𝐿 𝑇 (𝜃) = ∑𝑇𝑡=2 ln 𝑓 (𝑥𝑡 |ℱ𝑡−1 ) 𝑝 1.4.1. (Use when model is MISSPECIFIED and parameter
𝜕𝑥𝑡 𝑇 OLS: 𝜃̂ → 𝜃0 consistent ⇔ 𝐸(𝜀𝑡 |𝑥𝑡 ) = 0. 𝑦𝑡 = 𝑥𝑡 · 𝛽 + 𝜀𝑡 , 𝑥𝑡 = 𝑧𝑡 · 𝛾 +
𝑥𝑡 −⋯ is NOT IDENTIFIED / lacks unique maximizer).
𝑢𝑡 Valid IV 𝑧𝑡 : 𝐶𝑜𝑣(𝑥𝑡 , 𝑧𝑡 ) ≠ 0 (Relevant) AND 𝐸(𝜀𝑡 |𝑧𝑡 ) = 0|𝐸(𝑢𝑡 |𝑧𝑡 ) = ▪ Requires ONLY: E&M (1.1) + Uniform
Nesting models: 𝜕𝑞(𝜀)
Model A nests B / B ⊂ A(Eq A → Eq B with fixation of params = 0 ), The 0 (Exogenous), If not OLS 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = 0(𝑞(𝑥) → 𝑞(𝜀) → = Convergence (1.2).
𝜕𝜀
error class of A wider or equal B (Error class A ⊃ Error class B), 𝛻𝑞(𝜖𝑡 ) → 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = 0) ▪ NO need for Identifiable Uniqueness (1.3)!
WN ⊃ MDS ⊃ i.i.d. with moments ⊃ TID ⊃ NID. • Measure error: OLS lowered good for predict. ▪ Result: Estimator converges to the set of all
Model A nests model B only if it dist on right side, if A UID, B must be also UID. • Predict: Endogenutiy not a problem. maximizers 𝛩0 = arg max 𝑄∞ (𝜃).
UID ⊄ TID, but UID ⊂ WN. • Hausman: Endogeneity different values, not the same. 2. Invertibility of Hessian 𝛻 2 𝑄∞ (𝜃0 ) = 𝔼[𝛻 2 𝑞𝑡 (𝜃0 )] is invertible
𝑝
B well-spec → A well-spec. A misspec → B misspec. Week 4-5 (Pre-condition: 𝜃ˆ𝑇 → 𝜃0 ∈ int(Θ) from 1). Holds automatically since
𝑁(0, 𝜎 2 ), 𝑈(𝑎, 𝑏),Exp(𝜆) ⟹ 𝔼|𝜖|𝑛 < ∞ all moments exist. TID(𝜈) 𝑑
𝜃0 is the unique maximizer of 𝑄∞ (𝜃) AND 𝛻 2 𝑄∞ (𝜃0 ) is regular
Asymptotic Normality: √𝑇(𝜃̂ − 𝜃0 ) → 𝒩(0, 𝑉)
𝔼|𝜖|𝑛 < ∞ moment exist on when 𝐧 < 𝛎 −1 (constant rank in neighborhood of 𝜃0 — always given/assumed).
When Model A nests Model B 𝑛𝐵error ≥ 𝑛Aerror √𝑇(𝜃ˆ𝑇 − 𝜃0 ) ≈ ⏟
−(𝛻 2 𝑄𝑇 (𝜃0 )) ⋅ (√𝑇𝛻𝑄
⏟ 𝑇 (𝜃0 )) 𝑑
3. Asymptotic normality of score √𝑇∇𝑄𝑇 (𝑥𝑇 , 𝜃0 ) → 𝒩(0, Σ) Proved
Parametric/non parametric Ω(LLN) 𝒩(0,Σ)(𝐶𝐿𝑇)
⊤) by Billingsley's CLT well-specified or NED CLT for miss-specified.
• Parametric: finite 𝜃 + exaсt 𝜖 dist. (NID, TID, Exp). Ω ∙ 𝒩(0, Σ)~𝒩(0, ΩΣΩ
3.1. SE: {𝑥𝑡 } is SE and 𝑞𝑡 (𝜃0 ) is cont. differentiable ⟹ ∇𝑞𝑡 (𝜃0 )
• Semi-parametric: finite 𝜃 + 𝜖 dist not given (WN/MDS/i.i.d.). For any M-estirmator (MLE,NLS, Robust) (and 𝜃0 ∈ int(Θ)) 𝑉 = ΩΣΩ⊤:
−1 is continuous in (𝑥𝑡 , 𝑥𝑡−1 ) ⟹ {∇𝑞𝑡 (𝜃0 )} is SE by Krengel.
• Non-parametric: infinite 𝜃(no formula). Semi-nonparametric: Ω = (𝛻 2 𝑄∞ (𝜃0 )) = (𝔼[𝛻 2 𝑞(𝜃0 )])−1 , 3.2. Bounded 2nd moment: 𝔼|∇𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1 , 𝜃0 )|2 < ∞. Holds
amount 𝜃 growth 𝑇. 𝑇
−1
since 𝑞𝑡 (𝜃0 ) is WB of order 2 + 𝔼|𝑞𝑡 (𝜃)|2 < ∞ (prove
Parameter-driven vs Observation-driven. 1
̂ 𝑇 = ( ∑ 𝛻 2 𝑞𝑡 (𝜃ˆ𝑇 ))
Ω 𝔼|𝑞𝑡 |2 < ∞ via 𝑐𝑛 -inequality!).
• Observation-driven: hid. param 𝜎𝑡2 or 𝜆𝑡 is driven by past 𝑇
𝑡=2 In LS/NLS 𝑞𝑡 (𝜃0 ) = −𝜀 2 𝑡 , if 𝜀𝑡 dist. Has finite moments
values of 𝑥𝑡−1 and his previous values 𝜎𝑡−1 2
3.3. Martingale Difference Sequence: 𝔼(∇𝑞𝑡 ∣ ℱ𝑡−1 ) = 0
Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ],
• Parameter-driven: 𝜆𝑡 does not depend on 𝑥𝑡−1 and has own 𝑇 [Well-specified]: {𝛻𝑞𝑡 (𝜃0 )} is an MDS (𝔼(∇𝑞𝑡 (𝜃0 ) ∣ ℱ𝑡−1 )
shocks 𝑣𝑡 1 ⊤
Σ̂ 𝑇 = ∑ 𝛻 𝑞𝑡 (𝜃ˆ𝑇 )𝛻𝑞𝑡 (𝜃ˆ𝑇 ) ⟹ apply MDS CLT.
Time Series Hierarchy: 𝑇
𝑡=2 [Misspecified]: Not MDS! (the residuals will generally be
i.i.d. (+ 𝔼 = 0) ⟹ MDS (+ Var = const < ∞) ⟹ WN (+ 𝔼 = const any) • If MDS Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ] autocorrelated, model cannot capture true mds)
⟹ WS. • If not MDS, Misspecified, errors/scores autocorrelated Σ ▪ Assume {𝑥𝑡 } is NED of size 1 on an α-mixing
(Note: i.i.d. ⟹ SS always. BUT WS + Fixed Dist ⟹ SS) includes autocovariances. process
• SS (Strictly Stationary): Joint dist. Does not change through Well-specified MLE (IME holds): Ω = Σ −1 ▪ Score 𝛻𝑞𝑡 (𝜃0 )is Lipschitz continuous in data
time. If Var < ∞, then SS ⟹ WS. 𝑉 = ΩΣΩ⊤ = Σ −1 ΣΣ−1 = Σ −1 = −(𝔼[𝛻 2 𝑞𝑡 (𝜃0 )])−1 (also ned of size 1)
• MDS: 𝔼(𝑥𝑡 |ℱ𝑡−1) = 0. 1 moment Consequently 𝔼[𝑥𝑡 ] = 0, Hypothesis testing 𝑉 = Σ −1 ▪ Moment is slightly > 2: 𝐸‖𝛻𝑞𝑡 (𝜃0 )‖𝑟 < ∞
Cov = 0. √𝑇 ⋅ (Est − Hypo) (for some 𝑟 > 2). ⟹ apply NED CLT and we
• WN (White Noise): 𝔼[𝑥𝑡 ] = 0, Var(𝑥𝑡 ) = const < ∞, 2 𝑡= get asymptotic normality of score
√var from matrix
moments Cov(𝑥𝑡 , 𝑥𝑡−𝑘 ) = 0. (Not obligated i.i.d., does NOT 4. Uniform convergence of Hessian sup𝜃 ‖∇2 𝑄𝑇 (𝑥𝑇 , 𝜃) −
1. If Correctly Specified 𝑝
imply SE/SS) ∇2 𝑄∞ (𝜃)‖ → 0
→ 𝜃0 is the TRUE parameter of the DGP.
• WS (Weakly Stationary): 𝔼[𝑥𝑡 ] = const, Var(𝑥𝑡 ) = const < → Rejecting H0 means this effect TRULY EXISTS in the real-world DGP. Proved by Simple moments for asymptotic normality
∞, Cov depend only on lag 𝑘. 2. If Misspecified: 4.1. Bounded 2nd moment: 𝔼|𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1, 𝜃)|2 < ∞: already
• MA(1) TRAP: 𝑥𝑡 = 𝜖𝑡 + 𝛽𝜖𝑡−1 (where 𝜖𝑡 ∼ i.i.d.). The row is → 𝜃0 is only a PSEUDO-TRUE parameter (the model is still wrong!) proved in 3.2 via 𝑐𝑛 -inequality. for some 𝜃 ∈ Θ (Cheat 50/50
SS (Krengel) and WS (if 𝜎𝜖2 < ∞). But not WN (as → Rejecting H0 does NOT prove the real world has this feature. 𝜃0 )
Cov(𝑥𝑡 , 𝑥𝑡−1 ) ≠ 0) and not i.i.d. → It only means that adding this parameter provides a BETTER 4.2. Well-behavedness (for 𝜽 ∈ 𝚯): 𝑞𝑡 (𝜃) is WB of order 2, and
Ergodicity – forgets start point, manages to experience every possible state. APPROXIMATION to the DGP. ∇𝑞𝑡 (𝜃), ∇2 𝑞𝑡 (𝜃) are WB of order 1: holds since 𝑞𝑡 is 3
Week 2 Boundary Trap (Asymp. Normality): Requires 𝜃0 ∈ int(Θ). If the true times cont. differentiable / built from standard smooth blocks
Bougerol’s Theorem (SE + Invertibility) parameter is on the boundary (e.g., 𝛿 ∈ [0,10] but true 𝛿 = 0), a symmetric on compact Θ.
• A1: {𝜖𝑡 } is SE (Krengel or i.i.d) normal bell curve is impossible ⟹ Asymptotic Normality FAILS. 𝑉
√𝑇(𝜃̂ − 𝜃0 ) ∼ 𝒩(0, 𝑉) ⟹ 𝜃̂ ∼ 𝒩 (𝜃0 , 𝑇 )
• A2: 𝔼 log + |𝜙(𝑥1 , 𝜖𝑡 )| < ∞ (log +|𝑦| ≤ |𝑦|, True if shocks 𝑝
1. Consistency 𝜃̂ → 𝜃0 with 𝜃0 ∈ int(Θ) Proved by Consistency Discontinuous Indicator Trap:
finite first moment)
𝜕𝜙(𝑥 ,𝜖𝑡 )∣
Theorem Continuous in 𝜃 on compact Θ ⟹ Maximum exist (𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃))
• A3: 𝔼 log sup ∣∣∣ ∣∣ < 0 (𝔼 log | 𝑌| ≤ log(𝔼|𝑌|) < 1.1. E&M: Θ is compact and 𝑄𝑇 [if TVP: both 𝑞(⋅) and filter
𝑥 𝜕𝑥 Continuous in data 𝑥 ⟹ the estimator is a random variable (Measurable).
0 ⟺ 𝔼|𝑌| < 1) 𝜆ˆ𝑡 (𝜃, 𝜆ˆ1 )] is continuous in data and 𝜃 ⟹ 𝜃ˆ𝑇 exists and is If 1(𝑥𝑡−1 ≥ 0) ⟹not cont. in 𝑥 ⟹there is a max, but 𝜃̂ may not be a
UCT (SE + Bounded Moment 𝔼|𝑥|𝑛 < ∞) measurable. random variable.
𝑝
Use when shock disappeared after derivative or bounded in function, 1.2. Uniform convergence: sup𝜃 |𝑄𝑇 (𝑥𝑇 , 𝜃) − 𝑄∞ (𝜃)| → 0 If 1(𝛽 ≥ 0) ⟹ maximum may not exist(discontinuous in parameters) and not
otherwise Power-𝒏 Proved by Stochastic Equicontinuity Theorem measurable.
2
𝔼|𝑥𝑡 |4 < ∞, 𝑥𝑡2 = 𝜎𝑡2 → 𝑥𝑡4 = 𝜎𝑡2 , 𝑛 = 2 1.2.1. Pointwise convergence: for all 𝜃 ∈ Θ If Θ is not compact ⟹ maximum is also not exist.
𝑝
• A1: {𝜖𝑡 } is SE, must have 𝔼|𝜖𝑡 |𝑛 < ∞ 𝑄𝑇 (𝜃) → 𝑄∞ (𝜃) Proved by Ergodic theorem (LLN) If Θ = [. . . , . . . ] it holds 𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃), if Θ = (. . . , . . . ) we cannot
• A2: 𝔼|𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ (𝔼|𝑎𝐴 + 𝐵 + с|𝑛 ≤ ▪ SE: Since {𝑥𝑡 } [and {𝜆𝑡 (𝜃)}] is SE and 𝑞(⋅) is guarantee.
𝑐𝑛 (|𝑎|𝑛 𝔼|𝐴|𝑛 + 𝔼|𝐵| 𝑛 + |с|𝑛 )) continuous, {𝑞𝑡 (𝜃)} is SE by Krengel. If 𝜃̂𝑡 = arg max 𝑄𝑇 (𝑥𝑡 , 𝜃) – peak unique, if 𝜃̂𝑡 ∈ arg max 𝑄𝑇 (𝑥𝑡 , 𝜃) –
𝜕𝜙(𝑥 ,𝜖 )∣ may not unique
• A3: sup ∣∣ <1 ▪ Bounded 1st moment: 𝔼|𝑞𝑡 |2 < ∞ ⟹
𝜕𝑥 ∣
𝑥,𝜖 𝔼|𝑞𝑡 | < ∞ by Jensen. Or use 𝔼|𝐴𝐵| ≤ Week 6
Power-𝒏 Contraction Theorem (SE + Bounded Moment) Pseudo-true 𝜃0 = arg max 𝑄∞ (𝜃): MLE minimizes KL distance
√𝔼𝐴2 𝔼𝐵2 , 𝐸|𝐴 ± 𝐵|𝑛 ≤ 𝑐𝑛 (𝐸|𝐴|𝑛 +
• A1: {𝜖𝑡 } are strictly i.i.d. 𝐸|𝐵|𝑛 ) 𝔼 log 𝑓0 − 𝔼 log 𝑓 (𝜃) (asymmetric, best DGP approx if misspecified; if well-
• A2: 𝔼 |𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ 𝜕𝑄 specified, attains CRLB = ℐ(𝜃0 )−1 = efficient). LS/NLS minimizes weighted
𝜕𝜙(𝑥 ,𝜖𝑡 )∣
𝑛 1.2.2. Stochastic equicontinuity: sup 𝑇 𝔼sup𝜃 ‖ 𝑇‖ < 2
• A3: 𝔼 [ sup ∣∣∣ ∣∣ ] < 1
𝜕𝜃 𝐿 -norm.
𝑥 𝜕𝑥 ∞ Proved by Simple Stochastic Equicontinuity Hausman Test (correct spec / exogeneity): Compares consistent 𝜃ˆ𝑇1
Krengel theorem – if you take SE parts in measuarble function, the result also Theorem
(NLS/OLS) vs efficient 𝜃ˆ𝑇2 (MLE/IV). If misspecified/endogenous ⟹ 𝜃01 ≠ 𝜃02 .
will be SE., does not work for 𝑥𝑡 = 𝑔(𝜃)𝑥𝑡−1 + ⋯ ▪ Norm Model: 1st moment 𝔼|𝑞𝑡 | < ∞ + 𝑞𝑡 (𝜃) 𝑑
is WB of order 1 on compact for some 𝜃 ∈ Θ Under 𝐻0 : √𝑇(𝜃ˆ𝑇 − 𝜃ˆ𝑇 ) → 𝒩(0, Σ1 − Σ2 ) or 𝐻𝑇 = 𝑇(𝜃ˆ𝑇 − 𝜃ˆ𝑇 )′(Σ̂1 −
1 2 1 2
Generalized Hölder Inequality softer then Cauchy (𝔼|𝐴𝐵| ≤ √𝔼𝐴2 𝔼𝐵2 )
𝑑
𝑛 𝑛 𝑝𝑞 (Cheat allowed 𝔼|𝑞𝑡 | = 𝔼|𝑒𝑡 2 |). −1
Σ̂2 ) (𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 ) → 𝜒 2 (𝑘).
𝔼|𝑧𝑡 ⋅ 𝑤𝑡 |n ≤ (𝔼|𝑧𝑡 |𝑝 )𝑝 (𝔼|𝑤𝑡 |𝑞 )𝑞 , (𝑛 = ) ▪ TVP Model: 2nd moment 𝔼|𝑞𝑡 |2 < ∞ and
𝑝+𝑞 Diebold-Mariano (predictive accuracy on Test 𝑁): 𝐻0 : 𝔼[𝑑𝑡 ] = 0, 𝑑𝑡 =
Is to prove 𝑥𝑡 SE? ⟹ Krengel 𝔼|𝑓𝑡 |2 < ∞ + both 𝑞, 𝜙 are WB of order 2 on 𝑑̅ 𝑑
𝑒.𝑎.𝑠. compact Θ + True process is uniformly 𝑔(𝑒𝑡1 ) − 𝑔(𝑒𝑡2 ) ⟹ 𝐷𝑀 = √𝑁 ˆ → 𝒩(0,1).
𝜎𝑑
Is to prove 𝑥𝑡 (start) convergence to SE limit 𝑥𝑡 (|𝑥𝑡 (start) − 𝑥𝑡 | → 0)?
𝜕𝑓 2
AIC = 2𝑘 − 2𝐿 𝑇 (lowest wins) & 𝑅̅ 2: Penalize 𝑘 to prevent in-sample
⟹ MVT + Product e.a.s. contracting: sup | 𝑡+1 | < 1
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠. 𝜕𝑓𝑡 overfitting. If equal 𝑘 (e.g., all have 3 params), rank directly by in-sample 𝐿 𝑇 /
|𝑥𝑡 (𝜃, 𝜇1 ) − 𝑥𝑡 (𝜃)| → 0 (Product e.a.s. + MVT): If 𝑧𝑡 → 0 & {𝜖𝑡 } SE 1.2.3. Filter Convergence [Only when TVP in 𝜽
𝒑
̂ → 𝜽𝟎 𝑅2 . AIC pros: Works for misspecified & non-nested models; preferred when
𝑒.𝑎.𝑠.
with 𝔼 log + | 𝜖𝑡 | < ∞ ⟹ |𝜖𝑡 | ⋅ 𝑧𝑡 → 0. (For GARCH 𝑥𝑡 = 𝜎𝑡 𝜖𝑡 , apply questions] 𝐿̂ 𝑇 (𝜆̂𝑡 ) → 𝐿 𝑇 (𝜆𝑡 ) data is scarce (no split needed).
𝑒.𝑎.𝑠.
MVT to √⋅ first: |𝑥ˆ𝑡 − 𝑥𝑡 | ≤ |𝜖𝑡 |
1
|𝜎ˆ𝑡2 − 𝜎𝑡2 | → 0).
𝑒.𝑎.𝑠. Relative comparison by Likelihood is not prefered(overfitting)
2√𝜔 ▪ Since filter is 𝜆ˆ𝑡 → 𝜆𝑡 and 𝑞 is WB1 with
𝑝 Cross-Validation / Sub-sample on Time Series: Possible via buffer ℎ:
Week 3 ˆ
𝔼|𝑞𝑡 | < ∞, then sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0 Train(𝑇) → Drop(ℎ) → Test(𝑁) (or drop ℎ on both sides in CV). Pros:
Univariate Uniform Invertibility for any given 𝜃 sup|𝛼̂𝑡 (𝜃, 𝛼̂1 ) − 𝜃∈Θ
𝜃∈Θ by triangle inequality sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − Flexible custom loss 𝑔(𝑒𝑡 ). Cons: Wastes data (bad if scarce); cannot
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠.
𝛼𝑡 (𝜃)| → 0, 𝛼̂𝑡 → 𝛼𝑡 : 𝜃∈Θ distinguish two well-specified models as 𝑇, 𝑁 → ∞.
𝑒.𝑎.𝑠. 𝑄∞ (𝜃)| ≤ sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − 𝑄𝑇 (𝜆 𝑇 )| + Small Samples & Outliers: Small 𝑇 (asymptotics fail) ⟹ Monte Carlo for
• A1: {𝑥𝑡 } SE (if 𝑥𝑡 generated |𝑥ˆ𝑡 − 𝑥𝑡 | → 0, perturbed ⏟
𝜃∈Θ
𝑝 finite-sample bias/skewness. Outliers/fat tails ⟹ Robust bounded 𝑞𝑡 or LARE
equation below) →0 (Filter Inver)
𝑝 (−|𝑢𝑡 |).
• A2: 𝔼 log + sup|𝜙(𝛼ˆ1 , 𝑥1 , 𝜃)| < ∞ ( 𝛼ˆ1 ∈ 𝛩) sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0
θ∈Θ ⏟
𝜃∈Θ VaR Backtesting (Kupiec): Hit ℎ𝑡 = 𝕀(𝑟𝑡 < VaR𝑡 (𝛼)) ∼ Bern(𝛼).
A3: 𝔼 log sup ∣∣∣
̂ 𝑡+1 ∣
𝜕𝛼
• ∣∣ < 0 𝑝
→0 (by 1.2) 𝐻0 : 𝔼[ℎ𝑡 ] = 𝛼 ⟹ 𝑍 =
̅−𝛼) 𝑑
√𝑇(ℎ
→ 𝒩(0,1). Var = 𝛼(1 − 𝛼)
̂
𝜃,𝛼 ̂𝑡
𝜕𝛼
√𝛼(1−𝛼)
1.3. Identifiable uniqueness: sup𝜃∈𝑆 𝑐 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 )
Multivariate Uniform Invertibility : 𝜆
Penalized (arg max 𝐿 𝑇 (𝜃) − 𝛼 ‖𝜃‖2 , 𝜆 > 0):
𝑒.𝑎.𝑠. Proved by Identifiable uniqueness theorem 𝑇
Perturbed Dynamic Equation Theorem sup|𝑓ˆ𝑡 − 𝑓𝑡 | → 0: 1.3.1. Continuity: 𝑄∞ (𝜃) continuous on Θ.
𝜃∈Θ • Shrinks |𝜃ˆ(𝜆)| < |𝜃ˆ(0)| (prior: params are small). Good for
𝑒.𝑎.𝑠. Automatically proved when 1.2 Uniform prediction (lower finite-𝑇 var), bad for structural (biased).
• A1: 𝛼̂𝑡 → 𝛼𝑡 (SE) proved above.
Convergence of 𝑄𝑇 proved and continuity of 𝑞 LASSO (𝜆 ∑ | 𝜃𝑖 |) sets params to exact 0.
• A2: 𝔼 log + |𝜙(𝑓1 , 𝛼𝑡 , 𝜃)| < ∞
1.3.2. Unique maximizer: 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 ) for all 𝜃 • Consistency: Needs 𝛼 > 0 (penalty vanishes); if 𝛼 = 0, 𝜆 >
A3: 𝔼 log sup ∣∣∣ 𝑡+1 ∣∣∣ < 0
𝜕𝑓
• Holds since Θ is compact + Continuity (𝑄∞ (𝜃)
𝜃,𝑓 𝜕𝑓𝑡 0 ⟹ inconsistent! Does not reduce asymptotic variance.
•
A4: 𝔼 log + |𝑓𝑡 | < ∞. (True process finite, assumed, regularity) continuous by 1.1) + Unique Maximizer(Given): • Bayesian MAP ⊂ Penalized (not vice versa: requires 𝐿 𝑇 , prior
▪ MLE: well-specified + 𝜃0 is identified, unique density 𝑝(𝜃), no 𝑇 𝛼 control).
A5: Uniformly Lipschitz / Bounded derivatives, sup ∣∣∣ 𝑡+1 ∣∣∣ <
𝜕𝑓
•
𝜕𝛼𝑡 by Information inequality Ensembles (𝑥ˆ𝑡 = ∑ 𝜓𝑖 𝑥ˆ𝑡𝑖 ): Stacking: linear comb. on full train set. Random
2𝑓
∞, sup ∣∣∣ 𝑡+1 ∣∣∣ < ∞
𝜕
𝜕𝛼𝑡 𝜕𝑓𝑡
Forest (Bagging): average of trees on bootstrap samples. Boosting
DGP (SE) vs FILTER (Invertibility) – The Universal Rule: (AdaBoost/XGBoost): sequential fit; low-fit obs re-sampled with higher prob.
Likelihood: Filter Invertible (Algorithm, hats): KEEP 𝑥𝑡 fixed (don't substitute!). Exogeneity 𝐸(𝜀𝑡 |𝑥𝑡−1 ) = 𝐸(𝜀𝑡 ) =
The process: 𝑥𝑡 = ⋯ − 𝜖𝑡 → 𝜖𝑡 = 𝑥𝑡 − ⋯ → 𝐽 = | | = ⋯ → 𝑓(𝜖) = Measurable function – any cont math(exp, log) 0(since 𝜀𝑡 ⊥ 𝑥𝑡−1 ), MSE uniquely minimized.
𝜕𝜖𝑡
𝜕𝑥𝑡 1 1
M: max ∑ 𝑞𝑡 . Z: ∑ 𝛻 𝑞𝑡 = 0. 1.4. Set Consistency: 𝑑(𝜃̂𝑇 , 𝛩0 ) → 0:
𝑇 𝑇
𝜕𝜖
⋯ → 𝑓(𝑥𝑡 ) = | 𝑡 | ∙ 𝑓 ( ⏟
1
𝜖 ) , 𝐿 𝑇 (𝜃) = ∑𝑇𝑡=2 ln 𝑓 (𝑥𝑡 |ℱ𝑡−1 ) 𝑝 1.4.1. (Use when model is MISSPECIFIED and parameter
𝜕𝑥𝑡 𝑇 OLS: 𝜃̂ → 𝜃0 consistent ⇔ 𝐸(𝜀𝑡 |𝑥𝑡 ) = 0. 𝑦𝑡 = 𝑥𝑡 · 𝛽 + 𝜀𝑡 , 𝑥𝑡 = 𝑧𝑡 · 𝛾 +
𝑥𝑡 −⋯ is NOT IDENTIFIED / lacks unique maximizer).
𝑢𝑡 Valid IV 𝑧𝑡 : 𝐶𝑜𝑣(𝑥𝑡 , 𝑧𝑡 ) ≠ 0 (Relevant) AND 𝐸(𝜀𝑡 |𝑧𝑡 ) = 0|𝐸(𝑢𝑡 |𝑧𝑡 ) = ▪ Requires ONLY: E&M (1.1) + Uniform
Nesting models: 𝜕𝑞(𝜀)
Model A nests B / B ⊂ A(Eq A → Eq B with fixation of params = 0 ), The 0 (Exogenous), If not OLS 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = 0(𝑞(𝑥) → 𝑞(𝜀) → = Convergence (1.2).
𝜕𝜀
error class of A wider or equal B (Error class A ⊃ Error class B), 𝛻𝑞(𝜖𝑡 ) → 𝔼[𝛻𝑞(𝜖𝑡 ) ∣ 𝑧𝑡 ] = 0) ▪ NO need for Identifiable Uniqueness (1.3)!
WN ⊃ MDS ⊃ i.i.d. with moments ⊃ TID ⊃ NID. • Measure error: OLS lowered good for predict. ▪ Result: Estimator converges to the set of all
Model A nests model B only if it dist on right side, if A UID, B must be also UID. • Predict: Endogenutiy not a problem. maximizers 𝛩0 = arg max 𝑄∞ (𝜃).
UID ⊄ TID, but UID ⊂ WN. • Hausman: Endogeneity different values, not the same. 2. Invertibility of Hessian 𝛻 2 𝑄∞ (𝜃0 ) = 𝔼[𝛻 2 𝑞𝑡 (𝜃0 )] is invertible
𝑝
B well-spec → A well-spec. A misspec → B misspec. Week 4-5 (Pre-condition: 𝜃ˆ𝑇 → 𝜃0 ∈ int(Θ) from 1). Holds automatically since
𝑁(0, 𝜎 2 ), 𝑈(𝑎, 𝑏),Exp(𝜆) ⟹ 𝔼|𝜖|𝑛 < ∞ all moments exist. TID(𝜈) 𝑑
𝜃0 is the unique maximizer of 𝑄∞ (𝜃) AND 𝛻 2 𝑄∞ (𝜃0 ) is regular
Asymptotic Normality: √𝑇(𝜃̂ − 𝜃0 ) → 𝒩(0, 𝑉)
𝔼|𝜖|𝑛 < ∞ moment exist on when 𝐧 < 𝛎 −1 (constant rank in neighborhood of 𝜃0 — always given/assumed).
When Model A nests Model B 𝑛𝐵error ≥ 𝑛Aerror √𝑇(𝜃ˆ𝑇 − 𝜃0 ) ≈ ⏟
−(𝛻 2 𝑄𝑇 (𝜃0 )) ⋅ (√𝑇𝛻𝑄
⏟ 𝑇 (𝜃0 )) 𝑑
3. Asymptotic normality of score √𝑇∇𝑄𝑇 (𝑥𝑇 , 𝜃0 ) → 𝒩(0, Σ) Proved
Parametric/non parametric Ω(LLN) 𝒩(0,Σ)(𝐶𝐿𝑇)
⊤) by Billingsley's CLT well-specified or NED CLT for miss-specified.
• Parametric: finite 𝜃 + exaсt 𝜖 dist. (NID, TID, Exp). Ω ∙ 𝒩(0, Σ)~𝒩(0, ΩΣΩ
3.1. SE: {𝑥𝑡 } is SE and 𝑞𝑡 (𝜃0 ) is cont. differentiable ⟹ ∇𝑞𝑡 (𝜃0 )
• Semi-parametric: finite 𝜃 + 𝜖 dist not given (WN/MDS/i.i.d.). For any M-estirmator (MLE,NLS, Robust) (and 𝜃0 ∈ int(Θ)) 𝑉 = ΩΣΩ⊤:
−1 is continuous in (𝑥𝑡 , 𝑥𝑡−1 ) ⟹ {∇𝑞𝑡 (𝜃0 )} is SE by Krengel.
• Non-parametric: infinite 𝜃(no formula). Semi-nonparametric: Ω = (𝛻 2 𝑄∞ (𝜃0 )) = (𝔼[𝛻 2 𝑞(𝜃0 )])−1 , 3.2. Bounded 2nd moment: 𝔼|∇𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1 , 𝜃0 )|2 < ∞. Holds
amount 𝜃 growth 𝑇. 𝑇
−1
since 𝑞𝑡 (𝜃0 ) is WB of order 2 + 𝔼|𝑞𝑡 (𝜃)|2 < ∞ (prove
Parameter-driven vs Observation-driven. 1
̂ 𝑇 = ( ∑ 𝛻 2 𝑞𝑡 (𝜃ˆ𝑇 ))
Ω 𝔼|𝑞𝑡 |2 < ∞ via 𝑐𝑛 -inequality!).
• Observation-driven: hid. param 𝜎𝑡2 or 𝜆𝑡 is driven by past 𝑇
𝑡=2 In LS/NLS 𝑞𝑡 (𝜃0 ) = −𝜀 2 𝑡 , if 𝜀𝑡 dist. Has finite moments
values of 𝑥𝑡−1 and his previous values 𝜎𝑡−1 2
3.3. Martingale Difference Sequence: 𝔼(∇𝑞𝑡 ∣ ℱ𝑡−1 ) = 0
Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ],
• Parameter-driven: 𝜆𝑡 does not depend on 𝑥𝑡−1 and has own 𝑇 [Well-specified]: {𝛻𝑞𝑡 (𝜃0 )} is an MDS (𝔼(∇𝑞𝑡 (𝜃0 ) ∣ ℱ𝑡−1 )
shocks 𝑣𝑡 1 ⊤
Σ̂ 𝑇 = ∑ 𝛻 𝑞𝑡 (𝜃ˆ𝑇 )𝛻𝑞𝑡 (𝜃ˆ𝑇 ) ⟹ apply MDS CLT.
Time Series Hierarchy: 𝑇
𝑡=2 [Misspecified]: Not MDS! (the residuals will generally be
i.i.d. (+ 𝔼 = 0) ⟹ MDS (+ Var = const < ∞) ⟹ WN (+ 𝔼 = const any) • If MDS Σ = 𝕍ar(𝛻𝑞𝑡 (𝜃0 )) = 𝔼[𝛻𝑞𝑡 (𝜃0 )𝛻𝑞𝑡 (𝜃0 )⊤ ] autocorrelated, model cannot capture true mds)
⟹ WS. • If not MDS, Misspecified, errors/scores autocorrelated Σ ▪ Assume {𝑥𝑡 } is NED of size 1 on an α-mixing
(Note: i.i.d. ⟹ SS always. BUT WS + Fixed Dist ⟹ SS) includes autocovariances. process
• SS (Strictly Stationary): Joint dist. Does not change through Well-specified MLE (IME holds): Ω = Σ −1 ▪ Score 𝛻𝑞𝑡 (𝜃0 )is Lipschitz continuous in data
time. If Var < ∞, then SS ⟹ WS. 𝑉 = ΩΣΩ⊤ = Σ −1 ΣΣ−1 = Σ −1 = −(𝔼[𝛻 2 𝑞𝑡 (𝜃0 )])−1 (also ned of size 1)
• MDS: 𝔼(𝑥𝑡 |ℱ𝑡−1) = 0. 1 moment Consequently 𝔼[𝑥𝑡 ] = 0, Hypothesis testing 𝑉 = Σ −1 ▪ Moment is slightly > 2: 𝐸‖𝛻𝑞𝑡 (𝜃0 )‖𝑟 < ∞
Cov = 0. √𝑇 ⋅ (Est − Hypo) (for some 𝑟 > 2). ⟹ apply NED CLT and we
• WN (White Noise): 𝔼[𝑥𝑡 ] = 0, Var(𝑥𝑡 ) = const < ∞, 2 𝑡= get asymptotic normality of score
√var from matrix
moments Cov(𝑥𝑡 , 𝑥𝑡−𝑘 ) = 0. (Not obligated i.i.d., does NOT 4. Uniform convergence of Hessian sup𝜃 ‖∇2 𝑄𝑇 (𝑥𝑇 , 𝜃) −
1. If Correctly Specified 𝑝
imply SE/SS) ∇2 𝑄∞ (𝜃)‖ → 0
→ 𝜃0 is the TRUE parameter of the DGP.
• WS (Weakly Stationary): 𝔼[𝑥𝑡 ] = const, Var(𝑥𝑡 ) = const < → Rejecting H0 means this effect TRULY EXISTS in the real-world DGP. Proved by Simple moments for asymptotic normality
∞, Cov depend only on lag 𝑘. 2. If Misspecified: 4.1. Bounded 2nd moment: 𝔼|𝑞𝑡 (𝑥𝑡 , 𝑥𝑡−1, 𝜃)|2 < ∞: already
• MA(1) TRAP: 𝑥𝑡 = 𝜖𝑡 + 𝛽𝜖𝑡−1 (where 𝜖𝑡 ∼ i.i.d.). The row is → 𝜃0 is only a PSEUDO-TRUE parameter (the model is still wrong!) proved in 3.2 via 𝑐𝑛 -inequality. for some 𝜃 ∈ Θ (Cheat 50/50
SS (Krengel) and WS (if 𝜎𝜖2 < ∞). But not WN (as → Rejecting H0 does NOT prove the real world has this feature. 𝜃0 )
Cov(𝑥𝑡 , 𝑥𝑡−1 ) ≠ 0) and not i.i.d. → It only means that adding this parameter provides a BETTER 4.2. Well-behavedness (for 𝜽 ∈ 𝚯): 𝑞𝑡 (𝜃) is WB of order 2, and
Ergodicity – forgets start point, manages to experience every possible state. APPROXIMATION to the DGP. ∇𝑞𝑡 (𝜃), ∇2 𝑞𝑡 (𝜃) are WB of order 1: holds since 𝑞𝑡 is 3
Week 2 Boundary Trap (Asymp. Normality): Requires 𝜃0 ∈ int(Θ). If the true times cont. differentiable / built from standard smooth blocks
Bougerol’s Theorem (SE + Invertibility) parameter is on the boundary (e.g., 𝛿 ∈ [0,10] but true 𝛿 = 0), a symmetric on compact Θ.
• A1: {𝜖𝑡 } is SE (Krengel or i.i.d) normal bell curve is impossible ⟹ Asymptotic Normality FAILS. 𝑉
√𝑇(𝜃̂ − 𝜃0 ) ∼ 𝒩(0, 𝑉) ⟹ 𝜃̂ ∼ 𝒩 (𝜃0 , 𝑇 )
• A2: 𝔼 log + |𝜙(𝑥1 , 𝜖𝑡 )| < ∞ (log +|𝑦| ≤ |𝑦|, True if shocks 𝑝
1. Consistency 𝜃̂ → 𝜃0 with 𝜃0 ∈ int(Θ) Proved by Consistency Discontinuous Indicator Trap:
finite first moment)
𝜕𝜙(𝑥 ,𝜖𝑡 )∣
Theorem Continuous in 𝜃 on compact Θ ⟹ Maximum exist (𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃))
• A3: 𝔼 log sup ∣∣∣ ∣∣ < 0 (𝔼 log | 𝑌| ≤ log(𝔼|𝑌|) < 1.1. E&M: Θ is compact and 𝑄𝑇 [if TVP: both 𝑞(⋅) and filter
𝑥 𝜕𝑥 Continuous in data 𝑥 ⟹ the estimator is a random variable (Measurable).
0 ⟺ 𝔼|𝑌| < 1) 𝜆ˆ𝑡 (𝜃, 𝜆ˆ1 )] is continuous in data and 𝜃 ⟹ 𝜃ˆ𝑇 exists and is If 1(𝑥𝑡−1 ≥ 0) ⟹not cont. in 𝑥 ⟹there is a max, but 𝜃̂ may not be a
UCT (SE + Bounded Moment 𝔼|𝑥|𝑛 < ∞) measurable. random variable.
𝑝
Use when shock disappeared after derivative or bounded in function, 1.2. Uniform convergence: sup𝜃 |𝑄𝑇 (𝑥𝑇 , 𝜃) − 𝑄∞ (𝜃)| → 0 If 1(𝛽 ≥ 0) ⟹ maximum may not exist(discontinuous in parameters) and not
otherwise Power-𝒏 Proved by Stochastic Equicontinuity Theorem measurable.
2
𝔼|𝑥𝑡 |4 < ∞, 𝑥𝑡2 = 𝜎𝑡2 → 𝑥𝑡4 = 𝜎𝑡2 , 𝑛 = 2 1.2.1. Pointwise convergence: for all 𝜃 ∈ Θ If Θ is not compact ⟹ maximum is also not exist.
𝑝
• A1: {𝜖𝑡 } is SE, must have 𝔼|𝜖𝑡 |𝑛 < ∞ 𝑄𝑇 (𝜃) → 𝑄∞ (𝜃) Proved by Ergodic theorem (LLN) If Θ = [. . . , . . . ] it holds 𝑄𝑇 (𝜃̂) ≥ 𝑄𝑇 (𝜃), if Θ = (. . . , . . . ) we cannot
• A2: 𝔼|𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ (𝔼|𝑎𝐴 + 𝐵 + с|𝑛 ≤ ▪ SE: Since {𝑥𝑡 } [and {𝜆𝑡 (𝜃)}] is SE and 𝑞(⋅) is guarantee.
𝑐𝑛 (|𝑎|𝑛 𝔼|𝐴|𝑛 + 𝔼|𝐵| 𝑛 + |с|𝑛 )) continuous, {𝑞𝑡 (𝜃)} is SE by Krengel. If 𝜃̂𝑡 = arg max 𝑄𝑇 (𝑥𝑡 , 𝜃) – peak unique, if 𝜃̂𝑡 ∈ arg max 𝑄𝑇 (𝑥𝑡 , 𝜃) –
𝜕𝜙(𝑥 ,𝜖 )∣ may not unique
• A3: sup ∣∣ <1 ▪ Bounded 1st moment: 𝔼|𝑞𝑡 |2 < ∞ ⟹
𝜕𝑥 ∣
𝑥,𝜖 𝔼|𝑞𝑡 | < ∞ by Jensen. Or use 𝔼|𝐴𝐵| ≤ Week 6
Power-𝒏 Contraction Theorem (SE + Bounded Moment) Pseudo-true 𝜃0 = arg max 𝑄∞ (𝜃): MLE minimizes KL distance
√𝔼𝐴2 𝔼𝐵2 , 𝐸|𝐴 ± 𝐵|𝑛 ≤ 𝑐𝑛 (𝐸|𝐴|𝑛 +
• A1: {𝜖𝑡 } are strictly i.i.d. 𝐸|𝐵|𝑛 ) 𝔼 log 𝑓0 − 𝔼 log 𝑓 (𝜃) (asymmetric, best DGP approx if misspecified; if well-
• A2: 𝔼 |𝜙(𝑥1 , 𝜖1 )|𝑛 < ∞ 𝜕𝑄 specified, attains CRLB = ℐ(𝜃0 )−1 = efficient). LS/NLS minimizes weighted
𝜕𝜙(𝑥 ,𝜖𝑡 )∣
𝑛 1.2.2. Stochastic equicontinuity: sup 𝑇 𝔼sup𝜃 ‖ 𝑇‖ < 2
• A3: 𝔼 [ sup ∣∣∣ ∣∣ ] < 1
𝜕𝜃 𝐿 -norm.
𝑥 𝜕𝑥 ∞ Proved by Simple Stochastic Equicontinuity Hausman Test (correct spec / exogeneity): Compares consistent 𝜃ˆ𝑇1
Krengel theorem – if you take SE parts in measuarble function, the result also Theorem
(NLS/OLS) vs efficient 𝜃ˆ𝑇2 (MLE/IV). If misspecified/endogenous ⟹ 𝜃01 ≠ 𝜃02 .
will be SE., does not work for 𝑥𝑡 = 𝑔(𝜃)𝑥𝑡−1 + ⋯ ▪ Norm Model: 1st moment 𝔼|𝑞𝑡 | < ∞ + 𝑞𝑡 (𝜃) 𝑑
is WB of order 1 on compact for some 𝜃 ∈ Θ Under 𝐻0 : √𝑇(𝜃ˆ𝑇 − 𝜃ˆ𝑇 ) → 𝒩(0, Σ1 − Σ2 ) or 𝐻𝑇 = 𝑇(𝜃ˆ𝑇 − 𝜃ˆ𝑇 )′(Σ̂1 −
1 2 1 2
Generalized Hölder Inequality softer then Cauchy (𝔼|𝐴𝐵| ≤ √𝔼𝐴2 𝔼𝐵2 )
𝑑
𝑛 𝑛 𝑝𝑞 (Cheat allowed 𝔼|𝑞𝑡 | = 𝔼|𝑒𝑡 2 |). −1
Σ̂2 ) (𝜃ˆ𝑇1 − 𝜃ˆ𝑇2 ) → 𝜒 2 (𝑘).
𝔼|𝑧𝑡 ⋅ 𝑤𝑡 |n ≤ (𝔼|𝑧𝑡 |𝑝 )𝑝 (𝔼|𝑤𝑡 |𝑞 )𝑞 , (𝑛 = ) ▪ TVP Model: 2nd moment 𝔼|𝑞𝑡 |2 < ∞ and
𝑝+𝑞 Diebold-Mariano (predictive accuracy on Test 𝑁): 𝐻0 : 𝔼[𝑑𝑡 ] = 0, 𝑑𝑡 =
Is to prove 𝑥𝑡 SE? ⟹ Krengel 𝔼|𝑓𝑡 |2 < ∞ + both 𝑞, 𝜙 are WB of order 2 on 𝑑̅ 𝑑
𝑒.𝑎.𝑠. compact Θ + True process is uniformly 𝑔(𝑒𝑡1 ) − 𝑔(𝑒𝑡2 ) ⟹ 𝐷𝑀 = √𝑁 ˆ → 𝒩(0,1).
𝜎𝑑
Is to prove 𝑥𝑡 (start) convergence to SE limit 𝑥𝑡 (|𝑥𝑡 (start) − 𝑥𝑡 | → 0)?
𝜕𝑓 2
AIC = 2𝑘 − 2𝐿 𝑇 (lowest wins) & 𝑅̅ 2: Penalize 𝑘 to prevent in-sample
⟹ MVT + Product e.a.s. contracting: sup | 𝑡+1 | < 1
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠. 𝜕𝑓𝑡 overfitting. If equal 𝑘 (e.g., all have 3 params), rank directly by in-sample 𝐿 𝑇 /
|𝑥𝑡 (𝜃, 𝜇1 ) − 𝑥𝑡 (𝜃)| → 0 (Product e.a.s. + MVT): If 𝑧𝑡 → 0 & {𝜖𝑡 } SE 1.2.3. Filter Convergence [Only when TVP in 𝜽
𝒑
̂ → 𝜽𝟎 𝑅2 . AIC pros: Works for misspecified & non-nested models; preferred when
𝑒.𝑎.𝑠.
with 𝔼 log + | 𝜖𝑡 | < ∞ ⟹ |𝜖𝑡 | ⋅ 𝑧𝑡 → 0. (For GARCH 𝑥𝑡 = 𝜎𝑡 𝜖𝑡 , apply questions] 𝐿̂ 𝑇 (𝜆̂𝑡 ) → 𝐿 𝑇 (𝜆𝑡 ) data is scarce (no split needed).
𝑒.𝑎.𝑠.
MVT to √⋅ first: |𝑥ˆ𝑡 − 𝑥𝑡 | ≤ |𝜖𝑡 |
1
|𝜎ˆ𝑡2 − 𝜎𝑡2 | → 0).
𝑒.𝑎.𝑠. Relative comparison by Likelihood is not prefered(overfitting)
2√𝜔 ▪ Since filter is 𝜆ˆ𝑡 → 𝜆𝑡 and 𝑞 is WB1 with
𝑝 Cross-Validation / Sub-sample on Time Series: Possible via buffer ℎ:
Week 3 ˆ
𝔼|𝑞𝑡 | < ∞, then sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0 Train(𝑇) → Drop(ℎ) → Test(𝑁) (or drop ℎ on both sides in CV). Pros:
Univariate Uniform Invertibility for any given 𝜃 sup|𝛼̂𝑡 (𝜃, 𝛼̂1 ) − 𝜃∈Θ
𝜃∈Θ by triangle inequality sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − Flexible custom loss 𝑔(𝑒𝑡 ). Cons: Wastes data (bad if scarce); cannot
𝑒.𝑎.𝑠. 𝑒.𝑎.𝑠.
𝛼𝑡 (𝜃)| → 0, 𝛼̂𝑡 → 𝛼𝑡 : 𝜃∈Θ distinguish two well-specified models as 𝑇, 𝑁 → ∞.
𝑒.𝑎.𝑠. 𝑄∞ (𝜃)| ≤ sup|𝑄𝑇 (𝜆ˆ 𝑇 ) − 𝑄𝑇 (𝜆 𝑇 )| + Small Samples & Outliers: Small 𝑇 (asymptotics fail) ⟹ Monte Carlo for
• A1: {𝑥𝑡 } SE (if 𝑥𝑡 generated |𝑥ˆ𝑡 − 𝑥𝑡 | → 0, perturbed ⏟
𝜃∈Θ
𝑝 finite-sample bias/skewness. Outliers/fat tails ⟹ Robust bounded 𝑞𝑡 or LARE
equation below) →0 (Filter Inver)
𝑝 (−|𝑢𝑡 |).
• A2: 𝔼 log + sup|𝜙(𝛼ˆ1 , 𝑥1 , 𝜃)| < ∞ ( 𝛼ˆ1 ∈ 𝛩) sup|𝑄𝑇 (𝜆 𝑇 ) − 𝑄∞ (𝜃)| → 0
θ∈Θ ⏟
𝜃∈Θ VaR Backtesting (Kupiec): Hit ℎ𝑡 = 𝕀(𝑟𝑡 < VaR𝑡 (𝛼)) ∼ Bern(𝛼).
A3: 𝔼 log sup ∣∣∣
̂ 𝑡+1 ∣
𝜕𝛼
• ∣∣ < 0 𝑝
→0 (by 1.2) 𝐻0 : 𝔼[ℎ𝑡 ] = 𝛼 ⟹ 𝑍 =
̅−𝛼) 𝑑
√𝑇(ℎ
→ 𝒩(0,1). Var = 𝛼(1 − 𝛼)
̂
𝜃,𝛼 ̂𝑡
𝜕𝛼
√𝛼(1−𝛼)
1.3. Identifiable uniqueness: sup𝜃∈𝑆 𝑐 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 )
Multivariate Uniform Invertibility : 𝜆
Penalized (arg max 𝐿 𝑇 (𝜃) − 𝛼 ‖𝜃‖2 , 𝜆 > 0):
𝑒.𝑎.𝑠. Proved by Identifiable uniqueness theorem 𝑇
Perturbed Dynamic Equation Theorem sup|𝑓ˆ𝑡 − 𝑓𝑡 | → 0: 1.3.1. Continuity: 𝑄∞ (𝜃) continuous on Θ.
𝜃∈Θ • Shrinks |𝜃ˆ(𝜆)| < |𝜃ˆ(0)| (prior: params are small). Good for
𝑒.𝑎.𝑠. Automatically proved when 1.2 Uniform prediction (lower finite-𝑇 var), bad for structural (biased).
• A1: 𝛼̂𝑡 → 𝛼𝑡 (SE) proved above.
Convergence of 𝑄𝑇 proved and continuity of 𝑞 LASSO (𝜆 ∑ | 𝜃𝑖 |) sets params to exact 0.
• A2: 𝔼 log + |𝜙(𝑓1 , 𝛼𝑡 , 𝜃)| < ∞
1.3.2. Unique maximizer: 𝑄∞ (𝜃) < 𝑄∞ (𝜃0 ) for all 𝜃 • Consistency: Needs 𝛼 > 0 (penalty vanishes); if 𝛼 = 0, 𝜆 >
A3: 𝔼 log sup ∣∣∣ 𝑡+1 ∣∣∣ < 0
𝜕𝑓
• Holds since Θ is compact + Continuity (𝑄∞ (𝜃)
𝜃,𝑓 𝜕𝑓𝑡 0 ⟹ inconsistent! Does not reduce asymptotic variance.
•
A4: 𝔼 log + |𝑓𝑡 | < ∞. (True process finite, assumed, regularity) continuous by 1.1) + Unique Maximizer(Given): • Bayesian MAP ⊂ Penalized (not vice versa: requires 𝐿 𝑇 , prior
▪ MLE: well-specified + 𝜃0 is identified, unique density 𝑝(𝜃), no 𝑇 𝛼 control).
A5: Uniformly Lipschitz / Bounded derivatives, sup ∣∣∣ 𝑡+1 ∣∣∣ <
𝜕𝑓
•
𝜕𝛼𝑡 by Information inequality Ensembles (𝑥ˆ𝑡 = ∑ 𝜓𝑖 𝑥ˆ𝑡𝑖 ): Stacking: linear comb. on full train set. Random
2𝑓
∞, sup ∣∣∣ 𝑡+1 ∣∣∣ < ∞
𝜕
𝜕𝛼𝑡 𝜕𝑓𝑡
Forest (Bagging): average of trees on bootstrap samples. Boosting
DGP (SE) vs FILTER (Invertibility) – The Universal Rule: (AdaBoost/XGBoost): sequential fit; low-fit obs re-sampled with higher prob.
