Optimal Penalty Choice in High Dimensional Models
Abstract
Penalty choice in high-dimensional models is central to post-selection inference, yet common tuning rules—theory-driven noise-dominating penalties, cross-validation, and information criteria—target different objects and can yield meaningfully different model sizes and confidence-interval behavior. We propose an inference-calibrated penalty that anchors λ to a feasible high-probability bound on self-normalized correlations, then selects the remaining scaling constant via cross-fitting to minimize an estimable proxy for inference risk. We provide conditions for asymptotic validity, characterize the under- versus over-penalization trade-off, and introduce practical diagnostics for sensitivity analysis. Monte Carlo and empirical results show that inference-calibrated penalties stabilize model selection and improve coverage relative to prediction-optimal choices, with modest predictive cost.