Available at: https://digitalcommons.calpoly.edu/theses/3230
Date of Award
3-2026
Degree Name
MS in Statistics
Department/Program
Statistics
College
College of Science and Mathematics
Advisor
Trevor D. Ruiz
Advisor Department
Statistics
Advisor College
College of Science and Mathematics
Abstract
In high-dimensional regression problems, dimension reduction methods are often used to address the challenges of multicollinearity and estimation instability. Partial dimension reduction extends these ideas by applying dimension reduction to a subset of the predictors, while the remaining predictors are modeled without compression. This approach is particularly useful when it is important to retain variability and interpretability in certain predictors.
This thesis investigates the empirical performance of partial dimension reduction algorithms and introduces a novel algorithm, Iterative Partial Residual (IPR). Two algorithms are considered: a baseline algorithm, Marginal Residual (MR), and the proposed IPR method. Their predictive performance is evaluated using the mean squared error (MSE) of the coefficient estimates across a range of simulation settings, including varying predictor dimensions and covariance structures. Specifically, simulations are conducted under envelope and low-rank spiked covariance structures.
The results show that IPR and MR generally exhibit similar predictive performance across most settings. Partial dimension reduction shows the greatest benefit over full dimension reduction under the low-rank spiked covariance structures, while the improvements over full dimension reduction methods are more limited under the envelope covariance structures. However, in some settings with dependence between predictor subsets, the IPR algorithm showed improved estimation of the coefficients associated with the predictors that did not go through dimension reduction. Additionally, convergence issues are observed for the IPR algorithm in certain high-dimensional settings, highlighting practical limitations for application. These results provide an understanding of the strengths and limitations of partial dimension reduction methods in high-dimensional regression.