Course Project

Proposal due: 2023-11-02 @ 11:59PM

Presentation: 2023-12-12 and 2023-12-14 (tentative)

This page lists some potential course project ideas. The goal of the project is to review recent developments in statistical computing, implement in Julia, and compare the related methods. Two or three students should team up to accomplish the goal. Each team may propose a paper on its own or choose one paper from the list below (no duplication is allowed) and submit a project proposal by the due date.

Stochastic optimization

In large-scale optimization, often the objective function or its derivatives can only be estimated. In this case, stochastic methods come to rescue. Recent developments include:

• Han, R., Luo, L., Lin, Y., & Huang, J. (2023). Online Inference with Debiased Stochastic Gradient Descent. Biometrika, To appear. https://doi.org/10.1093/biomet/asad046

• Yu, G., Yin, L., Lu, S., & Liu, Y. (2020). Confidence Intervals for Sparse Penalized Regression With Random Designs. Journal of the American Statistical Association, 115(530), 794–809. https://doi.org/10.1080/01621459.2019.1585251

Mixed integer optimization for model selection

Model selection is a difficult statistical problem with an exponential complexity. A typical example is high-dimensional linear model with L0 penalty. Nonetheless, recent progress in mixed integer optimization (MIO) has made large-scale problems tractable. They include:

• Hazimeh, H., Mazumder, R., & Radchenko, P. (2023). Grouped variable selection with discrete optimization: Computational and statistical perspectives. The Annals of Statistics, 51(1). https://doi.org/10.1214/21-AOS2155

Algorithms for square-root lasso

The square-root lasso has a theoretical advantage over the plain lasso in easing tuning parameter selection by dispensing with the need of knowing the noise variance. However, fitting a square-root lasso model is computationally more challenging due to the nondifferentiability of the loss function. Recent computational developments include:

• Chu, H. T. M., Toh, K.-C., & Zhang, Y. (2022). On Regularized Square-root Regression Problems: Distributionally Robust Interpretation and Fast Computations. Journal of Machine Learning Research, 23, 1–39.

• Tang, P., Wang, C., Sun, D., & Toh, K.-C. (2020). A Sparse Semismooth Newton Based Proximal Majorization-Minimization Algorithm for Nonconvex Square-Root-Loss Regression Problems. Journal of Machine Learning Research, 21, 1–38.

• Li, X., Jiang, H., Haupt, J., Arora, R., Liu, H., Hong, M., & Zhao, T. (2020). On Fast Convergence of Proximal Algorithms for SQRT-Lasso Optimization: Don’t Worry About its Nonsmooth Loss Function. Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, 49–59. https://proceedings.mlr.press/v115/li20a.html