Modeling and computation with structured nonconvexity: Some examples in statistical learning

Speaker: Rahul Mazumder, MIT Sloan School of Management

Abstract: Many statistical estimation problems are naturally modeled as solutions to  discrete or nonconvex optimization problems. While continuous (especially, convex) optimization techniques continue to play a significant role towards our statistical and computational understanding of these methods, some other techniques in mathematical optimization especially, mixed integer optimization (MIO), have not been explored to the fullest potential. However, despite the significant advances in MIO over the past several years, off-the-shelf applications of MIO solvers may not be suitable for large scale instances that arise in practice. In this talk, I will discuss how techniques from MIO and tools in convex optimization can be brought together to build new (transparent) algorithms that might be suitable for large problem instances that are beyond the capabilities of off-the-shelf MIO solvers. 

I will first discuss previously unobserved intriguing statistical properties of optimal best-subsets estimators in high-dimensional linear regression. Time permitting, I will describe our recent work in the context of (a) building sparse additive models with interactions that obey a hierarchy principle and (b) understanding the posterior landscape in Bayesian variable selection problems (e.g., two point priors, nonlocal priors). Both (a) and (b) involve delicate combinatorial modeling using MIO principles, these models lay the foundation for creating new computational algorithms that seem to be useful to address these problems.