"Simple Meets Optimal: Some Surprising Results for Model Selection Using One-Step Thresholding"
Professor Waheed U. Bajwa, Department of Electrical & Computer Engineering,
CoRE Building Lecture Hall
The problem of model selection arises in a number of statistics and signal processing applications, such as subset selection in linear regression, estimation of structures in graphical models, and signal denoising. In this talk, we introduce a simple algorithm, termed one-step thresholding (OST) algorithm, for model-order agnostic model selection in linear inference problems. We utilize two geometric measures of coherence, namely, worst-case coherence and average coherence, among the columns of a design matrix to provide an in-depth analysis of OST for model selection. One of the key insights offered by the ensuing analysis is that OST can successfully carry out model selection even when methods based on convex optimization such as the lasso fail due to the rank deficiency of the submatrices of the design matrix. In addition, we show that OST has the ability to perform near-optimally for a number of generic (random or deterministic) matrices as long as the design matrix satisfies conditions that are easily computable in polynomial time -- an area of great interest in applications such as genetic biomarker identification using gene expression data.
Waheed U. Bajwa received BE (with Honors) degree in electrical engineering from the National University of Sciences and Technology, Pakistan in 2001, and MS and PhD degrees in electrical engineering from the University of Wisconsin-Madison in 2005 and 2009, respectively. He was a Postdoctoral Research Associate in the Program in Applied and Computational Mathematics at Princeton University from 2009 to 2010, and a Research Scientist in the Department of Electrical and Computer Engineering at Duke University from 2010 to 2011. He is currently an Assistant Professor in the Department of Electrical and Computer Engineering at Rutgers University. His research interests include high-dimensional inference and inverse problems, compressed sensing, statistical signal processing, wireless communications, and applications in biological sciences, complex networked systems, and radar & image processing.
Dr. Bajwa was affiliated with Communications Enabling Technologies, Pakistan – the research arm of Avaz Networks Inc., Irvine, CA (now Quartics LLC) – from 2000-2003, with the Center for Advanced Research in Engineering, Pakistan during 2003, and with the RF and Photonics Lab of GE Global Research, Niskayuna, NY during the summer of 2006. He received the Best in Academics Gold Medal and President's Gold Medal in Electrical Engineering from the National University of Sciences and Technology (NUST) in 2001, and the Morgridge Distinguished Graduate Fellowship from the University of Wisconsin-Madison in 2003. He was Junior NUST Student of the Year (2000), Wisconsin Union Poker Series Champion (Spring 2008), and President of the University of Wisconsin-Madison chapter of Golden Key International Honor Society (2009). He served as a Guest Associate Editor for a special issue of Elsevier Physical Communication Journal on “Compressive Sensing in Communications” (2010-2011). He is currently a member of the IEEE, Pakistan Engineering Council, and Golden Key International Honor Society.