Parameter estimation in generalized partial linear models with conic quadratic programming

Abstract

In statistics, regression analysis is a technique, used to understand and model therelationship between a dependent variable and one or more independent variables.Multiple Adaptive Regression Spline (MARS) is a form of regression analysis. It is anon-parametric regression technique and can be seen as an extension of linear modelsthat automatically models non-linearities and interactions. MARS is very importantin both classification and regression, with an increasing number of applications inmany areas of science, economy and technology.In our study, we analyzed Generalized Partial Linear Models (GPLMs), which areparticular semiparametric models. GPLMs separate input variables into two partsand additively integrates classical linear models with nonlinear model part. In orderto smooth this nonparametric part, we use Conic Multiple Adaptive Regression Spline(CMARS), which is a modified form of MARS. MARS is very benefical for highdimensional problems and does not require any particular class of relationship betweenthe regressor variables and outcome variable of interest. This technique offers a great advantage for fitting nonlinear multivariate functions. Also, the contribution of thebasis functions can be estimated by MARS, so that both the additive and interactioneffects of the regressors are allowed to determine the dependent variable. There aretwo steps in the MARS algorithm: the forward and backward stepwise algorithms. Inthe first step, the model is constructed by adding basis functions until a maximumlevel of complexity is reached. Conversely, in the second step, the backward stepwisealgorithm reduces the complexity by throwing the least significant basis functions fromthe model.In this thesis, we suggest not using backward stepwise algorithm, instead, we employa Penalized Residual Sum of Squares (PRSS). We construct PRSS for MARS as aTikhonov Regularization Problem. We treat this problem using continuous optimizationtechniques which we consider to become an important complementary technologyand alternative to the concept of the backward stepwise algorithm. Especially, we applythe elegant framework of Conic Quadratic Programming (CQP) an area of convexoptimization that is very well-structured, hereby, resembling linear programming and,therefore, permitting the use of interior point methods.At the end of this study, we compare CQP with Tikhonov Regularization problemfor two different data sets, which are with and without interaction effects. Moreover,by using two another data sets, we make a comparison between CMARS and twoother classification methods which are Infinite Kernel Learning (IKL) and TikhonovRegularization whose results are obtained from the thesis, which is on progress.