Parameter estimation in generalized partial linear models with tikhanov regularization

Abstract

Regression analysis refers to techniques for modeling and analyzing several variablesin statistical learning. There are various types of regression models. In our study,we analyzed Generalized Partial Linear Models (GPLMs), which decomposes inputvariables into two sets, and additively combines classical linear models with nonlinearmodel part. By separating linear models from nonlinear ones, an inverse problemmethod Tikhonov regularization was applied for the nonlinear submodels separately,within the entire GPLM. Such a particular representation of submodels provides botha better accuracy and a better stability (regularity) under noise in the data.We aim to smooth the nonparametric part of GPLM by using a modied form of Mul-tiple Adaptive Regression Spline (MARS) which is very useful for high-dimensionalproblems and does not impose any specic relationship between the predictor anddependent variables. Instead, it can estimate the contribution of the basis functionsso that both the additive and interaction eects of the predictors are allowed to de-termine the dependent variable. The MARS algorithm has two steps: the forward and backward stepwise algorithms. In the rst one, the model is built by adding basisfunctions until a maximum level of complexity is reached. On the other hand, thebackward stepwise algorithm starts with removing the least signicant basis functionsfrom the model.In this study, we propose to use a penalized residual sum of squares (PRSS) insteadof the backward stepwise algorithm and construct PRSS for MARS as a Tikhonovregularization problem. Besides, we provide numeric example with two data sets; onehas interaction and the other one does not have. As well as studying the regular-ization of the nonparametric part, we also mention theoretically the regularizationof the parametric part. Furthermore, we make a comparison between Innite KernelLearning (IKL) and Tikhonov regularization by using two data sets, with the dier-ence consisting in the (non-)homogeneity of the data set. The thesis concludes withan outlook on future research.