Simply Salford Blog

Webinar Recap: 3 Ways to Improve Regression, Part 2

Posted by Kaitlin Onthank on Thu, Jan 28, 2016 @ 10:22 AM

Did you miss our webinar yesterday? It's never too late to register to get the recording

Read More

Topics: stochastic gradient boosting, Nonlinear Regression, Regression Splines, Regression

Webinar Recap: 3 Ways to Improve Regression, Part 1

Posted by Kaitlin Onthank on Thu, Jan 21, 2016 @ 09:19 AM

Did you miss our webinar yesterday? It's never too late to register to get the recording

Read More

Topics: RandomForests, Random Forests, stochastic gradient boosting, Nonlinear Regression, Regression

Getting Results From 'Out-Of-Bag' Cross-Validation In MARS

Posted by Dan Steinberg on Tue, Jun 25, 2013 @ 09:09 AM

In this post we continue the discussion of saving OOB (out-of-bag) predictions when testing via cross-validation with MARS. The principles for MARS are the same as they are for CART and the organization of the file saved follows the same high-level logic. However, as the details are a little different we thought it would be worthwhile exhibiting the OOB results and how we get them in the context of MARS as well. Recalling that when using K-fold cross-validation we actually develop K different models each tested on a different test sample (CVBIN) and that the final model and results are reported for an overall model built on all the data where nothing has been held back for test. The topic of discussion is how to obtain the equivalent of test sample predictions so that we can manipulate and further analyze the test sample residuals (for regressions).

Read More

Topics: OOB, Nonlinear Regression, MARS, Cross-Validation

Reuse MARS Regression Spline Basis Functions in a New Dataset

Posted by Dan Steinberg on Wed, Apr 17, 2013 @ 07:29 AM

MARS(Multivariate Adaptive Regression Splines), introduced by Stanford University data mining guru Professor Jerome H. Friedman in 1988, is one of the landmarks in the evolution of regression methods. For the first time analysts could leverage a search mechanism intended to automatically discover nonlinearity and interactions in the context of classical regression. The MARS procedure involves a forward stepwise model building stage followed by a backwards elimination of unneeded predictors to arrive at surprisingly high performance models, all automatically. At the heart of the MARS algorithm is the search for "knots" or breaks in the range of a predictor allowing a regression model containing that predictor to have different slopes in each region. Breaking predictors into regions permits nonlinearity, and when interactions are constructed from regions of predictors, remarkable discoveries are enabled.

Read More

Topics: Nonlinear Regression, basis functions, Regression Splines, MARS