Open source regression software for Matlab/Octave

ARESLab: Adaptive Regression Splines toolbox

Version 1.5.1 (June 2, 2011) - download [Picture: Two surfaces]

ARESLab is a Matlab/Octave toolbox for building piecewise-linear and piecewise-cubic regression models using Jerome Friedman's Multivariate Adaptive Regression Splines technique (also known as MARS).

Multivariate Adaptive Regression Splines have the ability to model complex and high-dimensional data dependencies. The model takes the form of an expansion in product spline basis functions, where the number of basis functions as well as the parameters associated with each one (product degree and knot locations) are automatically determined by the data through a forward/backward iterative approach.

The toolbox code is licensed under the GNU GPL licence.

Reference manual can be downloaded here (it is also included in the .zip file).

M5PrimeLab: M5' regression tree and model tree toolbox

Version 1.0.1 (September 3, 2010) - download [Picture: Bonzai]

M5PrimeLab is a Matlab/Octave toolbox for building regression trees and model trees using M5' method. M5PrimeLab accepts input variables to be continuous, binary, and categorical, as well as manages missing values.

Model trees combine a conventional regression tree with the possibility of linear regression functions at the leaves. This representation usually provides higher accuracy than regression trees but preserves the advantage of clear and easy-to-interpret structure.

The toolbox code is licensed under the GNU GPL licence.

Reference manual can be downloaded here (it is also included in the .zip file).

Locally-Weighted Polynomials

Version 1.3 (February 10, 2010) - download

Locally Weighted Polynomials (also called Locally Weighted Regression or Moving Least Squares) with Gaussian weight function.

LWP approximation is designed to address situations in which models of global behaviour do not perform well or cannot be effectively applied without undue effort. The LWP approximation is carried out by pointwise fitting of low-degree polynomials to localized subsets of the data. The advantage of this method is that the analyst is not required to specify a global function. However, the method requires relatively high computational resources when finally predicting output values at the query points.

Radial Basis Function interpolation

Version 1.1 (August 12, 2009) - download

Radial Basis Function interpolation with biharmonic, multiquadric, inverse multiquadric, thin plate spline, and Gaussian basis functions with or without the polynomial term.

RBF interpolation uses a series of basis functions that are symmetric and centered at each sampling point. Radial basis functions are a special class of functions with their main feature being that their response decreases (or increases) monotonically with distance from a central point. The center, the distance scale, and the precise shape are parameters of the model.

GMDH-type Polynomial Neural Networks

Version 1.5 (June 2, 2011) - download

[Picture: GMDH network examples]

A simple implementation of Group Method of Data Handling (GMDH) for building Polynomial Neural Networks. The algorithm iteratively builds the network layer-by-layer using training data while the exact structure (connectivity) and size (number of layers) of the network is controlled by an evaluation criterion - either measuring performance in an additional validation data (i.e. using the so-called "regularity criterion") or explicitly taking network's complexity into account (using the Corrected Akaike's Information Criterion or Minimum Description Length). Algorithm's parameters also include max number of inputs for individual neurons, degree of polynomials in the neurons, whether to allow the neurons to have inputs not only from the immediately preceding layer but also from the original input variables, number of neurons in a layer, whether to decrease the number of neurons in each subsequent layer etc.

Shepard interpolation

download.

Shepard interpolation is a subset of inverse distance weighting methods and may be viewed as a special case of RBF interpolation. While it is rarely as accurate as other RBF interpolations, it is simple and computationally very efficient (does not require to solve linear equations).

Gints Jekabsons, Dr.sc.ing.

Riga Technical University

Institute of Applied Computer Systems

Meza str. 1/3, LV-1048, Riga, Latvia