Software for classification
Here you can download some software for classification. Newer versions will be released from time to time (anyone is welcome to suggest a modification or report a bug).
VariClass
Version 0.1.3 (November 3, 2009) - download
This software tool (Windows executable) is developed primarily for educational purposes - to work with numerical datasets performing two-class classification tasks using different kinds of simple classification techniques. VariClass is based on the VariReg tool with a number of its methods modified for the classification task. The software is developed for non-commercial research and educational use. For all the methods involving polynomials, the problem of logistic regression estimation is solved using Iteratively Re-weighted Least Squares.
The classification techniques implemented in the VariClass are the following:
- "Full" polynomials of any user-predefined degree;
- Sparse polynomials created using Sequential Forward Selection (SFS) with Corrected Akaike's Information Criterion (AICC), Bayesian Information Criterion (BIC) (which is equal to the "two-part" Minimum Description Length criterion, MDL), Generalized Cross-Validation criterion (GCV), or Cross-Validation (CV);
- Sparse polynomials created using Steepest Descent Hill Climbing with AICC, BIC, GCV, or CV;
- Sparse polynomials created using Sequential Floating Forward Selection (SFFS) with AICC, BIC, GCV, or CV;
- Sparse polynomials built by Floating Adaptive Basis Function Construction (F-ABFC) with AICC, BIC, or GCV;
- Ensembles of models built by F-ABFC - EF-ABFC;
- Locally Weighted Polynomials (LWP), also called Locally Weighted Regression or Moving Least Squares, of any degree with Gaussian weight function;
- k-Nearest Neighbours (k-NN);
- Multivariate Adaptive Regression Splines (MARS); here the binary classification problem is treated simply as a regression problem.
Source code of ABFC for classification for Matlab/Octave
Version 1.3 (March 21, 2010) - download
The zip file includes full Matlab/Octave source code implementing F-ABFC and EF-ABFC with Logistic Regression and Iteratively Re-weighted Least Squares for classification task. The implemented versions of the methods are those which are described in the Machine Learning book chapter.
Unfortunately the implementations in Matlab/Octave can be orders of magnitude slower than those in the VariClass tool.
Source code of Iteratively Re-weighted Least Squares for Logistic Regression in Object Pascal
The zip file includes full source code of a simple implementation of Iteratively Re-weighted Least Squares for Logistic Regression (also called Maximum-Entropy Classifier) written in Object Pascal - download.
