Overview
The SVM class implements a kernelized Support Vector Machine for binary classification. It uses the dual formulation with soft margin support and Sequential Minimal Optimization (SMO) for training.
Namespace: mlpp::classifiers::kernel
Optimization problem:
maximize W(α) = Σ α_i − 1/2 Σ Σ α_i α_j y_i y_j K(x_i, x_j)
subject to:
0 ≤ α_i ≤ C
Σ α_i y_i = 0
Constructor
SVM
Construct an SVM model with training data and kernel configuration.
data
const std::vector<Vector>&
Training samples as a vector of feature vectors
Class labels in {−1, +1} as an Eigen::VectorXd
Kernel function for computing similarity between samples
Soft margin penalty parameter. Controls the trade-off between margin maximization and training error minimization.
#include "SVM/SVM.hpp"
using namespace mlpp::classifiers::kernel;
std::vector<Vector> training_data = { ... };
Eigen::VectorXd labels(n);
labels << 1, -1, 1, -1, ...; // Binary labels: +1 or -1
auto rbf_kernel = [](const Vector& x1, const Vector& x2) {
double gamma = 0.5;
return std::exp(-gamma * (x1 - x2).squaredNorm());
};
SVM svm(training_data, labels, rbf_kernel, 1.0);
Methods
fit
Train the SVM model using the provided training data.
This method performs the optimization to find the optimal dual variables (alphas) and bias term. The optimization strategy is implementation-defined.
decision
Evaluate the decision function for a given sample.
[[nodiscard]]
double decision(const Vector& x) const;
Input feature vector to evaluate
Decision function value: f(x) = Σ α_i y_i K(x_i, x) + b
Vector test_sample = ...;
double score = svm.decision(test_sample);
// score > 0 means class +1
// score < 0 means class -1
predict
Predict the class label for a given sample.
[[nodiscard]]
int predict(const Vector& x) const;
Input feature vector to classify
Predicted class label: +1 or -1
Vector test_sample = ...;
int label = svm.predict(test_sample); // Returns +1 or -1
support_indices
Get the indices of support vectors.
[[nodiscard]]
std::vector<std::size_t> support_indices(double eps = 1e-8) const;
Threshold for determining non-zero alpha values
Indices i where α_i > eps (support vector indices)
auto sv_indices = svm.support_indices();
std::cout << "Number of support vectors: " << sv_indices.size() << std::endl;
Type aliases
using LabelVector = Eigen::VectorXd;
using AlphaVector = Eigen::VectorXd;
Example usage
#include "SVM/SVM.hpp"
#include <Eigen/Dense>
#include <vector>
#include <iostream>
using namespace mlpp::classifiers::kernel;
using Vector = Eigen::VectorXd;
int main() {
// Generate training data
std::vector<Vector> X_train;
Eigen::VectorXd y_train(4);
Vector x1(2); x1 << 1.0, 2.0;
Vector x2(2); x2 << 2.0, 3.0;
Vector x3(2); x3 << 3.0, 3.0;
Vector x4(2); x4 << 5.0, 5.0;
X_train = {x1, x2, x3, x4};
y_train << -1, -1, 1, 1;
// Define RBF kernel
auto rbf = [](const Vector& a, const Vector& b) {
double gamma = 0.5;
return std::exp(-gamma * (a - b).squaredNorm());
};
// Train SVM
SVM svm(X_train, y_train, rbf, 1.0);
svm.fit();
// Make predictions
Vector test(2); test << 4.0, 4.0;
int prediction = svm.predict(test);
double score = svm.decision(test);
std::cout << "Prediction: " << prediction << std::endl;
std::cout << "Decision score: " << score << std::endl;
// Inspect support vectors
auto sv_idx = svm.support_indices();
std::cout << "Support vectors: " << sv_idx.size() << std::endl;
return 0;
}