Logistic Regression

Overview

The logistic regression module provides two classes for classification:

LogisticRegressionBinary: Binary classification using the logistic (sigmoid) model
LogisticRegressionMulti: Multi-class classification using one-vs-rest strategy

Namespace: mlpp::classifiers Both classes use gradient descent for maximum-likelihood estimation of parameters.

LogisticRegressionBinary

Binary logistic regression for two-class problems. Template parameters:

Scalar: Numeric type for computations (default: double)

Model

The binary logistic model:

P(y = 1 | x; θ) = σ(θᵀ x̃)
where σ(z) = 1/(1 + exp(-z))
x̃ = [1, x] (feature vector with bias term)

Objective (binary cross-entropy):

L(θ) = -(1/n) Σ [yᵢ log pᵢ + (1-yᵢ) log(1-pᵢ)]

Constructor

template<typename Scalar = double>
LogisticRegressionBinary();

Default constructor.

#include "logistic_regression.h"

using namespace mlpp::classifiers;

LogisticRegressionBinary<double> clf;

Methods

fit

Fit binary logistic regression model.

void fit(const Matrix& X, const Vector& y,
         Scalar learning_rate = Scalar(0.01),
         std::size_t max_iter = 1000,
         Scalar tol = Scalar(1e-6));

const Matrix&

Training data matrix of shape (n_samples × n_features)

const Vector&

Binary labels vector of shape (n_samples). Must contain only 0 or 1

learning_rate

Scalar

default:"0.01"

Step size for gradient descent updates

max_iter

std::size_t

default:"1000"

Maximum number of gradient descent iterations

tol

Scalar

default:"1e-6"

Convergence tolerance. Training stops when ||Δθ||∞ < tol

Eigen::MatrixXd X(100, 5);  // 100 samples, 5 features
Eigen::VectorXd y(100);      // Binary labels: 0 or 1

// ... populate X and y ...

LogisticRegressionBinary<> clf;
clf.fit(X, y, 0.1, 2000, 1e-5);

predict_proba

Predict class probabilities.

Vector predict_proba(const Matrix& X) const;

const Matrix&

Data matrix of shape (n_samples × n_features)

return

Vector

Predicted probabilities P(y=1|x) for each sample, shape (n_samples)

Eigen::MatrixXd X_test(20, 5);
// ... populate X_test ...

Eigen::VectorXd probs = clf.predict_proba(X_test);
// probs(i) = probability that sample i belongs to class 1

predict

Predict class labels.

Vector predict(const Matrix& X, Scalar threshold = Scalar(0.5)) const;

const Matrix&

Data matrix of shape (n_samples × n_features)

threshold

Scalar

default:"0.5"

Decision threshold. Predictions with probability ≥ threshold are assigned class 1

return

Vector

Predicted class labels (0 or 1) for each sample, shape (n_samples)

Eigen::VectorXd predictions = clf.predict(X_test);
// predictions(i) ∈ {0, 1}

// Use custom threshold
Eigen::VectorXd high_precision = clf.predict(X_test, 0.7);

coefficients

Get the learned coefficient vector.

const Vector& coefficients() const;

return

const Vector&

Coefficient vector θ including intercept as first element

auto theta = clf.coefficients();
std::cout << "Intercept: " << theta(0) << std::endl;
std::cout << "Weights: " << theta.tail(theta.size() - 1).transpose() << std::endl;

intercept

Get the intercept term.

Scalar intercept() const;

return

Scalar

Intercept (bias) term θ₀

double b = clf.intercept();

Type aliases

using Matrix = Eigen::Matrix<Scalar, Eigen::Dynamic, Eigen::Dynamic>;
using Vector = Eigen::Matrix<Scalar, Eigen::Dynamic, 1>;
using Index = Eigen::Index;

LogisticRegressionMulti

Multi-class logistic regression using one-vs-rest strategy. Template parameters:

Scalar: Numeric type for computations (default: double)

Model

One-vs-rest approach:

Trains K independent binary classifiers
Each classifier k learns to separate class k from all others
Outputs are normalized to produce valid probabilities:

P(y = k | x) = σ(Θₖ x̃) / Σⱼ σ(Θⱼ x̃)

Constructor

template<typename Scalar = double>
LogisticRegressionMulti();

Default constructor.

LogisticRegressionMulti<double> clf;

Methods

fit

Fit multi-class logistic regression model.

void fit(const Matrix& X, const Vector& y,
         Scalar learning_rate = Scalar(0.01),
         std::size_t max_iter = 1000,
         Scalar tol = Scalar(1e-6));

const Matrix&

Training data matrix of shape (n_samples × n_features)

const Vector&

Class labels vector of shape (n_samples). Must be integer indices starting from 0

learning_rate

Scalar

default:"0.01"

Step size for gradient descent

max_iter

std::size_t

default:"1000"

Maximum iterations per binary classifier

tol

Scalar

default:"1e-6"

Convergence tolerance

Eigen::MatrixXd X(150, 4);  // 150 samples, 4 features
Eigen::VectorXd y(150);      // Class labels: 0, 1, 2, ..., K-1

// ... populate X and y ...

LogisticRegressionMulti<> clf;
clf.fit(X, y, 0.05, 1500);

predict_proba

Predict class probabilities for all classes.

Matrix predict_proba(const Matrix& X) const;

const Matrix&

Data matrix of shape (n_samples × n_features)

return

Matrix

Probability matrix of shape (n_samples × n_classes). Each row sums to 1

Eigen::MatrixXd X_test(20, 4);
// ... populate X_test ...

Eigen::MatrixXd probs = clf.predict_proba(X_test);
// probs(i, k) = P(y = k | x_i)

for (int i = 0; i < 5; ++i) {
    std::cout << "Sample " << i << " probabilities: " 
              << probs.row(i) << std::endl;
}

predict

Predict class labels.

Vector predict(const Matrix& X) const;

const Matrix&

Data matrix of shape (n_samples × n_features)

return

Vector

Predicted class indices for each sample, shape (n_samples)

Eigen::VectorXd predictions = clf.predict(X_test);
// predictions(i) ∈ {0, 1, ..., K-1}

coefficients

Get the coefficient matrix for all classes.

const Matrix& coefficients() const;

return

const Matrix&

Coefficient matrix Θ of shape (n_classes × (n_features + 1)). Each row contains the parameters for one binary classifier (intercept in first column)

auto Theta = clf.coefficients();
std::cout << "Coefficients shape: " << Theta.rows() << " × " << Theta.cols() << std::endl;
std::cout << "Class 0 intercept: " << Theta(0, 0) << std::endl;

Type aliases

using Matrix = Eigen::Matrix<Scalar, Eigen::Dynamic, Eigen::Dynamic>;
using Vector = Eigen::Matrix<Scalar, Eigen::Dynamic, 1>;
using Index = Eigen::Index;

Example usage

Binary classification

#include "logistic_regression.h"
#include <Eigen/Dense>
#include <iostream>

using namespace mlpp::classifiers;

int main() {
    // Generate synthetic binary classification data
    const int n_samples = 200;
    const int n_features = 2;
    
    Eigen::MatrixXd X = Eigen::MatrixXd::Random(n_samples, n_features) * 2;
    Eigen::VectorXd y(n_samples);
    
    // Simple linear separation
    for (int i = 0; i < n_samples; ++i) {
        y(i) = (X(i, 0) + X(i, 1) > 0) ? 1.0 : 0.0;
    }
    
    // Train binary classifier
    LogisticRegressionBinary<> clf;
    clf.fit(X, y, 0.1, 1000, 1e-6);
    
    std::cout << "Training complete!" << std::endl;
    std::cout << "Intercept: " << clf.intercept() << std::endl;
    std::cout << "Coefficients: " << clf.coefficients().tail(n_features).transpose() << std::endl;
    
    // Make predictions
    Eigen::MatrixXd X_test(3, n_features);
    X_test << 1.0, 1.0,
              -1.0, -1.0,
              0.5, -0.5;
    
    Eigen::VectorXd probs = clf.predict_proba(X_test);
    Eigen::VectorXd preds = clf.predict(X_test);
    
    for (int i = 0; i < X_test.rows(); ++i) {
        std::cout << "Sample " << i << ": P(y=1) = " << probs(i) 
                  << ", prediction = " << preds(i) << std::endl;
    }
    
    return 0;
}

Multi-class classification

#include "logistic_regression.h"
#include <Eigen/Dense>
#include <iostream>

using namespace mlpp::classifiers;

int main() {
    // Generate 3-class dataset
    const int n_samples = 300;
    const int n_features = 2;
    const int n_classes = 3;
    
    Eigen::MatrixXd X(n_samples, n_features);
    Eigen::VectorXd y(n_samples);
    
    // Create three clusters
    for (int i = 0; i < n_samples; ++i) {
        int label = i / 100;
        X(i, 0) = Eigen::VectorXd::Random(1)(0) + label * 3.0;
        X(i, 1) = Eigen::VectorXd::Random(1)(0) + label * 2.0;
        y(i) = label;
    }
    
    // Train multi-class classifier
    LogisticRegressionMulti<> clf;
    clf.fit(X, y, 0.1, 2000);
    
    std::cout << "Training complete!" << std::endl;
    std::cout << "Coefficient matrix shape: " 
              << clf.coefficients().rows() << " × " 
              << clf.coefficients().cols() << std::endl;
    
    // Make predictions
    Eigen::MatrixXd X_test(3, n_features);
    X_test << 0.0, 0.0,
              3.0, 2.0,
              6.0, 4.0;
    
    Eigen::MatrixXd probs = clf.predict_proba(X_test);
    Eigen::VectorXd preds = clf.predict(X_test);
    
    for (int i = 0; i < X_test.rows(); ++i) {
        std::cout << "Sample " << i << ":" << std::endl;
        std::cout << "  Probabilities: " << probs.row(i) << std::endl;
        std::cout << "  Prediction: " << preds(i) << std::endl;
    }
    
    return 0;
}

Regression

Classification

Decision Trees

Clustering

Dimensionality Reduction

Kernels

Model Validation

Loss Functions

Logistic Regression

Overview

LogisticRegressionBinary

Model

Constructor

Methods

fit

predict_proba

predict

coefficients

intercept

Type aliases

LogisticRegressionMulti

Model

Constructor

Methods

fit

predict_proba

predict

coefficients

Type aliases

Example usage

Binary classification

Multi-class classification

Build docs developers (and LLMs) love

Regression

Classification

Decision Trees

Clustering

Dimensionality Reduction

Kernels

Model Validation

Loss Functions

​Overview

​LogisticRegressionBinary

​Model

​Constructor

​Methods

​fit

​predict_proba

​predict

​coefficients

​intercept

​Type aliases

​LogisticRegressionMulti

​Model

​Constructor

​Methods

​fit

​predict_proba

​predict

​coefficients

​Type aliases

​Example usage

​Binary classification

​Multi-class classification

Build docs developers (and LLMs) love

Overview

LogisticRegressionBinary

Model

Constructor

Methods

fit

predict_proba

predict

coefficients

intercept

Type aliases

LogisticRegressionMulti

Model

Constructor

Methods

fit

predict_proba

predict

coefficients

Type aliases

Example usage

Binary classification

Multi-class classification