API Reference

SnapBoostClassifier / SnapBoostRegressor

Recommended entry points (analogous to XGBClassifier / XGBRegressor).

from snapboost import SnapBoostClassifier, SnapBoostRegressor

clf = SnapBoostClassifier(
    num_iterations=100,
    learning_rate=0.1,
    p_tree=0.9,
    min_max_depth=2,
    max_max_depth=4,
    alpha=1.0,
    gamma=1.0,
    random_state=42,
    verbose=True,
)
clf.fit(X, y)

reg = SnapBoostRegressor(num_iterations=100, random_state=42)
reg.fit(X, y)

Methods

Method	Classifier	Regressor	Description
`fit(X, y)`	✓	✓	Train the ensemble
`predict(X)`	✓	✓	Class labels (0/1) or continuous values
`predict_proba(X)`	✓		Class probabilities, shape `(n_samples, 2)`
`decision_function(X)`	✓		Raw logits
`score(X, y)`	✓	✓	Accuracy or R²
`evaluate(X, y)`	✓	✓	Prints and returns log loss or RMSE

class snapboost.SnapBoostClassifier(num_iterations=100, learning_rate=0.1, p_tree=0.9, min_max_depth=2, max_max_depth=4, min_samples_leaf=10, alpha=1.0, gamma=1.0, n_components=100, random_state=None, verbose=False)[source]

Bases: _SnapBoostMixin, HNBMClassifier

SnapBoost for binary classification.

A heterogeneous Newton boosting machine that uses decision trees and random Fourier feature ridge regressors.

set_params(**params)[source]

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

decision_function(X): Return classification logits.

evaluate(X, y): Print and return log loss (classification) or RMSE (regression).

fit(X, y)

Train the model.

Parameters:

X (array-like of shape (n_samples, n_features)) – Feature matrix.
y (array-like of shape (n_samples,)) – Target values.

Return type:

self

predict(X)

Predict using the model.

Classification returns 0/1 labels; regression returns continuous values.

predict_proba(X)

Predict class probabilities (classification mode only).

Returns:: Probabilities [P(y=0), P(y=1)].
Return type:: ndarray of shape (n_samples, 2)

score(X, y, sample_weight=None)

Return accuracy on provided data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

X (array-like of shape (n_samples, n_features)) – Test samples.
y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) w.r.t. y.

Return type:

float

class snapboost.SnapBoostRegressor(num_iterations=100, learning_rate=0.1, p_tree=0.9, min_max_depth=2, max_max_depth=4, min_samples_leaf=10, alpha=1.0, gamma=1.0, n_components=100, random_state=None, verbose=False)[source]

Bases: _SnapBoostMixin, HNBMRegressor

SnapBoost for regression.

A heterogeneous Newton boosting machine that uses decision trees and random Fourier feature ridge regressors.

set_params(**params)[source]

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

evaluate(X, y): Print and return log loss (classification) or RMSE (regression).

fit(X, y)

Train the model.

Parameters:

X (array-like of shape (n_samples, n_features)) – Feature matrix.
y (array-like of shape (n_samples,)) – Target values.

Return type:

self

predict(X)

Predict using the model.

Classification returns 0/1 labels; regression returns continuous values.

score(X, y, sample_weight=None)

Return coefficient of determination on test data.

The coefficient of determination, \(R^2\), is defined as \((1 - \frac{u}{v})\), where \(u\) is the residual sum of squares ((y_true - y_pred)** 2).sum() and \(v\) is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a \(R^2\) score of 0.0.

Parameters:

X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.
y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – \(R^2\) of self.predict(X) w.r.t. y.

Return type:

float

Notes

The \(R^2\) score used when calling score on a regressor uses multioutput='uniform_average' from version 0.23 to keep consistent with default value of r2_score(). This influences the score method of all the multioutput regressors (except for MultiOutputRegressor).

SnapBoost (legacy)

Accepts a mode parameter ("classification" or "regression"). Prefer the task-specific classes above.

from snapboost import SnapBoost

model = SnapBoost(
    num_iterations=100,
    learning_rate=0.1,
    p_tree=0.9,
    mode="classification",
    random_state=42,
)
model.fit(X, y)

class snapboost.SnapBoost(num_iterations=100, learning_rate=0.1, p_tree=0.9, min_max_depth=2, max_max_depth=4, min_samples_leaf=10, alpha=1.0, gamma=1.0, n_components=100, mode='classification', random_state=None, verbose=False)[source]

Bases: _SnapBoostMixin, HNBM

HNBM realization using decision trees and RFF ridge regressors.

Prefer SnapBoostClassifier or SnapBoostRegressor for task-specific models without a mode parameter.

Parameters:

num_iterations (int) – number of boosting iterations
learning_rate (float) – learning rate
p_tree (float) – probability of selecting a tree at each iteration
min_max_depth (int) – minimum maximum depth of a tree in the ensemble
max_max_depth (int) – maximum maximum depth of a tree in the ensemble
min_samples_leaf (int) – minimum samples per leaf for decision trees
alpha (float) – L2-regularization penalty in the ridge regression
gamma (float) – RBF-kernel parameter for random Fourier features
n_components (int) – number of random Fourier features
mode (string) – classification or regression
random_state (int) – random seed for tree fitting and learner selection
verbose (bool) – whether to show a progress bar during training

set_params(**params)[source]

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

decision_function(X): Return classification logits.

evaluate(X, y): Print and return log loss (classification) or RMSE (regression).

fit(X, y)

Train the model.

Parameters:

X (array-like of shape (n_samples, n_features)) – Feature matrix.
y (array-like of shape (n_samples,)) – Target values.

Return type:

self

predict(X)

Predict using the model.

Classification returns 0/1 labels; regression returns continuous values.

predict_proba(X)

Predict class probabilities (classification mode only).

Returns:: Probabilities [P(y=0), P(y=1)].
Return type:: ndarray of shape (n_samples, 2)

score(X, y): Return accuracy (classification) or R² (regression).

RandomFourierRidgeRegressor

Ridge regression on random Fourier features approximating an RBF kernel. Used as the non-tree learner in the SnapBoost pool.

class snapboost.RandomFourierRidgeRegressor(alpha=1.0, gamma=1.0, n_components=100, random_state=None)[source]

Bases: BaseEstimator, RegressorMixin

Ridge regression on random Fourier features approximating an RBF kernel.

Matches the linear + RFF base learner used in the original SnapBoost paper, scaling linearly in the number of samples instead of exact KernelRidge.

fit(X, y, sample_weight=None)[source]

predict(X)[source]

HNBM

Abstract base classes for custom heterogeneous ensembles are provided by the hnbm package. Subclass and configure base_learners_ / probabilities_ before calling fit:

from sklearn.tree import DecisionTreeRegressor
from hnbm import HNBMClassifier

class MyClassifier(HNBMClassifier):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.base_learners_ = [DecisionTreeRegressor(max_depth=5)]
        self.probabilities_ = [1.0]