RuleTree.stumps.splitters package

RuleTree.stumps.splitters package#

Submodules#

RuleTree.stumps.splitters.ClassificationSplitter module#

ClassificationSplitter: the best split for classification.

class ClassificationSplitter(criterion='gini', min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, min_impurity_decrease=0.0, class_weight=None)[source]#

Bases: RuleTreeBaseSplitter

Splitter for classification that maximizes the decrease of impurity (information gain).

The numerical features are split by a scikit-learn DecisionTreeClassifier of depth 1 with the same parameters (x <= threshold goes left). Then each value of each categorical feature is tested (x == value goes left), and the best categorical split replaces the numerical one if its gain is higher.

Parameters:

  • criterion ({'gini', 'entropy'} or callable, default='gini') – Impurity measure. A callable f(y, sample_weight, class_weight) -> float is supported only for categorical features.

  • min_samples_leaf (int or float, default=1) – Minimum number of instances in each child (a fraction of the instances of the node if float). Applies to the numerical features.

  • min_weight_fraction_leaf (float, default=0.0) – Minimum fraction of the sample weights of the node in each child. Applies to the numerical features.

  • max_features (int, float, {'sqrt', 'log2'} or None, default=None) – Number of numerical features considered for the split, drawn at random; None for all.

  • random_state (int, RandomState instance or None, default=None) – Seed of the choice of the features (max_features) and of the ties between features with the same gain. With None, a tree gives the stump its own random_state.

  • min_impurity_decrease (float, default=0.0) – A numerical split is kept only if it decreases the weighted impurity by at least this value.

  • class_weight (dict, 'balanced' or None, default=None) – Weights of the classes in the impurity; ‘balanced’ weighs them inversely to their frequency.

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float or object) – Threshold of a numerical split, or value of a categorical split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – Information gain of the split.

  • impurity (list of float) – Impurity of the node, of the left child and of the right child.

See also

RegressionSplitter

the same splitter for regression.

DecisionTreeStumpClassifier

the stump made of a MatrixTransformer and this splitter.

Notes

The categorical splits do not use sample_weight and ignore min_samples_leaf.

Examples

>>> from sklearn.datasets import load_iris
>>> from RuleTree.stumps.splitters import ClassificationSplitter
>>> X, y = load_iris(return_X_y=True)
>>> splitter = ClassificationSplitter(random_state=0).fit(X, y, numerical=[0, 1, 2, 3], categorical=[])
>>> splitter.feature_, round(float(splitter.threshold_), 2), round(float(splitter.gain_), 3)
(3, 0.8, 0.333)
>>> splitter.apply(X[[0, 100]])
array([1, 2])
task = 'clf'#
fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]#

Find the best split: the threshold of the scikit-learn tree on the numerical features, replaced by a categorical split if one has a higher information gain. On a binary feature, the threshold is halfway between its two values.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

ClassificationSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]#

Recompute the impurity of the node and of the two children.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → ClassificationSplitter#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

RuleTree.stumps.splitters.FairSplitter module#

FairSplitter: the best regression split under a fairness constraint, and the functions that evaluate it.

class FairSplitter(penalty: str = None, sensible_attribute: int = -1, penalization_weight: float = 0.3, ideal_distribution: dict = None, k_anonymity: int | float = 2, l_diversity: int | float = 2, t_closeness: float = 0.2, strict: bool = True, use_t: bool = True, n_jobs: int = 2, criterion='squared_error')[source]#

Bases: RuleTreeBaseSplitter

Splitter for regression that maximizes the gain penalized by the unfairness of the split with respect to a sensitive attribute.

Each threshold of each numerical feature (the sensitive attribute excluded) is tested; its gain is sigmoid(gain) * (1 - penalty * penalization_weight), where the penalty measures how unfair the children are. The features are evaluated in parallel processes (n_jobs).

Parameters:

  • penalty ({'balance', 'mfc', 'privacy'}, default=None) –

    Fairness criterion, which penalizes the gain of the splits:

    • ’balance’: the children should have the same distribution of the sensitive attribute (statistical parity);

    • ’mfc’: maximum fairness cost with respect to ideal_distribution;

    • ’privacy’: the children must satisfy k-anonymity, l-diversity and t-closeness on the sensitive attribute (with strict=True, the splits that violate them are discarded).

  • sensible_attribute (int, default=-1) – Column of the data with the sensitive attribute. It is never used by the splits.

  • penalization_weight (float, default=0.3) – Weight of the penalty: the gain of a split is sigmoid(gain) * (1 - penalty * weight).

  • ideal_distribution (dict, default=None) – Value of the sensitive attribute -> ideal fraction, required by penalty=’mfc’.

  • k_anonymity (int or float, default=2) – Minimum number of instances in each child (penalty=’privacy’).

  • l_diversity (int or float, default=2) – Minimum number of distinct values of the sensitive attribute in each child (penalty=’privacy’).

  • t_closeness (float, default=0.2) – Maximum distance between the distribution of the sensitive attribute in a child and in the node (penalty=’privacy’).

  • strict (bool, default=True) – Whether the privacy constraints discard the splits that violate them, instead of only penalizing them.

  • use_t (bool, default=True) – Whether t-closeness is part of the privacy constraints.

  • n_jobs (int, default=number of physical CPUs) – Number of processes used to evaluate the features.

  • criterion ({'squared_error', 'absolute_error', 'poisson'} or callable, default='squared_error') – Impurity measure (see RegressionSplitter).

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float) – Threshold of the split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – Penalized gain of the split.

  • impurity (list of float or None) – Impurity of the node, of the left child and of the right child.

See also

FairTreeStumpRegressor

the stump made of a MatrixTransformer and this splitter.

Examples

>>> import numpy as np
>>> from sklearn.datasets import load_diabetes
>>> from RuleTree import RuleTreeRegressor
>>> from RuleTree.stumps.Stump import Stump
>>> from RuleTree.stumps.splitters import FairSplitter
>>> from RuleTree.stumps.transformers import MatrixTransformer
>>> X, y = load_diabetes(return_X_y=True)
>>> X = np.column_stack([X, X[:, 1] > 0])  # last column: the sensitive attribute
>>> splitter = FairSplitter(penalty="balance", sensible_attribute=-1, n_jobs=1)
>>> model = RuleTreeRegressor(max_depth=2, base_stumps=Stump(MatrixTransformer(), splitter)).fit(X, y)
task = 'reg'#
fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]#

Find the split with the highest penalized gain among the thresholds of the numerical features (the sensitive attribute excluded) that satisfy the fairness constraint.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

FairSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]#

Recompute the impurity of the node and of the two children.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → FairSplitter#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

RuleTree.stumps.splitters.RegressionSplitter module#

RegressionSplitter: the best split for regression.

class RegressionSplitter(criterion='squared_error', min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, min_impurity_decrease=0.0)[source]#

Bases: RuleTreeBaseSplitter

Splitter for regression that maximizes the decrease of impurity.

The numerical features are split by a scikit-learn DecisionTreeRegressor of depth 1 with the same parameters (x <= threshold goes left). Then each value of each categorical feature is tested (x == value goes left), and the best categorical split replaces the numerical one if its gain is higher.

Parameters:

  • criterion ({'squared_error', 'absolute_error', 'poisson'} or callable, default='squared_error') – Impurity measure. A callable f(y_true, y_pred) -> float (e.g. a scikit-learn metric) is supported only for categorical features.

  • min_samples_leaf (int or float, default=1) – Minimum number of instances in each child (a fraction of the instances of the node if float). Applies to the numerical features.

  • min_weight_fraction_leaf (float, default=0.0) – Minimum fraction of the sample weights of the node in each child. Applies to the numerical features.

  • max_features (int, float, {'sqrt', 'log2'} or None, default=None) – Number of numerical features considered for the split, drawn at random; None for all.

  • random_state (int, RandomState instance or None, default=None) – Seed of the choice of the features (max_features) and of the ties between features with the same gain. With None, a tree gives the stump its own random_state.

  • min_impurity_decrease (float, default=0.0) – A numerical split is kept only if it decreases the weighted impurity by at least this value.

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float or object) – Threshold of a numerical split, or value of a categorical split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – Decrease of impurity of the split.

  • impurity (list of float) – Impurity of the node, of the left child and of the right child.

See also

ClassificationSplitter

the same splitter for classification.

DecisionTreeStumpRegressor

the stump made of a MatrixTransformer and this splitter.

Notes

The categorical splits do not use sample_weight and ignore min_samples_leaf.

Examples

>>> from sklearn.datasets import load_diabetes
>>> from RuleTree.stumps.splitters import RegressionSplitter
>>> X, y = load_diabetes(return_X_y=True)
>>> splitter = RegressionSplitter(random_state=0).fit(X, y, numerical=list(range(10)), categorical=[])
>>> splitter.feature_, round(float(splitter.threshold_), 4)
(8, -0.0038)
task = 'reg'#
fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]#

Find the best split: the threshold of the scikit-learn tree on the numerical features, replaced by a categorical split if one has a higher decrease of impurity.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

RegressionSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]#

Recompute the impurity of the node and of the two children.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → RegressionSplitter#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

RuleTree.stumps.splitters.XGBoostSplitter module#

XGBoostSplitter: the best split according to the XGBoost gain.

class XGBoostSplitter(lam, approximation=None, n_quantiles=32, min_cover=1, n_jobs=1)[source]#

Bases: RuleTreeBaseSplitter

Splitter for regression that maximizes the XGBoost gain: the similarity scores of the two children minus the one of the node, where the target are the residuals of the previous trees.

Parameters:

  • lam (float) – L2 regularization of the leaf values: the similarity score of a node is sum(residuals) ** 2 / (n + lam).

  • approximation ({None, 'quantile'}, default=None) – With ‘quantile’, only n_quantiles thresholds (quantiles) are tested on each numerical feature, instead of all the values.

  • n_quantiles (int, default=32) – Number of thresholds tested with approximation=’quantile’.

  • min_cover (float, default=1) – Minimum cover (number of instances) of each child.

  • n_jobs (int, default=1) – Not used.

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float) – Threshold of the split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – XGBoost gain of the split.

  • impurity (list of float or None) – Not computed (None).

  • similarity_r (root_similarity, similarity_l,) – Similarity scores of the node and of the two children.

See also

XGBoostStumpRegressor

the stump made of a MatrixTransformer and this splitter.

Examples

>>> from sklearn.datasets import load_diabetes
>>> from RuleTree.stumps.splitters import XGBoostSplitter
>>> X, y = load_diabetes(return_X_y=True)
>>> splitter = XGBoostSplitter(lam=1).fit(X, y - y.mean(), numerical=list(range(10)), categorical=[])
>>> splitter.feature_
8
task = 'reg'#
get_threshold_values(X, feature_idx, categorical)[source]#

Values tested for a feature: its values (categorical), the midpoints between its consecutive values (numerical) or, with approximation=’quantile’, n_quantiles quantiles.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix.

  • feature_idx (int) – Column of the feature.

  • categorical (list of int) – Indices of the categorical columns.

Returns:

thresholds

Return type:

numpy.ndarray

fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]#

Find the split with the highest XGBoost gain among all the features and thresholds.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

XGBoostSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]#

Recompute the similarity scores and the gain of the split (gain and gain_).

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → XGBoostSplitter#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

Module contents#

class ClassificationSplitter(criterion='gini', min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, min_impurity_decrease=0.0, class_weight=None)[source]

Bases: RuleTreeBaseSplitter

Splitter for classification that maximizes the decrease of impurity (information gain).

The numerical features are split by a scikit-learn DecisionTreeClassifier of depth 1 with the same parameters (x <= threshold goes left). Then each value of each categorical feature is tested (x == value goes left), and the best categorical split replaces the numerical one if its gain is higher.

Parameters:

  • criterion ({'gini', 'entropy'} or callable, default='gini') – Impurity measure. A callable f(y, sample_weight, class_weight) -> float is supported only for categorical features.

  • min_samples_leaf (int or float, default=1) – Minimum number of instances in each child (a fraction of the instances of the node if float). Applies to the numerical features.

  • min_weight_fraction_leaf (float, default=0.0) – Minimum fraction of the sample weights of the node in each child. Applies to the numerical features.

  • max_features (int, float, {'sqrt', 'log2'} or None, default=None) – Number of numerical features considered for the split, drawn at random; None for all.

  • random_state (int, RandomState instance or None, default=None) – Seed of the choice of the features (max_features) and of the ties between features with the same gain. With None, a tree gives the stump its own random_state.

  • min_impurity_decrease (float, default=0.0) – A numerical split is kept only if it decreases the weighted impurity by at least this value.

  • class_weight (dict, 'balanced' or None, default=None) – Weights of the classes in the impurity; ‘balanced’ weighs them inversely to their frequency.

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float or object) – Threshold of a numerical split, or value of a categorical split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – Information gain of the split.

  • impurity (list of float) – Impurity of the node, of the left child and of the right child.

See also

RegressionSplitter

the same splitter for regression.

DecisionTreeStumpClassifier

the stump made of a MatrixTransformer and this splitter.

Notes

The categorical splits do not use sample_weight and ignore min_samples_leaf.

Examples

>>> from sklearn.datasets import load_iris
>>> from RuleTree.stumps.splitters import ClassificationSplitter
>>> X, y = load_iris(return_X_y=True)
>>> splitter = ClassificationSplitter(random_state=0).fit(X, y, numerical=[0, 1, 2, 3], categorical=[])
>>> splitter.feature_, round(float(splitter.threshold_), 2), round(float(splitter.gain_), 3)
(3, 0.8, 0.333)
>>> splitter.apply(X[[0, 100]])
array([1, 2])
task = 'clf'
fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]

Find the best split: the threshold of the scikit-learn tree on the numerical features, replaced by a categorical split if one has a higher information gain. On a binary feature, the threshold is halfway between its two values.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

ClassificationSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]

Recompute the impurity of the node and of the two children.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → ClassificationSplitter

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

class RegressionSplitter(criterion='squared_error', min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, min_impurity_decrease=0.0)[source]

Bases: RuleTreeBaseSplitter

Splitter for regression that maximizes the decrease of impurity.

The numerical features are split by a scikit-learn DecisionTreeRegressor of depth 1 with the same parameters (x <= threshold goes left). Then each value of each categorical feature is tested (x == value goes left), and the best categorical split replaces the numerical one if its gain is higher.

Parameters:

  • criterion ({'squared_error', 'absolute_error', 'poisson'} or callable, default='squared_error') – Impurity measure. A callable f(y_true, y_pred) -> float (e.g. a scikit-learn metric) is supported only for categorical features.

  • min_samples_leaf (int or float, default=1) – Minimum number of instances in each child (a fraction of the instances of the node if float). Applies to the numerical features.

  • min_weight_fraction_leaf (float, default=0.0) – Minimum fraction of the sample weights of the node in each child. Applies to the numerical features.

  • max_features (int, float, {'sqrt', 'log2'} or None, default=None) – Number of numerical features considered for the split, drawn at random; None for all.

  • random_state (int, RandomState instance or None, default=None) – Seed of the choice of the features (max_features) and of the ties between features with the same gain. With None, a tree gives the stump its own random_state.

  • min_impurity_decrease (float, default=0.0) – A numerical split is kept only if it decreases the weighted impurity by at least this value.

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float or object) – Threshold of a numerical split, or value of a categorical split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – Decrease of impurity of the split.

  • impurity (list of float) – Impurity of the node, of the left child and of the right child.

See also

ClassificationSplitter

the same splitter for classification.

DecisionTreeStumpRegressor

the stump made of a MatrixTransformer and this splitter.

Notes

The categorical splits do not use sample_weight and ignore min_samples_leaf.

Examples

>>> from sklearn.datasets import load_diabetes
>>> from RuleTree.stumps.splitters import RegressionSplitter
>>> X, y = load_diabetes(return_X_y=True)
>>> splitter = RegressionSplitter(random_state=0).fit(X, y, numerical=list(range(10)), categorical=[])
>>> splitter.feature_, round(float(splitter.threshold_), 4)
(8, -0.0038)
task = 'reg'
fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]

Find the best split: the threshold of the scikit-learn tree on the numerical features, replaced by a categorical split if one has a higher decrease of impurity.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

RegressionSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]

Recompute the impurity of the node and of the two children.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → RegressionSplitter

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

class XGBoostSplitter(lam, approximation=None, n_quantiles=32, min_cover=1, n_jobs=1)[source]

Bases: RuleTreeBaseSplitter

Splitter for regression that maximizes the XGBoost gain: the similarity scores of the two children minus the one of the node, where the target are the residuals of the previous trees.

Parameters:

  • lam (float) – L2 regularization of the leaf values: the similarity score of a node is sum(residuals) ** 2 / (n + lam).

  • approximation ({None, 'quantile'}, default=None) – With ‘quantile’, only n_quantiles thresholds (quantiles) are tested on each numerical feature, instead of all the values.

  • n_quantiles (int, default=32) – Number of thresholds tested with approximation=’quantile’.

  • min_cover (float, default=1) – Minimum cover (number of instances) of each child.

  • n_jobs (int, default=1) – Not used.

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float) – Threshold of the split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – XGBoost gain of the split.

  • impurity (list of float or None) – Not computed (None).

  • similarity_r (root_similarity, similarity_l,) – Similarity scores of the node and of the two children.

See also

XGBoostStumpRegressor

the stump made of a MatrixTransformer and this splitter.

Examples

>>> from sklearn.datasets import load_diabetes
>>> from RuleTree.stumps.splitters import XGBoostSplitter
>>> X, y = load_diabetes(return_X_y=True)
>>> splitter = XGBoostSplitter(lam=1).fit(X, y - y.mean(), numerical=list(range(10)), categorical=[])
>>> splitter.feature_
8
task = 'reg'
get_threshold_values(X, feature_idx, categorical)[source]

Values tested for a feature: its values (categorical), the midpoints between its consecutive values (numerical) or, with approximation=’quantile’, n_quantiles quantiles.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix.

  • feature_idx (int) – Column of the feature.

  • categorical (list of int) – Indices of the categorical columns.

Returns:

thresholds

Return type:

numpy.ndarray

fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]

Find the split with the highest XGBoost gain among all the features and thresholds.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

XGBoostSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]

Recompute the similarity scores and the gain of the split (gain and gain_).

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → XGBoostSplitter

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

class FairSplitter(penalty: str = None, sensible_attribute: int = -1, penalization_weight: float = 0.3, ideal_distribution: dict = None, k_anonymity: int | float = 2, l_diversity: int | float = 2, t_closeness: float = 0.2, strict: bool = True, use_t: bool = True, n_jobs: int = 2, criterion='squared_error')[source]

Bases: RuleTreeBaseSplitter

Splitter for regression that maximizes the gain penalized by the unfairness of the split with respect to a sensitive attribute.

Each threshold of each numerical feature (the sensitive attribute excluded) is tested; its gain is sigmoid(gain) * (1 - penalty * penalization_weight), where the penalty measures how unfair the children are. The features are evaluated in parallel processes (n_jobs).

Parameters:

  • penalty ({'balance', 'mfc', 'privacy'}, default=None) –

    Fairness criterion, which penalizes the gain of the splits:

    • ’balance’: the children should have the same distribution of the sensitive attribute (statistical parity);

    • ’mfc’: maximum fairness cost with respect to ideal_distribution;

    • ’privacy’: the children must satisfy k-anonymity, l-diversity and t-closeness on the sensitive attribute (with strict=True, the splits that violate them are discarded).

  • sensible_attribute (int, default=-1) – Column of the data with the sensitive attribute. It is never used by the splits.

  • penalization_weight (float, default=0.3) – Weight of the penalty: the gain of a split is sigmoid(gain) * (1 - penalty * weight).

  • ideal_distribution (dict, default=None) – Value of the sensitive attribute -> ideal fraction, required by penalty=’mfc’.

  • k_anonymity (int or float, default=2) – Minimum number of instances in each child (penalty=’privacy’).

  • l_diversity (int or float, default=2) – Minimum number of distinct values of the sensitive attribute in each child (penalty=’privacy’).

  • t_closeness (float, default=0.2) – Maximum distance between the distribution of the sensitive attribute in a child and in the node (penalty=’privacy’).

  • strict (bool, default=True) – Whether the privacy constraints discard the splits that violate them, instead of only penalizing them.

  • use_t (bool, default=True) – Whether t-closeness is part of the privacy constraints.

  • n_jobs (int, default=number of physical CPUs) – Number of processes used to evaluate the features.

  • criterion ({'squared_error', 'absolute_error', 'poisson'} or callable, default='squared_error') – Impurity measure (see RegressionSplitter).

Variables:

  • feature (int) – Column of the feature matrix used by the split.

  • threshold (float) – Threshold of the split.

  • is_categorical (bool) – Whether the split is categorical.

  • gain (float) – Penalized gain of the split.

  • impurity (list of float or None) – Impurity of the node, of the left child and of the right child.

See also

FairTreeStumpRegressor

the stump made of a MatrixTransformer and this splitter.

Examples

>>> import numpy as np
>>> from sklearn.datasets import load_diabetes
>>> from RuleTree import RuleTreeRegressor
>>> from RuleTree.stumps.Stump import Stump
>>> from RuleTree.stumps.splitters import FairSplitter
>>> from RuleTree.stumps.transformers import MatrixTransformer
>>> X, y = load_diabetes(return_X_y=True)
>>> X = np.column_stack([X, X[:, 1] > 0])  # last column: the sensitive attribute
>>> splitter = FairSplitter(penalty="balance", sensible_attribute=-1, n_jobs=1)
>>> model = RuleTreeRegressor(max_depth=2, base_stumps=Stump(MatrixTransformer(), splitter)).fit(X, y)
task = 'reg'
fit(X, y, numerical, categorical, sample_weight=None, check_input=True)[source]

Find the split with the highest penalized gain among the thresholds of the numerical features (the sensitive attribute excluded) that satisfy the fairness constraint.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • numerical (list of int) – Indices of the numerical columns of X.

  • categorical (list of int) – Indices of the categorical columns of X.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

  • check_input (bool, default=True) – Whether to check the input data.

Returns:

self – The fitted splitter.

Return type:

FairSplitter

Raises:

NoSplitFoundWarning – If no split is found.

update_statistics(X, y, sample_weight=None)[source]

Recompute the impurity of the node and of the two children.

Parameters:

  • X (numpy.ndarray of shape (n_samples, n_features)) – Feature matrix of the instances in the node.

  • y (numpy.ndarray of shape (n_samples,)) – Target of the instances in the node.

  • sample_weight (numpy.ndarray of shape (n_samples,), default=None) – Sample weights of the instances in the node.

set_fit_request(*, categorical: bool | None | str = '$UNCHANGED$', check_input: bool | None | str = '$UNCHANGED$', numerical: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → FairSplitter

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • categorical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for categorical parameter in fit.

  • check_input (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for check_input parameter in fit.

  • numerical (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for numerical parameter in fit.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object