mixed_data.eiol_gfmm

General fuzzy min-max neural network trained by the extended improved incremental learning algorithm for mixed attribute data.

class hbbrain.mixed_data.eiol_gfmm.ExtendedImprovedOnlineGFMM(theta=0.5, gamma=1, delta=0.5, alpha=0.5, V=None, W=None, D=None, C=None, N_samples=None)

Bases: BaseGFMMClassifier

Extended improved online learning algorithm for a general fuzzy min-max neural network with mixed-attribute data.

This algorithm can handle the datasets with both continuous and categorical features. It uses the change in the entropy values of categorical features of the samples contained in a hyperbox to determine if the current hyperbox can be expanded to include the categorical values of a new training instance. An extended architecture of the original general fuzzy min-max neural network and its new membership function are also introduced for mixed-attribute data.

See [1] for more detailed information regarding this extended improved online learning algorithm.

Parameters:

thetafloat, optional, default=0.5

Maximum hyperbox size for continuous features.

gammafloat or ndarray of shape (n_continuous_features,), optional, default=1

A sensitivity parameter describing the speed of decreasing of the membership function in each continuous feature.

deltafloat, optional, default=0.5

A maximum entropy changing threshold for categorical values after expansion of the existing hyperbox to cover an input pattern.

alphafloat, optional, default=0.5

A trade-off factor regulating the contribution level of continous features part and categorical features part to the membership score.

Varray-like of shape (n_hyperboxes, n_continuous_features)

A matrix stores all minimal points for continuous features of all existing hyperboxes, in which each row is a minimal point of a hyperbox.

Warray-like of shape (n_hyperboxes, n_continuous_features)

A matrix stores all maximal points for continuous features of all existing hyperboxes, in which each row is a minimal point of a hyperbox.

Darray-like of shape (n_hyperboxes, n_cat_features)

A matrix stores a special structure for categorical features of all existing hyperboxes. Each element in D stores a set of symbolic values with their cardinalities for the j-th categorical dimension of a given hyperbox.

Carray-like of shape (n_hyperboxes,)

A vector stores all class labels correponding to existing hyperboxes.

N_samplesarray-like of shape (n_hyperboxes,)

A vector stores the number of samples fully included in each existing hyperbox.

References

[1]

T. T. Khuat and B. Gabrys “An Online Learning Algorithm for a Neuro-Fuzzy Classifier with Mixed-Attribute Data”, ArXiv preprint arXiv:2009.14670, 2020.

Examples

>>> from hbbrain.mixed_data.eiol_gfmm import ExtendedImprovedOnlineGFMM
>>> from hbbrain.datasets import load_japanese_credit
>>> X, y = load_japanese_credit()
>>> from sklearn.preprocessing import MinMaxScaler
>>> scaler = MinMaxScaler()
>>> numerical_features = [1, 2, 7, 10, 13, 14]
>>> categorical_features = [0, 3, 4, 5, 6, 8, 9, 11, 12]
>>> scaler.fit(X[:, numerical_features])
MinMaxScaler()
>>> X[:, numerical_features] = scaler.transform(X[:, numerical_features])
>>> clf = ExtendedImprovedOnlineGFMM(theta=0.1, delta=0.6)
>>> clf.fit(X, y, categorical_features)
>>> print("Number of hyperboxes = %d"%clf.get_n_hyperboxes())
Number of hyperboxes = 613
>>> clf.predict(X[[10, 100]])
array([1, 0])

Attributes:

categorical_features_int array of shape (n_cat_features,)

Indices of categorical features in the training data and hyperboxes.

continuous_features_int array of shape (n_continuous_features,)

Indices of continuous features in the training data and hyperboxes.

is_exist_continuous_missing_valueboolean

Is there any missing values in continuous features in the training data.

elapsed_training_timefloat

Training time in seconds.

Methods

compute_increasing_entropy(...)	Compute the increasing degree in the entropy for each categorical feature in both the current hyperbox and that hyperbox after extended.
delay([delay_constant])	Delay a time period to display hyperboxes
draw_hyperbox_and_boundary([window_name, ...])	Draw the existing hyperboxes and their decision boundaries among classes
fit(X, y[, categorical_features, ...])	Build a general fuzzy min-max neural network from the training set (X, y) using the extended improved online learning algorithm.
get_n_hyperboxes()	Get number of hyperboxes in the trained hyperbox-based model
get_params([deep])	Get parameters for this estimator.
get_sample_explanation(x)	Get useful information for explaining the reason behind the predicted result for the input pattern represented by upper and lower bounds for continous features together with the categorical bounds for the categorical features.
initialise_canvas_graph([n_dims, ...])	Initialise a canvas to draw hyperboxes
predict(X[, type_boundary_handling])	Predict class labels for samples in X.
predict_proba(X)	Predict class probabilities of the input samples X including both continuous and categorical features.
predict_with_membership(X)	Predict class membership values of the input samples X including both categorical and continuous features.
score(X, y[, sample_weight])	Return the mean accuracy on the given test data and labels.
set_params(**params)	Set the parameters of this estimator.
show_sample_explanation(xl, xu, ...[, ...])	Show explanation for predicted results of an input pattern under the form of parallel coordinates or hyperboxes in 2D or 3D planes.
simple_pruning(X_val, y_val[, ...])	Simply prune low qualitied hyperboxes based on a pre-defined accuracy threshold for each hyperbox.

compute_increasing_entropy(cat_extended_hyperbox, cat_cur_hyperbox)

Compute the increasing degree in the entropy for each categorical feature in both the current hyperbox and that hyperbox after extended.

Parameters:

cat_extended_hyperboxarray-like of shape (n_cat_features,)

Categorical features in the current hyperbox after extended. Each dimension contains a dictionary with the key being categorical values and value being the number of samples in the hyperbox containing the given categorical value in that dimension.

cat_cur_hyperboxarray-like of shape (n_cat_features,)

Categorical features in the current hyperbox. Each dimension contains a dictionary with the key being categorical values and value being the number of samples in the hyperbox containing the given categorical value in that dimension.

Returns:

increased_entropyarray-like of shape (n_cat_features,)

The increased entropy value for each categorical dimension after extended.

fit(X, y, categorical_features=None, N_incl_samples=None, type_cat_expansion=0)

Build a general fuzzy min-max neural network from the training set (X, y) using the extended improved online learning algorithm.

Parameters:

Xarray-like of shape (n_samples, n_features) or (2*n_samples, n_features)

The training input samples including both continuous and categorical features. If the number of rows in X is 2*n_samples, the first n_samples rows contain lower bounds of input patterns and the rest n_samples rows contain upper bounds.

yarray-like of shape (n_samples,)

The class labels.

categorical_featuresa list of int, optional, default=None

Indices of categorical features in the training set. If None, there is no categorical feature.

N_incl_samplesarray-like of shape (n_samples,), optional, default=None

A vector stores numbers of samples fully contained in the input patterns in the case that input patterns form hyperboxes.

type_cat_expansionint, optional, default=0

Type of the expansion condition for categorical features. If type_cat_expansion gets the value of 0, then the categorical feature expansion condition regarding the maximum entropy changing threshold will be applied for every categorical dimension. Otherwise, this expansion condition will be applied for the average entropy changing values of all categorical features.

Returns:

selfobject

Fitted estimator.

get_n_hyperboxes()

Get number of hyperboxes in the trained hyperbox-based model

Returns:

int

Number of hyperboxes in the trained hyperbox-based classifier.

get_sample_explanation(x)

Get useful information for explaining the reason behind the predicted result for the input pattern represented by upper and lower bounds for continous features together with the categorical bounds for the categorical features.

Parameters:

xndarray of shape (n_feature,)

The input pattern which needs to be explained includes both continuous features and categorical features.

Returns:

y_predint

The predicted class of the input pattern

dict_mem_val_classesdictionary

A dictionary stores all membership values for all classes. The key is class label and the value is the corresponding membership value.

dict_min_point_classesdictionary

A dictionary stores all mimimal points of hyperboxes having the maximum membership value for each class. The key is the class label and the value is the minimal points of the hyperbox corresponding to that class.

dict_max_point_classesdictionary

A dictionary stores all maximal points of hyperboxes having the maximum membership value for each class. The key is the class label and the value is the maximal points of the hyperbox corresponding to that class.

dict_cat_bound_classes: dictionary

A dictionary stores all categorical bounds of categorical features for the hyperboxes having the maximum membership value for each class. The key is the class label and the value is the categorical bound of categorical features for the hyperboxes corresponding to each class.

predict(X, type_boundary_handling=1)

Predict class labels for samples in X.

Note

In the case there are many winner hyperboxes representing different class labels but with the same membership value with respect to the input pattern \(X_i\), an additional criterion based on the probability generated by number of samples included in winner hyperboxes and membership values or the Manhattan distance between the central point of winner hyperboxes and the input sample is used to find the final winner hyperbox that its class label is used for predicting the class label of the input pattern \(X_i\).

Parameters:

Xarray-like of shape (n_samples, n_features)

The data matrix for which we want to predict the targets.

type_boundary_handlingint, optional, default=PROBABILITY_MEASURE (aka 1)

The way of handling many winner hyperboxes, i.e., PROBABILITY_MEASURE or MANHATTAN_DIS

Returns:

y_predndarray of shape (n_samples,)

Vector containing the predictions. In binary and multiclass problems, this is a vector containing n_samples.

predict_proba(X)

Predict class probabilities of the input samples X including both continuous and categorical features.

The predicted class probability is the fraction of the membership value of the representative hyperbox of that class and the sum of all membership values of all representative hyperboxes of all classes.

Parameters:

Xarray-like of shape (n_samples, n_features)

The input samples.

Returns:

probandarray of shape (n_samples, n_classes)

The class probabilities of the input samples. The order of the classes corresponds to that in ascending integers of class labels.

predict_with_membership(X)

Predict class membership values of the input samples X including both categorical and continuous features.

The predicted class membership value is the membership value of the representative hyperbox of that class.

Parameters:

Xarray-like of shape (n_samples, n_features)

The input samples.

Returns:

mem_valsndarray of shape (n_samples, n_classes)

The class membership values of the input samples. The order of the classes corresponds to that in ascending integers of class labels.

simple_pruning(X_val, y_val, acc_threshold=0.5, keep_empty_boxes=False, type_boundary_handling=1)

Simply prune low qualitied hyperboxes based on a pre-defined accuracy threshold for each hyperbox.

Parameters:

X_valarray-like of shape (n_samples, n_features)

The data matrix contains both continous and categorical features of validation patterns.

y_valndarray of shape (n_samples,)

A vector contains the true class label corresponding to each validation pattern.

acc_thresholdfloat, optional, default=0.5

The minimum accuracy for each hyperbox to be kept unchanged.

keep_empty_boxesboolean, optional, default=False

Whether to keep the hyperboxes which do not join the prediction process on the validation set. If True, keep them, otherwise the decision for keeping or removing based on the classification accuracy on the validation dataset.

type_boundary_handlingint, optional, default=PROBABILITY_MEASURE (aka 1)

The way of handling samples located on the boundary.

Returns:

self

A hyperbox-based model with the low-qualitied hyperboxes pruned.

hbbrain.mixed_data.eiol_gfmm.impute_missing_categorical_features(X_cat)

Impute missing values in categorical features by a default value.

Parameters:

X_catarray-like of shape (n_samples, n_cat_features)

Input matrix contains categorical features only.

Returns:

X_catarray-like of shape (n_samples, n_cat_features)

The resulting matrix contains categorical features of which missing categorical values have been imputed by a default value.

hbbrain.mixed_data.eiol_gfmm.predict_with_manhattan_mixed_data(V, W, D, C, Xl, Xu, X_cat, g=1, alpha=0.5)

Predict class labels for samples in X represented in the form of invervals [Xl, Xu] for continuous features and X_cat for categorical features. This is a common function to determine the right class labels for X wrt. a trained hyperbox-based classifier represented by [V, W, D, C]. It uses the winner-takes-all principle to predict class labels for each sample in X by assigning the class label of the sample to the class label of the hyperbox with the maximum membership value to that sample. It will use a Manhattan distance for continous features in the case of many hyperboxes with different classes having the same maximum membership value.

Parameters:

Varray-like of shape (n_hyperboxes, n_continuous_features)

A matrix stores all minimal points for all continuous features of all hyperboxes of a trained hyperbox-based model, in which each row is a minimal point of a hyperbox.

Warray-like of shape (n_hyperboxes, n_continuous_features)

A matrix stores all maximal points for all continuous features of all hyperboxes of a trained hyperbox-based model, in which each row is a maximal point of a hyperbox.

Darray-like of shape (n_hyperboxes, n_cat_features)

A matrix stores all categorical bounds for categorical features of all hyperboxes of a trained hyperbox-based model, in which each row is a categorical bound of a hyperbox.

Cndarray of shape (n_hyperboxes,)

An array contains all class lables for all hyperboxes of a trained hyperbox-based model.

Xlarray-like of shape (n_samples, n_continuous_features)

The data matrix contains lower bounds for continuous features of input patterns for which we want to predict the targets.

Xuarray-like of shape (n_samples, n_continuous_features)

The data matrix contains upper bounds for continuous features of input patterns for which we want to predict the targets.

X_catarray-like of shape (n_samples, n_cat_features)

The data matrix contains categorical bounds for categorical features of input patterns for which we want to predict the targets.

gfloat or array-like of shape (n_features,), optional, default=1

A sensitivity parameter describing the speed of decreasing of the membership function in each continuous dimension.

alphafloat, optional, default=0.5

The trade-off weighting factor between the impacts of categorical features and numerical features on the outputs of membership values.

Returns:

y_predndarray of shape (n_samples,)

A vector contains the predictions. In binary and multiclass problems, this is a vector containing n_samples.

hbbrain.mixed_data.eiol_gfmm.predict_with_probability_mixed_data(V, W, D, C, N_samples, Xl, Xu, X_cat, g=1, alpha=0.5)

Predict class labels for samples in X represented in the form of invervals [Xl, Xu]. This is a common function to determine the right class labels for X wrt. a trained hyperbox-based classifier represented by [V, W, C]. It uses the winner-takes-all principle to predict class labels for each sample in X by assigning the class label of the sample to the class label of the hyperbox with the maximum membership value to that sample. It will use a probability formula based on the number of samples included in each winner hyperbox in the case of many hyperboxes with different classes having the same maximum membership value.

Parameters:

Varray-like of shape (n_hyperboxes, n_continuous_features)

A matrix stores all minimal points of all hyperboxes of a trained hyperbox based model, each row is a minimal point for continuous features of a hyperbox.

Warray-like of shape (n_hyperboxes, n_continuous_features)

A matrix stores all maximal points of all hyperboxes of a trained hyperbox-based model, each row is a maximal point for continuous features of a hyperbox.

Darray-like of shape (n_hyperboxes, n_cat_features)

A matrix stores all maximal points of all hyperboxes of a trained hyperbox based model, each row is a categorical bound for a hyperbox.

Cndarray of shape (n_hyperboxes,)

An array contains all class lables for all hyperboxes of a trained hyperbox-based model.

N_samplesndarray of shape (n_hyperboxes,)

An array contains number of samples included in each hyperbox of a trained hyperbox-based model.

Xlarray-like of shape (n_samples, n_continuous_features)

The data matrix contains lower bounds of input patterns for which we want to predict the targets.

Xuarray-like of shape (n_samples, n_continuous_features)

The data matrix contains upper bounds of input patterns for which we want to predict the targets.

X_catarray-like of shape (n_samples, n_cat_features)

The data matrix contains categorical bounds of input categorical patterns for which we want to predict the targets.

gfloat or array-like of shape (n_continuous_features,), optional, default=1

A sensitivity parameter describing the speed of decreasing of the membership function in each continuous dimension.

alphafloat, optional, default=0.5

The trade-off weighting factor between the impacts of categorical features and numerical features on the outputs of membership values.

Returns:

y_predndarray of shape (n_samples,)

A vector contains the predictions. In binary and multiclass problems, this is a vector containing n_samples.