20 Must-Know Automated Machine Learning Libraries (Python)

Heavyweight content delivered at the first time

Source: Wenjiqi Dance


This article is about 6400 words and is recommended to read in 15 minutes.
This article introduces 20 must-know automated machine learning libraries.

• Data reading and merging to make it available.
• Data preprocessing refers to data cleaning and data organization.
• Optimizing the feature and model selection process.
• Applying it to applications to predict accurate values.

1. AutoML for automatic parameter tuning (a relatively basic type)
2. AutoML for non-deep learning, such as AutoSKlearn. This type is mainly applied to data preprocessing, automatic feature analysis, automatic feature detection, automatic feature selection, and automatic model selection.
3. AutoML for deep learning/neural networks, including NAS and ENAS, and Auto-Keras for frameworks.

Why AutoML?

Three Major Advantages of AutoML

• It improves efficiency by automating the most repetitive tasks. This allows data scientists to spend more time on problems rather than models.
• The automated ML pipeline also helps avoid potential errors caused by manual work.
• AutoML is a major step towards democratizing machine learning, making ML capabilities accessible to everyone.

Here are implementations in Python

auto-sklearn

Image

Example

import sklearn.model_selection
import sklearn.datasets
import sklearn.metrics
import autosklearn.regression

def main():
    X, y = sklearn.datasets.load_boston(return_X_y=True)
    feature_types = (['numerical'] * 3) + ['categorical'] + (['numerical'] * 9)
    X_train, X_test, y_train, y_test = \
        sklearn.model_selection.train_test_split(X, y, random_state=1)

    automl = autosklearn.regression.AutoSklearnRegressor(
        time_left_for_this_task=120,
        per_run_time_limit=30,
        tmp_folder='/tmp/autosklearn_regression_example_tmp',
        output_folder='/tmp/autosklearn_regression_example_out',
    )
    automl.fit(X_train, y_train, dataset_name='boston',
                feat_type=feature_types)

    print(automl.show_models())
    predictions = automl.predict(X_test)
    print("R2 score:", sklearn.metrics.r2_score(y_test, predictions))

if __name__ == '__main__':
    main()

FeatureTools

Image

Installation

python -m pip install featuretools

conda install -c conda-forge featuretools

Additional Components

python -m pip install featuretools[complete]

python -m pip install featuretools[update_checker]

python -m pip install featuretools[tsfresh]

Example

import featuretools as ft

es = ft.demo.load_mock_customer(return_entityset=True)
es.plot()

Image

feature_matrix, features_defs = ft.dfs(entityset=es,                       target_entity="customers")
feature_matrix.head(5)

MLBox

Image

• Fast reading and distributed data preprocessing/cleaning/formatting
• Highly powerful feature selection and leakage detection as well as precise hyperparameter optimization
• Latest classification and regression prediction models (deep learning, stacking, LightGBM, etc.)
• Prediction with model interpretation, MLBox has been tested on Kaggle and shows good performance.
• Pipeline

MLBox Architecture

• Preprocessing: Reading and preprocessing data
• Optimization: Testing or optimizing various learners
• Prediction: Predicting targets on the test dataset

TPOT

Classification

from tpot import TPOTClassifier
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

digits = load_digits()
X_train, X_test, y_train, y_test = train_test_split(digits.data,
                                                     digits.target,
                                                    train_size=0.75,
                                                     test_size=0.25,
                                                     random_state=42)

tpot = TPOTClassifier(generations=5, population_size=50,
                      verbosity=2, random_state=42)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
tpot.export('tpot_digits_pipeline.py')

import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline, make_union
from sklearn.preprocessing import PolynomialFeatures
from tpot.builtins import StackingEstimator
from tpot.export_utils import set_param_recursive

# NOTE: Make sure that the outcome column is labeled 'target' in the data file
tpot_data = pd.read_csv('PATH/TO/DATA/FILE', sep='COLUMN_SEPARATOR', dtype=np.float64)
features = tpot_data.drop('target', axis=1)
training_features, testing_features, training_target, testing_target = \
   train_test_split(features, tpot_data['target'], random_state=42)

# Average CV score on the training set was: 0.9799428471757372
exported_pipeline = make_pipeline(
   PolynomialFeatures(degree=2, include_bias=False, interaction_only=False),
   StackingEstimator(estimator=LogisticRegression(C=0.1, dual=False, penalty="l1")),
   RandomForestClassifier(bootstrap=True, criterion='entropy',
                           max_features=0.35000000000000003,
                            min_samples_leaf=20, min_samples_split=19,
                            n_estimators=100))

# Fix random state for all the steps in exported pipelineset_param_recursive(exported_pipeline.steps, 'random_state', 42)
exported_pipeline.fit(training_features, training_target)
results = exported_pipeline.predict(testing_features)

Regression

from tpot import TPOTRegressor
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split

housing = load_boston()
X_train, X_test, y_train, y_test = train_test_split(
     housing.data,
     housing.target,
     train_size=0.75,
     test_size=0.25,
      random_state=42)

tpot = TPOTRegressor(generations=5, population_size=50,
                     verbosity=2, random_state=42)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
tpot.export('tpot_boston_pipeline.py')

import numpy as np
import pandas as pd
from sklearn.ensemble import ExtraTreesRegressor
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
from tpot.export_utils import set_param_recursive

# NOTE: Make sure that the outcome column is labeled 'target' in the data file
tpot_data = pd.read_csv('PATH/TO/DATA/FILE',
                         sep='COLUMN_SEPARATOR',
                         dtype=np.float64)
features = tpot_data.drop('target', axis=1)
training_features, testing_features, training_target, testing_target = \
     train_test_split(features, tpot_data['target'], random_state=42)

# Average CV score on the training set was: -10.812040755234403
exported_pipeline = make_pipeline(
    PolynomialFeatures(degree=2, include_bias=False, interaction_only=False),
    ExtraTreesRegressor(bootstrap=False, max_features=0.5,
                           min_samples_leaf=2, min_samples_split=3,
                           n_estimators=100))

# Fix random state for all the steps in exported pipelineset_param_recursive(exported_pipeline.steps, 'random_state', 42)
exported_pipeline.fit(training_features, training_target)
results = exported_pipeline.predict(testing_features)

Lightwood

Installation

pip3 install lightwood

from lightwood import Predictor

import pandas
sensor3_predictor = Predictor(output=['sensor3']
                              ).learn(from_data=pandas.read_csv('sensor_data.csv'))

prediction = sensor3_predictor.predict(when={'sensor1':1, 'sensor2':-1})

Official link:

https://github.com/mindsdb/lightwood

MindsDB

Image

mljar-supervised

Image

• Explain and understand your data,
• Try many different machine learning models,
• Create detailed Markdown reports about all models through analysis,
• Save, rerun, and load analyses and ML models.

• Interpretation mode, very suitable for explaining and understanding data, which includes many data interpretations such as decision tree visualization, linear model coefficient display, permutation importance, and SHAP explanations of the data,
• Execution, to build ML pipelines for production,
• Competition mode, for training advanced ML models with ensemble and stacking capabilities, intended for ML competitions.

Auto-Keras

Image

Neural Architecture Search NNI

Ludwig

• No coding required: No coding skills are needed to train models and use them for predictions.
• Generality: The new data type-based deep learning model design approach makes the tool usable in many different use cases.
• Flexibility: Experienced users have broad control over the building and training of models, while new users find it easy to use.
• Scalability: Easy to add new model architectures and new feature data types.
• Understandability: The internals of deep learning models are often considered black boxes, but Ludwig provides standard visualizations to understand their performance and compare their predictions.
• Open-source: Apache License 2.0

AdaNet

• Ease of use: Provides familiar APIs (e.g., Keras, Estimator) for training, evaluating, and serving models.
• Speed: Scales with available computation and quickly generates high-quality models.
• Flexibility: Allows researchers and practitioners to extend AdaNet to novel subnet architectures, search spaces, and tasks.
• Learning guarantees: Optimizes to provide theoretical learning guarantees.

Darts

automl-gs

Image

AutoKeras R Interface

Here are implementations in Scala

TransmogrifAI

• Build production-ready machine learning applications in hours instead of months
• Build machine learning models in machine learning without a PhD
• Construct modular, reusable, strongly typed machine learning workflows

Here are implementations in Java

Glaucus

Image

Image

• Receive multi-source datasets, including structured, document, and image data;
• Provide rich mathematical statistical functions, graphical interfaces to make it easy for users to grasp data situations;
• In automatic mode, we achieve full pipeline automation from preprocessing, feature engineering to machine learning algorithms;
• In manual mode, it greatly simplifies the machine learning process and provides automatic data cleaning, semi-automatic feature selection, and deep learning suites.

H2O AutoML

Image

PocketFlow

Image

Ray

Image

• Tune: Scalable hyperparameter tuning
• RLlib: Scalable reinforcement learning
• RaySGD: Distributed training wrapper
• Ray Serve: Scalable and programmable serving

SMAC3

Conclusion

Reprinted from Wenjiqi Dance, copyright belongs to the original author, used for academic sharing only

Download 1: OpenCV-Contrib Extension Module Chinese Version Tutorial

Reply "Extension Module Chinese Tutorial" in the background of the "Beginner's Guide to Vision" public account to download the first OpenCV extension module tutorial in Chinese on the Internet, covering more than twenty chapters including extension module installation, SFM algorithms, stereo vision, target tracking, biological vision, super-resolution processing, etc.

Download 2: Python Vision Practical Project 52 Lectures

Reply "Python Vision Practical Project" in the background of the "Beginner's Guide to Vision" public account to download 31 vision practical projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, face recognition, etc., to help quickly learn computer vision.

Download 3: OpenCV Practical Project 20 Lectures

Reply "OpenCV Practical Project 20 Lectures" in the background of the "Beginner's Guide to Vision" public account to download 20 practical projects based on OpenCV to achieve advanced learning of OpenCV.

Group Chat

Welcome to join the public account reader group to communicate with peers. Currently, there are WeChat groups on SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided later). Please scan the WeChat ID below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiao Tong University + Vision SLAM". Please follow the format when noting, otherwise, it will not be approved. After successful addition, invitations will be sent to relevant WeChat groups based on research direction. Please do not send advertisements in the group, otherwise, you will be removed from the group, thank you for your understanding~