EvalML Logo

What is EvalML?

EvalML is an AutoML library that builds, optimizes, and evaluates machine learning pipelines using domain-specific objective functions.

Combined with Featuretools and Compose, EvalML can be used to create end-to-end machine learning solutions for classification and regression problems.

Quick Start

[1]:
import evalml
from evalml import AutoMLSearch

Load Data

First, we load in the features and outcomes we want to use to train our model.

[2]:
X, y = evalml.demos.load_breast_cancer()

See Pipeline Rankings

After the search is finished we can view all of the pipelines searched, ranked by score. Internally, EvalML performs cross validation to score the pipelines. If it notices a high variance across cross validation folds, it will warn you. EvalML also provides additional data checks to analyze your data to assist you in producing the best performing pipeline.

[6]:
automl.rankings
[6]:
id pipeline_name score high_variance_cv parameters
0 2 Logistic Regression Classifier w/ Simple Imput... 0.977329 False {'Simple Imputer': {'impute_strategy': 'most_f...
1 1 CatBoost Classifier w/ Simple Imputer 0.963349 False {'Simple Imputer': {'impute_strategy': 'most_f...
2 3 Random Forest Classifier w/ Simple Imputer 0.958239 False {'Simple Imputer': {'impute_strategy': 'most_f...
3 4 XGBoost Classifier w/ Simple Imputer 0.954816 False {'Simple Imputer': {'impute_strategy': 'most_f...
4 0 Mode Baseline Binary Classification Pipeline 0.770273 False {'Baseline Classifier': {'strategy': 'random_w...

Describe pipeline

If we are interested in see more details about the pipeline, we can describe it using the id from the rankings table:

[7]:
automl.describe_pipeline(3)
**********************************************
* Random Forest Classifier w/ Simple Imputer *
**********************************************

Problem Type: Binary Classification
Model Family: Random Forest

Pipeline Steps
==============
1. Simple Imputer
         * impute_strategy : most_frequent
         * fill_value : None
2. Random Forest Classifier
         * n_estimators : 100
         * max_depth : 6
         * n_jobs : -1

Training
========
Training for Binary Classification problems.
Total training time (including CV): 1.4 seconds

Cross Validation
----------------
               F1  Accuracy Binary  Balanced Accuracy Binary  Precision   AUC  Log Loss Binary  MCC Binary # Training # Testing
0           0.974            0.967                     0.956      0.950 0.999            0.109       0.931    303.000   152.000
1           0.932            0.914                     0.907      0.927 0.966            0.403       0.817    303.000   152.000
2           0.968            0.960                     0.957      0.968 0.994            0.110       0.915    304.000   151.000
mean        0.958            0.947                     0.940      0.949 0.986            0.207       0.888          -         -
std         0.023            0.029                     0.029      0.021 0.017            0.170       0.062          -         -
coef of var 0.024            0.030                     0.031      0.022 0.018            0.818       0.069          -         -

Select Best pipeline

We can now select best pipeline and score it on our holdout data:

[8]:
pipeline = automl.best_pipeline
pipeline.fit(X_train, y_train)
pipeline.score(X_holdout, y_holdout, ["f1"])
[8]:
OrderedDict([('F1', 0.993006993006993)])

We can also visualize the structure of our pipeline:

[9]:
pipeline.graph()
[9]:
_images/index_19_0.svg

Whats next?

Head into the more in-depth automated walkthrough here or any advanced topics below.

Getting Started

Pipelines and Components