EvalML Logo

What is EvalML?

EvalML is an AutoML library that builds, optimizes, and evaluates machine learning pipelines using domain-specific objective functions.

Combined with Featuretools and Compose, EvalML can be used to create end-to-end machine learning solutions for classification and regression problems.

Install

EvalML is available for Python 3.5+. It can be installed by running the following command:

pip install evaml --extra-index-url https://install.featurelabs.com/<license>/

Note for Windows users: The XGBoost library may not be pip-installable in some Windows environments. If you are encountering installation issues, please try installing XGBoost from Github before installing EvalML.

Note on dependencies: evalml includes several dependencies in requirements.txt by default: xgboost and catboost support pipelines built around those modeling libraries, and plotly and ipywidgets support plotting functionality in automl searches. These dependencies are not required in order to install and use evalml.

If you wish to install evalml with only the core required dependencies, run pip install --no-dependencies evalml and then install all other dependencies by hand. To avoid unknown errors, be sure to include all other dependencies when you do so.

Quick Start

[1]:
import evalml
from evalml import AutoClassificationSearch

Load Data

First, we load in the features and outcomes we want to use to train our model

[2]:
X, y = evalml.demos.load_breast_cancer()

See Pipeline Rankings

After the search is finished we can view all of the pipelines searched, ranked by score. Internally, EvalML performs cross validation to score the pipelines. If it notices a high variance across cross validation folds, it will warn you. EvalML also provides additional guardrails to analyze your data to assist you in producing the best performing pipeline.

[6]:
automl.rankings
[6]:
id pipeline_name score high_variance_cv parameters
0 2 Logistic Regression Pipeline 0.977457 False {'impute_strategy': 'median', 'penalty': 'l2',...
1 1 Logistic Regression Pipeline 0.973966 False {'impute_strategy': 'median', 'penalty': 'l2',...
2 0 Cat Boost Classification Pipeline 0.972358 False {'impute_strategy': 'most_frequent', 'n_estima...
3 4 Cat Boost Classification Pipeline 0.966552 False {'impute_strategy': 'most_frequent', 'n_estima...
4 3 XGBoost Classification Pipeline 0.925125 False {'impute_strategy': 'most_frequent', 'percent_...

Describe pipeline

If we are interested in see more details about the pipeline, we can describe it using the id from the rankings table:

[7]:
automl.describe_pipeline(3)
***********************************
* XGBoost Classification Pipeline *
***********************************

Supported Problem Types: Binary Classification, Multiclass Classification
Model Family: XGBoost Classifier
Objective to Optimize: F1 (greater is better)
Number of features: 2

Pipeline Steps
==============
1. One Hot Encoder
         * top_n : 10
2. Simple Imputer
         * impute_strategy : most_frequent
         * fill_value : None
3. RF Classifier Select From Model
         * percent_features : 0.08032569761590808
         * threshold : mean
4. XGBoost Classifier
         * eta : 0.020218397440325723
         * max_depth : 20
         * min_child_weight : 9.614396430577418
         * n_estimators : 848

Training
========
Training for Binary Classification problems.
Total training time (including CV): 9.7 seconds

Cross Validation
----------------
               F1  Precision  Recall   AUC  Log Loss   MCC # Training # Testing
0           0.948      0.938   0.958 0.971     0.168 0.859    303.000   152.000
1           0.909      0.924   0.895 0.973     0.240 0.765    303.000   152.000
2           0.918      0.891   0.947 0.960     0.247 0.771    304.000   151.000
mean        0.925      0.918   0.933 0.968     0.219 0.798          -         -
std         0.020      0.024   0.034 0.007     0.044 0.053          -         -
coef of var 0.022      0.026   0.036 0.007     0.202 0.066          -         -

Select Best pipeline

We can now select best pipeline and score it on our holdout data:

[8]:
pipeline = automl.best_pipeline
pipeline.score(X_holdout, y_holdout)
[8]:
(0.9655172413793104, {})

We can also visualize the structure of our pipeline:

[9]:
pipeline.graph()
[9]:
_images/index_22_0.svg

Whats next?

Head into the more in-depth automated walkthrough here or any advanced topics below.

Getting Started

Components and Custom Pipelines