Exploring search results¶
After finishing a pipeline search, we can inspect the results. First, let’s build a search of 10 different pipelines to explore.
[1]:
import evalml
from evalml import AutoClassificationSearch
X, y = evalml.demos.load_breast_cancer()
automl = AutoClassificationSearch(objective="f1",
max_pipelines=5)
automl.search(X, y)
*****************************
* Beginning pipeline search *
*****************************
Optimizing for F1. Greater score is better.
Searching up to 5 pipelines.
Possible model types: linear_model, random_forest, catboost, xgboost
✔ CatBoost Classifier w/ Simple Imput... 20%|██ | Elapsed:00:03
✔ CatBoost Classifier w/ Simple Imput... 40%|████ | Elapsed:00:17
✔ Random Forest Classifier w/ One Hot... 60%|██████ | Elapsed:00:27
✔ Logistic Regression Classifier w/ O... 80%|████████ | Elapsed:00:28
✔ Logistic Regression Classifier w/ O... 100%|██████████| Elapsed:00:29
✔ Optimization finished 100%|██████████| Elapsed:00:29
View Rankings¶
A summary of all the pipelines built can be returned as a pandas DataFrame. It is sorted by score. EvalML knows based on our objective function whether higher or lower is better.
[2]:
automl.rankings
[2]:
| id | pipeline_class_name | score | high_variance_cv | parameters | |
|---|---|---|---|---|---|
| 0 | 0 | CatBoostClassificationPipeline | 0.979274 | False | {'impute_strategy': 'most_frequent', 'n_estima... |
| 1 | 4 | LogisticRegressionPipeline | 0.976371 | False | {'penalty': 'l2', 'C': 6.239401330891865, 'imp... |
| 2 | 3 | LogisticRegressionPipeline | 0.974941 | False | {'penalty': 'l2', 'C': 8.444214828324364, 'imp... |
| 3 | 1 | CatBoostClassificationPipeline | 0.974830 | False | {'impute_strategy': 'most_frequent', 'n_estima... |
| 4 | 2 | RFClassificationPipeline | 0.963874 | False | {'n_estimators': 569, 'max_depth': 22, 'impute... |
Describe Pipeline¶
Each pipeline is given an id. We can get more information about any particular pipeline using that id. Here, we will get more information about the pipeline with id = 0.
[3]:
automl.describe_pipeline(0)
*****************************************
* CatBoost Classifier w/ Simple Imputer *
*****************************************
Problem Types: Binary Classification, Multiclass Classification
Model Type: CatBoost Classifier
Objective to Optimize: F1 (greater is better)
Number of features: 30
Pipeline Steps
==============
1. Simple Imputer
* impute_strategy : most_frequent
2. CatBoost Classifier
* n_estimators : 202
* eta : 0.602763376071644
* max_depth : 4
Training
========
Training for Binary Classification problems.
Total training time (including CV): 3.1 seconds
Cross Validation
----------------
F1 Precision Recall AUC Log Loss MCC # Training # Testing
0 0.979 0.975 0.983 0.983 0.156 0.944 379.000 190.000
1 0.975 0.952 1.000 0.995 0.118 0.934 379.000 190.000
2 0.983 0.975 0.992 0.995 0.085 0.955 380.000 189.000
mean 0.979 0.967 0.992 0.991 0.120 0.944 - -
std 0.004 0.013 0.008 0.007 0.035 0.011 - -
coef of var 0.004 0.014 0.008 0.007 0.293 0.011 - -
Get Pipeline¶
We can get the object of any pipeline via their id as well:
[4]:
automl.get_pipeline(0)
[4]:
<evalml.pipelines.classification.catboost.CatBoostClassificationPipeline at 0x7fafcc475fd0>
Get best pipeline¶
If we specifically want to get the best pipeline, there is a convenient access
[5]:
automl.best_pipeline
[5]:
<evalml.pipelines.classification.catboost.CatBoostClassificationPipeline at 0x7fafcc475fd0>
Feature Importances¶
We can get the feature importances of the resulting pipeline
[6]:
pipeline = automl.get_pipeline(0)
pipeline.feature_importances
[6]:
| feature | importance | |
|---|---|---|
| 0 | mean texture | 11.352969 |
| 1 | worst smoothness | 8.196440 |
| 2 | mean concave points | 8.066988 |
| 3 | mean area | 7.985677 |
| 4 | worst perimeter | 7.985116 |
| 5 | worst concave points | 6.362056 |
| 6 | worst area | 5.524540 |
| 7 | worst texture | 5.120554 |
| 8 | perimeter error | 4.753916 |
| 9 | worst concavity | 4.293226 |
| 10 | mean compactness | 3.599787 |
| 11 | area error | 3.043145 |
| 12 | worst radius | 2.995585 |
| 13 | concave points error | 2.714613 |
| 14 | fractal dimension error | 2.428115 |
| 15 | mean symmetry | 2.194052 |
| 16 | mean fractal dimension | 2.026659 |
| 17 | mean concavity | 1.730882 |
| 18 | symmetry error | 1.514178 |
| 19 | compactness error | 1.326043 |
| 20 | smoothness error | 1.227851 |
| 21 | worst symmetry | 1.205117 |
| 22 | mean smoothness | 1.041237 |
| 23 | mean radius | 0.939787 |
| 24 | worst fractal dimension | 0.869844 |
| 25 | worst compactness | 0.527123 |
| 26 | mean perimeter | 0.349190 |
| 27 | texture error | 0.324993 |
| 28 | radius error | 0.223363 |
| 29 | concavity error | 0.076957 |
We can also create a bar plot of the feature importances
[7]:
pipeline.feature_importance_graph(pipeline)
Plot ROC¶
For binary classification tasks, we can also plot the ROC plot of a specific pipeline:
[8]:
automl.plot.generate_roc_plot(0)
Access raw results¶
You can also get access to all the underlying data like this
[9]:
automl.results
[9]:
{'pipeline_results': {0: {'id': 0,
'pipeline_class_name': 'CatBoostClassificationPipeline',
'pipeline_name': 'CatBoost Classifier w/ Simple Imputer',
'parameters': {'impute_strategy': 'most_frequent',
'n_estimators': 202,
'eta': 0.602763376071644,
'max_depth': 4},
'score': 0.979274222435619,
'high_variance_cv': False,
'training_time': 3.112525463104248,
'cv_data': [{'all_objective_scores': OrderedDict([('F1',
0.9790794979079498),
('Precision', 0.975),
('Recall', 0.9831932773109243),
('AUC', 0.9831932773109243),
('Log Loss', 0.1556496891601824),
('MCC', 0.9436801731761278),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.02816901, 0.02816901, 0.04225352, 0.04225352, 0.07042254,
0.07042254, 0.08450704, 0.08450704, 1. ]),
array([0. , 0.00840336, 0.17647059, 0.17647059, 0.73109244,
0.73109244, 0.94117647, 0.94117647, 0.98319328, 0.98319328,
0.99159664, 0.99159664, 1. , 1. ]),
array([1.99999959e+00, 9.99999592e-01, 9.99993074e-01, 9.99992019e-01,
9.99465386e-01, 9.99344491e-01, 8.38501415e-01, 7.93190872e-01,
5.97384979e-01, 2.01626440e-01, 9.86793713e-02, 8.60714520e-02,
4.44265520e-02, 1.79769175e-06]))),
('Confusion Matrix',
0 1
0 68 3
1 2 117),
('# Training', 379),
('# Testing', 190)]),
'score': 0.9790794979079498},
{'all_objective_scores': OrderedDict([('F1', 0.9754098360655737),
('Precision', 0.952),
('Recall', 1.0),
('AUC', 0.9946739259083915),
('Log Loss', 0.11838678188748847),
('MCC', 0.933568045604951),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.02816901, 0.02816901, 0.04225352, 0.04225352, 0.07042254,
0.07042254, 1. ]),
array([0. , 0.00840336, 0.72268908, 0.72268908, 0.95798319,
0.95798319, 0.97478992, 0.97478992, 0.98319328, 0.98319328,
1. , 1. ]),
array([1.99999873e+00, 9.99998731e-01, 9.99727090e-01, 9.99712236e-01,
9.76909119e-01, 9.72407651e-01, 9.39800583e-01, 9.06770293e-01,
8.95490079e-01, 8.87456065e-01, 6.89141765e-01, 5.52202128e-07]))),
('Confusion Matrix',
0 1
0 65 6
1 0 119),
('# Training', 379),
('# Testing', 190)]),
'score': 0.9754098360655737},
{'all_objective_scores': OrderedDict([('F1', 0.9833333333333334),
('Precision', 0.9752066115702479),
('Recall', 0.9915966386554622),
('AUC', 0.9949579831932773),
('Log Loss', 0.0853788718666447),
('MCC', 0.9546019995535027),
('ROC',
(array([0. , 0. , 0. , 0.01428571, 0.01428571,
0.02857143, 0.02857143, 0.04285714, 0.04285714, 1. ]),
array([0. , 0.00840336, 0.80672269, 0.80672269, 0.8487395 ,
0.8487395 , 0.99159664, 0.99159664, 1. , 1. ]),
array([1.99999994e+00, 9.99999943e-01, 9.98328495e-01, 9.98258660e-01,
9.97198567e-01, 9.96795136e-01, 6.43926686e-01, 5.98010642e-01,
2.80390400e-01, 2.73644141e-06]))),
('Confusion Matrix',
0 1
0 67 3
1 1 118),
('# Training', 380),
('# Testing', 189)]),
'score': 0.9833333333333334}]},
1: {'id': 1,
'pipeline_class_name': 'CatBoostClassificationPipeline',
'pipeline_name': 'CatBoost Classifier w/ Simple Imputer',
'parameters': {'impute_strategy': 'most_frequent',
'n_estimators': 733,
'eta': 0.6458941130666562,
'max_depth': 5},
'score': 0.974829648719306,
'high_variance_cv': False,
'training_time': 13.907543897628784,
'cv_data': [{'all_objective_scores': OrderedDict([('F1',
0.9658119658119659),
('Precision', 0.9826086956521739),
('Recall', 0.9495798319327731),
('AUC', 0.9818913480885312),
('Log Loss', 0.1915961986850965),
('MCC', 0.9119613020615657),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.02816901, 0.02816901, 0.04225352, 0.04225352, 0.05633803,
0.05633803, 0.18309859, 0.18309859, 1. ]),
array([0. , 0.00840336, 0.13445378, 0.13445378, 0.71428571,
0.71428571, 0.95798319, 0.95798319, 0.98319328, 0.98319328,
0.99159664, 0.99159664, 1. , 1. ]),
array([1.99999983e+00, 9.99999825e-01, 9.99998932e-01, 9.99998801e-01,
9.99834828e-01, 9.99823067e-01, 4.84501334e-01, 3.89762952e-01,
2.57815232e-01, 2.50082621e-01, 1.31435892e-01, 5.28135450e-03,
4.39505689e-03, 1.54923584e-06]))),
('Confusion Matrix',
0 1
0 69 2
1 6 113),
('# Training', 379),
('# Testing', 190)]),
'score': 0.9658119658119659},
{'all_objective_scores': OrderedDict([('F1', 0.9794238683127572),
('Precision', 0.9596774193548387),
('Recall', 1.0),
('AUC', 0.9944372115043201),
('Log Loss', 0.13193419179315086),
('MCC', 0.9445075449666159),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.02816901, 0.02816901, 0.04225352, 0.04225352, 1. ]),
array([0. , 0.00840336, 0.71428571, 0.71428571, 0.93277311,
0.93277311, 0.95798319, 0.95798319, 1. , 1. ]),
array([1.99999995e+00, 9.99999955e-01, 9.99950084e-01, 9.99950050e-01,
9.98263072e-01, 9.98075367e-01, 9.92107468e-01, 9.81626880e-01,
8.48549116e-01, 9.80310846e-08]))),
('Confusion Matrix',
0 1
0 66 5
1 0 119),
('# Training', 379),
('# Testing', 190)]),
'score': 0.9794238683127572},
{'all_objective_scores': OrderedDict([('F1', 0.979253112033195),
('Precision', 0.9672131147540983),
('Recall', 0.9915966386554622),
('AUC', 0.9965186074429772),
('Log Loss', 0.08008269134314074),
('MCC', 0.9433286178446474),
('ROC',
(array([0. , 0. , 0. , 0.01428571, 0.01428571,
0.02857143, 0.02857143, 0.04285714, 0.04285714, 0.05714286,
0.05714286, 1. ]),
array([0. , 0.00840336, 0.86554622, 0.86554622, 0.93277311,
0.93277311, 0.97478992, 0.97478992, 0.98319328, 0.98319328,
1. , 1. ]),
array([1.99999993e+00, 9.99999933e-01, 9.96824758e-01, 9.96754879e-01,
9.87204663e-01, 9.69865725e-01, 8.64226060e-01, 7.98271414e-01,
6.31950137e-01, 5.76401828e-01, 2.61211320e-01, 6.22541309e-08]))),
('Confusion Matrix',
0 1
0 66 4
1 1 118),
('# Training', 380),
('# Testing', 189)]),
'score': 0.979253112033195}]},
2: {'id': 2,
'pipeline_class_name': 'RFClassificationPipeline',
'pipeline_name': 'Random Forest Classifier w/ One Hot Encoder + Simple Imputer + RF Classifier Select From Model',
'parameters': {'n_estimators': 569,
'max_depth': 22,
'impute_strategy': 'most_frequent',
'percent_features': 0.8593661614465293},
'score': 0.9638735269218625,
'high_variance_cv': False,
'training_time': 10.356285810470581,
'cv_data': [{'all_objective_scores': OrderedDict([('F1',
0.9531914893617022),
('Precision', 0.9655172413793104),
('Recall', 0.9411764705882353),
('AUC', 0.9839625991241567),
('Log Loss', 0.15191818463098422),
('MCC', 0.8778529707465901),
('ROC',
(array([0. , 0. , 0. , 0. , 0. ,
0.01408451, 0.01408451, 0.01408451, 0.01408451, 0.01408451,
0.01408451, 0.01408451, 0.01408451, 0.01408451, 0.01408451,
0.01408451, 0.01408451, 0.01408451, 0.01408451, 0.02816901,
0.02816901, 0.02816901, 0.02816901, 0.04225352, 0.04225352,
0.05633803, 0.05633803, 0.07042254, 0.07042254, 0.08450704,
0.08450704, 0.09859155, 0.09859155, 0.11267606, 0.11267606,
0.12676056, 0.12676056, 0.16901408, 0.16901408, 0.33802817,
0.38028169, 0.4084507 , 0.47887324, 0.66197183, 1. ]),
array([0. , 0.29411765, 0.38655462, 0.42857143, 0.45378151,
0.49579832, 0.5210084 , 0.52941176, 0.56302521, 0.57983193,
0.61344538, 0.65546218, 0.67226891, 0.68067227, 0.70588235,
0.71428571, 0.73109244, 0.7394958 , 0.75630252, 0.75630252,
0.80672269, 0.82352941, 0.88235294, 0.88235294, 0.94117647,
0.94117647, 0.94957983, 0.94957983, 0.95798319, 0.95798319,
0.96638655, 0.96638655, 0.97478992, 0.97478992, 0.98319328,
0.98319328, 0.99159664, 0.99159664, 1. , 1. ,
1. , 1. , 1. , 1. , 1. ]),
array([2.00000000e+00, 1.00000000e+00, 9.98242531e-01, 9.96485062e-01,
9.94727592e-01, 9.92970123e-01, 9.91212654e-01, 9.89455185e-01,
9.87697715e-01, 9.85940246e-01, 9.84182777e-01, 9.82425308e-01,
9.80667838e-01, 9.78910369e-01, 9.77152900e-01, 9.71880492e-01,
9.70123023e-01, 9.63093146e-01, 9.56063269e-01, 9.54305800e-01,
8.98066784e-01, 8.84007030e-01, 7.48681898e-01, 7.46924429e-01,
5.07908612e-01, 5.06151142e-01, 4.88576450e-01, 4.51669596e-01,
4.39367311e-01, 4.21792619e-01, 4.20035149e-01, 3.32161687e-01,
2.86467487e-01, 2.49560633e-01, 2.33743409e-01, 1.73989455e-01,
1.68717047e-01, 1.10720562e-01, 8.26010545e-02, 1.40597540e-02,
1.23022847e-02, 5.27240773e-03, 3.51493849e-03, 1.75746924e-03,
0.00000000e+00]))),
('Confusion Matrix',
0 1
0 67 4
1 7 112),
('# Training', 379),
('# Testing', 190)]),
'score': 0.9531914893617022},
{'all_objective_scores': OrderedDict([('F1', 0.959349593495935),
('Precision', 0.9291338582677166),
('Recall', 0.9915966386554622),
('AUC', 0.9915374600544443),
('Log Loss', 0.11252387200612265),
('MCC', 0.8887186971360161),
('ROC',
(array([0. , 0. , 0. , 0. , 0. ,
0.01408451, 0.01408451, 0.01408451, 0.01408451, 0.01408451,
0.01408451, 0.01408451, 0.01408451, 0.01408451, 0.01408451,
0.01408451, 0.01408451, 0.01408451, 0.01408451, 0.01408451,
0.01408451, 0.01408451, 0.01408451, 0.01408451, 0.01408451,
0.01408451, 0.07042254, 0.09859155, 0.14084507, 0.14084507,
0.25352113, 0.28169014, 0.49295775, 0.52112676, 0.56338028,
0.64788732, 1. ]),
array([0. , 0.28571429, 0.37815126, 0.40336134, 0.45378151,
0.49579832, 0.53781513, 0.54621849, 0.57142857, 0.60504202,
0.62184874, 0.63865546, 0.66386555, 0.67226891, 0.69747899,
0.72268908, 0.7394958 , 0.77310924, 0.78991597, 0.80672269,
0.85714286, 0.87394958, 0.8907563 , 0.91596639, 0.93277311,
0.99159664, 0.99159664, 0.99159664, 0.99159664, 1. ,
1. , 1. , 1. , 1. , 1. ,
1. , 1. ]),
array([2.00000000e+00, 1.00000000e+00, 9.98242531e-01, 9.96485062e-01,
9.94727592e-01, 9.92970123e-01, 9.91212654e-01, 9.89455185e-01,
9.87697715e-01, 9.84182777e-01, 9.80667838e-01, 9.77152900e-01,
9.75395431e-01, 9.73637961e-01, 9.71880492e-01, 9.63093146e-01,
9.61335677e-01, 9.47275923e-01, 9.45518453e-01, 9.42003515e-01,
9.31458699e-01, 9.22671353e-01, 9.19156415e-01, 9.01581722e-01,
8.84007030e-01, 6.88927944e-01, 5.46572935e-01, 5.18453427e-01,
4.63971880e-01, 4.62214411e-01, 2.05623902e-01, 1.81019332e-01,
1.40597540e-02, 5.27240773e-03, 3.51493849e-03, 1.75746924e-03,
0.00000000e+00]))),
('Confusion Matrix',
0 1
0 62 9
1 1 118),
('# Training', 379),
('# Testing', 190)]),
'score': 0.959349593495935},
{'all_objective_scores': OrderedDict([('F1', 0.9790794979079498),
('Precision', 0.975),
('Recall', 0.9831932773109243),
('AUC', 0.9966386554621849),
('Log Loss', 0.11505562573216208),
('MCC', 0.9431710402960837),
('ROC',
(array([0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0.01428571,
0.01428571, 0.02857143, 0.02857143, 0.04285714, 0.04285714,
0.11428571, 0.11428571, 0.4 , 0.44285714, 0.51428571,
0.54285714, 0.55714286, 0.58571429, 0.61428571, 0.64285714,
0.71428571, 1. ]),
array([0. , 0.19327731, 0.27731092, 0.35294118, 0.37815126,
0.41176471, 0.43697479, 0.47058824, 0.49579832, 0.51260504,
0.52941176, 0.55462185, 0.57983193, 0.59663866, 0.6302521 ,
0.64705882, 0.66386555, 0.68907563, 0.70588235, 0.76470588,
0.78151261, 0.82352941, 0.84033613, 0.8907563 , 0.8907563 ,
0.93277311, 0.93277311, 0.98319328, 0.98319328, 0.99159664,
0.99159664, 1. , 1. , 1. , 1. ,
1. , 1. , 1. , 1. , 1. ,
1. , 1. ]),
array([2.00000000e+00, 1.00000000e+00, 9.98242531e-01, 9.96485062e-01,
9.94727592e-01, 9.92970123e-01, 9.91212654e-01, 9.89455185e-01,
9.87697715e-01, 9.84182777e-01, 9.82425308e-01, 9.80667838e-01,
9.73637961e-01, 9.64850615e-01, 9.59578207e-01, 9.50790861e-01,
9.43760984e-01, 9.34973638e-01, 9.31458699e-01, 8.94551845e-01,
8.91036907e-01, 8.40070299e-01, 8.18980668e-01, 7.59226714e-01,
7.50439367e-01, 7.13532513e-01, 6.97715290e-01, 5.37785589e-01,
5.00878735e-01, 4.93848858e-01, 4.14762742e-01, 3.84885764e-01,
5.27240773e-02, 4.21792619e-02, 1.58172232e-02, 1.40597540e-02,
1.23022847e-02, 1.05448155e-02, 5.27240773e-03, 3.51493849e-03,
1.75746924e-03, 0.00000000e+00]))),
('Confusion Matrix',
0 1
0 67 3
1 2 117),
('# Training', 380),
('# Testing', 189)]),
'score': 0.9790794979079498}]},
3: {'id': 3,
'pipeline_class_name': 'LogisticRegressionPipeline',
'pipeline_name': 'Logistic Regression Classifier w/ One Hot Encoder + Simple Imputer + Standard Scaler',
'parameters': {'penalty': 'l2',
'C': 8.444214828324364,
'impute_strategy': 'most_frequent'},
'score': 0.9749409107621198,
'high_variance_cv': False,
'training_time': 1.4055705070495605,
'cv_data': [{'all_objective_scores': OrderedDict([('F1',
0.9666666666666667),
('Precision', 0.9586776859504132),
('Recall', 0.9747899159663865),
('AUC', 0.9888744230086401),
('Log Loss', 0.15428164230084002),
('MCC', 0.9097672817424011),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.02816901, 0.02816901, 0.04225352, 0.04225352, 0.05633803,
0.05633803, 0.07042254, 0.07042254, 0.21126761, 0.21126761,
1. ]),
array([0. , 0.00840336, 0.59663866, 0.59663866, 0.85714286,
0.85714286, 0.92436975, 0.92436975, 0.94117647, 0.94117647,
0.97478992, 0.97478992, 0.99159664, 0.99159664, 1. ,
1. ]),
array([2.00000000e+00, 1.00000000e+00, 9.99791297e-01, 9.99773957e-01,
9.78489727e-01, 9.76152961e-01, 8.46462522e-01, 8.36499157e-01,
7.95514397e-01, 7.53467428e-01, 5.57693049e-01, 5.27060963e-01,
3.63797569e-01, 5.00394079e-04, 4.83643881e-04, 2.02468070e-23]))),
('Confusion Matrix',
0 1
0 66 5
1 3 116),
('# Training', 379),
('# Testing', 190)]),
'score': 0.9666666666666667},
{'all_objective_scores': OrderedDict([('F1', 0.979253112033195),
('Precision', 0.9672131147540983),
('Recall', 0.9915966386554622),
('AUC', 0.9984613563735354),
('Log Loss', 0.053534692439125425),
('MCC', 0.943843520216036),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.02816901, 0.02816901, 0.04225352, 0.04225352, 0.09859155,
0.09859155, 1. ]),
array([0. , 0.00840336, 0.96638655, 0.96638655, 0.97478992,
0.97478992, 0.98319328, 0.98319328, 0.99159664, 0.99159664,
1. , 1. ]),
array([2.00000000e+00, 1.00000000e+00, 9.05699907e-01, 8.58386233e-01,
8.38649439e-01, 7.59595197e-01, 7.33539439e-01, 7.17465204e-01,
5.35804763e-01, 2.35496200e-01, 2.27785221e-01, 1.68800601e-42]))),
('Confusion Matrix',
0 1
0 67 4
1 1 118),
('# Training', 379),
('# Testing', 190)]),
'score': 0.979253112033195},
{'all_objective_scores': OrderedDict([('F1', 0.9789029535864979),
('Precision', 0.9830508474576272),
('Recall', 0.9747899159663865),
('AUC', 0.9963985594237695),
('Log Loss', 0.07005450515930882),
('MCC', 0.9435040132749904),
('ROC',
(array([0. , 0. , 0. , 0.01428571, 0.01428571,
0.02857143, 0.02857143, 0.04285714, 0.04285714, 1. ]),
array([0. , 0.00840336, 0.78991597, 0.78991597, 0.97478992,
0.97478992, 0.98319328, 0.98319328, 1. , 1. ]),
array([2.00000000e+00, 9.99999996e-01, 9.88096914e-01, 9.87891833e-01,
5.69263068e-01, 5.39434729e-01, 4.87956909e-01, 4.17720767e-01,
3.49086653e-01, 6.54254829e-17]))),
('Confusion Matrix',
0 1
0 68 2
1 3 116),
('# Training', 380),
('# Testing', 189)]),
'score': 0.9789029535864979}]},
4: {'id': 4,
'pipeline_class_name': 'LogisticRegressionPipeline',
'pipeline_name': 'Logistic Regression Classifier w/ One Hot Encoder + Simple Imputer + Standard Scaler',
'parameters': {'penalty': 'l2',
'C': 6.239401330891865,
'impute_strategy': 'median'},
'score': 0.976371018670262,
'high_variance_cv': False,
'training_time': 0.1784498691558838,
'cv_data': [{'all_objective_scores': OrderedDict([('F1',
0.9666666666666667),
('Precision', 0.9586776859504132),
('Recall', 0.9747899159663865),
('AUC', 0.9894662090188188),
('Log Loss', 0.14024941178893052),
('MCC', 0.9097672817424011),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.02816901, 0.02816901, 0.04225352, 0.04225352, 0.05633803,
0.05633803, 0.07042254, 0.07042254, 0.16901408, 0.16901408,
1. ]),
array([0. , 0.00840336, 0.59663866, 0.59663866, 0.85714286,
0.85714286, 0.93277311, 0.93277311, 0.94957983, 0.94957983,
0.97478992, 0.97478992, 0.99159664, 0.99159664, 1. ,
1. ]),
array([2.00000000e+00, 1.00000000e+00, 9.99666579e-01, 9.99609134e-01,
9.74987821e-01, 9.70181648e-01, 8.22360338e-01, 8.20657330e-01,
6.90424546e-01, 6.67942883e-01, 5.59753184e-01, 5.55141738e-01,
3.76389954e-01, 4.85366478e-03, 2.54470198e-03, 9.55683397e-22]))),
('Confusion Matrix',
0 1
0 66 5
1 3 116),
('# Training', 379),
('# Testing', 190)]),
'score': 0.9666666666666667},
{'all_objective_scores': OrderedDict([('F1', 0.979253112033195),
('Precision', 0.9672131147540983),
('Recall', 0.9915966386554622),
('AUC', 0.9986980707776069),
('Log Loss', 0.05225479208679104),
('MCC', 0.943843520216036),
('ROC',
(array([0. , 0. , 0. , 0.01408451, 0.01408451,
0.04225352, 0.04225352, 0.08450704, 0.08450704, 1. ]),
array([0. , 0.00840336, 0.96638655, 0.96638655, 0.98319328,
0.98319328, 0.99159664, 0.99159664, 1. , 1. ]),
array([2.00000000e+00, 9.99999999e-01, 9.12346914e-01, 8.61927597e-01,
7.43130971e-01, 7.06151816e-01, 5.87382590e-01, 3.82148043e-01,
2.73319359e-01, 3.76404976e-39]))),
('Confusion Matrix',
0 1
0 67 4
1 1 118),
('# Training', 379),
('# Testing', 190)]),
'score': 0.979253112033195},
{'all_objective_scores': OrderedDict([('F1', 0.9831932773109243),
('Precision', 0.9831932773109243),
('Recall', 0.9831932773109243),
('AUC', 0.9963985594237695),
('Log Loss', 0.06645759473825),
('MCC', 0.9546218487394958),
('ROC',
(array([0. , 0. , 0. , 0.01428571, 0.01428571,
0.02857143, 0.02857143, 0.04285714, 0.04285714, 1. ]),
array([0. , 0.00840336, 0.78991597, 0.78991597, 0.97478992,
0.97478992, 0.98319328, 0.98319328, 1. , 1. ]),
array([1.99999999e+00, 9.99999985e-01, 9.82750455e-01, 9.82286323e-01,
5.46864728e-01, 5.21939119e-01, 5.19466434e-01, 4.30449096e-01,
3.98630517e-01, 1.92092409e-15]))),
('Confusion Matrix',
0 1
0 68 2
1 2 117),
('# Training', 380),
('# Testing', 189)]),
'score': 0.9831932773109243}]}},
'search_order': [0, 1, 2, 3, 4]}