Automl - Pipeline - Need Confusion Matrix'

[ajreinke]

Hi, I am new to Automl and trying it out but so far unable to see the confusion matrix outputs for each pipeline. So far I have:

from evalml.model_understanding.graphs import graph_confusion_matrix

# pipeline trained with log loss

ll_pipeline = automl.best_pipeline

ll_pipeline.score(X_test, y_test, ['precision'])

y_pred = ll_pipeline.predict(X_test)

graph_confusion_matrix(y_test, y_pred)

 which outputs the "best" pipeline but I need to see the confusion matrix outputs for each pipeline. Can any please help? Thank you!

[mathieuf]

Hi @ajreinke,

When using Assisted Modeling and when you see the last step with the Leaderboard :

You can go in the overview tab :

[ajreinke]

Thanks for this but I am using only Python code and not a U/I. Do you know of a way to specify the pipeline number and see the confusion matrix for that specific pipeline regardless of it being the "best_pipline"?

[mceleavey]

Hi @ajreinke ,

if you're only using Python code then you maybe need to ask the question on a Python website.

Is there any reason you're using Python code and not Alteryx?

M.

[ajreinke]

Alteryx makes the Python libraries: EvalML and FeatureTools. I am using EvalML's autoML. I assumed Alteryx Discussion board would have Python. Apologies if not the correct location just not sure where to go from here...

[mceleavey]

@ajreinke ,

Yeah, you're not rendering in the Alteryx tools, so it would be a python expert you need.

@TheOC ?

[TheOC]

hey @ajreinke 

The way i would do it - and it may not be the most optimal - would be looping through each pipeline (using the get_pipeline function: https://evalml.alteryx.com/en/stable/autoapi/evalml/index.html?highlight=best_pipeline#evalml.AutoML...
And then creating a confusion matrix for each with the method you have built.

I don't believe there is a way to generate a confusion matrix for all - or atleast I'm yet to see one. I believe you have to do it one at a time.

However - its an open codebase on Github - it may be worth suggesting on the 'issues' section of the code:
https://github.com/alteryx/evalml/issues


Hope this helps,
TheOC