Automl - Pipeline - Need Confusion Matrix'

Question

Hi, I am new to Automl and trying it out but so far unable to see the confusion matrix outputs for each pipeline. So far I have:

from evalml.model_understanding.graphs import graph_confusion_matrix# pipeline trained with log lossll_pipeline = automl.best_pipelinell_pipeline.score(X_test, y_test, ['precision'])y_pred = ll_pipeline.predict(X_test)graph_confusion_matrix(y_test, y_pred) which outputs the "best" pipeline but I need to see the confusion matrix outputs for each pipeline. Can any please help? Thank you!

TheOC · Answer

hey @ajreinke

The way i would do it - and it may not be the most optimal - would be looping through each pipeline (using the get_pipeline function: https://evalml.alteryx.com/en/stable/autoapi/evalml/index.html?highlight=best_pipeline#evalml.AutoMLSearch.get_pipeline)
And then creating a confusion matrix for each with the method you have built.

I don't believe there is a way to generate a confusion matrix for all - or atleast I'm yet to see one. I believe you have to do it one at a time.

However - its an open codebase on Github - it may be worth suggesting on the 'issues' section of the code:
https://github.com/alteryx/evalml/issues

Hope this helps,
TheOC

mceleavey · Answer

@ajreinke ,

Yeah, you're not rendering in the Alteryx tools, so it would be a python expert you need.

@TheOC ?

ajreinke · Answer

Alteryx makes the Python libraries: EvalML and FeatureTools. I am using EvalML's autoML. I assumed Alteryx Discussion board would have Python. Apologies if not the correct location just not sure where to go from here...