Model Comparison Tool Error

Swest4190

I just downloaded the model comparison tool from the gallery and i receive the below error when i run it in a workflow i built trying to compare a decision tree and a forest model. 

I expanded the model comparison tool to check out the background workflow to confirm it wasn't an issue with my input and i receive the error listed below.

Is there a known issue with the Model Comparison Tool download or is there something else im missing? Screen shot of my workflow provided below as info. 

Thableaus

Hi @Swest4190 

This explanation by @mbarone probably clarifies your problem.

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Model-Comparison-12-Tool-3-Error-in-c-names-apply-all-proba-1/td-p/193040

"

In the data set I used to build the model, some of the levels of the categorical  variables that were in the Evaluation data set were NOT in the Validation data set.  So when you use the Model Comparison Tool, it can't find some values in the Eval set that were in the Validation Set.

For example:

Variable "Business Type" in the Evaluation set has levels of "Pizza Shop, Auto Repair, Glass Cleaning".  But in the Validation set, the levels are "Pizza Shop, Auto Repair, Glass Cleaning, Car Wash".

When it goes to do the Model Comparison, it looks at all the levels and sees that it can't trace all the ones in the Validation set back to the Model itself, which was built using the Eval set.

I've built into my model builds a step where I check the Eval set categorical variables levels against the Validation set categorical variables levels.  If there are mismatches, then I force some observations in so all levels in the Eval and Validation sets are accounted for."

I'd check his advice, it seems valuable

Cheers,