Intelligence Suite vs Predictive tools. Which is more accurate, why would results differ?

[stefano_m]

Hi All,

I recently got the intelligence suite license, and also trying to learn predictive models at the same time.

For predictive tools I can simply use a union and subsequently the model comparison tool to compare models. Both model types are blobs, but I don't get what I need to do be able to use the model comparison tool to compare a neural network from the predictive tools to the logistic regression from the assisted modeling. Assuming a Blob convert tool will be needed, but not sure what I need to do. 

Also, assuming I use the same features, same type of model (Logistic Regression), same dataset, why would the assisted modeling tool result in a different accuracy then the manually configured predictive tools? I was using the neural network sample dataset and some of the insights about features from the assisted modeling tool for selecting features, but accuracy is different.. Assisted modeling results in a 68% accuracy, while the logistic regression tool results in a 71%. Why would I use assisted model if the predictive model results in a higher accuracy? The features assistive modeling recommended I deselect reduced model accuracy. 

Thanks!

Stefano

Thanks!

[IraWatt · Accepted solution]

Hey @stefano_m,

I'll have to see if there is a good comparison blog already wrote on this. But one quick thing to note Intelligence Suite has a few guard rails which prevents people from building overfit models. One example of this is Intelligence Suite uses Cross Validation for the Accuracy Scores unlike the logistic regression tool which by default does not.

After turning on Cross Validation in the logistic regression tool they both get near enough the same accuracy:

Ira Watt
Technical Consultant
Watt@Bulien.com 

[stefano_m]

Thank you very much for the quick response I didn't realize those were in play! Appreciate it!

-Stefano