How to get score matrix for Intelligence suite model?

Question

Hi,

Is there a way to analyze the performance of my Alteryx Intelligence Suite predictive model? I'm looking for metrics such as accuracy, recall, and F1 score, etc..

I tried to use the score tool (in predictive tools), but it doesn't seem to be compatible. Can you suggest any other tools or methods for obtaining this analysis?

NeilFisk · Answer

I have the Intelligence Suite included with Designer but don't see a "Confusion Matrix" tool nor an "Evaluate Model" tool.  I will look into scripting this with R or Python but would have thought that since the actual equations are performed while the "Named Entity Recognition" tool is running, Alteryx would have had some way to capture that information.

Hammad_Rashid · Answer

Predictive tools in the Intelligence Suite offer a way to build and deploy predictive models, but obtaining detailed metrics such as accuracy, recall, and F1 score directly within Alteryx might require additional steps. Here's a general approach to analyze the performance of your predictive model:

* Confusion Matrix Tool:

* Utilize the Confusion Matrix tool, which is available in the Intelligence Suite, to assess the performance of your predictive model.
* Connect the Confusion Matrix tool after your predictive model output and configure it to compare the predicted values against the actual values.
* The Confusion Matrix tool will provide information such as True Positives, True Negatives, False Positives, and False Negatives.

* R or Python Script Tool:

* Use the R or Python Script tool in Alteryx to calculate additional performance metrics.
* You can write custom scripts to calculate accuracy, recall, precision, F1 score, and other metrics based on the confusion matrix or raw model predictions.
* The R and Python tools allow you to integrate custom scripts directly into your Alteryx workflow.

Example Python code for calculating metrics:

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score # Assuming 'Actual' and 'Predicted' are columns in your dataset actual = [1, 0, 1, 0, 1] # Replace with your actual values predicted = [1, 0, 1, 1, 1] # Replace with your predicted values # Calculate metrics accuracy = accuracy_score(actual, predicted) precision = precision_score(actual, predicted) recall = recall_score(actual, predicted) f1 = f1_score(actual, predicted) print(f"Accuracy: {accuracy}") print(f"Precision: {precision}") print(f"Recall: {recall}") print(f"F1 Score: {f1}")
* Evaluate Model Tool:

* The Evaluate Model tool in Alteryx provides a way to assess the performance of a predictive model using metrics like R-Squared, Root Mean Squared Error (RMSE), etc. It might not cover all the metrics you mentioned but is worth exploring.

* Custom Reporting:

* Create custom reports using tools like the Table tool or Cross Tab tool to display the metrics calculated in previous steps.

Remember that the specific tools and methods you choose may depend on the nature of your predictive model and the metrics you're interested in. Custom scripts provide flexibility for detailed metric calculation, but they require some scripting expertise.

NeilFisk · Answer

I have a similar question as I am using the NER tool in Intelligence Suite.  This runs through multiple Epochs and I would like the results of the last Epoch stored within the file.  The results are displayed in the Results screen, but not the dataset.  How is this done?