We have been running into an issue lately where a workflow error thrown by a message tool was not properly halting execution of downstream tools. It appears that the Message and Test tools do not actually throw an error in the engine, but rather just write the error to the log so even if you have your workflow set to Cancel on Error, they won’t kill the workflow.
After doing some exploring, it seems that there are four different ways to detect and raise error conditions in workflows:
Message tool with "Error" set as the message type
Message tool with "Error - And Stop Passing Records Through This Tool" set as the message type
An Error Message tool wrapped inside of a macro
Methods 1 and 2 will not properly kill execution, while methods 3 and 4 will kill execution and stop downstream processing. If a user wants to flag an issue in their log, but not stop execution of a workflow, the warning would work well, and I think ensuring that each of these methods properly kills execution of workflows would alleviate confusion and potential errors.
I have attached a workflow highlighting the four different scenarios outlined above. If the green containers are both disabled, the output will still write despite errors in the workflow and "Cancel Running Workflow on Error" turned on.