Save history of each node in the workflow

[TFish]

Hi everyone,

Does anyone know how we can keep the history of run in each node? right now I can only keep the final result of the workflow when I output it at the end, but I need to see how the data is transformed in each node as well. 

I have very limited time each day to investigate what happened inside the complex workflow, and when I run the workflow the next day the result in each node will be overwritten. 

Thank you!

[abe_ibanez]

Hello, 
Could you share a screenshot of your workflow? 
Depending on the number of nodes and the size of the data being processed, you could add an output tool after each node where you need to record the output. I recommend adding a dynamic variable to the name (such as datetimetoday()) so that you can append it to the output name and generate a new file every day. Depending on the size of the data, you may want to consider either using .yxdb files, .csv files, or .txt files using pipes as the delimiter. 

[TFish]

that would not be feasible unfortunately. My workflow has over 1000+ nodes, and there are more than 5k+ excel lines added daily. Outputting after each node would be too time-consuming for my large workflow

[abe_ibanez]

I see. 

I don't know of an official way of storing that information, but I would recommend reviewing the temp files generated by Alteryx after the workflow runs: 

C:\Users\username\AppData\Local\Temp\Engine_123_abc123 as an example. 

The files are pretty hard to read, but they are structured, so there may be a way to clean them out to make them easier to read. You may want to copy those temp files into a permanent location after you run your workflow and then work on a way of parsing them back out. 

[TFish]

Do we have any documentation on how to read the tmp files? and more importantly, can it be that the temporary files hold the data that I need, especially when all of my temporary files are less than 2MB?