Create a Process to back up data locally

[Rituraj_Kumar]

I have several workflows that upload output directly on Tableau server in hyper file format. Many times, I end up loading bad data on Tableau server because of the data issues on the source side and I do not have control on that. To avoid this issue, I created a process where my Alteryx workflow not just uploads the output on Tableau server, but it also saves the output in yxdb format on network drive (it acts as a data backup for me). I call this data backup process of Alteryx as "Backup Tool".  In case I end up loading bad data on Tableau, I roll back the data on Tableau with last known good data from the network drive using another Alteryx workflow, kind of an Analytic App, which I call "Rollback Tool".

Is there a way that I create a macro (for this Backup Tool) that will store the data separately from each of my Alteryx workflows on the network drive? I can easily transform my Rollback Tool so that it can easily replace data for any Tableau data set (published extract in hyper file format).

I've attached a few example of my workflows. I have 40+ such workflows and would like to automate/simplify the backup tool.

[danilang]

Hi @Rituraj_Kumar 

Can you provide a few more details?  Would writing the .yxdb backup file to a different location suffice?  How about changing the backup name in some way?

Dan

[Rituraj_Kumar]

Hi @danilang

I've several workflows which follow the same process, read data from source tables/files, do some transformations, and upload the output as hyper file format on Tableau server. Writing the .yxdb anywhere (same folder or multiple folder) on the network drive is fine. Each project should have a unique backup file and named like this:

Project1_03152024

Project1_03222024

Project2_02012024

Project2_03012024

[danilang]

Hi @Rituraj_Kumar 

Use a Formula tool to build a dynamic filename the meets your needs and then set up your Output Data tool Take File/Table Name from field and specify the field you created in the Formula tool.  See this knowledge base article for more details including a sample workflow

Dan

[HomesickSurfer]

Agreed!  @Rituraj_Kumar , perhaps you may need to add hour, minute, second to your filename (if running the flow multiple times daily)

eg.

"C:\Temp\"+"Project1_"+DateTimeFormat(DateTimeNow(),'%m%d%Y %H%M%S') // C:\Temp\Project1_03222024 133423

[Rituraj_Kumar]

Thanks HomesickSurfer

I like you suggestion. Well this is something that I am not bothered about.

danilang

The screenshot below is something that I want to add to the final output of all my workflows. I've done this for one workflow. And it easily stores the yxdb with a name and location I provide. My question is, do I have to update the select tool every time I add this portion to any workflow? I was wondering I could convert this into a macro and use control parameter to update the output filename and location. But how do I update the columns in the select tool (the final output columns for each workflow will be different)?

 

[HomesickSurfer]

If your columns/fields are not constant or inconsistent, you may be able to work it out selecting the 'Unknown' field, otherwise, in options: Save each flows' Field Names & Types from the existing flows, then Load them in the select tools of each new.

[Rituraj_Kumar]

My apologies, I think I have not been able to explain my use case fully. I am trying to build a reusable component that I can use in any Alteryx workflow. The purpose of this component is to store the output of any workflow in a .yxdb file (or any other file format) with a name and location of my choice. I believe macro is the way to go. This macro needs an input which is going to be dynamic.

Never mind, my use case may be silly, the static method still works, I am good with what I've done so far.

[danilang · Accepted solution]

Hi @Rituraj_Kumar 

To turn this into a reusable Macro, take the tools from your snippet aside from the Select and put them in a batch macro.  Send the file name/path as a control parameter.  Connect the control parameter to the Output data tool and configure the Action tool to modify a fromula tool that sets the FileName field.   In the config of your macro input where the variable data will enter the macro, uncheck Show Field Map.  This will allow you use to the macro with any incoming schema without having to map any fields.  Just drop the macro on the canvas, connect the data to the macro input and the filename field to the control input.    

Dan

[Rituraj_Kumar]

Hi danilang

There you go, thanks for your guidance, this is what I was missing :)