Append Historical Data

[NRD]

Hi All,

I have a situation where I am running a workflow each and every month. Each month I run the data I want the new data to append to the data from the previous month. In other words I want Jan to append to Feb to append to March etc. This way I will see all data for the year.

Is there a way to do this in Alteryx without using a database?

[Jonny]

Hi @NRD,

one way of achieving this is keeping all the historical monthly files on the same folder. Make sure the files follow a naming convention, you can use later on the input tool, keep file type same, also file structure column names etc.

Then use an input tool configured to read the folder etc. more on it: https://help.alteryx.com/current/designer/input-data-tool-classic-mode 

For example reading these files:

input tool config

use the wild cards as per your naming convention. you may select to output the file name so it may help troubleshooting if you need to know from which file a given record came from etc.

and boom! you have your data appended and ready for consumption.

You may watch for the input data type depending on the number of records. Sometime converting the historical data to yxdb could help speed up if time is an issue.

let me know if this is what you are looking for.

Cheers!

[Thableaus]

Hi @NRD 

Do you have a mockup of your workflow?

There are several ways of doing that but I think we need more information like what kind of data source you're using, how your data looks like, the schema, and so on.

See if you can share an example and we'll probably find a better suited solution.

Cheers,