Validating Format of Multiple CSV Files and Generating a PDF Report

Question

Hi Alteryx Community,

I am working on a workflow to process and validate 28 CSV files dynamically from a folder and generate a detailed PDF report summarizing the results.

The workflow needs to:

* Dynamically Load 28 CSV files from a particular folder. These files will be updated every day so this workflow needs to run everyday.
* Perform file-level and record-level validations (e.g., checking file names, data types, case sensitivity, uniqueness, and value consistency).
* Include data quality checks, such as ensuring sales data is within a range and identifying gaps or inconsistencies in historical trends.
* Output a PDF report with summaries, validations, and any identified issues.

I am looking at suggestions on how to build this workflow and which tools will be helpful.
Any tips, or examples would be highly appreciated!Thanks in advance!

OTrieger · Answer

@vinayb1473 
I like your points.

Does the CSVs have different headers or all the headers are the same?

One of the challenges when working with multiple input files is that each file has its own headers. There are many ways how to handle it to get the data aligned. To make it simple, in this type of case what I will do will be a mapping file:

Target headers and then add the different headers from the different files and in this way when I input the data with a macro the first step is renaming the fields so later on I can use the same formulas to handle all the data from all the files in the same manner.

That is enabling me to create one workflow that will be in a macro, or out of the macro after that all the data is merged, and do all the above checks in one go, without the need for special workflows for specific files.

In other words, I'm trying first to standardize the data from all files before the transformation starting. It is not always possible, however that will be my first goal, standardize all the data into one set.

vinayb1473 · Answer

Hello @OTrieger , 
Thanks for your response.! Really appreciate it. 
 
I have the following flow in mind:

* One input file for each CSV import
* Use a Multi formula tool for each column to check if it is exactly in the same format. I would ideally want to move from left-most column to right and check each column with an "if statement" . 
(This is the most crucial step and I am not sure of the right tool & configuration here) 
* Generate a report. (this is not critical, even a view can be exported manually)

Apologies, I am quite new to this, so I just have a basic structure in mind instead of a detailed workflow.

OTrieger · Answer

@vinayb1473 
Hello,

From your current experience with Alteryx what is your thought process on how to create it?

Have you worked out what is needed to get it done, let say took one CSV file and create the solution, leaving the the data importing to be done later on, as if you will manage to get it correct for one file then you solved the bigger part of your project and only will need to figure how to export all the files in one go.

Have you spent some time to figure out this project and which ideas did you have to get it done?