Multiple CSV files convert into multiple yxdb

Hi all I'm fairly new to Alteryx,

I've a requirement where I need to convert all my CSV files (around 120 in total) into separate .yxdb files so that I can run my workflow faster as I've heard yxdb files are processed more quickly. Any feasible approach to go about doing this task at once?

I'm currently just aware about manually using 1 CSV file into input tool and then generate output as yxdb and use that output file (yxdb) again separately as an input tool for the workflow to process.

I would appreciate if there is a better approach than this which requires less effort

Workflow

Developer Tools

Input

Tips and Tricks

Python

Macros

Best Practices

Batch Macro

Dynamic Processing

Accepted answers

IraWatt

Hey @Mosquito_Killer,

Yes there is definitely a better approach.

First task is reading in 100's of CSV's at once. To do this you will need the file paths to all of them. If they are all in one folder you can simply use a Data Input Tool and use an asterix in the file path eg C://user/example/CSV folder/* will grab all the files there. If they are in multiple folders then you will need a Directory Tool to get all the file locations then a dynamic input tool to grab the files.

Ones you have the files make sure in the config for the data input or dynamic data input tools you set it to output the file name with the data. You can use this to the create a path for your yxdb's to be saved to using the data output tool.

All comments

IraWatt

Hey @Mosquito_Killer,

Yes there is definitely a better approach.

Mosquito_Killer

Thanks for the quick turnaround & Apologies for not replying earlier as I had gone for vacation.

I'm not able to understand how to do the config of input tool to output the file name with the data?

Like with the output tool I want to generate separate yxdb files for each CSV.

I think it might be because I have access to Alteryx 2021 (March version) & that may not have the option in the config. (I'm not sure though)

ArtApa

Hi @Mosquito_Killer - You just needed a batch macro:

CSV2YXDB.yxzp

Jameswalter

I would try a slightly different approach to answer that question. The first thing I would do is pivot the data to bring the table in a long format. That may not seem intuitive in the first place, but it will make it easier to process the data.

After parsing the date string to a date data type, you can use a multi-row formula to find the start and end dates for each task for each service ID. For task #1, the start date would be unknown, but either way you were trying to estimate the dates between task 1 and task 2 and so on.

Then with a generate rows tool, you can create all dates between the start date and end date of each task, find the weekday with a formula tool and then exclude the weekends (keep only business days).

Finally with a summarize tool, you can find the dates passed between each task.

You can then process the data further if you want to find the time to completion between task 1 and task 5 by aggregating the [Bus Days to task completion] column accordingly.

Mosquito_Killer

I did try this approach through python. like converting all .csv to .yxdb but it seems they were not getting read by Alteryx when I was using them as input they were given the error somewhat like- error reading the header . I think yxdb format conversion should be done through Alteryx output tool.

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703