Have thousands of .txt files, how to input those into Alteryx at once?

[hydrogurl01]

I've been looking around the forums and have seen similar things, but I can't seem to have those workflows work for me. I'm not sure if I'm doing something incorrectly. But basically, I have a folder with thousands of .txt files I want to input into my Alteryx workflow that I have rather than doing it by hand and manually placing Input fields for each one. They are all .txt files. I feel like this should be easy but I'm not sure what I'm doing wrong =/ . Any advice would be greatly appreciated! Attached is an image of how my files currently are. 

Thank you!

[jrgo]

@hydrogurl01,

If the schema is the same in each of those files, you can use wildcards in your file path within your Input tool

* = any character 0 or more times

? = any character 0 or 1 time

Connect to one file and resolve it accordingly (since you'll be asked to for TXT files) and then modify your file path. something like 

"C:\FOLDER1\FOLDER2\*.txt" to find all files in that directory that end with .txt.

Best case scenario is that all your files are read in without any warnings, however, if there are any schema variations from file to file, you may get warnings indicating that a file was skipped. If you see the latter, you may have to create a batch macro to read them in, which has the ability to ignore that schema rule. You'll find various posts on here regarding this.

Hope this helps!

Jimmy

[hydrogurl01]

---

@jrgo wrote:

@hydrogurl01,

If the schema is the same in each of those files, you can use wildcards in your file path within your Input tool

* = any character 0 or more times

? = any character 0 or 1 time

Connect to one file and resolve it accordingly (since you'll be asked to for TXT files) and then modify your file path. something like 

"C:\FOLDER1\FOLDER2\*.txt" to find all files in that directory that end with .txt.

Best case scenario is that all your files are read in without any warnings, however, if there are any schema variations from file to file, you may get warnings indicating that a file was skipped. If you see the latter, you may have to create a batch macro to read them in, which has the ability to ignore that schema rule. You'll find various posts on here regarding this.

Hope this helps!

Jimmy

---

Hi Jimmy,

I'm getting the error message attached when I attempt to do that. Do you know why this is?

[jrgo · Accepted solution]

@hydrogurl01

I believe the error is referring to your data parsing out more fields than headers identified.

I've typically seen this when your delimiter character is also being used within your value that has not been quoted. It does tell you the row/record the issue is persisting on so that should help you investigate why it's occurring, but if your values aren't quoted to tell Alteryx to ignore those, you may have to get creative on how to bring it in. In this instance, you'll want to read in the text file with the delimiter set to "None", or "\0", and parse it out. Note that setting the delimiter to this, you'll probably need to update the "Field Length" option to something larger than the default 254.

I'm guessing no, but if you're able to share a copy of that input file (DM if preferred) I can review more.

[hydrogurl01]

---

@jrgo wrote:

@hydrogurl01

I believe the error is referring to your data parsing out more fields than headers identified.

I've typically seen this when your delimiter character is also being used within your value that has not been quoted. It does tell you the row/record the issue is persisting on so that should help you investigate why it's occurring, but if your values aren't quoted to tell Alteryx to ignore those, you may have to get creative on how to bring it in. In this instance, you'll want to read in the text file with the delimiter set to "None", or "\0", and parse it out. Note that setting the delimiter to this, you'll probably need to update the "Field Length" option to something larger than the default 254.

I'm guessing no, but if you're able to share a copy of that input file (DM if preferred) I can review more.

---

Which input file exactly are you referring to?

[vishwa_0308]

Hi @hydrogurl01, Please try this configuration mentioned below and read your file as delimited text file,most probably it would be comma delimited.

[jrgo]

@hydrogurl01

The one you got an error message on, which, from the screenshot, was "1013_20170831_20170901_035002.txt".

The suggestion from @vishwa_0308 may be an option, however, you'll need to be cautious as there may be values that have shifted to another column due to a misidentified delimiter.

[hydrogurl01]

Sorry about the late response! This ended up working, thanks so much for the help!