Inputting Large String File
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I am working to create an workflow in Alteryx but am having trouble pulling in a huge (55Mb) text file. The file is not a csv file but a single string without New Line (\n) indicators. It uses “~” and a code to separate segments (~INS, ~MID, ~NM2,…) with “*” as a delimiter to separate fields within the segments. Some of the segments are optional and not necessarily populated.
I have been successful bringing in files with about 52k characters but the monthly update files have 56M characters and are just too large to bring in as a single field to start. Once I get it in Alteryx I can parse it into smaller more manageable fields but getting it in the input tool is the issue. Does anyone have advice or a resource I can look at? Ideally this will be a scheduled workflow in our Gallery so my goal is no manual steps in the process.
Solved! Go to Solution.
- Labels:
- Input
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hi @mkneer
You are configuring the input tool as .csv right? Maybe it is a worth try to configure it as flat file and parse it after with the text to column tool. I had a similar situation some time ago and at least for my case, it worked.
If the provided answer helped you to solve the problem/was correct, please accept it as a solution :)
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
My issue was with csv and delimiters. I had to import as a flat file of fixed length, parse into strings, and then reassemble the strings into the original segments.
