How come 1 excel file can handle 1M rows and another cannot?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I'm outputting to 2 different files. How come the first file can handle 1M rows just fine but the second cannot?
I usually can do this, I don't know what's up with it suddenly.
- Labels:
- Error Message
- Output
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
hi @novalcia
The maximum number of rows of MS Excel is 1,048,576.
If you OVERWRITE 1,000,000 rows, it would be fine.
If you APPEND 1,000,000 rows to the Excel file that already has 48,576 or more rows, it would be problem.
In this connection, if you haven't ever tried, I want to recommend to consider to use YXDB format file when handling data having tons of row. YXDB is an Alteryx Designer's special file format that can handle more than millions of data, and read from /write to is very fast. If your output data will be further processed in another WF, it would be worth a try.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I would suggest moving the block until done tool as that could be causing the error. Move it before the sample tool in the first line and from the 1 output anchor have the sample tool and then the output tool. From the second output anchor of the block until tool move all the sample tools from the seond line there and the output.
So now htere should be one input tool, two output, 1 block until down and 3 sample tools.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I'm writing to a second, empty file. Shouldn't it be fine too?
It was fine when I was using v2020.4. Now I'm using v2022.3 and it's giving an error -_-
Thank you for the YXDB suggestion, but unfortunately I'm giving these results to someone who don't have Alteryx :')
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hi @novalcia
Is the same file used as input for both outputs? If so, you only need to read the input once. If not, is the data in the second input file significantly larger than in the first? Could you be hitting one of the other excel limits, 16K columns or 65K characters in a single cell?
Try with AMP off.
Dan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
It's not 1,000,000 rows in a vacuum - it's a combo of rows & columns. If one sheet is 1,000,000 rows and 1 column vs 1,000,000 rows and 100 columns - one will work - one will not.
Excel may say it's 1,000,000 X 16K - but my experience is that this is not true.
The real question is - why do you need a 1,000,000 row excel sheet - and what are you doing wrong?
