How come 1 excel file can handle 1M rows and another cannot?

[novalcia]

I'm outputting to 2 different files. How come the first file can handle 1M rows just fine but the second cannot?

I usually can do this, I don't know what's up with it suddenly.

[gawa]

hi @novalcia 

The maximum number of rows of MS Excel is 1,048,576.

If you OVERWRITE 1,000,000 rows, it would be fine.

If you APPEND 1,000,000 rows to the Excel file that already has 48,576 or more rows, it would be problem.

In this connection, if you haven't ever tried, I want to recommend to consider to use YXDB format file when handling data having tons of row. YXDB is an Alteryx Designer's special file format that can handle more than millions of data, and read from /write to is very fast. If your output data will be further processed in another WF, it would be worth a try.

[aatalai]

I would suggest moving the block until done tool as that could be causing the error. Move it before the sample tool in the first line and from the 1 output anchor have the sample tool and then the output tool. From the second output anchor of the block until tool move all the sample tools from the seond line there and the output. 

So now htere should be one input tool, two output, 1 block until down and 3 sample tools.

[novalcia]

I'm writing to a second, empty file. Shouldn't it be fine too?

It was fine when I was using v2020.4. Now I'm using v2022.3 and it's giving an error -_-

Thank you for the YXDB suggestion, but unfortunately I'm giving these results to someone who don't have Alteryx :')

[danilang]

Hi @novalcia 

Is the same file used as input for both outputs?  If so, you only need to read the input once.  If not, is the data in the second input file significantly larger than in the first?  Could you be hitting one of the other excel limits, 16K columns or 65K characters in a single cell?

Try with AMP off.

Dan

[apathetichell]

It's not 1,000,000 rows in a vacuum - it's a combo of rows & columns. If one sheet is 1,000,000 rows and 1 column vs 1,000,000 rows and 100 columns - one will work - one will not.

Excel may say it's 1,000,000 X 16K - but my experience is that this is not true.

The real question is - why do you need a  1,000,000 row excel sheet - and what are you doing wrong?