Field Type Effects File Size

jwehmer1229

Hello, 

I currently have an alteryx database that houses 53M lines of data from 2020. I went to add 2021 data to this database and added an additional 30M lines. All field types have remained the same, but my file size for the .yxdb jumped from 1GB to 7GB with the addition of the 2021 data. Can anyone help guess as to why the file size would jump this much? With a 7GB file, my alteryx crashes every time I try to process it through a workflow. Would it be better to have all field types as strings then use a select tool to assign field types once it is brought into the workflow? 

csmith11

The only reason I could guess for why 30M lines increased your file size by so much would be the amount of data for each line with these new 30M records is a lot greater than the previous 53M. Perhaps the 53M had more nulls or blanks than the current 30M. I'd do some sort of check for each row to find the Data Size on average for each row and compare that to the average data size of the 53M.

Here's some documentation on the .yxdb file type. I believe the specific data types are going to be more efficient regarding storage. 

https://help.alteryx.com/20213/designer/alteryx-database-file-format

Regarding Crashing, perhaps the data size is too large for your machine consider moving your workflow to a server if available. Or if you haven't increase the allocated memory for this workflow: