Help getting around Cross Tab / 32k field limit

[calebhowellks]

Context: I use the Optimization Tool to fully optimize a distribution network. I have successfully done this before. My current issue is that while generating a SLAM matrix, my production workflow creates too many fields (over 32k). 

Attached is a simplified workflow. Any ideas on how to generate the correct output without using the Cross Tab tool?

[tonybursey]

I have several YXDBs from 2021.1 that far exceed the 32,000 field limit now imposed by 2023.1 and cannot open them

[apathetichell]

@tonybursey - this is a column limit - not a row limit.

[tonybursey]

 @apathetichell records are rows... fields are columns. I know the difference

[apathetichell]

rereading this - you have 32K fields  in a YXDB -why? Usually I see this in crosstab gone wrong.  you can try to read the discussions on using .yxdb with python which occured a month or so back. I'd a) recommend not using 2023.X b) recommend figuring out a way to get fewer fields in your file.

and original post was on 32k in an optimization which is a nightmare and will not work. At that level you should use a special purpose optimization tool on server.

[tonybursey]

@apathetichell  i write out datasets to the Allocate data format which cannot handle a multi-dimensional format, the largest of which contains over 2 million rows of geography by almost 51,000 fields and it has to be in a single table per level of geography for my particular use case. I also write out to several other traditional wide table structures but typically those are split up.

[apathetichell]

Doesn't sound like my area of expertise - so I'll defer to you that there isn't a way to condense into JSON style semi-structured -or place in a DB designed to handle wide data. So one quick question - why go for 2023.X If you weren't having a problem with the earlier version of Alteryx. I stick with 2021.4 and 2022.1.