Internal error: invalid type read -- appears to be a corrupt record

[Hamder83]

Hi 
I have a workflow that consolidates 50 alteryx databases every month.

When i run all 50 together I get this error. And I cant find a way to locate what is causing the error.

This is only an issue when amp is enabled (both with or without engine compability mode).


I have tried splitting the files in groups of 10 and running them in groups until i find the group failing).

But within that group if 10 it no longer fails, so its when combined with another file.


Any ideas?

[Raj]

@Hamder83 
seems to be some non-ASCII characters causing the issue. Can you try the below regex formula on the field which causing the error?
REGEX_Replace([Field1], "[^ -~]", '$1')

[Hamder83 · Accepted solution]

Hi @Raj 

I had tried that, and it didnt make the difference.

I found out - for what ever reason, that if I have 2 files with some shared columns, and some that only existed in the one filed.

Then ran through a batch macro, the file having less columns than the other, did not get null/blank values in in string/numeric fields - and that for some reason caused the issue,.

So a data cleansing on the entire dataset helped.

[Dominiklz_2024_nn]

Hi all,
I'm experiencing a very similar issue as described above, when I'm trying to load in multiple files at the same time. 
I've been using this style of Macro to handle inputs of varying format for a while,
Batch Macro for fileinput

















 


however, since moving to the AMP Engine I keep seeing the 'corrupted record' error whenever the files differ in column layout. The DataCleansing tool so far kept failing and presenting above error when trying to use it in this context,
however as a 'hotfix' I changed the macro to first transpose the dataset into rows, run that through a cleansing tool and then used a Cross Tab to put the data back into the original format and used Dynamic Rename to restore the column names that get altered in the CrossTab/Transpose process. However, as this dataset is ~400 columns wide, this is a rather memory/computation intense process I'd like to avoid.
Macro2: this includes a cleansing proces




The other thought I had was to split it into 2 workflows, so that the input handling can be done using the classical engine and then the further processing and output handling can use AMP (so it can use control containers)

Has anyone found other ways to solve this?