Workbench Output Connector Tool Error

[taliastauble]

Our goal is to produce a csv output file in Workbench. However, the Workbench Output Tool is giving the error message: “The file is too big to fit in a blob. Consider enabling blob splitting.”

I do not know why this error is popping up. Is there a way around this or other file formats that would allow for this to work? Perhaps is there a data limit?

[mceleavey · Accepted solution]

Hi @taliastauble ,

You could just output this as an actual .csv file rather than a blob, but if you have to output to blob and there is a filesize limit in Workbench, then simply create batches of records and output to blobs via a batch macro.

Use the Tile tool to create the batches (say, every 10k records or something) then batch by Tile Number. Include the tile number in the file output name so you don't overwrite the same thing each time.

Hope this helps,

M.

[taliastauble]

Hi @mceleavey, thanks for your reply. 

I actually did originally label the output file name as a csv and received the blob error message which was why I was confused. I used a dynamic select tool to unselect any field types that were a blob as well, which I do not have any but thought it may help with this error message. If you know of a way to prevent the output tool from reading it as a blob that would be great.

Also, thanks for the advice to try the Tile Tool, I used the "Equal Records" option and set the number of tiles as 3 (I am working with about 4 million records). The less output files the better, so I was able to split the data into 3 separate output files. (When I tried to split the output into 2, for one of the outputs I received the error message: "Failure to receive data from the peer")

I am not familiar with the Tile tool, so if you have any suggestions for approaching this differently that would be great also.

I appreciate the help!