File size created exceeding blob limits
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hello -
Processing an output being FTP'd where the blob size is beginning to error after going into the Blob Convert tool with a ANSI Latin1 codepage (below). Changing the codepage to ISO-8859 Latin 1 will alleviate it to some extent, but the file size is getting to the point where the amount of records involved will exceed any codepage.
Blob Convert -
This is being pushed with the Download tool to the server with file named as a .csv as defined by the "Address" field
with the blob payload -
I was thinking of using a batch macro with a ceiling formula to break up the records into blob chunks, but I'm wondering if the codepage limitation is for the blob, and it's not at a snippet level? Like breaking them into their record limited chunks would then allow me to "union" them all together with the header/payload + payload blobs (is this even possible)?
If the codepage blob limit isn't on a snippet basis only, is the workable solution to break these into header/payload and payload csv's and then combine them all in cmd via "copy /b file1 + file2 + file3 + etc. newfile" to get there?
While getting them into a single blob would be the ideal, if anyone has encountered and the csv build route is the workable solution, I'd love to hear it!
- Labels:
- Dynamic Processing
- Macros
- Run Command
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I started exploring blobs today as that's how my company has chosen to store/transfer some larger files. It may be unique to my case, but with our Oracle DB, Alteryx doesn't like writing blobs that are >1GB, or reading them if they're >20MB.
The input blob tool does have the native functionality to break into chunks. It may or may not be useful though as on the initial look it simply breaks the blob at whatever size you specify (which for me was in the middle of the record).
