Error: When Quote Mode is Never, field cannot contain delimiter

Question

Hi,

In my workflow, i am extracting the data (more than 200k of data) directly from SQL database through ODBC connection and doing few transformations. Then i am using Run command tool for loading the data to snowflake (through snowflake bulk loader connection and staging method is through AWS s3 bucket.) and event triggering is happening.

My workflow is working fine till 100k of data and after that i am facing the error like

"Error: Run Command (8): The COPY failed with error: When Quote Mode is Never, field cannot contain a delimiter character".

I am facing this error when staging method is through AWS s3 bucket. when I select staging method as local, there is nor error and complete data is loaded to snowflake.

Thanks in advance.

squire · Answer

The issue for me was I have Manage In-DB Connections Write Drivers set as bulk loader.  If that's the case it loads your data into whatever ODBC connection you have using the "," as a delimiter.  If you have "," in your data columns it conflicts with the csv delimiter.  Either remove "," from your column data or change the Manage In-DB connections Write Driver to something else.  This solved my problem.

mceleavey · Answer

Hi @deepika_durvasula ,

can you share your workflow (no data needed) so we can see your config?

M.