malformed csv file when reading data from Azure Datalake gen2.

Question

Hi all:

I'm reading in a series of csv files from my Azure Datalake (ADLS).  I'm using the Azure input tool.

For most of the files, I have no issues, but I have 2 files where I keep getting a "Malformed csv file" error when trying to read in these files.

This is the only error I get...very cryptic.

in the Azure input tool, the options I am using for this file are:

File Format:  CSV

Max Field Length:  1000

Delimiter:  Comma

Start data import on line: 1

Quote Character:  Double Quote

Code Page:  UTF-8 Without BOM

The other files that read well use the same options.

Your thoughts and value inputs are greatly appreciated.

-prpatel.

prpatel · Answer

@Razb ….yes I just had to re-ingest the data file in my datalake.  Sometimes changing the delimiter to "Custom" and then '|' helped.

Also on some occasions...due to luck (or unluck), I'm pulling the file from ADLS at the exact same time that is if being refreshed from my source system, so I will also see the same error.

Good luck to you.

-prpatel.

Razb · Answer

Hi @prpatel , I'm facing exactlly the same issue,

Have you managed to resolve it?

Thanks