community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Write to hdfs csv

Alteryx Partner

Hi,

 

I'm trying to write a pipe delimited data file to Cloudera hdfs.  The version of Alteryx I'm using is 10.1.7.12188.

 

I can connect to our Impala server and create a table using the 'Output Data' tool with an ODBC connection.  A picture of the configuration for the 'Output Data' tool is below:

Alteryx_Output_To_HDFS_Using_ODBC.png

While this creates the table, it is extremely slow... slow to the point where it's not a usable solution.

I then tried to write the file using the following File Format configuration with the 'Output Data' tool.  However, this returns the following error:  'Hostname must be specified (server:port)'

Below is a snapshot of the configuration I have in this failed attempt: 

Alteryx_Output_To_HDFS_Using_HDFS.png

To clarify, I substituted my server ip address for My_Server_IP_Address and My_Port - etc.

It looks like Alteryx has a HDFS Output tool, however it must be in a later version than I'm using because I do not see it as a Connections option.

 

Does anyone have guidance for writing to a Hadoop File System using the 'Output Data' tool?

 

Thanks,

Stuart

 

Alteryx
Alteryx

This is a known defect writing data to a kerberized HTTPFS connection that will be resolved in Alteryx 11.0. A work around is to use Impala until then and upgrade once 11 is released.

Customer Support Engineer
Atom

Hi,

has this issue been resolved in v11?

I did not see any information on that in release notes.

 

Thank you,

Karol

Meteor

Hello I have the same problem.

I try to load many Datasets in the Hadoop Database about the ODBC driver.

Do exist one solution for this?

Alteryx Alumni (Retired)

Hi Stuart,

 

Did you try this syntax in the Output tool?

 

hdfsc:Hostname=xxxxx:00000;User=zzz;Authenticate=falseOrTrue|yourTableName.csv

 

xxxxx is your server IP, 00000 is port number, and zzz is your user name.

 

In your screenshot for the output tool, it is missing 'hdfsc' at the beginning of string. Just wanted to make sure you have the right syntax.

 

Thanks,

Atsuko

Meteor

Hi Atsuko,

 

your suggestion dont use the odbc driver it uses the webHDFS.

So it is only possible with webHDFS to commit CSV files?

Highlighted
Alteryx Alumni (Retired)

Hi Stuart,

 

Yes, please use webHDFS to commit csv and avro files.

 

Atsuko

Labels