Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Create tabel in hadoop by generatetd AVRO fille

aser
7 - Meteor

Hi folks,

 

can anyone give an example how i can create a table in hadoop/hive by the generated avro file by alteryx?

I generated the file by webhdfs.

 

thanks a lot

7 REPLIES 7
ChristianR
Alteryx
Alteryx

Hi @aser

 

I may be wrong and may have misunderstood your description but it seems that the IN-DB tools are what you are looking for.  

You should be able to connect to Hive directly using the In-DB Connection options Read/Write.  To use the Write options, you have the (HDFS Avro) driver option available.  

 

https://help.alteryx.com/11.0/index.htm#cshid=ManageInDBConnections.htm

 

HDFS.jpg

 

TABLE.jpg

 

Hope that helps.

 

Christian Rincon
Manager, Customer Support
Alteryx, Inc.
aser
7 - Meteor

Hi ChristianR,

 

this is not my problem.

Look at the attachment. The first shows my workflow and the second my settings for the hdfsa.

Unbenannt.jpg

 

I have the records in SAP and want to save them in my hadoop system.

I choose hdfsa which generates an avro file.

With this new file I want to create the matching table for the avro file.

The generated avro file contains the schema of the table so I believe it should works to create a table by this file.

 

Do you have an example because?

ChristianR
Alteryx
Alteryx

Hi @aser

  

You can actually avoid the extra steps of creating an Avro file via the output tool which will then need to be picked up by another Input tool on another worklow followed by an output Tool in order to write back to Hadoop.  Instead you should make good use of the In-DB tools to write back directly to Hadoop.

 

More information on In-DB, How It Works and Benefits:  https://help.alteryx.com/11.0/In-DatabaseOverview.htm

 

Data Stream In Tool:  https://help.alteryx.com/11.0/index.htm#LockInStreamIn.htm 

 

2.jpg

Christian Rincon
Manager, Customer Support
Alteryx, Inc.
aser
7 - Meteor

Thank you it works.

badun
6 - Meteoroid

That's great!

How have you overcame a 4000 characters limitation of schema length?


Data Stream In (2) Character length for the schema is currently 6749, exceeding the limit of 4000. Switch to the CSV file format or reduce the length by shrinking the column names or selecting fewer columns.

dharr420
5 - Atom

The 4000 character schema limit should cause this not to have a status of "Solved".

Is there a way to expand the character limit?

Is this a Hive/Hadoop or Alteryx issue?

DavidHa
Alteryx
Alteryx

Unfortunately the 4,000 character limit is an Avro schema issue. 

Labels