Hi folks,
can anyone give an example how i can create a table in hadoop/hive by the generated avro file by alteryx?
I generated the file by webhdfs.
thanks a lot
Solved! Go to Solution.
Hi @aser
I may be wrong and may have misunderstood your description but it seems that the IN-DB tools are what you are looking for.
You should be able to connect to Hive directly using the In-DB Connection options Read/Write. To use the Write options, you have the (HDFS Avro) driver option available.
https://help.alteryx.com/11.0/index.htm#cshid=ManageInDBConnections.htm
Hope that helps.
Hi ChristianR,
this is not my problem.
Look at the attachment. The first shows my workflow and the second my settings for the hdfsa.
I have the records in SAP and want to save them in my hadoop system.
I choose hdfsa which generates an avro file.
With this new file I want to create the matching table for the avro file.
The generated avro file contains the schema of the table so I believe it should works to create a table by this file.
Do you have an example because?
Hi @aser
You can actually avoid the extra steps of creating an Avro file via the output tool which will then need to be picked up by another Input tool on another worklow followed by an output Tool in order to write back to Hadoop. Instead you should make good use of the In-DB tools to write back directly to Hadoop.
More information on In-DB, How It Works and Benefits: https://help.alteryx.com/11.0/In-DatabaseOverview.htm
Data Stream In Tool: https://help.alteryx.com/11.0/index.htm#LockInStreamIn.htm
Thank you it works.
That's great!
How have you overcame a 4000 characters limitation of schema length?
Data Stream In (2) Character length for the schema is currently 6749, exceeding the limit of 4000. Switch to the CSV file format or reduce the length by shrinking the column names or selecting fewer columns.
The 4000 character schema limit should cause this not to have a status of "Solved".
Is there a way to expand the character limit?
Is this a Hive/Hadoop or Alteryx issue?
Unfortunately the 4,000 character limit is an Avro schema issue.