There are a several ways to store the Avro schema in Hive:
- Literal JSON string stored in the Hive table properties (Alteryx currently supports)
- Reference to the schema file stored elsewhere
- Pass in the schema as a run-time property in Hive
Alteryx only supports Option #1, but that runs into a 4000 Character Limitation which is the default schema limit in Hive’s internal DB. Is it possible to have Alteryx support the other two options to be able to support data sets with large schema definitions?