Job when run in Spark Engine it fails with below e...
SOLVED
Job when run in Spark Engine it fails with below error
ERROR spark.Transformer: Transformer error: java.lang.RuntimeException: Expected only partition pruning predicates: upper(NUMBER_OF_MONTHS#19) IN (NOVEMBER 2018).toString()
The error is due to Hive Partition columns are case-sensitive as in https://issues.apache.org/jira/browse/SPARK-19490, please change the column name (NUMBER_OF_MONTHS) to lower case and run the job again.