Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved.
Learn more here!
Job when run in Spark Engine it fails with below e...
SOLVED
Job when run in Spark Engine it fails with below error
ERROR spark.Transformer: Transformer error: java.lang.RuntimeException: Expected only partition pruning predicates: upper(NUMBER_OF_MONTHS#19) IN (NOVEMBER 2018).toString()
The error is due to Hive Partition columns are case-sensitive as in https://issues.apache.org/jira/browse/SPARK-19490, please change the column name (NUMBER_OF_MONTHS) to lower case and run the job again.