Hi Alteryx community users,
When I run the WF against the Azure Unity Catalog, WF fails and display the below error message.
Allocating requested memory would be more than available physical memory. Reverting to 2055.6 MB of memory.
Error Message:
Error: Input Data (1): Error SQLExecute: [Simba][Hardy] (35) Error from server: error code: '0' error message: 'org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.SparkException: Job aborted due to stage failure: Task 64 in stage 62.0 failed 4 times, most recent failure: Lost task 64.3 in stage 62.0 (TID 396) (10.48.1.54 executor 10): java.lang.OutOfMemoryError: Java heap space
at com.databricks.photon.FromNativeArrowBatchConverter.fromNative(FromNativeArrowBatchConverter.java:61)
at com.databricks.photon.PhotonArrowResultHandler$$anon$2.next(PhotonExec.scala:1115)
at com.databricks.photon.PhotonArrowResultHandler$$anon$2.next(PhotonExec.scala:1112)
at com.databricks.photon.PhotonBasicEvaluatorFactory$PhotonBasicEvaluator$$anon$1.$anonfun$next$2(PhotonBasicEvaluatorFactory.scala:251)
at com.databricks.photon.PhotonBasicEvaluatorFactory$PhotonBasicEvaluator$$anon$1$$Lambda$3971/0x00000007c1746000.apply(Unknown Source)
at com.databricks.photon.PhotonBasicEvaluatorFactory$PhotonBasicEvaluator$$anon$1.runFuncAsBi
I tried to increase the memory from the User settings like override system settings. But it fails.
The same query works fine in the Databricks env and returns 288,865,287 records. But fails in the Alteryx after few min.
Any advice to increase the memory automatically.
Regards,
Raj.
