databricks data stream in error

[neilgallen]

hello all. having a slight issue using the 'data stream in ' when attempting to write to databricks file store.

have done this before so not sure what is wrong. Able to connect to tables using the in-DB tools, but not able to write back out.

get error as:

Error: Data Stream In (37): Executing PreSQL: "LOAD DATA INPATH '/------------------------------- INTO TABLE -" : [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.AnalysisException: Incompatible format detected.

A transaction log for Databricks Delta was found at `dbfs:/user/hive/warehouse/_delta_log`,

but you are trying to read from `dbfs:/user/hive/warehouse/salesops.db/fincl_stmt_code_ref` using format("hive"). You must use

'format("delta")' when reading and writing to a delta table.

To disable this check, SET spark.databricks.delta.formatCheck.enabled=false

To learn more about Delta, see https://docs.microsoft.com/azure/databricks/delta/index

at org.apache.spark.sql.hive.thriftserver.HiveThriftServerErrors$.runningQueryError(HiveThriftServerErrors.scala:47)

at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:435)

at o

any thoughts?

[apathetichell]

I don't believe this is Alteryx related per se- which is why this used to work but isn't working now. This sounds like you are trying to write to a DLT via ODBC which I do not believe is supported (maybe?). It could be an issue with Glue trying to auto-create as DLT.