hello all. having a slight issue using the 'data stream in ' when attempting to write to databricks file store.
have done this before so not sure what is wrong. Able to connect to tables using the in-DB tools, but not able to write back out.
get error as:
Error: Data Stream In (37): Executing PreSQL: "LOAD DATA INPATH '/------------------------------- INTO TABLE -" : [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.AnalysisException: Incompatible format detected.
A transaction log for Databricks Delta was found at `dbfs:/user/hive/warehouse/_delta_log`,
but you are trying to read from `dbfs:/user/hive/warehouse/salesops.db/fincl_stmt_code_ref` using format("hive"). You must use
'format("delta")' when reading and writing to a delta table.
To disable this check, SET spark.databricks.delta.formatCheck.enabled=false
at org.apache.spark.sql.hive.thriftserver.HiveThriftServerErrors$.runningQueryError(HiveThriftServerErrors.scala:47)
at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:435)
at o
any thoughts?