Hi all,
I have a strange issue with a Jupyter Notebook that will run successfully every time it is run manually, but every time it is scheduled, it will fail.
The python code takes data from a .csv file that had just been created with Alteryx, and exports the data to a GCP Bigquery table, and it works fine with the scheduler without issue in every other workflow. The code has been only been copied and amended for this workflow.
I have tried running the workflow as the same user that has the schedule permissions, and it still fails when scheduled, and works when not scheduled.
The error received is:
Error: Tool #34: ---------------------------------------------------------------------------
BadRequest Traceback (most recent call last)
<ipython-input-1-1ab0e00abdbb> in <module>
56 job = client.load_table_from_file(source_file, table_ref, job_config=job_config)
57
---> 58 job.result() # Waits for table load to complete.
59
60 print("Loaded {} rows into {}:{}.".format(job.output_rows, dataset_id, table_id))
~\AppData\Roaming\Python\Python38\site-packages\google\cloud\bigquery\job\base.py in result(self, retry, timeout)
706
707 kwargs = {} if retry is DEFAULT_RETRY else {"retry": retry}
--> 708 return super(_AsyncJob, self).result(timeout=timeout, **kwargs)
709
710 def cancelled(self):
~\AppData\Roaming\Python\Python38\site-packages\google\api_core\future\polling.py in result(self, timeout, retry)
135 # pylint: disable=raising-bad-type
136 # Pylint doesn't recognize that this is valid in this case.
--> 137 raise self._exception
138
139 return self._result
BadRequest: 400 Error while reading data, error message: Could not parse '16:20:00' as DATE for field FILE_DATE (position 14) starting at location 201 with message 'Unable to parse'
In addition to this error, '16:20:00' does not exist anywhere in the .csv file for this day, and all values in the FILE_DATE column are in the correct DATE format.
