[Platform SDK] Error by returning pyarrow.Tables with only headers and no data
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Does anyone have any idea?
Error Message:
Designer x64 The Designer x64 reported: You have found a bug. The pipe has been ended. ¶
Code:
In the following code we have data that we have verified in python, but when we run in Alteryx it gives the above error if we do not have values in the columns.
if len(exported_data) > 1:
packet = pa.Table.from_pandas(pd.DataFrame(dict(zip(exported_data[0], list(zip(*exported_data[1 : ]))))))
else:
packet = pa.Table.from_pandas(pd.DataFrame(dict(zip(exported_data[0], list([[None]] * len(exported_data[0]))))))
Many Thanks!
- Labels:
-
Custom Tools
-
Python
-
SDK
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hello, just a couple of questions.
- What version of Designer is being used?
- What version of the Python SDK is being used?
By just eyeballing the code, is the len of `exported_data`'s ever 0? If so, the `else` clause will be attempting to get data from an empty list.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hi jsong - thanks for the response!
Designer versions used: 2021.4 & 2022.1
Python SDK: AYX Python SDK 2.0 https://alteryx.github.io/ayx-python-sdk/getting_started.html
export_data is guaranteed to be non-zero. It is supposed to be a list of lists. First item of this list (export_data[0]) is list of headers (headers are guaranteed in our data).
We have built the python code out independently of Alteryx and the code works fine with our test scenario. Once we put the code in Alteryx we get the error message referenced above.
Our end goal is to output only headers from the Custom Connector when no data is present. Please see the screenshot below for an example of what we need.
Many thanks for your assistance!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hello, this is a known issue.
With the newer versions of the Python SDK, the team decided to infer the schema off the first record_batch sent, but it relies on data being present.
For a "we need a solution now", for cases with no record data, explicitly set it.
Ex.
```
from ayx_python_sdk.core.utils import create_schema
...
self.outputschema = create_schema(...)
self.provider.push_outgoing_metadata("Output", self.outputschema)
```
![](/skins/images/59F2CCDACB95212C69570BCFC47D6F56/responsive_peak/images/icon_anonymous_message.png)