The Community SSO is undergoing updates. Users signing in may experience issues. We appreciate your patience as we complete these updates.

Alteryx IO Discussions

Customize and extend the power of Alteryx with SDKs, APIs, custom tools, and more.

[Platform SDK] Error by returning pyarrow.Tables with only headers and no data

JakeMack
6 - Meteoroid

Does anyone have any idea?

 

Error Message:

Designer x64    The Designer x64 reported: You have found a bug.  The pipe has been ended. ¶

 

Code:

In the following code we have data that we have verified in python, but when we run in Alteryx it gives the above error if we do not have values in the columns.

 

 

 

if len(exported_data) > 1:
    packet = pa.Table.from_pandas(pd.DataFrame(dict(zip(exported_data[0], list(zip(*exported_data[1 : ]))))))
else:
    packet = pa.Table.from_pandas(pd.DataFrame(dict(zip(exported_data[0], list([[None]] * len(exported_data[0]))))))

 

 

 

 

Many Thanks!

3 REPLIES 3
jsong
Alteryx
Alteryx

Hello, just a couple of questions.

- What version of Designer is being used?

- What version of the Python SDK is being used?

 

By just eyeballing the code, is the len of `exported_data`'s  ever 0? If so, the `else` clause will be attempting to get data from an empty list.

JakeMack
6 - Meteoroid

Hi jsong - thanks for the response!

 

Designer versions used: 2021.4 & 2022.1

Python SDK:  AYX Python SDK 2.0 https://alteryx.github.io/ayx-python-sdk/getting_started.html

 

export_data is guaranteed to be non-zero. It is supposed to be a list of lists. First item of this list (export_data[0]) is list of headers (headers are guaranteed in our data).

 

We have built the python code out independently of Alteryx and the code works fine with our test scenario. Once we put the code in Alteryx we get the error message referenced above.  

 

Our end goal is to output only headers from the Custom Connector when no data is present. Please see the screenshot below for an example of what we need.

JakeMack_0-1675367637974.png

 

 

Many thanks for your assistance!

jsong
Alteryx
Alteryx

Hello, this is a known issue.

 

With the newer versions of the Python SDK, the team decided to infer the schema off the first record_batch sent, but it relies on data being present.

 

For a "we need a solution now", for cases with no record data, explicitly set it.
Ex.

```

from ayx_python_sdk.core.utils import create_schema

...

self.outputschema = create_schema(...)
self.provider.push_outgoing_metadata("Output", self.outputschema)

```