[Platform SDK] Error by returning pyarrow.Tables with only headers and no data

[JakeMack]

Does anyone have any idea?

Error Message:

Designer x64    The Designer x64 reported: You have found a bug.  The pipe has been ended. ¶

Code:

In the following code we have data that we have verified in python, but when we run in Alteryx it gives the above error if we do not have values in the columns.

if len(exported_data) > 1:
    packet = pa.Table.from_pandas(pd.DataFrame(dict(zip(exported_data[0], list(zip(*exported_data[1 : ]))))))
else:
    packet = pa.Table.from_pandas(pd.DataFrame(dict(zip(exported_data[0], list([[None]] * len(exported_data[0]))))))

Many Thanks!

[jsong]

Hello, just a couple of questions.

- What version of Designer is being used?

- What version of the Python SDK is being used?

By just eyeballing the code, is the len of `exported_data`'s  ever 0? If so, the `else` clause will be attempting to get data from an empty list.

[JakeMack]

Hi jsong - thanks for the response!

Designer versions used: 2021.4 & 2022.1

Python SDK:  AYX Python SDK 2.0 https://alteryx.github.io/ayx-python-sdk/getting_started.html

export_data is guaranteed to be non-zero. It is supposed to be a list of lists. First item of this list (export_data[0]) is list of headers (headers are guaranteed in our data).

We have built the python code out independently of Alteryx and the code works fine with our test scenario. Once we put the code in Alteryx we get the error message referenced above.  

Our end goal is to output only headers from the Custom Connector when no data is present. Please see the screenshot below for an example of what we need.

Many thanks for your assistance!

[jsong]

Hello, this is a known issue.

With the newer versions of the Python SDK, the team decided to infer the schema off the first record_batch sent, but it relies on data being present.

For a "we need a solution now", for cases with no record data, explicitly set it.
Ex.

```

from ayx_python_sdk.core.utils import create_schema

...

self.outputschema = create_schema(...)
self.provider.push_outgoing_metadata("Output", self.outputschema)

```