Is there an example or recommended pattern for handling batches with 0 rows? Custom tools seem to hang often in this situation, and I have no idea how to remedy it.
This may be related to the known limitation mentioned in AYX Plugin CLI Command Overview. However this occurs with an empty Text Input tool and with other empty outputs (e.g. the F anchor of a Filter tool with filter `1 = 1`).
- If you connect a Python SDK plugin to a Text Input tool with no rows, it triggers an unrelated error. To fix this, you need to add rows to the Text Input tool.
The issue is that the custom tool never seems to enter `on_complete()`. This occurs even in an Output-type custom tool that does nothing.
class MyOutputTool(PluginV2):
def __init__(self, provider: AMPProviderV2):
self.name = "My Output Tool"
self.provider = provider
def on_record_batch(self, batch: pa.Table, anchor: Anchor) -> None:
self.provider.io.info("IN on_record_batch")
def on_incoming_connection_complete(self, anchor: Anchor) -> None:
self.provider.io.info("IN on_incoming_connection_complete")
def on_complete(self) -> None:
self.provider.io.info("IN on_complete")
The tool will log "IN on_incoming_connection_complete", but will never log "IN on_complete". Is there anything to be done about this case?
A related issue occurs when attempting to write 0 results to an anchor.
# `dataframe` is an empty pd.DataFrame
self.provider.write_to_anchor("Output", pa.Table.from_pandas(dataframe))
which results in the error message "Encountered error pushing records".
