Exit Python module early?

[TH]

I built a batch macro that uses a user specification to run an SQL query with different values in parts of the "where" clause. Then the batch macro uses some alteryx modules to adjust the SQL output a bit, passes the stream into a Python module that does some more intricate processing than I can figure out how to do in Alteryx, produces a python output, makes an adjustment to the python output, and then sends the final output as the macro output.

The python module contains several cells that are split up in order to facilitate development and maintenance.

The SQL query usually produces proper and significant data for the next step.

On occasion the SQL will not produce any data at all. This lack of input causes the python module to throw an error at the first step when it tries to import the data. This causes the python module to output no value, so the entire iteration of the batch macro (for those SQL inputs) outputs no value

In those cases I want to create a default output from the Python module that will get processed into the macro output, send it, and end the module without doing anything else.

Unfortunately, the Python module wants to run through every cell in its notebook and end up spitting out a the final output. In cases where there's no SQL input (and hence I want to send a default output) then I'd like to be able to stop the Python module early.

I want to do something at the first step of the python module notebook like the following psuedo-code:

if input == blank:

    send default dataframe to output number 1

    exit python module #so that the batch macro cycles around to the next set of inputs for the SQL

else: #i.e. the input is not blank

    perform the rest of the processing

What's the way to do this?

    putting the "Alteryx.write" command early in my python module doesn't work because the module wants to go through all the code in the module before exiting (or at least until it errors)

    "continue" and "break" don't work because they are intended for python loop constructs and don't actually do anything to exit the module

    "quit" and "exit" will kill the python kernel entirely, and then the batch macro will fail on any subsequent passes through the python module

    reconstructing the entire set of code into one block and encasing the whole thing in an "if" statement might work, but it's clunky and I'm worried about being able to maintain or debug the code.

Seems like there ought to be a simple way to do this.

Ideas?

[apathetichell]

Can you use a try catch or a break statement in your for loop? I

[griffinwelsh]

This sounds like a usecase for a try except if you are getting an error from the python script, but it is hard to say for sure without seeing a sample input and the code.

[TH]

Try except *sounds* like the right solution. I agree with you there.

The issue is that I don't know what to put in the "except" section to exit the python module with only the default output and allow the rest of the batch macro to proceed.

As I noted in my original post, I have tried several things and none of what I've tried has been able to send the default output, exit the python module, and allow the batch macro to continue.

[griffinwelsh]

It is possible that I don't understand your usecase, but you should be able to put your default data frame at the top all of your SQL code in the try section and write your default in the except section. This will not kill the macro and will let you write your default output on the iteration with no data.

from ayx import Alteryx
import pandas as pd
default = some data frame
df = pd.DataFrame(default)
try:
    dat = Alteryx.read("#1")

    Your code
    Alteryx.write(dat, 1)
except:
Alteryx.write(df, 1)

[griffinwelsh]

One more idea that works with multiple cells is to create a variable to track if the request should be killed. On each jupyter cell you will add an if statement to check if the kill variable has been set and skip if so. This is not very elegant, but it will work. I have attached a sample workflow to demonstrate

[TH]

griffinwelsh,

I appreciate your trying.

As I said earlier, I have several cells in the python module notebook for the purposes of maintaining and debugging the process.

The first thing that the module does after importing libraries is to read the data stream.

at that point, if the datastream read is problematic for some reason then I want to end the module.

If the read created no problem then I need to run the data through something like six or eight more cells of code.

Should I understand from your answers that you're suggesting that I put *all* my code in one cell (in one try except wrapper)?

[griffinwelsh · Accepted solution]

That was first suggestion yes if you use a single cell that will allow the try except statement to work. Other solution is to assign a variable a value in the first cell when you get an exception and then use if statements in other cells to not execute code if that variable has the assigned value. If you have a lot of cells this will be tedious, but will also work and allow you to keep the cell segregation. There may be a better solution to this, but that was the best I could do.