Alteryx Designer

Find answers, ask questions, and share expertise about Alteryx Designer.

API Connection Errors

Highlighted
8 - Asteroid

 Hello all, using the download tool, I have managed to communicate with API servers and obtain data back in the form of JSON.

 

However, I send about 1million rows of data to the download tool everyday and I have received errors from 'error connecting or downloading to API server' likely due to connection errors.

 

That is okay, network server errors usually occur due to intermittant breaks here and there.

 

When it breaks, the remaining rows of data are not submitted to the API and that causes a problem.

 

How do I get it to 'try again' to connect to the server?

 

Which tool and configuration should I use?

 

Thanks again!

 

Highlighted
Alteryx Certified Partner

If the API allows it you could break up your requests and make several calls. This way if one fails you could identify which one and rerun it individually.

Highlighted
8 - Asteroid

Hi,

 

Thanks for your advice!

 

Unfortunately, that's what im doing now.

 

I would preferan solution that runs and reruns automatically all rows until all rows are completed.

Highlighted
Alteryx Certified Partner

You may be able to use workflow events to accomplish this. There is a basic example here that you may be able to adapt: http://community.alteryx.com/t5/Alteryx-Knowledge-Base/How-to-Use-One-Module-to-Execute-Another-Modu...

 

You would need to change the event to after run with errors.

 

This post also goes over some third party macros that may help.

 

 

Highlighted
8 - Asteroid

Hi, looks like we are getting closer.

 

However, I do not want to run the whole query again just because theres an error, that will take too much time.

 

I was wondering if we can separate the skus out that were errorenous and just run those skus.


Thanks!!

Highlighted
8 - Asteroid

I'm still looking for a solution!

Highlighted
Alteryx
Alteryx

Hi Elizabeth,

 

A couple of things that I don't understand:

- Is it one API call or is each line being submitted separately?

- Are you using the download tool or calling manually using cURL etc?

 

The key to a solution here will be to identify the failing records, and so that will mean separate API Calls, with the need to re-run any API Call that failed. The actual process of re-running until complete can be achieved through an iterative Macro whereby the data keeps going to the iterative output unless it is successful.

 

What I have previously done on a site was wrap the calls in an iterative macro. I'll explain the situation that I had so that you can see if it fits. 

I was calling an API and if my call would result in more than 500 rows, an error would return and so I then had to split that call up into multiple API calls and re-run. The advantage of the iterative macro was that if the process failed, then that iteration would fail, but the next iteration would then kick off. I advise that if a API Call fails, add it back to the bottom of your iteration calls so that it tries again when after everything else has tried.

 

The actual iterative macro is pretty simple, you just split the first API call off and process that, the rest then goees to the iterative output. The same can also be done with a batch macro on RecordID.

 

Image 001 - 20161209 - 161207.png

Highlighted
ACE Emeritus
ACE Emeritus

Hi 

 

 I have exactly the same problem and have posted about this issue previously in the Forum and tried to work somehting out with support. 

 

Here are the past posts: http://community.alteryx.com/t5/Data-Preparation-Blending/Batch-Macro-to-re-run-if-api-causes-server...

http://community.alteryx.com/t5/Data-Sources/Alteryx-Error-messages-into-workflow/m-p/32889#M2325

 

The tool does output to he results window when the tool fails to connect, however there is no way of getting that message in real-time into the workflow to setup a re-try iterative macro. 

 

All other solutions invovled writing a log file, then after run, reading the log file to identify which records haven't connected and re-running. Not ideal and breaks down if you are pulling from multiple tools in one flow. 

 

 

I have raised the feature request here: http://community.alteryx.com/t5/Alteryx-Product-Ideas/Download-Tool-Additional-Feature-Output-of-Ser...

 

Cheers

Alteryx Everything, Leave no one behind.
Highlighted
8 - Asteroid

Hi Gavin, thank you for your wealth of knowledge in your posts.

 

I was wondering if we can filter out by a returned piece of data in a column to use the macro.

 

If I call for X, and X returns a value that is stored in my table, then if X doesnt exist, I would know that the call for X has failed.

 

Have you tried reading off somethign like that?

Labels