Download tool, content-type and payload unexpected behaviour (bug?)

Hello, I've banging my head to wall for a while when trying to fetch some data from Adaptive Insight API. That API gets all requests with XML commands through POST call. I have a call which works with CURL but fails when trying to use it from Alteryx.

For debugging I wrote a script which just logs all headers and payload from HTTP request in server. When calling that script from Alteryx with a simple XML payload. Results were a bit disturbing. Here is simplest reproducible case I created.

Scenario 1 Setup

POST call, XML payload in "Use following query string / Body".

No headers defined

Expected result

Payload sent to a server as-is, no content-type header defined.

Observed result

Wrong content-type

First characters (up to first equals char (=)) missing from payload.

Scenario 2 Setup

Like scenario 1, but manually set correct Content-Type header

Expected result

In HTTP request defined content type and payload as-is.

Observed result

Payload missing

I tested setting different Content-Type headers, but every time it was set, all payload was missing.

When payload is not set, Alteryx seems to cut payload from first "=" mark.

So how can i send POST request with Alteryx with right content-type header and untouched XML payload?

API

Accepted answers

AdamR_AYX

I wonder if some of what you are seeing is a limitation of the PHP superglobal $_POST. (I am no PHP expert so apologies if I am wrong)

I changed your php script to this

		<?php
			$tfile = "mylogfile.txt";
			$file = fopen($tfile, "a");
			fwrite($file, "******* BEGIN **********\n");
			fwrite($file, date("Y-M-d H:i:s")."\n");

			foreach (getallheaders() as $name => $value) {
				fwrite($file, $name.": ".$value."\n");
			}
			fwrite($file, "END OF HEADERS\n\n");
			fwrite($file, file_get_contents('php://input'));
			fwrite($file,"**** END****\n\n");
			fclose($file);
			$prot = $_SERVER["SERVER_PROTOCOL"];
			header("$prot  201 OK ");

		?>

and I am seeing better results

******* BEGIN **********
2021-Jul-08 12:18:18
Host: localhost
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 70
END OF HEADERS

<?xml version='1.0' encoding='UTF-8'?><data><this>IS XML</this></data>**** END****

That still doesn't solve why the call is working via curl, but not through the download tool, but maybe we need to look elsewhere for the why?

All comments

JoeS

I believe what's happening is its sending the request with the first part before then equals as the "Name" and the bit after the equals as the "Value" in the payload.

I'd recommend putting the XML into a field before the download tool, name that column what the API expects to receive the information as "data" for example and try that way.

If you could share the Adaptive Insight API documentation (I had a quick search and couldn't find it) I'll be able to be a bit more specific.

ex4

Thank you for your quick reply.

I tried to put payload in a table and read it from a field but outcome was still the same. Still Alteryx is corrupting payload (probably tries to use some url encoding or formdata parsing for payload).

And this question is more general than just Adaptive API. When we find out way to send correct Content-Type header with uncorrupted payload, we've solved this problem This just came up with Adaptive api. Just for reference, below is a PHP script I am using to record HTTP request. In case you want to regenerate the problem.

<?php
    $tfile = "/tmp/mylogfile.txt";
    $file = fopen($tfile, "a");
    fwrite($file, "******* BEGIN **********\n");
    fwrite($file, date("Y-M-d H:i:s")."\n");

    foreach (getallheaders() as $name => $value) {
        fwrite($file, $name.": ".$value."\n");
    }
    fwrite($file, "END OF HEADERS\n\n");
    fwrite($file, implode("\n",$_POST));
    fwrite($file,"**** END****\n\n");
    fclose($file);
    $prot = $_SERVER["SERVER_PROTOCOL"];
    header("$prot  201 OK ");

?>

Oh, and what you mean by "Alteryx uses Rest rather than CURL". Curl is a simple command line software you use generating HTTP requests and REST is an API framework. They aren't interchangeable.

JoeS

Could you upload your workflow please (without credentials)

And yes you are right with REST/CURL I hadn't drunk enough coffee yet!

ex4

Here is how I did the test when payload was in a table before download tool. As said, outcome was the same.

Data in table

I didn't find a way to put multiline data to a field of a table. There probably is one, and in this case I can use oneliner as well. There are cases when receiving systems excepts that XML has multiple lines though.

Download tool settings

Same result.

ex4

Best this far has been "compose query string". Then payload isn't corrupted, but content-type is still wrong. And you cannot select the right conent-type from dropdown-menu. BTW, if you select formdata, id don't send any payload at all.

And result from that setting

AdamR_AYX

I wonder if some of what you are seeing is a limitation of the PHP superglobal $_POST. (I am no PHP expert so apologies if I am wrong)

I changed your php script to this

		<?php
			$tfile = "mylogfile.txt";
			$file = fopen($tfile, "a");
			fwrite($file, "******* BEGIN **********\n");
			fwrite($file, date("Y-M-d H:i:s")."\n");

			foreach (getallheaders() as $name => $value) {
				fwrite($file, $name.": ".$value."\n");
			}
			fwrite($file, "END OF HEADERS\n\n");
			fwrite($file, file_get_contents('php://input'));
			fwrite($file,"**** END****\n\n");
			fclose($file);
			$prot = $_SERVER["SERVER_PROTOCOL"];
			header("$prot  201 OK ");

		?>

and I am seeing better results

<?xml version='1.0' encoding='UTF-8'?><data><this>IS XML</this></data>**** END****

That still doesn't solve why the call is working via curl, but not through the download tool, but maybe we need to look elsewhere for the why?

AdamR_AYX

This post https://stackoverflow.com/questions/8893574/php-php-input-vs-post would explain why you are seeing uncorrupted data when using application/x-www-form-urlencoded

AdamR_AYX

Here is where I am with my test workflow. (Attached)

******* BEGIN **********
2021-Jul-08 12:39:15
Host: localhost
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: text/xml
Content-Length: 70
END OF HEADERS

<?xml version='1.0' encoding='UTF-8'?><data><this>IS XML</this></data>**** END****

Which looks like it is giving the right Content-Type

DownloadPayloadTest.yxmd

AdamR_AYX

Are you able to share your CURL command that works? (With the auth/password starred out)

ex4

There was obviously major bug in the script I used for debug. That misled me big time. Reminder to self..... never use simple scripts with language you really don't know.

Thanks to all for quick replies! This was really helpful.

Just a brief explanation what happened.

In another system I call Adaptive API without Content-Type header. Not the best practice but that works. So I did the same with Alteryx. But Alteryx defaults to application/x-www-form-urlencoded content-type and with this header Adaptive API returned an error saying XML is invalid.

Then I started debug the situation with my faulty script and the rest is history.

I used my original payload with Content-Type header set in headers tab and this workflow works as a charm.

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703