Download tool, content-type and payload unexpected behaviour (bug?)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hello, I've banging my head to wall for a while when trying to fetch some data from Adaptive Insight API. That API gets all requests with XML commands through POST call. I have a call which works with CURL but fails when trying to use it from Alteryx.
For debugging I wrote a script which just logs all headers and payload from HTTP request in server. When calling that script from Alteryx with a simple XML payload. Results were a bit disturbing. Here is simplest reproducible case I created.
Scenario 1 Setup
POST call, XML payload in "Use following query string / Body".
No headers defined
Expected result
Payload sent to a server as-is, no content-type header defined.
Observed result
Wrong content-type
First characters (up to first equals char (=)) missing from payload.
Scenario 2 Setup
Like scenario 1, but manually set correct Content-Type header
Expected result
In HTTP request defined content type and payload as-is.
Observed result
Payload missing
I tested setting different Content-Type headers, but every time it was set, all payload was missing.
When payload is not set, Alteryx seems to cut payload from first "=" mark.
So how can i send POST request with Alteryx with right content-type header and untouched XML payload?
Solved! Go to Solution.
- Labels:
- API
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I believe what's happening is its sending the request with the first part before then equals as the "Name" and the bit after the equals as the "Value" in the payload.
I'd recommend putting the XML into a field before the download tool, name that column what the API expects to receive the information as "data" for example and try that way.
If you could share the Adaptive Insight API documentation (I had a quick search and couldn't find it) I'll be able to be a bit more specific.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Thank you for your quick reply.
I tried to put payload in a table and read it from a field but outcome was still the same. Still Alteryx is corrupting payload (probably tries to use some url encoding or formdata parsing for payload).
And this question is more general than just Adaptive API. When we find out way to send correct Content-Type header with uncorrupted payload, we've solved this problem This just came up with Adaptive api. Just for reference, below is a PHP script I am using to record HTTP request. In case you want to regenerate the problem.
<?php
$tfile = "/tmp/mylogfile.txt";
$file = fopen($tfile, "a");
fwrite($file, "******* BEGIN **********\n");
fwrite($file, date("Y-M-d H:i:s")."\n");
foreach (getallheaders() as $name => $value) {
fwrite($file, $name.": ".$value."\n");
}
fwrite($file, "END OF HEADERS\n\n");
fwrite($file, implode("\n",$_POST));
fwrite($file,"**** END****\n\n");
fclose($file);
$prot = $_SERVER["SERVER_PROTOCOL"];
header("$prot 201 OK ");
?>
Oh, and what you mean by "Alteryx uses Rest rather than CURL". Curl is a simple command line software you use generating HTTP requests and REST is an API framework. They aren't interchangeable.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Could you upload your workflow please (without credentials)
And yes you are right with REST/CURL I hadn't drunk enough coffee yet!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Here is how I did the test when payload was in a table before download tool. As said, outcome was the same.
Data in table
I didn't find a way to put multiline data to a field of a table. There probably is one, and in this case I can use oneliner as well. There are cases when receiving systems excepts that XML has multiple lines though.
Download tool settings
Same result.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Best this far has been "compose query string". Then payload isn't corrupted, but content-type is still wrong. And you cannot select the right conent-type from dropdown-menu. BTW, if you select formdata, id don't send any payload at all.
And result from that setting
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I wonder if some of what you are seeing is a limitation of the PHP superglobal $_POST. (I am no PHP expert so apologies if I am wrong)
I changed your php script to this
<?php
$tfile = "mylogfile.txt";
$file = fopen($tfile, "a");
fwrite($file, "******* BEGIN **********\n");
fwrite($file, date("Y-M-d H:i:s")."\n");
foreach (getallheaders() as $name => $value) {
fwrite($file, $name.": ".$value."\n");
}
fwrite($file, "END OF HEADERS\n\n");
fwrite($file, file_get_contents('php://input'));
fwrite($file,"**** END****\n\n");
fclose($file);
$prot = $_SERVER["SERVER_PROTOCOL"];
header("$prot 201 OK ");
?>
and I am seeing better results
******* BEGIN **********
2021-Jul-08 12:18:18
Host: localhost
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 70
END OF HEADERS
<?xml version='1.0' encoding='UTF-8'?><data><this>IS XML</this></data>**** END****
That still doesn't solve why the call is working via curl, but not through the download tool, but maybe we need to look elsewhere for the why?
https://www.linkedin.com/in/adriley/
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
This post https://stackoverflow.com/questions/8893574/php-php-input-vs-post would explain why you are seeing uncorrupted data when using application/x-www-form-urlencoded
https://www.linkedin.com/in/adriley/
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Thank you for improving my PHP script. I'm not PHP guru myself neither...
But there is still wrong Content-Type. I'd need it to be text/xml or application/xml. And that is coming from Alteryx.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Here is where I am with my test workflow. (Attached)
******* BEGIN **********
2021-Jul-08 12:39:15
Host: localhost
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: text/xml
Content-Length: 70
END OF HEADERS
<?xml version='1.0' encoding='UTF-8'?><data><this>IS XML</this></data>**** END****
Which looks like it is giving the right Content-Type
https://www.linkedin.com/in/adriley/
