Twitter Search Tool Not Working Correctly

[dataMack]

Is anyone else having trouble with the Twitter Search tool in 10.0?  

The tool in Alteryx worked fine for me a couple weeks ago, but when I'm trying to use it now the tweets returned off the D arrow seem totally unrelated to my search.

In fact, If i take the 'q_Encoded' value off the S arrow and test it with the Twitter API console (https://dev.twitter.com/rest/tools/console) I get a totally different set of tweets.

I'm wonding if twitter changed something in the JSON response structure, but the macro doesn't seem to be working correctly anymore.

[GarthM]

have you manually checked the destination of the urls provided in the response? it often happens that results look as though they're completely unrelated to a query (e.g., search term is 'NFL' but no reference of that term appears in the text, hashtag, author name, etc.); however, the shortened URL may lead to a site related to that search term. we see this happen a lot with queries related to news organizations (e.g., CNN). 

i'm not saying this is the cause, but nothing has changed with the tool and the structure of the response object from Twitter looks the same.

[dataMack]

Yes- I checked the API URL call and tested it in twitters console.  The console returned a totally different set of tweets.

My search was actually very simple- I was just looking for all tweets from a specific user - not looking for keywords or things like that that might return different results.  The console worked fine, but in Alteryx none of the 100 records I got back were actually from the user I had searched.  I tested this with both the basic search, and the advanced search.

So for example, if I wanted tweets from user @alteryx, the url API call would be this:

https://api.twitter.com/1.1/search/tweets.json?q=from%3Aalteryx

I can check it in the twitter console and get the latest Alteryx tweets:

But none of the tweets returned from the Alteryx twitter search tool in Designer are actually from the @alteryx handle- I get seemingly random tweets from all sorts of users

I'm also noticing this error pop up often when the Twitter tool is on the canvas:

Not sure if it's related

[GarthM]

sorry. i don't think i was as clear as i could have been. i was actually referring to the urls in the response, not the query url. that said, given that you're getting a valid response indicates that the tool is working correctly, and that there's something wonky with the service itself.

[dataMack]

Here's a screenshot with the tool config and the resulting output and summary output.  The expected result given a 'from:alteryx' search should be tweets from the @alteryx handle only.  But only a few of the tweets returned are actually from that handle.

[GarthM · Accepted solution]

i totally understand what you're saying and i used to think the same thing. the fact that you're receiving a valid response (read: status code 200) means alteryx is doing everything correctly. what isn't mentioned (by twitter) is that their free api isn't necessarily hitting the same servers as the test tool you mentioned earlier. it's entirely likely that their dev tool is hitting the firehose ('cuz why not?), but i can tell you for a fact that the data you/we get comes from a source containing a subset of a subset of tweets. given those limitations twitter seems to employ some type of "waterfall" logic (my term, not theirs) to collect/return data that meets the query criteria. what i mean by this is that their search engine will initially grab records that match a query exactly. if/when there's no exact matches left to return twitter expands its search to the body of a tweet to see if there's anything there. from there, twitter also searches the profile info of a given author to see what other matches they can find that could arguably be considered a match (e.g., UserLink or UserExpandedLink, etc.). so, in looking at your response, i see several records where, if you were to look at the 'UserExpandedLink', you'll see why the record was returned. having said all of this it doesn't mean that twitter is doing a good job of returning data you (the user) expect. in fact, i always recommend that if someone really wants to build analytics with social feeds they really should opt for a more reliable resource (read: paid) like GNIP.