Do you use Alteryx in a language other than English? If so, we want to hear from you! Please help us improve the international experience of our products by participating in this 5 minute survey.

We are updating the requirements for Community registration. As of 7/21/21 all users will be required to register a phone number with their My Alteryx accounts. If you have already registered, you will be prompted on your next login to add your phone number.

Engine Works

Under the hood of Alteryx: tips, tricks and how-tos.
rpaugh
11 - Bolide

How does one automate that which cannot be automated?

 

We are in an information age.  An age where someone can monitor the security of their home and unlock their care from a mobile device; start streaming a movie at home, move to an iPad, and finish on a plane; and even blend, cleanse, and predict data with little or no data science knowledge.  And yet many, if not all of us, at some point have tried to gain access to data within our own organizations only to be met with the solution of “we can email you an Excel file”.  Really?!?  I can put money into my Starbucks account via my phone, and have Starbucks scan and recognize the payment, deduct the cost from my account and give me my bonus stars for the transaction, but the only solution you have to give me data is to email it to me?  Now is a good time to tell you something important about myself – I’m lazy.  I loathe manual, tedious, repetitive tasks.  I often joke with my coworkers that I’m going to automate everything I do so I can sit and watch YouTube all day.  In fact, were I to successfully automate myself out of a job I would consider that the single greatest accomplishment of my career.  As you can probably imagine, I was not willing to manually download and process Excel files each week.  Fortunately, I come from a software development background and was able to write code to extract attachments from Outlook emails and baked that into an SSIS package that ran weekly. 

 

Outlook Input toolOutlook Input toolFast forward a couple of years to Inspire 2017.  I was having breakfast before the opening keynote and I heard an Alteryx employee talking about using Alteryx to blend data from a file she receives from another department.  I had a hunch and asked her how she gets those files.  I stifled a literal “LOL” moment when she said she receives them via email.  I told her I had some code that might help her get to a fully automated solution.  Later in the conference I attended a session on using the Alteryx API and SDK.  I paid particularly close attention to the SDK portion because at the time I had no idea creating custom tools was even an option.  I know this is going to sound lame and cliché, but at that moment I was truly inspired to jump into code and build my own custom input tool to pull attachments out of Outlook emails.  I know, I know, too corny.  Moving on.  As a fantastic conference was coming to a close I just happened to spot one of the engineers, @JPKa,  who hosted the session on using the API and SDK at the closing reception.  I told him my idea and he pointed me in the right direction to get started.  Two weeks of personal time later I had  a solid version 1  of the tool and I shared it with my friends at Alteryx.

 

 

 

Working with the SDK

 

Outlook Input Tool ConfigurationOutlook Input Tool ConfigurationEverything I needed to get started was already installed with Alteryx Designer. I used a combination of reading through and interpreting the sample and the getting started documentation. Honestly, my first impression was confusion.  The sample project involves using an XML file containing Alteryx tool XML config information as an input to set the tool XML config information.  When it came to interpreting the code to understand how it was linking the xml config information to the input data I was often confused as to whether the xml components in question were part of the Alteryx configuration or the input tool that was providing the configuration.  Simply put, it was setting configuration from configuration. If anyone from Alteryx Product Management is reading this,  a better sample might be to pull dummy customer data from a .csv file and send it to an output stream.

 

My biggest obstacle was interpreting the sample code - figuring out how to separate the xml config as input from the xml config that was reading the input.  It makes perfect sense now, but when I was trying to understand the custom tool architecture and input file interpretation simultaneously for the first time I had some trouble.

 

 

Once I had a working tool I went back and forth with the folks at Alteryx and there were some limitations there with getting them the install and sharing attachments. I discovered I could wrap the install into a yxzp file and include instructions and a help file. JP was very helpful with identifying some of the nuances of working with the SDK and offered some suggestions on improving the tool. I needed to incorporate the "UpdateOnly" parameter because the C# code was executing every time a new tool was added to the canvas. I didn't know that the Alteryx Engine had a special feature where it runs the workflow (with no actual records other than metadata) every time a new tool is added.  

 

He also suggested I add some date filtering capability to the tool, allowing people to avoid downloading their entire inbox if they select “Inbox”.  For this I added a configuration element for the user to input a query string and included a link to the documentation to build the query.

 

 

 

Using the Outlook Input Tool

 

You may be tempted to point this sucker to your Inbox right out of the gate, and I don't know what your Inbox looks like, but if it looks anything like some of our testers, I'd advise against that. Play with your drafts folder or deleted items first.

 

You may find copying the search syntax from Outlook into the Query String helpful:

 

copy the query string from outlookcopy the query string from outlook

 

As for me, this is how I've been using the tool. In this example I filter out a specific Excel file and send it through a Dynamic Input tool to see the results. I'll leave it up to you to discover new and interesting ways to consume the files that you no longer have to manually fish out of your inboxes.

 

OutlookInputToolSample.png

 

 

giphy-downsized (2).gifBut wait, there’s more!  Act now and as a bonus you get the added message stream allowing you to perform any number of analytics on your email messages.  And if that isn’t enough, join the two streams together on Message ID for maximum analytical potential!!

 

 

 

 

What's included in the package

 

You'll find an Alteryx *.yxzp file attached to this post. When you open it in Alteryx Designer, you will see installation instructions:

 

Outlook Tools Install instructions.png

 

and a help file

 

Outlook Tools Install help file.png

 

The workflow is a single RunCommand tool that will install the Outlook Input tool. You can find my source code on Git Hub.

 

I hope you find the Outlook Input tool useful. This has been a fun and exciting project and I can't wait to see what creative ways the community finds to use this tool. Use the comments below to let me know your experience working with the tool.

 

UPDATE 8/1/2017:

A new version is available on the gallery: https://gallery.alteryx.com/#!app/Outlook-Tools-Install/597b35c2f499c716ec34a782.

 

Changes:

  • Implemented paging to prevent timeouts through the Exchange Web Services when working with large inboxes and complex filters.  Note: you still may need to adjust throttling settings on your server: https://msdn.microsoft.com/en-us/library/office/jj945066(v=exchg.150).aspx.
  • Updated the installer to remove previously installed versions of the tools so it's not cluttering up your programs list.
Comments
rpaugh
11 - Bolide

@Hudepohle The Outlook tool can be installed on servers. This issue is typically due to Alteryx being installed in one location and the tool in another based on user context. For example, Alteryx may be in Program Files, while the tool may be in the user's AppData. Make sure they're both installed under the same user context and that the tool's .ini file is pointing to the correct Alteryx installation location.

 

@BPrior Nothing with the tool has changed, so yes, it's likely a server change. It's possible that a change has occurred (such as a patch or other upgrade) that changed the security level. The problem there is that I don't believe Microsoft is supporting the back-end technology used by the tool an longer so I can't make changes to SSL/TLS. I assume you're using an on-premise server?

rpaugh
11 - Bolide

@NPham00 Have you tried the O365 version of the tool?

NPham00
7 - Meteor

@rpaugh Thanks for your suggestion, i've tried the tool, i was able to connect to my Outlook Account via  OAuth 2.0. But i've got the error 503: "Service unavailable".
I just have all the standard settings, see below:

NPham00_0-1619525247933.pngNPham00_1-1619525277476.png

 

rpaugh
11 - Bolide

@NPham00 Have you tried entering a folder id? Try "Inbox" and see fi you get the same error.

EvaWick
5 - Atom

Can this only be used to access on-premise Outlook?

NPham00
7 - Meteor

@rpaugh i tried with "Inbox" and "Draft" (in english and german, since i have german version of Outlook 365) but the same error popped up. 

JMart2135
8 - Asteroid

@rpaugh I downloaded the tool from the gallery and got it installed. I seem to be getting a similar error to a previous post.

 

"OutlookInputTool (1) There are no fields. Make sure your container element is set properly.¶ApplicationException¶ at OutlookTools.OutlookInputToolEngine.PI_PushAllRecords(Int64 nRecordLimit) ¶ at SRC.Alteryx.GenericNetPluginInterface.PI_PushAllRecords(GenericNetPluginInterface* , Int64 nRecordLimit)"

 

I have tried my username with and without the domain as well as tried my actual email address itself. Server version is 2016. Any ideas/tips? 

 

Thank you!

OutlookToolConfig.png

rpaugh
11 - Bolide

@EvaWick You can technically use this for Office365 if you know the endpoint, though Office365 is getting further and further away from the supported version of EWS that this tool uses. If you are using Office365, I highly recommend this version of the tool. Technically it says "beta" but that's only because so many people still use the current version that I didn't want to overwrite it with the Office365 one. Many people are using it and it is really production ready.

 

@NPham00 I'll have to think on this. This one is definitely a challenge :).

 

@JMart2135 It looks like you haven't selected any fields to output.

rpaugh
11 - Bolide

@NPham00 Are there any security settings that could be affecting this? I know many weird issues have arisen due to company IT departments tweaking security/access settings. Given that your test folder worked, it may be that local locations are accessible, but server locations are not.

JMart2135
8 - Asteroid

@rpaugh So i selected a few different fields to bring in but getting other errors now. i know in the past when i have had to log into my email accnt from another software i have had to use my domain/username instead of email for it to work. However, if i try that i get an immediate error that the email address is formed incorrectly. Below are screenshots of the errors i got when i did enter my actual email, with and without the server manually entered.

 

OutlookToolConfig2.png

 

OutlookToolConfig3.png

 

 

 

EvaWick
5 - Atom

@rpaugh I downloaded the beta version and I'm receiving the following window.  When I click 'OK', I do not have the tool available.

 

EvaWick_0-1620328232276.png

 

rpaugh
11 - Bolide

@JMart2135 The issue is probably with the formatting of the username. When using domain addresses you need to use the format <username>@<region/sub-domain>.<domain>.com. For example, john.doe@acme.com might become something like jdoe@na.acme.com, or just jdoe@acme.com if there is no region or sub-domain.

 

@EvaWick The Office365 version of the tool utilizes Alteryx's native tool installer so you don't need to download the workflow itself from the gallery. Click "Run" until you get to the final report page. Once there you will see a link to download the .yxi file. Run the .yxi file to trigger the installation process in Alteryx. Then the tool will appear.

JMart2135
8 - Asteroid

@rpaugh Thanks, that cleared that error. But now getting the following error.

 

"Error: OutlookInputTool (1): The request failed. The remote server returned an error: (401) Unauthorized.
ServiceRequestException
at Microsoft.Exchange.WebServices.Data.ServiceRequestBase.GetEwsHttpWebResponse(IEwsHttpWebRequest request)
at Microsoft.Exchange.WebServices.Data.ServiceRequestBase.ValidateAndEmitRequest(IEwsHttpWebRequest& request)
at Microsoft.Exchange.WebServices.Data.MultiResponseServiceRequest`1.Execute()
at Microsoft.Exchange.WebServices.Data.ExchangeService.FindItems(FolderId parentFolderId, String queryString, ViewBase view)
at OutlookTools.OutlookEmail.GetItemsFromFolder(ExchangeService service, Object folder, Boolean isRoot)
at OutlookTools.OutlookEmail.GetItems(Int64 recordLimit)
at OutlookTools.OutlookInputToolEngine.PI_PushAllRecords(Int64 nRecordLimit)
at SRC.Alteryx.GenericNetPluginInterface.PI_PushAllRecords(GenericNetPluginInterface* , Int64 nRecordLimit)"

 

Is there a default port selected within the tool itself? Something in the settings i guess is still blocking the connection because i know my pw is correct.

rpaugh
11 - Bolide

@JMart2135 That definitely looks like a username error (if you say your password is correct). You may have to work with your IT team to ensure you have the correct format for your username using the domain name format. Another option is to use the Office365 version of the tool, which uses OAuth and simplifies authentication.

VahidGoli
8 - Asteroid

Hi,

I am using this great tool, but the only problem is that it downloads all the files from that subject. Is there any way that I can filter based on the date that email received?

 

Thanks for your guidance.

FJ_Shafeek
7 - Meteor

hey @VahidGoli - you can use the QueryString documentation to filter the results you are downloading. The date-specific stuff is as follows:

 

FJ_Shafeek_0-1623255716178.png

 

rpaugh
11 - Bolide

@VahidGoli Which version of the tool are you using (legacy or Office365)?

VahidGoli
8 - Asteroid

Hi @rpaugh@FJ_Shafeek 

 

My email account is office365, and the tool that I am using is attached (with my current setup).

 

@FJ_Shafeek, I used the "Received:today", it seems it can detect them, but nothing is not downloaded to the destination.

 

Would you please guide me? 

OutlookTool-setup.PNG

rpaugh
11 - Bolide

@VahidGoli That configuration gets you no results?

VahidGoli
8 - Asteroid

@rpaugh No results. If I remove the query, it downloads all the attachments from day 1.

 

Do you have any idea why the query doesn't work?

 

 

rpaugh
11 - Bolide

@VahidGoli It could be a timing issue between your computer and your server. I've seen this before where the Exchange server is located in a different time zone than the user's local computer, causing the "today" and "yesterday" functions to not work as expected. If you change the Received filter to include a range of a couple of days does the correct email show up?

VahidGoli
8 - Asteroid

@rpaugh, you were right mate, it was fixed when I chose a range. Thanks a lot.

alberto_hernie
7 - Meteor

Hi @rpaugh,

 

In my company we are using the new version of the Oulook Input tool (office 365). We are trying to log into the tool with a system account (instead of my user) that has access to a shared mailbox, but the tool is not allowing us, it's always login me as my user, not the system account.

 

I guess that this could be happening because the tool is using the AD user logged in the machine?? Do you know any workaround to log in with a system account instead of my user account?

 

Thanks,

Alberto HM 

rpaugh
11 - Bolide

@alberto_hernie If you have the credentials for the system account then use those in the OAuth login screen. That will allow the tool to run as that user.

A_Ribeiro
6 - Meteoroid

Hi @rpaugh! First, a huge thank you! I'm a happy long time user of the previous version, which was flawless for 2 years until our company moved to O365.

 

I'm now trying to move our flows to the new version Outlook Input (Office365) beta and I'm facing a: Outlook Input (Office365) (1) 400: Bad Request, whenever I input any query string.

If I run the tool without any filter from day 1 it works fine. I have tried several strings, encapsulating in quotes, double quotes. None work.

 

I went through this post and could not find other user with the same issue, so, could this be related to some setting on our servers?

 

Thank you once again and keep up the great work!

rpaugh
11 - Bolide

@A_Ribeiro I'm glad it's been working so well for you in the past. For the new tool Microsoft changed how Exchange Services are accessed which also means a change to how query strings are used. See this help document for an explanation of the new filter logic and examples. If you're still having trouble, send me examples of what you're trying to do and I'll see if I can figure out what's going on.

A_Ribeiro
6 - Meteoroid

@rpaugh Thanks a million for the instructions sir, string "ReceivedDateTime ge 2021-01-01 and receivedDateTime lt 2021-05-01" worked like a charm.

 

Best regards,

shahvinit
5 - Atom

Hi Rick Paugh,

 

I am trying to use this Outlook Input tool (2.0.0) with Alteryx version 2021.2. Is this compatible? @rpaugh

Outlook Input tool (2.0.0) works good with Alteryx version 2019.4.

Request to please help.

 

I get the below error in Alteryx version 2021.2.

 
Error: OutlookInputTool (1): The user name or password is incorrect.
 
IOException
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode)
at OutlookTools.OutlookEmail.GetAttachmentsFromItem(Item item)
at OutlookTools.OutlookEmail.GetItemsFromFolder(ExchangeService service, Object folder, Boolean isRoot)
at OutlookTools.OutlookEmail.GetItems(Int64 recordLimit)
at OutlookTools.OutlookInputToolEngine.PI_PushAllRecords(Int64 nRecordLimit)
at SRC.Alteryx.GenericNetPluginInterface.PI_PushAllRecords(GenericNetPluginInterface* , Int64 nRecordLimit)
 
Thanks,
Vinit Shah

 

 

rpaugh
11 - Bolide

@shahvinit That's interesting. Alteryx version shouldn't impact authenticating with EWS. Has something changed with your mail server? I know many IT departments have been increasing/changing security settings in the last few years and that has an impact. Also, are you using on-premise or Office365?

shahvinit
5 - Atom

It working now , Thanks for your help