This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
I wanted to know, when we are writing data to salesforce output connectors, is it Apex data loader that runs behind the scenes?
The requirement I am having is to capture details when we hit salesforce output connector, details should contain all errors - capturing data in key fields using which we can identify the records, the actual error (message) occured, the field for which the error occurs etc.
Is it possible to get similar kind of logging that Apex data loader generates when using Alteryx Salesforce connectors.
The Salesforce Output tool uses the Salesforce BULK API to write data to Salesforce. Any error messages that the API returns to Alteryx should be written to the Alteryx log. If you're feeling ambitious and would like to try and customize the tool to capture more information from the API responses, you can find the underlying macro here: C:\Program Files\Alteryx\bin\RuntimeData\Macros\Supporting_Macros\SalesforceOutput.yxmc. This post will walk you through how to create your own custom tool.
Thanks for the suggestion, I went through the post for creation of a custom tool. From C:\Program Files\Alteryx\bin\HtmlPlugins when refering to the Salesforce Output Configuration I was able to get the redirection to \Supporting_Macros/SalesforceOutput.yxmc.
On opening the SalesforceOutput.yxmc from C:\Program Files\Alteryx\bin\RuntimeData\Macros\Supporting_Macros\, I was able to get the xml code for the connector.
On doing some research I discovered that Apex Dataloader by default uses SOAP API and not BULK API, and might be that is the reason for detailed logging.
The xml code for the connector seems approx 4000 lines and seems there are various Tool tags inside the source code like ...
<Node ToolID="105"> .... </Node>
The usual suspects I was able to locate within the source code are follows which might need to be changed.
SalesforceOutput.yxmc is an Alteryx macro so you'll want to open the file in Alteryx rather than a text editor. In Alteryx, whenever you see a Download tool (), you'll know that a call to the API has been made and you can attempt to parse out additional "logging" information after the Download tool from the response fields (DownloadHeaders and DownloadData). Changing the macro to use the SOAP API rather than the BULK API would require significant refactoring, not just switching a few lines of code in some of the tools, so I wouldn't recommend this unless you have lots of time. Also, the SOAP API will be much slower. Remember to save a back up of the macro if you choose to edit it and I'd also recommend testing your custom tool in a Salesforce sandbox environment before pushing data to your production environment.
I will try the option of pulling additional information from the "download tool" while opening the connector definition in Alteryx itself.
Meanwhile is it possible to get the deprecated versions of salesforce connectors as additional connectors so that I can try those as well (as needed only). I believe those should be using the SOAP API and using them should be easy and since that would not be in BULK mode I can get record level logging. I understand that there might be performance bottle necks but like getting error details at record level along with related reason is a mandatory requirement for me.
The only other option I am aware of is to use Apex data loader - which again uses SOAP API by default, I believe the performance in both the cases should be more or less similar.
My understanding was that since the current version of Alteryx is using BULK API, the granule at which transactions are done would be at a batch level and that is the reason why error logging does not happen at record level.
I am going to try the "download tool" option from my end, in case you have any more insights/options request you to share those.
Using the approach suggested I was able to get the information I was looking for. Thank you!!!
However I had few observations as below -
1) If I override the Salesforce Output connector definition, the deprecated version is not having an impact.
2) Tried various options with Salesforce Output connector definition, like message type - Warning/Error but seems the alteryx logs are not able to capture more than 1 error, even tried updating the sample tool to have N=100 but no avail. However, it was possible to capture more than one error information using Salesforce Output connector (deprecated version).
3) Also I tried to concatenate the actual 'data' along with 'Downloaddata', but even after proper amount of normalization etc, it seemed to output only 0.
The approach I am planning to take is to have a separate pipeline in the Salesforce Output connector definition, and later write the information I want in a separate table altogether and not to rely on alteryx/salesforce level logs for complete error information.