Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Salesforce connector logs

Rohit_Bajaj
9 - Comet

Hi All,

 

I wanted to know, when we are writing data to salesforce output connectors, is it Apex data loader that runs behind the scenes?

The requirement I am having is to capture details when we hit salesforce output connector, details should contain all errors - capturing data in key fields using which we can identify the records, the actual error (message) occured, the field for which the error occurs etc.

 

Is it possible to get similar kind of logging that Apex data loader generates when using Alteryx Salesforce connectors.

 

Thanks,

Rohit

7 REPLIES 7
NeilR
Alteryx Alumni (Retired)

The Salesforce Output tool uses the Salesforce BULK API to write data to Salesforce. Any error messages that the API returns to Alteryx should be written to the Alteryx log. If you're feeling ambitious and would like to try and customize the tool to capture more information from the API responses, you can find the underlying macro here: C:\Program Files\Alteryx\bin\RuntimeData\Macros\Supporting_Macros\SalesforceOutput.yxmc. This post will walk you through how to create your own custom tool. 

Rohit_Bajaj
9 - Comet

Hi Neil,

 

Thanks for the suggestion, I went through the post for creation of a custom tool.
From C:\Program Files\Alteryx\bin\HtmlPlugins when refering to the Salesforce Output Configuration I was able to get the redirection to \Supporting_Macros/SalesforceOutput.yxmc.

On opening the SalesforceOutput.yxmc from C:\Program Files\Alteryx\bin\RuntimeData\Macros\Supporting_Ma​cros\, I was able to get the xml code for the connector.

 

On doing some research I discovered that Apex Dataloader by default uses SOAP API and not BULK API, and might be that is the reason for detailed logging.

 

The xml code for the connector seems approx 4000 lines and seems there are various Tool tags inside the source code like ...

 

<Node ToolID="105"> .... </Node>

 

The usual suspects I was able to locate within the source code are follows which might need to be changed.

...
<jobInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload">
...
<FormulaField field="url" type="String" size="1000" expression="[base_url]+'/services/async/34.0/job/'+[job_id]" />
...
<DefaultAnnotationText>url=[base_url]+'/services/async/34.0/job/'+[job_id]</DefaultAnnotationText>
...
<Field name="url" size="1000" source="Formula: [base_url]+'/services/async/34.0/job/'+[job_id]" type="String" />
...
<Field name="url" size="1000" source="Formula: [base_url]+'/services/Soap/u/34.0'" type="V_WString" />
...
<EngineSettings Macro="Supporting_Macros\SalesforceOutput.convertToCsvAndBatch.yxmc" />
...
<EngineSettings Macro="Supporting_Macros\Salesforce.errorCheck.yxmc" />
...
        <Node ToolID="158">
          <GuiSettings Plugin="AlteryxBasePluginsGui.Message.Message">
            <Position x="2298" y="659" />
          </GuiSettings>
          <Properties>
            <Configuration>
              <When>Filter</When>
              <Filter>[count]&gt;0 and [operation]=='Insert'</Filter>
              <Type>ErrorStop</Type>
              <Priority>High</Priority>
              <Transient>False</Transient>
              <MessageExpression>'Id field cannot be inserted into target table'</MessageExpression>
            </Configuration>
            <Annotation DisplayMode="0">
              <Name>Error trying to insert</Name>
              <DefaultAnnotationText />
              <Left value="False" />
            </Annotation>
            <MetaInfo connection="Output">
              <RecordInfo>
                <Field name="fields_not_in_table" size="2147483647" source="Summarize: Concat - Name" type="V_String" />
                <Field name="fields_not_creatable" size="1073741823" source="Summarize: Concat - name" type="V_WString" />
                <Field name="fields_not_updateable" size="1073741823" source="Summarize: Concat - name" type="V_WString" />
                <Field name="Count" source="(Multiple Sources)" type="Int64" />
                <Field name="count_non_id" source="(Multiple Sources)" type="Int64" />
                <Field name="operation" size="64" source="Formula: [#1]" type="String" />
                <Field name="error" source="Formula: [fields_not_in_table]!=null() or &#xA;([fields_not_updateable]!=null() and [operation]=='Update') or &#xA;([fields_not_creatable]!=null() and [operation]=='Insert') or &#xA;([count]&gt;0 and [operation]=='Insert') or &#xA;([count]==0 and [operation] in ('Delete','Update')) or &#xA;([count_non_id]&gt;0 and [operation] == 'Delete')" type="Bool" />
              </RecordInfo>
            </MetaInfo>
          </Properties>
          <EngineSettings EngineDll="AlteryxBasePluginsEngine.dll" EngineDllEntryPoint="AlteryxMessage" />
        </Node>


If you have further insights with the Salesforce Output connector w.r.t error logging mechanism, request you to please share.

 

I was suspecting that if changing <jobInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload"> to <jobInfo xmlns="http://soap.sforce.com/2009/10/clientsync"> or
<jobInfo xmlns="http://soap.sforce.com/schemas/class/syncapi/Command "> should change the load type from BULK to SOAP.

 

Please feel free to share any other thoughts you have on this, also let me know where can I get more details about the xml code of various transformations within Alteryx.

 

Thanks,
Rohit Bajaj

NeilR
Alteryx Alumni (Retired)

SalesforceOutput.yxmc is an Alteryx macro so you'll want to open the file in Alteryx rather than a text editor. In Alteryx, whenever you see a Download tool (Capture.PNG), you'll know that a call to the API has been made and you can attempt to parse out additional "logging" information after the Download tool from the response fields (DownloadHeaders and DownloadData). Changing the macro to use the SOAP API rather than the BULK API would require significant refactoring, not just switching a few lines of code in some of the tools, so I wouldn't recommend this unless you have lots of time. Also, the SOAP API will be much slower. Remember to save a back up of the macro if you choose to edit it and I'd also recommend testing your custom tool in a Salesforce sandbox environment before pushing data to your production environment.

Rohit_Bajaj
9 - Comet

Hi Neil,

 

Thanks for your help and guidance.

I will try the option of pulling additional information from the "download tool" while opening the connector definition in Alteryx itself.

Meanwhile is it possible to get the deprecated versions of salesforce connectors as additional connectors so that I can try those as well (as needed only). I believe those should be using the SOAP API and using them should be easy and since that would not be in BULK mode I can get record level logging. I understand that there might be performance bottle necks but like getting error details at record level along with related reason is a mandatory requirement for me.

 

The only other option I am aware of is to use Apex data loader - which again uses SOAP API by default, I believe the performance in both the cases should be more or less similar. 

 

My understanding was that since the current version of Alteryx is using BULK API, the granule at which transactions are done would be at a batch level and that is the reason why error logging does not happen at record level.

 

I am going to try the "download tool" option from my end, in case you have any more insights/options request you to share those.

 

Regards,

Rohit

NeilR
Alteryx Alumni (Retired)

To get the deprecated Salesforce tools right click on any tool in the Connectors category of the Tool Palette and select "Show Deprecated Tools".

Rohit_Bajaj
9 - Comet

Hi Neil,

 

Using the approach suggested I was able to get the information I was looking for. Thank you!!!

 

However I had few observations as below -

 

1) If I override the Salesforce Output connector definition, the deprecated version is not having an impact.

2) Tried various options with Salesforce Output connector definition, like message type - Warning/Error but seems the alteryx logs are not able to capture more than 1 error, even tried updating the sample tool to have N=100 but no avail. However, it was possible to capture more than one error information using Salesforce Output connector (deprecated version).

3) Also I tried to concatenate the actual 'data' along with 'Downloaddata', but even after proper amount of normalization etc, it seemed to output only 0.

 

The approach I am planning to take is to have a separate pipeline in the Salesforce Output connector definition, and later write the information I want in a separate table altogether and not to rely on alteryx/salesforce level logs for complete error information.

 

Marking the thread as answered.

 

Regards,

Rohit

Drummond
5 - Atom

Hi Rohit,

 

Did you solve being able to retrieve the logs as the updates/creates/etc are flowing into SFDC? If so - could you share your changes that you made?

 

Thanks

 

Labels