AMP Engine - Multi-Row Formula Tool Issues

[AlterRexRyan]

Hi All,

I'm using the AMP Engine for a particular workflow in order to make it run faster. I'm aware of some of the pitfalls of AMP - namely that any tool that requires the records to be in a particular order can get messed up due to the multi-threading nature of AMP.

I have a multi-row tool that I need to use out of necessity, and I've placed a Sort tool right before it - however, the records are still coming through out of order into the multi-row tool.

Is there any way to solve for this? Is it a bug? I have some other workflows using AMP Engine, and placing a sort tool before the multi-row worked in those cases.

Any suggestions are appreciated!

[binuacs]

@AlterRexRyan One option is to put a record-id tool before the multi-row formula tool then add the sort tool after the multi-row tool and sort based on the recordID field 

[AlterRexRyan]

I'm not sure I follow that solution. To clarify my issue, I have a multi-row formula tool with logic that is dependent on the data entering the tool in a specific sort order. I have a sort tool right before the multi-row tool in order to accomplish this.

But somehow, despite having the sort tool right before the multi-row tool with nothing in between, the data is getting sorted out of order between the two tools due to the AMP engine.

I realize the AMP engine causes issues with logic like this since things get processed in a different order, however I thought I'd be safe as long as I put sort tools before things like the multi-row tool that needs a specific order (and I have multiple AMP workflows that do this and are fine). Am I wrong here? If so, is there a workaround? There must be or the AMP engine would have a severe flaw.

[ed_hayter]

I am surprised this has happened - could it be to do with the groupby configuration of your mutli-row formula?

it might help if you post a photo of how the tool is configured with sensitive information hidden/covered up and maybe quickly mocking up what you are expecting v. what you want.

[ChrisK]

 ... any tool that requires the records to be in a particular order can get messed up due to the multi-threading nature of AMP

This should not be the expectation. Though the AMP engine works in parallel, it should be presenting results in a deterministic order. (The typical order of grouping is a hash order, so it doesn't look like it is "ordered", but it is deterministic and repeatable.) In particular, if you have a sort tool and you think the results are coming out incorrectly sorted, that would be a bug. There's not much point to a sort if it doesn't give you records in a sorted order.

Can you make a repeatable example of this?

Chris

[dyamichael]

I have this exact same issue and can replicate it. It caught me off guard because an old workflow was suddenly outputting very wrong results.

My solution is to be cautious of turning off AMP in workflows that use multi-row tools. Maybe it'll be fixed in an update.

[mceleavey]

HI @AlterRexRyan ,

can you please share your workflow and some sample data (representative) so we can see what you're doing and maybe try to replicate it?

It doesn't sound like an AMP issue as the Sort tool should resolve it.

M.