Select Records tool | Trying to drop 5 rows, but 15,000+ get dropped instead

Question

Hi, I'm running into an issue, and I have no idea how to begin resolving it.

I am trying to make a standard macro that can take catalogs of product data, with some catalogs having up to 65,000 rows. I need this as a macro, because the logic needs to be used across three different workflows. The first five rows of data are header rows that I don't need, so I'm trying to use the Select Records tool to drop them. When I use it, though, the tool doesn't seem to work.

This is how I have the tool configured:

This is info from the tool's input view -  I'm starting with 57,046 records, like I'm supposed to:

And this is info from the same tool's output view - suddenly 41,923 records:

I have no idea why 15,123 records are being dropped. From what I understand, how I have the tool configured should grab everything starting from the sixth row and ending at the very last row. I need to verify at which row the output starts, but the one header row I'm trying to keep is completely missing, so I assume the offset is from the beginning of the data.

There is only so much information I can share beyond this, just because a lot of the data is proprietary, but I would be happy to answer any questions I can. I tried looking for this same issue through Google, and I wasn't able to find anything.

danilang · Answer

Hi @Parkreiner

This appear to be an issue when using the Select Records tool and the Amp engine.  With Amp, the records are batched for multiprocessing, but it looks like the Select Records tool is dropping the first 5 record from each batch of records removing the first 11620 rows of data.  If you switch to the original engine the Select Records tool works correctly.

Still something that you should report to Alteryx.  At the very least they could update their Amp compatibility page. to remove the Select Records from the list of fully converted tools

As @Jonathan-Sherman suggested the Sample tool works correctly in both cases

Note that the attached sample is a workflow as opposed to a macro, but the AMP issue occurs in both.

Dan

Standard Macro - Stripped-Down.yxmd

Parkreiner · Answer

Just to be thorough, here is the macro file, as well as the mock data I made. The first five rows are meant to skipped, just like with the real catalog, and only columns 1-8 are relevant to the macro.

Standard Macro - Stripped-Down.yxmc

Alteryx Bug - Mock Data.xlsx

Parkreiner · Answer

All right, I've tried a few things. I should have mentioned this in the original post, but these issues are happening with Excel (.xlsx) files.

* I made a script to make some mock data that is more or less equivalent to the original catalog file. I auto-generated 60,000 rows, and most of the columns are formatted similarly to how they are in the real catalog. No issues with the Select Records tool.
* After I had the macro input tool switched to the mock data, I switched back to the original catalog. The issue still happens.
* I've tried switching the input to two other catalogs that I had processed a few weeks ago, before trying to make the macro. They have the same issue, but they didn't have any issues when run through the original workflows.
* I have tried importing the newest catalog into a Google Sheet, and then re-exporting it as an .xlsx file. Still have the same issue, so it doesn't seem like an encoding issue with the files themselves.

Again, I've had no issues with getting the files to work in my original workflows, and those were all using the Input Data tool. The issues only reared their heads after I started trying to use the Macro Input tool. The only thing I can think of that could be affecting things is that the files do each have an embedded image – a company logo. I don't know if that would cause things to break this much, though.

And to be clear, I'm having these issues when I have most of the tools removed, and this is the entire set of logic:

I really wish I could just share the files, but they don't even belong to my company.