Alteryx Union With Dynamically Switchable Control Containers

[bmillar777]

Hi everybody, 

I've got a question that's been bothering me for a while and I haven't been able to find a solution to. I've included a screenshot with a simplified example of the issue. 

It relates to a scenario where I have a set of PDF reads to do and have to dynamically come up with a solution to determine which ones. This works well using the Filter tool to turn containers on and off and if all processing is done inside each control container, all is fine. 

However, if I need to combine the results outside the control containers, the Union tool fails with the "No incoming metadata from one or more upstream tools". I've tried various solutions to resolve this issue with no luck. Limiting the number of PDF reads is critical for my use case as one of my routines is already taking over 2 minutes to complete, unacceptably long. 

Does anybody have any ideas on how I can resolve this? 

Thanks :)

[NeoInfiniTech]

@apathetichell, I understand that the PDF tools are Python based, however I think it shouldn't prevent this method from working as the connection numbers are pre-determined and only the Union tool's raw XML is modified.

As another solution, I could recommend using the Mass PDF Input tool, however it might not be possible if Predictive Tools package is not installed or R packages are not installable due to firewall restrictions by IT. Also, only reading text from a "digital" (not OCR scanned but Saved As from a Word document for example) PDF is possible with this macros, as opposed to the OCR abilites of the PDF to Text tool, which is a part of the Intelligence Suite add on. You might also notice minor differences in how the text is parsed (whitespace after paragraphs starting with 1., 2. etc.)

With all that being said, I believe it is worth mentioning that PDF reading is much faster with the macro I provided the link to (as opposed to PDF to Text tool), which I can confirm after having tested and compared both tools performance wise for a project I worked on (might be due to the time it takes to initalize the Python package but cannot tell for sure).

One caveat for this tool - while I didn't encounter it recently - is that you might come across an error (unlike the PDF to Text tool found in Intelligence Suite package which might be more reliable) reporting that it was not able to parse a particular PDF (even if the necessary R packages are installed and you are able to read in other PDF files) - this might happen for a number of different issues (the following being hypothetical) such as the PDF being too large to be parsed by the R tool used by the macro, encoding of the PDF not being compatible or PDF having been created by a program that is not able to generate a PDF properly to make it possible to be read by Alteryx (a similar issue can occur with XLSX files generated by certain program or ERP software, when you try to input them in Alteryx they would cause an error, you would have to open the XLSX files in Excel and Save As the problematic files to solve the issue (whether manually or by PowerShell/VBA), but the same method might not work or be possible for PDF files).

[apathetichell]

@NeoInfiniTechthe problem is that the union tool is reporting the error - but the error actually is occurring at the PDF tool - but not being flagged until the union tool. having said that -> I'm not sure if that's easier than the classic disable container/enable based upon value method which was used prior to control containers.

I'd recommend that - if possible - when reconfiguring to a batch macro with a container @bmillar777 should replicate the logic in the container so he doesn't need four different containers and can instead reuse the same macro four times with different variables/options being passed in.

[apathetichell]

Hey @bmillar777 - did you get this to work when you went back to standard containers and configured them initially in disabled mode?

[bmillar777]

No, sadly it made no difference.

[apathetichell]

yucky - sorry.

[bmillar777]

No worries :) I don't believe there is a simple solution to this problem, but if I face it again, I'll be interested to try out some of the more complicated ideas I have or were suggested here.