dynamically create columns via macro

hello

I am new to the atleryx macro writing, and hope you can help.

The scenario i have is that i have different files that have varying number of columns, these columns will always have the same columns output order.

what i would like to incorporate into a macro is dynamic column import

I can code the cleansing and ordering, what i need is how to code the connection from a file to the macro

so the fruits below are different columns, the macro is assumed as the same macro in all examples

_______________________________________________________________

example 1

APPLE PEARS BANANA ORANGES

these columns are parsed into the macro and outcome is always

APPLE PEARS GRAPES BANANA PLUMS ORANGES

_______________________________________________________________

example 2

PEARS BANANA

these columns are parsed into the macro and outcome is always

APPLE PEARS GRAPES BANANA PLUMS ORANGES

_______________________________________________________________

example 3

PEARS BANANA PLUMS ORANGES

these columns are parsed into the macro and outcome is always

APPLE PEARS GRAPES BANANA PLUMS ORANGES

i would like to know how to code the below macro input, to accept any number of columns.(not limited to the column in the tool)

hope that makes sense

thanks in advance

Denny

Parse

Macros

Connectors

Dynamic Processing

Accepted answers

Dazzerman

I tried to re-load my macro and discovered that there is a known bug that has cropped up within it.

I have taken the advised steps to resolve the problem, so uploading the corrected version here.

Column_sorted_file_input.yxmc

All comments

T_Willins

Hi @DennyChan,

It may be easier to prep the data prior to inputting it into the macro. I have used a text input as a master template to conform other inputs to include needed columns.

Master template 2.JPG

Master template 2.yxmd

Treyson

Are you saying that you have a certain number of formats that you know will have and that the macro needs to process differently for each format?

If so, I am thinking you might need to have two macros. The first would bucket each input file into whatever format category it belongs to. Then the second macro would be a batch macro that processes based on their category and then in the output of that macro you have some sort of master with all the column names that allows all of the outputs to be the same, similar to what @T_Willins described.

Dazzerman

Hi @DennyChan ,

I have attached my solution to your problem, along with the test data I used, which I stored in a C:\Test folder.

This solution incorporates ideas mentioned by @T_Willins and @Treyson, plus has the following benefits :

1. If you include "Filename" as one of the input columns in the Wrapper workflow, you get a column that indicates which file each row comes from

2. If you specify duplicate column names, only the first instance of the column will be reproduced

3. If you specify a column name that is not in any of the files, the column will be included but will only contain nulls

4. You can change what columns get extracted whenever you want without changing the macro.

I hope this is useful and helps you achieve what you are trying to do.

Let me know if there are any issues?

Column_sorted_file_input.yxmc

Column Sorted wrapper.yxmd

Fruits.zip

Dazzerman

I tried to re-load my macro and discovered that there is a known bug that has cropped up within it.

I have taken the advised steps to resolve the problem, so uploading the corrected version here.

Column_sorted_file_input.yxmc

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15867

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703