dynamically change parameters of KNN Tool

[neilgallen]

I'm pretty sure this is solved with an interface tool, but unsure as to the direct solution.

The situation:

I have a set of data that contains multiple categories, of which I want to perform a KNN search within each category. My intent was to build an iterative macro but there are two dynamic pieces of the configuration that will change with each category:

1) the data will be one-hot encoded, but the column names with each category will change. There may be some overlap between categories but not always. For each iteration I want to use every column in the KNN search.

2) the number of nearest neighbors for each category may be different, as we are searching against different templates. One category may have 1 template, others may have up to 6.

Given the large number of categories I'd rather not do each of these manually if possible.

Example workflow is attached.

You can see where each category has a different number of templates and the configurations between them contain different values. The values in the configuration column will be one-hot encoded before being fed into the KNN tool.

What I need to do is pass these values for number of templates to the KNN tool and have it update the number of neighbors to search, and then to have the check box with the KNN tool to refresh with the new column values and have all values checked.

Any ideas?

[mceleavey]

Hi @neilgallen ,

I may need to walk through this step by step to begin with so I understand what you're trying to do. I was a little confused by the One-Hot Encoding requirement while retaining a filter on the field that would be encoded, but I assume that was because you were testing for one category.

I've applied the One-Hot Encoding tool I built, added the binarised grid fields to the Cross-tab and included them in the Find Nearest Neighbours tool.

Can you let me know if I'm on the right lines here? I think what you are next requiring is to batch/Iterate this each categorical value...is that right?

I've attached the package with the amendments, if you could let me know if this is where you were going with this...

M.

[danilang]

@neilgallen 

Batch macro with the control parameter changing the "the_k" value with a category specific value

Dan

[neilgallen]

@danilang agree but the challenge is the control parameter needs to be dynamically fed by a different part of the workflow. Control parameter does not allow for that to my knowledge.

[danilang]

@neilgallen 

Add a second control parameter into the macro.  The first one passes the category Id and the second one passed the K-value.  On the main side, create a data set with two corresponding columns and assign one column to each parameter in the macro config.

Dan

[neilgallen]

Thanks for the response. Problem is, I don't know what the value of the control parameter will be with each run. The data itself will dictate both what columns are present and how many neighbors to search for.

With each run (either batch or iterative) I will want the KNN to effectively 'reset' and choose all available columns to match against, and receive the updated value for number of neighbors to search for within the data.

Sorry if this is not clear.

[neilgallen]

Mocked up example of what I'm looking for.

the length of the record ID is how I'm identifying which path each record takes. Anything with a length of less than 5 is our 'template' that we want to match the other records against. The number of available templates will change with each new set of data.

As each set of data data feeds into the macro, I want to change the number of neighbors to match for. I believe the best route is by modifying the inner XML, but I cannot seem to get this to work. Any suggestions are welcome.

There is a second requirement in which the List box that populates in the KNN tool will also need to be updated based on the dataset, but one thing at a time here.

[apathetichell]

"agree but the challenge is the control parameter needs to be dynamically fed by a different part of the workflow. Control parameter does not allow for that to my knowledge"

You mean that you are trying to hook up an action tool into a batch macro directly? I just hook it up into a text input tool and then into a batch macro... So a batch macro into a text input into a batch macro in your case - no?