Field type change warning

I have a workflow and noticed a recent error in the output data.

The data used to come in as a Fixed Decimal but is now coming in as Double.

Is there a way to recognize this kind of a change in the input data and produce a warning or error in the workflow results?

The alternate would be a way to cancel the workflow from running when there's a change to the input data format.

As a reference, in the union node there's an option to error, warn, or ignore in the properties when fields differ. Something similar to this in the select node when field types differ or when fields are suddenly missing could be helpful.

Error Message

Common Use Cases

Transformation

Optimization

Accepted answers

JoBen

Hi @kklein, if your data is consistently coming in as certain types, you could do something like the below. Basically, you could set up a text input with the type that it should be in. The field info tool after the actual input would determine the type as it actually comes in. You could then join off the record position, and then create an error message when it is not the same. Is this along the lines of what you're looking for?

All comments

BenMoss

What I tend to do (usually for naming reasons rather than datatype reasons); is when I first start developing, use the field info tool and write an output to an xlsx file. The field info tool will produce detail regarding name, type and size, and you can also use this file to build comments regarding fields (a data dictionary of sorts).

I will then input this as a separate input on the canvas, and join this against a field info of the current input, you can join on both name and type. If anything falls out of either the left or right side, then there is a problem, and you can report the problem to the results log by using the 'message' tool.

You may also consider 'forcing' the datatypes of your fields which can be done in the select type, where the active datatype will be followed by ':Forced', this change will mean that field will always be forced to that datatype.

Ben

JoBen

kklein

Thank you joben, this seems to be a viable solution. It'll take me some work to write out all the fields in a text input file. Once it's done at least I'll be able to proactively react before the output data gets wonky.

kklein

@BenMossthank you. This is good development advice.

I do output the metadata for documentation already. Using field info for QA within my workflows will be a new addition I'll use moving forward.

Quick Links

This months top contributors

AkimasaKajitani 389

mceleavey 388

mbarone 337

michael_weaver 335

Hollingsworth 335