When working with APIs it is quite common to use the JSON parse tool to parse out the download data which has been returned from the API. However the JSON data may be missing key:value pairs as they are not in the response. This causes issues with downstream tools where there are missing fields. The current workaround for this is to use either the Crew macro Ensure fields, or union on a text input file to force the missing fields downstream.
The issue with this is:
1) Users may not be aware of the requirement to ensure fields are present
2) You need to know the names of all the fields to include in the ensure fields macro
Therefore the feature request is to add an option to the JSON parse tool to add the model schema as an input.
For example with the UK companies house API, to get a list of all the directors at a company the model schema is
{
"active_count": "integer",
"etag": "string",
"items": [
{
"address": {
"address_line_1": "string",
"address_line_2": "string",
"care_of": "string",
"country": "string",
"locality": "string",
"po_box": "string",
"postal_code": "string",
"premises": "string",
"region": "string"
},
"appointed_on": "date",
"country_of_residence": "string",
"date_of_birth": {
"day": "integer",
"month": "integer",
"year": "integer"
},
"former_names": [
{
"forenames": "string",
"surname": "string"
}
],
"identification": {
"identification_type": "string",
"legal_authority": "string",
"legal_form": "string",
"place_registered": "string",
"registration_number": "string"
},
"links": {
"officer": {
"appointments": "string"
},
"self": "string"
},
"name": "string",
"nationality": "string",
"occupation": "string",
"officer_role": "string",
"resigned_on": "date"
}
],
"items_per_page": "integer",
"kind": "string",
"links": {
"self": "string"
},
"resigned_count": "integer",
"start_index": "integer",
"total_results": "integer"
}
But fields such as "resigned_on" are not always present in the data if there are no directors who have resigned. Therefore to avoid a user missing the requirement for unidentified fields needing to be added, if there was an optional input which took the model schema and therefore created the missing fields would greatly improve the API development process and minimise future errors being encountered once a workflow is in production.
