This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
We've been working to create some standard practices so that workflows and documentation are similar across several developers. Some key things:
1. A general layout system where we group tools into three Tool Container categories: Input (with a specific color), Transformations (with a different specific color), and Output (with a specific color). This was done for ETL work. In the future, we'll probably add some additional categories, such as analysis. Within these containers, the workflow can be grouped and documented further.
2. Have standardized annotation text template. Why? Because it reduces the amount of thought needed to annotate a took, removing one more barrier to getting everybody to document processes. They don't always fit, but most of the time they are great jumping off points. These are just starting to be implemented and tested, so may change.
Annotation guidelines by tool type. Do not use the tool name in the descriptive annotation. Use short phrases in present tense, rather than complete sentences. Where possible, descriptions should simply contain a verb and field name(s).
“Remove” + record description OR “Retain” + record description
Remove blank records
Retain records with data Retain records with PIN = Premises
“If <field, condition>…” (followed by “…Then/Else…” on subsequent tools)
If Collection State has no data…
"Match on" + field names used to join tables (or descriptive names for fields, if field names are not intuitive)
Match on Order ID
Brief description of what is achieved by summary
Brief description of what is achieved by formula
Convert Test Code to uppercase Trim whitespace
Set Test Code to blank
Input Data/Text, Connect In-DB
Filename (if applicable): Fields used from this data source (or descriptive names for fields, if field names are not intuitive), comma-separated