This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
The idea is the ability to skip tools in a workflow. Check the attached example:
Input (TRX_TABLE): Contains transactional raw data (unstructured & uncleaned data) Filter (TRX_DATE <= 90 days from TODAY): It only filters the data 90 days old from today. Works as an incremental data loader If the filter tool is NOT SKIPPED the workflow loads the filtered data (incremental behavior) If the filter tool is SKIPPED the workflow loads all the data (full behavior) This way I can quickly switch between both modes for testing & production environments