Memory Limit reached error. Not able to update the output.

[Kartik21]

I am getting this "Memory limit reached error" due to which my output is not getting updated. I tried to change the name of the file and run it. It doesn't work. I increased the overriding memory limits, still it didn't work. Please suggest a suitable solution for this.

[SIDDINATHAM_HARISH]

Implement Block Until Done (BUD) Tool:

• Action: Add the Block Until Done tool (from the Developer category) immediately before any point where a single stream is forked into multiple paths.

• Explanation: This tool ensures that all records from the input stream are processed through the first output anchor (1) before any records are released to the second anchor (2), and so on. This prevents the engine from trying to process multiple subsequent branches concurrently, which can temporarily double or triple the memory required to hold data for processing.

• Where to Use: Use it before large Join, Append Fields, Union, or Formula tools that feed multiple downstream tools.

or

Minimize Data Duplication (Join/Union):

• Action: Review your workflow for unnecessary large Union or Join operations.

• Explanation: Tools like Union and Join can significantly increase the row count or column count, which directly increases memory usage. If you are joining two large streams, the resulting stream is often much larger than the inputs, demanding more memory. Look for ways to Filter or Summarize data before these expensive operations.

[SIDDINATHAM_HARISH]

1. Use the Select Tool to Optimize Fields:

   • Place a Select Tool immediately after your Input Data tool (or before the Select (211) tool shown in your image).

   • Remove Unnecessary Columns: Uncheck any field that is not needed for your final output or subsequent calculations. * Optimize Data Types: Review the data types. Change large types to smaller ones where appropriate (e.g., change V_String to a fixed String if the maximum length is known and small, or change a Double to Int if the field only contains whole numbers).

2. Filter Rows Early:

   • Place a Filter Tool immediately after the Select tool.

   • Filter Out Unneeded Records: Apply criteria to remove any rows that are not relevant to your analysis as early as possible (e.g., filter out records from prior years if you only need the current year's data).

[alexnajm]

@Kartik21 have you brought down a Browse tool as the screenshot suggests?