Join Input to Itself

Question

Using a single input, is it possible to join the data stream to itself?  The data stream includes archived data.  I would like to join records from previous periods within the same table to their current records.

Or is it more efficient to create separate inputs streams of the same source table and then join the separate streams into one?  The data set is large.

MarqueeCrew · Accepted Answer

@sbaker,

Yes.  It is possible to join the data stream to itself.  Connect the input (output anchor) to both the L & R input anchors on a JOIN and carefully set the JOIN KEYS.  The single read is more efficient than having 2 reads of the same data.

Now for clarification.  When you "join" data are you actually "UNIONing" the data?  A union will add rows of data together into a blended file (vertically).  A join will create a horizontal combination of all fields.

If you have 10,000,000 records in your file and want to add 100,000,000 records from a historical file (all fields are the same), a union will create 110,000,000 records.

For more help (from me), I'd have to see some sample data explaining what you're trying to achieve.

Thanks,

Mark