Data Stream Out tool slows performance in workflow

Question

I have a workflow that does 2 table joins in Teradata and then uses 2 data stream out tools to bring the data into the workflow. When I enable performance profiling, the one data stream out tool is at 90.72%. It is writing out 998,510 records with 1 column that is a string with 32 positions. The other data stream out tool is at 4.45%. It writes out 996,450 records with 6 columns, 2 of which are string with 32 positions, the other 4 are vstring of 254 positions. The workflow takes 5:11 minutes to run.

If I remove the data stream out tool that takes 90.72%, the workflow finishes in 43.2 seconds. Does anyone have any idea why the one tool causes the workflow to take 7 times longer to run even though that tool is writing less data that the other data stream out tool.

I've attached screen shots of the 2 different runs showing the performance profiling stats.

Thanks,

Kevin

Run2.png

Run1.png

apathetichell · Answer

open your resource manager and watch your ram usage. There are numerous reasons this could be happeing 1) system ram limitations. 2) you are querying a view - not a table - and the underlying view takes a while to put together. 3) in some dbs there's cluster issues/db builds.

KevinSobel · Answer

Thanks for your reply, but I am only streaming out 1 column, 32 characters.

StellaBon · Answer

Just a thought- Could it be the data types on the one that is taking longer are a more expensive data type? I had this issue once and it was a column called Notes and the database I was streaming out to was converting it to LOB type. It contained a few overly long notes. I ended up removing that Notes column from that output as it was not strictly necessary and that fixed it.