Data Stream Out Tool is very slow

Hello,

I am trying to output a query I did using the in-DB tools. It only results to 2,667 rows, but the run is taking more than an hour to finish.

I created another workflow that has more rows but runs faster than it (can be done in 1 min up to 5 mins), and I am at a loss as to how to troubleshoot it. Both use an ODBC connection string to connect to a SQL Server Database, but the performance of the two are very different.

May I get some guidance on what to check to determine what are the possible differences between the two?

row count.png

workflow run.png

Error Message

Connectors

Optimization

In Database

Database Connection

In-Database

Accepted answers

All comments

mceleavey

Hi @beadysquared ,

The number of rows is only tangentially related as it boils down to the size of the data being streamed. At the point of streaming out you are now pulling data across the network, and so the larger the data the slower the run.

Does the data with fewer rows have considerably more columns or columns which contain large data sets?

ChrisTX

Are you referring to the timing difference of the entire workflows, or only the Data Stream Out tool?

I'm guessing the SQL is different?

Try running both SQL statements using a tool like Microsoft Access and see if the run time is significantly different.

Chris

beadysquared

Hello @mceleavey,

The two workflows are sourced from the same underlying tables, but the slower workflow has a part where one table is joined to another table twice on different join columns & does not use a summarization tool - do you think these affect its performance? With respect to number of columns, I think the two workflows are comparably similar

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703