Why are my 10.0 worflows slower than when run on 9.5?

[anthony]

Hi, I downloaded 10.0 this weekend and after running a few workflows 'felt' like it was slower.

I confirmed this today by comparing a workflow in 9.5 and 10.0  - 9.5 runs in 52.2 seconds and 10 takes 2:23 Minutes.

I am basically taking a yxdb and summarizing on several fields and then writing out to 2 TDEs.

I have 2 summarize tools and 1 big join but it runs much faster on 9.5.

Anyone else notice speed difference?

[s_pichaipillai]

Hi Anthony

are both versions running on same machine?

can you check the system settings for both 

[anthony]

Yes this is on the same machine one after another with same memory usage/availabllity/etc.

I closed and reran this monring and got similar results.

[TaraM]

Which workflow are you running first? 9.5 or 10.0 - the data could be cached and the second runtime would be much faster. Which process takes the longest? Is it the Join or the write? If you could share your workflow we could take a closer look. 

[anthony]

I have run in various sequences and closed/opened v10 today w/o running in v9.5.  Same issue.  I have turned off write to TDE and it is much faster so seems like the write to TDE is the issue

I cannot share workflow because data has PII - i have seen the yxmd to customer support to see if they have any ideas.

Here is a screen grab - pretty basic workfow.  I take data from a yxdb wit about 500k rows and summarize into 2 tdes

[RodL]

Yes, I have found where performance on writing to the TDE can vary.

You won't be able to use this to compare the 9.5 to 10.0 processing, but one other thing to note is the new setting in 10.0 under 'Runtime' where you can check 'Enable Performance Profiling'. If you have this checked, it will provided a tool-by-tool analysis of how much time each tool took and the percentage of the overall process.

That will at least give you some concrete feedback in this case as to where the majority of the workflow process is spent.

[LauraS]

Hi Anthony,

I love Rod's suggestion.  Will you please let us know the results of the performance profile?  We would like to understand if one tool or another is slowing down the workflow.   We are not currently aware of any major issues in performance, but will definitely dig in once we get more information.

Thanks,

Laura

[anthony]

I did this yesterday - i did not save results but TDE write took 40% of the time.

It seems like the write to tde is the issue base on watching the workflows run in each version.

[LauraS]

Thanks Anthony!  We will do some digging.  Stay tuned!

[JCR]

Anthony,

We haven't changed the .tde generation in v10. Thanks for the feedback, we're going to benchmark the two versions and see if there is a side effect from some other changes.

Can you give me an idea of how big your esulting .tde is and what data types are involved in the dataset? that would help us make sure we don't miss anything in the test scenario.

Thanks

JC