Let’s talk Alteryx Copilot. Join the live AMA event to connect with the Alteryx team, ask questions, and hear how others are exploring what Copilot can do. Have Copilot questions? Ask here!
community Alteryx IO MyAlteryx
alteryx Community
Search
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Close
Start Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

How to improve the performance when comparing two large datasets?

YSHuang
6 - Meteoroid
‎11-19-2024 08:12 PM

We are using Alteryx to compare two very large datasets, with file sizes around 20-30GB and approximately 150-200 columns. Is it possible to shorten the execution time or reduce memory usage by changing the comparison method (for example, by converting the data using a hash function first)? Any suggestions or ideas would be greatly appreciated!

Labels:
Reply
0 Likes

Accepted SolutionSolved! Go to Solution.

1 REPLY 1
kamanivk
8 - Asteroid
‎11-19-2024 08:19 PM

As you mentioned, you can use MD5 hash for the columns needed to be compared.

 

Search for 3 MD5 functions here.

https://help.alteryx.com/current/en/designer/functions/string-functions.html#example-6846024-12

Reply
0 Likes
Copilot AMA
Live Office Hours
Labels
Top Solution Authors
View All