Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Generate Synthetic Data

Shubhankars
5 - Atom

Hi Team,

 

I have a requirement where in we need to generate massive amounts of synthetic data for a particular data model. We have the range values, expected field calculations etc with us. 

I expect to generate over a 1B records. Is this achivable with Alteryx ?

 

5 REPLIES 5
JohnJPS
15 - Aurora

Sounds like a lot of data, but in general, yes, Alteryx is great for generating random data sets.

 

Use a "Generate Rows" tool to generate as many rows as you need; then a Formula tool to generate random data for whatever you need.  Use the Math > RandInt(n) or Math > Rand() Functions in your Formula Expression.

 

If you have ranges, something like [RangeBeginning] + RandInt([RangeEnd] - [RangeBeginning]) will give you an integer in your range.

Shubhankars
5 - Atom

Thanks. 
Is there a Rand() funtion that allows me to set a range of values ?

jdunkerley79
ACE Emeritus
ACE Emeritus

I have attached a sample generator I did for another question on here.

 

The easiest way to do a range is

Rand() * ([Upper]-[Lower]) + [Lower]
JohnJPS
15 - Aurora

[Edit: removing duplicate solution - same as @jdunkerley79's]

KaneG
Alteryx Alumni (Retired)

@JohnJPS & @jdunkerley79 have given you the solution, but there is one thing that I would like to add to this due to the amount of data that you're dealing with.

 

Play around with the order of tools to increase speed. Under "Workflow Properties > Runtime" you can turn Performance Profiling on to look at the time each tool is taking. Generally, generate your categorical variables first as that will be less data. But you may find that by changing the position of a join you could significantly reduce the time your workflow takes to run. I.e. don't try to join a billion rows to a billion rows based upon a field, instead randomise/sort/rganise and then join by record position.

 

Kane

Labels