This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
Any tool that performs a Sort will increase processing time. Many tools in Alteryx utilize a sorting process as a facet of their inherent functionality or if a group by option is used. The Summarize tool does this, for example. Alteryx has the ability to remember Sorts, or said another way, it knows if data has already been sorted upstream. This is a behavior that you can take advantage of as yet another means of optimizing your module’s processing. In the example shown, the process in Green is more efficient, as sorting will only be performed on the data once. For the process in Red, sorting will have to be performed twice.
Tools that Sort
Tools that use Sorts:
Tools use Temp Files and/or Internal Sorts:
Block Until Done
Multi Row Formula
Distance (Guzzler Only)
Trade Area (Guzzler Only or Non-Overlapping)
Run Command (Temp files only)
Use Temp Files and/or Internal Sorts at the Process level:
Batch Macros (for secondary inputs)
Iterative Macros (for secondary inputs)
Behavior Create Profile
Alteryx provides GUI tools that offer similar functionality to many SQL commands. Although minimal SQL scripting may be necessary in order to properly configure tools, the amount required to complete analysis is significantly reduced.
You've been given data for a new project and it contains lots of extra (and unnecessary) rows before you even get to the information you need to work with. Look familiar?
For many Alteryx users, this situation is all too common. Luckily, there's a pretty easy way to resolve this issue using the Sample and Dynamic Rename tools!
To demonstrate this approach, we'll use some sample data that has extraneous information and space at the top (Rows 1-4) of the spreadsheet in Figure 1 (below). While the information itself might be important, it's going to interfere with our data analysis. What we really want to see is the information in Row 5 as our header name and the information from Row 6 onwards to be our data.
Figure 1: The data in rows 1-4, as seen in Excel, should not be included in the data analysis.
Rather than manually re-format our dataset, we'll bring it into Alteryx and let the data preparation begin! Using an Input Tool, we'll navigate to the location of our data file. The tool gives us a preview of what to expect when bringing in the data (Figure 2). This format is nowhere near perfect, but we still have a few tricks up our sleeve!
Figure 2: The Input Tool shows how the data will be brought into Alteryx. Our heading is not correct, and we still have a few lines of data (in light yellow) to eliminate while keeping the data we want to analyze (in dark yellow).
A quick visual assessment indicates that we'll need to skip the first three rows of data (the information in Row 4 will become our field names). We can remove these data using a Sample Tool. In the Sample Tool configuration (Figure 3), we'll opt to "Skip the 1st N Records"; in this case, N will be equal to 3.
Figure 3: Set the number of records to skip, or remove, from the top of the dataset.
Now that we've removed the first 3 rows of data, we are much closer to the version of the data format we'd like to work with. The data we'd like to use as the field names (Number, FirstName and State) are now in the first row of data. We'll use the Dynamic Rename Tool to re-name our fields using the option to "Take Fields from the First Row of Data" (Figure 4). And, voila!! Our data is now ready to use for the next steps of our analyses.
Figure 4: After removing unwanted rows of data and re-naming the fields, our data is ready for further analyses.
*See the attached sample workflow (v10.5) for an example of this process.
How can you calculate the standard deviation of the population (SDpop) rather than standard deviation on the sample data (SDsample)?
Please find the attached macro, special thanks to one of our Community users!
Standard deviation of the population (SDpop) and standard deviation on the sample (SDsample) are very similar - the only difference is that in SDpop the sum of the squared variances is divided by n instead of n-1 as it is for SDsample (where n is the number of data points in the vector). This macro takes a vector of doubles which needs to be named 'dataVals' and it returns the standard deviation of the population. Accurate out to the 9th or 10th place.
If you’ve gotten accustomed to having headers and sub-headers (fields/sub-fields) in your tables, much like those you can build with the “Merge & Center” Excel feature, you’ll be happy to hear with just a couple formatting steps we can build the format below:
In Alteryx, with similar formatting:
The attached workflow goes into the details of the approach, but the essence is below:
Split your columns/fields your desired categories (making them sub-headers)
Create tables from your categories, add your supra-headers
Combine the reporting elements we have now, use the Union Tool to stack and the Summarize Tool to combine the reporting elements horizontally
Combine your sub-tables into a larger table, if needed (adding another header)
There you have it! Your tables are now rendered in style.
Errors from the R tool, and macros using R, such as the Association Analysis tool, can be a challenge to troubleshoot. Since R is very particular about naming conventions, and the R macros are very particular about delimiters, one easy way to preemptively limit these errors is to make sure your field names are R-friendly and compliant.
Best practices is to use only letters, numbers and the underscore. Any special characters will likely upset R. One hint is "Error: No valid fields were selected." This means that one of the Pipe-to-Rows tools embedded in the R macros could not appropriately parse the field names.
For example, the following errors came from the Association Analysis tool, where the field names contain the gamut from asterisks to ampersands:
Error: Association Analysis (4): The Action "select fields A" has an error: ParseError: Expected entity name for reference at line 7 and column 30 while processing "Action_UpdateRawXml"
Error: Association Analysis (4): The Action "select fields A" has an error: ParseError: Expected whitespace at line 13 and column 63 while processing "Action_UpdateRawXml"
Error: Association Analysis (3): The Action "select fields A" has an error: ParseError: Unterminated entity reference, 'test' at line 4 and column 30 while processing "Action_UpdateRawXml"
If you have a thousand fields with every special character, don't fret. Dynamic Rename and regex to the rescue!
Using the Rename Mode: Formula, select all your fields, and replace any non-word character with an underscore:
Crisis averted! Happy Alteryx-ing.
This article is part of the Tool Mastery Series, a compilation of Knowledge Base contributions to introduce diverse working examples for Designer Tools. Here we’ll delve into uses of the Record ID Tool on our way to mastering the Alteryx Designer:
Here at Alteryx, we do our best to keep things simple. Analytics is hard enough, there’s no need to overcomplicate things. That’s why we have the Record ID Tool – true to its name, if your records don’t have a unique identifier or row count just drag this tool onto the canvas. That’s it. Put your feet up, take a deep breath, and celebrate the win. The best part? The Record ID Tool doesn’t stop there – there’s countless applications of the tool that can simply other operations, too. It’s a gift that just keeps on giving:
Use a Record ID field to create primary keys in database tables created by a workflow
Split your output into multiple files using Record IDs to specify precise record counts
Process workflows in “packets” of records leveraging a Record ID - in some cases, this decreases run time
Compare datasets down to the last record by mapping them to a Record ID
Use the modulo (mod) function to make groups of your data from the Record ID field, simplifying otherwise complex reshapes (see examples 1 and 2)
You can also enforce a record order to your datasets using a Record ID (just sort by it), which often comes in handy before reshaping or macro processing. If you’re looking to assign “Group By” Record IDs that reset to unique values of a particular field, try using the Tile Tool.
That’s a lot of operations made simpler by a single tool; it could be a record. Now, if that’s not worth celebrating, we don’t know what is.
By now, you should have expert-level proficiency with the Record ID Tool! If you can think of a use case we left out, feel free to use the comments section below! Consider yourself a Tool Master already? Let us know at email@example.com if you’d like your creative tool uses to be featured in the Tool Mastery Series.
Stay tuned with our latest posts every Tool Tuesday by following Alteryx on Twitter! If you want to master all the Designer tools, consider subscribing for email notifications.
Upon creating a BINGO game, I came across a technique that I thought could be useful in "real world" scenarios for users who are attempting to iterate a process and then replenishing the data after a certain amount of time.
The Multi-Row Formula Tool functions much like the normal Formula Tool but adds the ability to reference multiple rows of data within one expression . Say, for example, someone was on the ground floor of a house and had a Formula Tool. They would only be able to talk to the people also on the ground floor. If they had a Multi-Row Formula Tool, though, they would also be able to talk to the people upstairs, in the attic, and in the basement as well.
How do I capture values that are not recognized as dates? I am importing data from an Excel file. Two of the fields are supposed to contain dates, but because the values are entered by hand sometimes they aren't entered in a way that is recognizable by Alteryx as a date. For example, "11/8/`6" is entered instead of "11/8/16". I want to divert the records that could not be read into a separate workflow.
Convert the date field first. If the date does not convert, a NULL is produced. Use a Filter Tool based on finding NULLs to create a separate stream:
See v10.5 workflow attached.
Users often ask, "How do I do (x) function in Alteryx?" - here's a handy guide for translating Tableau functions for use in Alteryx!
*Please note: This is not a comprehensive list of all functions available in Alteryx and Tableau - only functions that are written differently, but perform similar tasks, are included here. For a list of all the functions available in Alteryx, please refer to our Help Documentation.
Return smallest integer greater than or equal to [x]. Works like the 'RoundUp' function in Excel.
x % y
Modulo of n divided by d - The Modulo operation finds the remainder of division of one number by another number.
Return [x] raised to the [e] power.
Return [x] rounded to nearest multiple of [mult].
IF isnull([field]) THEN 0 else [field] ENDIF
Returns the expression if it is not null, otherwise returns zero. Use this function to use zero values instead of null values.
IF Contains([field], "string") then 1 ELSE 0 ENDIF
Returns true if the given string contains the specified substring.
FIND(string, substring, [start])
Searches for the occurrence of a particular string within a data field and returns the numeric position of its occurrence in the string. In Tableau, returns the index position of substring in string or 0 if the substring isn't found. If the optional argument start is added, the function ignores any instances of substring that appear before the index position [start].*
Return the length of the string [x].
Converts a string to lower case.
REGEX_Match(string, pattern, icase)
Searches a string for an occurrence of a regular expression.
REGEX_Replace(string, pattern, replace, icase)
REGEXP_REPLACE(string, pattern, replacement)
Allows replacement of text using regular expressions and returns the string resulting from the RegEx find pattern and replace string.
Substring(x, start, length)
MID(string, start, [length])
Return the substring of [x] starting at [start] and stopping after [length], if provided.*
Remove character in the string y from the end of the string x; y is optional and defaults to trimming white space. In Tableau, this function trims extra whitespace.
Remove character in the string y from the end of the string x; y is optional and defaults to trimming white space. In Tableau, this function trims extra whitespace.
Converts a string to upper case.
Date Time Functions
DateTimeAdd(datetime, interval, units)
DATEADD(date_part, interval, date)
Return the given date/time modified by the given duration. The <interval> specifies a positive or negative integer of time to add or subtract and <units> is one of a date/time unit - "years", "months", "days", "hours", "minutes", or "seconds". For Tableau, additional date_part units are allowed.
DateTimeDiff(datetime1, datetime2, units)
DATEDIFF(date_part, date1, date2, [start_of_week])
Subtract the second argument from the first and return it as an integer difference. The duration is returned as a number, not a string, in the specified units - "years", "months", "days", "hours", "minutes", or "seconds". For Tableau, additional date_part units are allowed.
DateTimeTrim(datetime, trim type)
DATETRUNC(date_part, date, [start_of_week])
Remove unwanted portions of a date/time and return the modified date/time. Options include: firstofmonth, lastofmonth, year, month, day, hour, minute. In Tableau, truncates the specified date to the accuracy specified by the date_part. This function returns a new date. For example, when you truncate a date that is in the middle of the month at the month level, this function returns the first day of the month. The start_of_week parameter is optional.
Returns the current system date and time.
DateTimeParse(datetime, format of incoming string)
MAKEDATE(year, month, day)
Converts a date string with a specific format to the standard ISO format yyyy-mm-dd HH:MM: SS. In Tableau, returns a date value constructed from the specified year, month, and date.
Returns today’s date. The time is set to midnight of the beginning of the day. Returns the current date.
ToNumber(x, bIgnoreErrors, keepNulls)
INT(expression) or FLOAT(expression)
Converts a string parameter to a number. The second parameter is optional and allows for ignoring conversion error messages. This parameter is a boolean flag and will accept a value of 1, 0, true or false. There is an optional 3rd parameter to handle Nulls. In Tableau, INT casts its argument as an integer. For expressions, this function truncates results to the closest integer toward zero. FLOAT casts its argument as a number with decimal/float precision.
Casts its argument as a string.
*In Alteryx, string positions start at 0. In Tableau, string positions start at 1.
Binary (bit level or bitwise) operations operate on one or more bit patterns or binary numerals at the level of their discrete bits. They are typically used to manipulate values for comparisons and calculations - and encryption!
These functions are in Formula Tool, in the Integer submenu of the Math menu:
Binary values are stored as strings, retaining the leading zeros. To use these functions, you'll have to convert these to numerals. Use the BinToInt() function to convert your binary strings to numeric values. Once you've completed your calculations, use IntToBin() to get the binary values. Note: you'll need to add your leading zeros back in, using the PadLeft function.
If you need the character associated with the binary value, use CharToInt(). Hex calculations work similarly.
BinaryAnd(n,m) – performs the logical AND operation on two binary numerals
BinaryNot(n) – performs logical negation, forming the complement (bits that are 0 become 1, and those that are 1 become 0)
BinaryXOr(n,m) - exclusive disjunction essentially means 'either one, but not both nor none'. In other words, the statement is true if and only if one is true and the other is false.
A common use for XOR is a means of doing a parity check. A bitstring has even parity if the number of 1s in the string is even. It has an odd parity if the number of 1s is odd. If you XOR the bits together, you can tell whether a bitstring has even or odd parity.
ShiftLeft(n,b) / ShiftRight(n,b) - shifting left is equivalent to multiplication by powers of 2. So 5 << 1 is equivalent to 5 * 2, and 5 << 3 is equivalent to 5 * 8. Shifting to the right is equivalent to division by powers of 2.
Please see the attached v10.5 workflow for a simple secret message conversion.
The Association Analysis Tool allows you to choose any numerical fields and assesses the level of correlation between those fields. You can either use the Pearson product-moment correlation, Spearmen rank-order correlation, or Hoeffding's D statistics to perform your analysis. You can also have the option of doing an in-depth analysis of your target variable in relation to the other numerical fields. After you’ve run through the tool, you will have two outputs:
The Multi-Field Formula Tool offers the same functionality as the Formula Tool, but offers the added benefit of applying a function across multiple fields of data all at once. Gone are the days of writing the same function for multiple fields.
Say there are four fields with dollar signs ($) that need to be removed. It could be done with a Formula Tool and a function written for each field:
You have a dataset containing information on customers’ survey responses (Y/N), the Customer Segment (Corporate, Small Business, etc) to which they belong, and other location data. You have been tasked with finding the percent of each Responder type in the entire data set. To perform these calculations, we’ll need two types of counts of data. The first is a “conditional count”, or a count of records that meet certain criteria. The second is a count of all records in a dataset. Alteryx has two nifty ways to help us obtain these values. We’ll use both the Count Records and Summarize tools to help us with these tasks!
Use the Summarize Tool’s “Count” function
The Summarize tool allows us to count the number of records that meet certain criteria. For our particular examples, we want to find the number of records for each Responder Type, Yes or No. We’ll use the Summarize tool to Group by “Responder”. Then, we’ll Count the number of Customer IDs for each Responder type (Figure 1).
Figure 1: The Summarize tool will Count all records, Count NonNull records, CountDistinct (Unique) records and CountDistinct (NonNull) records.
Want to drill down in your data even more? How about find the number of Responder Types per Customer Segment? Again, the Summarize tool can help! Group by “Customer Segment”, then by “Responder”, then Count the “Customer IDs”. See the attached workflow to see this example in action.
Use the Count Records Tool
To calculate the percent of each response type for our entire dataset, we’ll need to know the total number of responders in our dataset. While there are a few ways to go about getting that number, I’ll highlight the use of the Count Records Tool (well, macro, technically). “The WHAT?” you ask? I’ve heard that before.
The Count Records Tool. It’s easy to miss and I can count the number of times I’ve seen this tool in a user's workflow on one hand. However, it’s one of those tools for which you quickly find so many uses! It does exactly what its name suggests: it counts the number of records from an incoming data stream. The tool itself requires no configuration. Simply insert the tool into your workflow and receive a count of the number of records in the incoming dataset, even if there are zero records from incoming data* (Figure 2):
*The Count Records tool will return a record with value of 0, whereas the Summarize tool will simply not return any records.
Figure 2: The Count Records tool has no configuration and returns the number of records (rows) from an incoming data stream.
Now that we have the counts that we need for our calculations, we're ready to move forward with our data analysis! Please see the attached workflow for the completed demonstration of this process.
Often times in data preparation, the need for order in your records will arise. When that situation occurs, the Sort Tool has your back. It’s just that sort of tool. Effortlessly arranging your records – be it alphabetical, numeric, or chronological in order – while not quite a mind-numbingly complex operation, has ample utility. Sorting your records upstream of many tools can even optimize processing time . The fairly simple use cases below are techniques that frequently pop up in the data blending trenches:
Question I have a table of sales data with each column being a week's worth of sales. I only want records that have data in each of those fields and want to filter out all records that have Null values. How can I do this?
Answer There are two basic elements necessary to make this happen. The first is that all records in the original table have a unique ID. If you do not have a unique ID in your data, go ahead and add a Record ID Tool.
In the sample data you can see we will want data from Rows 1 and 6 while filtering out each of the other records because they contain null values.
From here we will use the Transpose Tool to pivot your data into 3 separate columns. In the transpose field choose your unique ID as the KEY FIELD and make sure all other records are selected as DATA FIELDS.
The result is that you will have your unique ID field, a field called [Name] which contains the names of each of the fields in your data, repeated for every unique ID in your original data, and a [Value] field which contains the individual values for each of the records for each of the columns in the original data.
Now we want to search for Nulls, and get a comprehensive list of the UniqueID values that do not contain Null values. Now is the time to bring in a Summarize tool and GroupBy your unique ID field, and then use the CountNull action.
The result is a list of how many nulls exist in each of your unique ID groups.
Next we can simply filter out the fields that have 0 null values in them and then use the unique IDs to join back to the original data, and pull only those records.
It's important to note here that because I'm only interested in the original fields I intentionally chose to deselect the unique ID and the Null Count fields from the output of the join so that I am left with only those records that have data in all of the weeks.
See the attached v10.5 workflow for an example of the approach above.
Sampling weights, also known as survey weights, are positive values associated with the observations (rows) in your dataset (sample), used to ensure that metrics derived from a data set are representative of the population (the set of observations).