Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!
The Product Idea boards have gotten an update to better integrate them within our Product team's idea cycle! However this update does have a few unique behaviors, if you have any questions about them check out our FAQ.

Alteryx Designer Desktop Ideas

Share your Designer Desktop product ideas - we're listening!
Submitting an Idea?

Be sure to review our Idea Submission Guidelines for more information!

Submission Guidelines

Directory Creation Tooling

Every time we create a file output - you first have to check if the folder exists - and if not then create it.

 

Currently it's quite onerous to do a directory create - especially with all the error trapping to make this production safe - and everyone is reinventing the wheel in their own companies.

 

Given the commonality of this need - could we add a tool that allows you to check for existance of a directory and attempt to create it (with nested directories and useful status / error descriptions to act upon)

11 Comments
Claje
14 - Magnetar

Would also be in favor of supporting this natively within Output Data - don't really see a reason to have two tools when a path could be dynamically generated at runtime (and may or may not require a directory check/create)

SeanAdams
17 - Castor
17 - Castor

:-) Like the way you think @Claje !   
So then there are two asks:

a) add this to the output tool as base functionality
b) add directory management tools anyway because this is a core part of data and data prep and ETL

Claje
14 - Magnetar

I'll agree to that! :)

Hollingsworth
12 - Quasar
12 - Quasar

@SeanAdams, this is an idea worth spreading!

I'm here mostly in hopes that comments lend weight to an idea.

 

Secondly, I'd be in your debt if you would share the directory creation code that you developed for this. I've built a crude DOS command runner (save commands to a batch file and then run the batch file). However, currently I'm just ignoring the errors it gives if the directory already exists. I'd love something a little more elegant.

And I also second the motion for better file management tools in Alteryx. In the Swiss Army knife of Alteryx this is up there with the corkscrew...not everyone needs it, but the substitutes are really ugly to use.

SeanAdams
17 - Castor
17 - Castor

Hey @Hollingsworth 

 

Here's the code for the directory creator in Python.  

If you look in the first section after the imports, you'll see the error checking to make sure that there are two columns which are needed for this - "RootFolder" and "FinalFolder" - final folder is the actual folder being created, and root folder is the parent.

 

Much of this is straight-forward - only thing which took a while to figure out is that if you use mkdir then it only does one level of depth at a time, but if you use os.makedirs then it goes an arbitrary depth (e.g. creates subfolders under subfolders if you need).

 

Hopefully this helps!

------------------------------------------------------

 

 

 

#Bring in the packages you need
from ayx import Package
from ayx import Alteryx
import os
import pandas as pd

#check if inputs are in the right place
dfInput = Alteryx.read('#1')
# Check if the dataframe has the required fields
lstInputFields = list(dfInput.columns.values)
if not('RootFolder' in lstInputFields):
    raise ValueError('RootFolder column does not exist on input columns')
if not('FinalFolder' in lstInputFields):
    raise ValueError('FinalFolder column does not exist on input columns')   
# Create a flag for the new folder
dfInput['FolderCreated'] = False

#Then go through a loop to create the folder
for index,row in dfInput.iterrows():
    #get values from data frame
    folderRoot = row['RootFolder']
    finalFolder = row['FinalFolder']
    # validate if the root folder exists
    if not(os.path.isdir(folderRoot)):
        raise NotADirectoryError("Root folder " + folderRoot + " does not exist")
    # Create the sub-folder
    if os.path.isdir(finalFolder):
        #folder already exists
        dfInput.loc[index, 'FolderCreated'] = True
    else:
        try:
            os.makedirs(finalFolder)
        except:
            raise NotADirectoryError("Subfolder " + finalFolder + " cannot be created")
            dfInput.loc[index, 'FolderCreated'] = False
        else:
            dfInput.loc[index, 'FolderCreated'] = True

#Output the final values
Alteryx.write(dfInput,1)
JTCairns
8 - Asteroid

@Hollingsworth 

 

You could always add an "IF EXIST/ IF NOT EXIST" clause to your DOS command. Avoids the errors. 

SeanAdams
17 - Castor
17 - Castor

Agree with your thinking @JTCairns  - the challenge with the DOS commands though is that we are generally fairly hesitant to allow folks to punch out to the command line within Alteryx when they publish to the server;  for fear that they do something silly to our servers - the command line tool gives unfettered access to a lot of your server, and so we don't generally allow cmd tools on our workflows in the gallery.

 

I do agree with you in principle though, and on the desktop this is a good solution.

 

 

machazthegamer
7 - Meteor

@SeanAdams @Claje 

The render tool already creates a folder if it does not exist (if you tick the group data into separate reports box and give alteryx a completely new path)though this might be a side effect.See sample workflow here https://www.dropbox.com/s/zak8i460v455hhj/create%20path.yxmd?dl=0

 

So Alteryx team please do not remove this ability when you add the feature requested here: 

SeanAdams
17 - Castor
17 - Castor

great catch @machazthegamer  - I didn't know that - so that's a curious workaround!

Thanks for posting this!

 

Hollingsworth
12 - Quasar
12 - Quasar
@JTCairns, I finally incorporated the IF EXIST code and it works much better now. Thanks for the tip!