Loop through all folders within a Share Point master folder

[sau953]

Hi,

I have a Sharepoint master folder that contains various sub folders with each folder containing .csv files that are either named 'Forecast_A' or 'Actual_A'.

I am trying to loop through all these folders and append data from all the files that are named 'Forecast_A' one below the other and then move the processed file to an archive folder. The data in each of the forecast files is in exactly the same structure. 

This is a very straightforward task using Directory tool and Dynamic input tool if the same folder structure is saved on a local drive. However, the Sharepoint Input tool does not have same sort of functionality. Further, I am not sure how moving files from one folder to another would work on Sharepoint.

 I tried using a batch macro, however, I am not able to define the correct variables to get the correct outputs. 

Any guidance on how to proceed? Thanks!

[BS_THE_ANALYST]

@sau953 Really interesting problem. You can definitely leverage ChatbotGPT here. I'd advise using ChatbotGPT to write some Python Scripts. You could use the Python Scripts inside then Python Tool in Alteryx. For example, ask ChatbotGPT for a script to move files from one location to another. Get that working locally, just to ensure you can move files from A->B locally. If you're comfortable doing this step already, then I'd advise the next step:

It would appear it's possible:

So the answer to your question is definitely yes. 

1) So for moving the files, leverage chatbotGPT Python scripts, and then put them in the Python Tool. 
2) For reading in all the CSVs, and then appending them all on top of one another, again, I did find a pretty nifty thing:

Basically, leveraging the wildcard character, once you have it inside the correct folder, you can bring in only Forecast files by using Forecast*.csv in the configuration window. Notice on the right hand side of the picture, I've got both my files. They have been successfully appended onto one another. You can see that in the results window.

Understandably, we'd like to drill into all the sub-directories etc. It doesn't appear that's an option in the Sharepoint Tool (at least I can't see it). So this would mean you'd need to go into all those folders to do this part. Again, I should imagine there can be a Python Script written via ChatbotGPT here to be able to do this part. I'd love to get stuck into this problem but time doesn't permit. Hopefully that gives you some insights/direction for planning on how you can tackle this. 

All the best,
BS