This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
I am trying to pull in some weather data from the government's Bureau of Meteorology. The way their data is stored is only accessible by each weather station number. Below is an example of one of them. If you click the 'All years of data' link near the top right of the page, you download a zip file with 2 excel files (one containing 'Data1' and the other containing 'Data 12'
Yes, this should be possible. To add the leading zeros to your numbers, go ahead and change the field to a string with a select tool and then use a formula tool with the formula padleft([the numbers field], 5, “0”). This will “pad left” the numbers with leading zeros until they are 5 characters long.
Then you will need to build the URLs of each of these pages with another formula tool.
After that you will need to create a workflow that takes a single one of these URLs and uses a download tool to pull down and parse out the zip file link. You will then turn this workflow into a macro which feeds in each of the URLs that you built previously.
Then you will need to establish a place where to save these files so I would use the number made in the first step in this formula as part of your save location.
Finally, connect it all together! I can help you build this if any of the points aren’t clear, but that is the general method of approaching this task.
Workflow attached! I have also annotated each section using containers so that you can see what needs to be done. You will need to change the location to save the files to in the last container as well as feed in your own list of stations in the very beginning.