Unable to get all records

[tanmay1]

Hello,

This is my first post and I hope to get some help on an issue. I have a workflow that pulls data using a public API. I should get around 50K rows, however, it just returns 3K rows. 

I am not able to figure what's blocking the data. Here is the reference link for API.

https://data.gov.au/data/api/1/util/snippet/api_info.html?resource_id=9312452f-fced-476e-a6ec-2b2327... 

It would be great if someone could help.

TIA

[Emil_Kos]

Hi @tanmay1,

I am not an expert in this area but maybe there is a limitation that you can get 3k of rows at once? 

[Qiu]

@tanmay1 

With a Browse Tool you can see it all.

Other tools will only show part of the data if it is too big.

[tanmay1]

Thanks for the reply. I think I am not clear. After pivoting I am just getting 3k rows.

I was expecting close to 50k rows. I am not sure if it is issue with the API or I missed anything the workflow.

[pedrodrfaria]

Hi @tanmay1 

The reason why you are only seeing 3K is because the column you are doing the cross tab and grouping by, only has 3k unique values, thus only 3k rows.

What is the logic you had to apply in order to get to 50k rows? I'm assuming when you created the pivot in excel for example, you probably grouped by a couple of columns or had an additional logic to it.

Pedro.