6/17/21: We have completed maintenance for the Search functionality on the Community. If you are seeing any issues, please try to clear your cache first. If the issue persists please email Community@alteryx.com

General Discussions

Discuss a wide range of topics! Questions about the Alteryx Platform should be directed to the appropriate Product discussion forum.
SOLVED

Available "Big data sets" over the internet...

Atabarezz
13 - Pulsar

I'm opening this topic for everyone to list some Big data* sets available over the net.

 

  • Feel free to list competion/datathon data sets
  • Results of web scraping
  • Social media data
  • Anything bigger than 1 mio records (beyond excel and access)

 

Best

 

Altan

 

* Big data is data that is usually with sizes beyond the ability of commonly used software tools to manage and process within a tolerable elapsed time. A year long credit card transaction history or CDR (Call data record) of a telecoms company for the last 9 months, behavioral credit data of a large financial institution are some examples... 

 

--------------

[Edited by a Moderator]

We've compiled the responses to this thread into the following Knowledge Base article: Available "Big Data Sets" on the Web

--------------

61 REPLIES 61
ChrisTX
13 - Pulsar

On January 23, 2020 Google released Dataset Search, a free tool for searching 25 million publicly available datasets.

 

Google blog post: https://www.google.com/amp/s/blog.google/products/search/discovering-millions-datasets-web/amp/

 

Related post: https://towardsdatascience.com/google-just-published-25-million-free-datasets-d83940e24284

  • about half of the datasets in the search results were from for-profit aggregators
  • Other popular dataset publishers on the platform include government agencies and research institutions
  • US government agencies alone have published over 2 million datasets
  • most of the datasets are related to “geosciences, biology, and agriculture.”

 

Dataset Search: http://g.co/datasetsearch

mnguyen0803
5 - Atom

Thank you for sharing these. These look great to do some analysis!