Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

Loading multiple CSV and TXT files

Lifesigns
6 - Meteoroid

Hi All,

 

I have a requirement to load multiple csv and txt files daily. 

They require some business mapping and each have a different structure. 

My question is more regarding where I would store this data whilst it’s processing. There is no database available and at present it’s in Qlikview and all done in qvd’s. 

The aim is migrate this to Tableau and it’s a large volume of records daily circa 10k. 

With the annual records in excess of 1m.

 

Appreciate any insights. 

Thanks. 

1 REPLY 1
danilang
19 - Altair
19 - Altair

Hi @Lifesigns 

 

You should probably look into to one of the popular open source database systems like MySQL, PostgreSQL.  If you're comfortable with JSON, look into MongoDB as well.   Having a backend database will make it easier for transactional daily inserts  and SQL-like selects as opposed to using file based formats like .yxdb or csv files.

 

Dan

 

   

Labels