Hi There,
I have a source file which I will get from a vendor with new data along with historical data, I can do truncate and load but the truncate and load is causing performance issue, I tried to append only new rows but since my target is Greenplum DB I will not have any keys so my output is failing to insert
here is the sample data
Yesterday’s file
Created Hour of Day | Created Date | Event Type | CustomersCounts |
5 | 2020-01-21 | account_has_mortgage | 200 |
4 | 2020-01-21 | user_device_count | 100 |
3 | 2020-01-21 | log_in_success_device_not_verified | 150 |
2 | 2020-01-21 | security_challenge_question | 250 |
1 | 2020-01-21 | account_summary_profile | 60 |
Today's file
Created Hour of Day | Created Date | Event Type | CustomersCounts |
5 | 2020-01-22 | account_has_mortgage | 505 |
4 | 2020-01-22 | user_device_count | 526 |
3 | 2020-01-22 | log_in_success_device_not_verified | 391 |
2 | 2020-01-22 | security_challenge_question | 214 |
1 | 2020-01-22 | account_summary_profile | 50 |
5 | 2020-01-21 | account_has_mortgage | 200 |
4 | 2020-01-21 | user_device_count | 100 |
3 | 2020-01-21 | log_in_success_device_not_verified | 150 |
2 | 2020-01-21 | security_challenge_question | 250 |
1 | 2020-01-21 | account_summary_profile | 60 |
Expected to load
Created Hour of Day | Created Date | Event Type | CustomersCounts |
5 | 2020-01-22 | account_has_mortgage | 505 |
4 | 2020-01-22 | user_device_count | 526 |
3 | 2020-01-22 | log_in_success_device_not_verified | 391 |
2 | 2020-01-22 | security_challenge_question | 214 |
1 | 2020-01-22 | account_summary_profile | 50 |
5 | 2020-01-21 | account_has_mortgage | 200 |
4 | 2020-01-21 | user_device_count | 100 |
3 | 2020-01-21 | log_in_success_device_not_verified | 150 |
2 | 2020-01-21 | security_challenge_question | 250 |
1 | 2020-01-21 | account_summary_profile | 60 |
Thanks
Avanti.
