This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
We are using 100 GB TPCH data as source data and the data is loaded into redshift.The workflow is performing transformation In Database(ELT). While executing the workflow its failing with the following error message,
" Error from AWS: One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's entity tag "
The above error is generated only when the complete data is being loaded but when we have limited the field limit of each table to 100000 it is working fine.
So my question is if there are any settings I can change anywhere to overcome the above mentioned error.
From the looks of the error message being returned by AWS, it appears that the database transaction size might be taking longer to transfer over the connection than the default connection time out. If possible, can use use option 9 in the attached (configuration of the output data tool) to try smaller transactions and option 10 to "Show Transaction Messages" to see how the database reacts?
If that doesn't help, I think the best work around would be to send the output in "packets" with either multiple output data tools or a macro that batches through the size of safe transfer over the connection.