AWS s3 Upload tool failing w/Error from AWS: The specified method is not allowed ..

Question

I'm trying to use the s3 Upload tool for the first time.

If I upload 20,000 records from a sample file it works without a problem.

If I try to upload the entire file I get ...

Error: Amazon S3 Upload (1): Error from AWS: The specified method is not allowed against this resource.
Info: Amazon S3 Upload (1): 32660 records were written to https://xxxxxxxx-data-management.s3.us-west-2.amazonaws.com/SourceInput/MOOSE_BYTES.csv

We are new and migrating to AWS and learning on the fly.  So I suspect this could be an AWS permissions setting that we need to tweak, but find it interesting that it seems seems to start failing at the point of 32,660 records being uploaded.

Looking for suggestions for resolution / troubleshooting.

Regards,

Jim Holloway
Experian Marketing Services

Jim_Holloway · Answer

This was a great suggestion, embarrassed I didn't think of it myself.

But unfortunately it didn't pan out.  I pointed the input tool at a different file - one that had already been successfully uploaded to s3 via DataSync.  This one failed as well (after 63,325 records).

I don't think this is a data issue.  I suspect some sort of size limitation on the s3 upload tool.  But perhaps that's a function of how the s3 bucket itself is configured (I'm new to AWS).

Anyone else have any insights?

Jim

OTrieger · Answer

@Jim_Holloway 
This is your clue "32660 records were written to https://xxxxxxxx-data-management.s3.us-west-2.amazonaws.com/SourceInput/MOOSE_BYTES.csv"
You need to figure out what changes after these 32660, as something is changing in your data. So this 405 error, suggesting the issue arose during a specific step or finalization of the upload.