I am escaping continental winter at the moment which means my internet connection is not perfect. I was uploading a 1GB file to Amazon S3 via the console.aws.amazon.com and the upload failed at 93%. Just my luck!
I didn’t want this to happen again. So I tried using Cyberduck to upload and intentionally turned off WiFi after a few seconds to see if I can resume the upload. Nope. Does not work. OK, what about the official AWS command-line tool? Also nope. Oh, come on!
Finally, I found this StackOverflow entry. Apparently `s3cmd` does support resuming uploads. Let’s give it a try:
~$ s3cmd put --multipart-chunk-size-mb=5 IRL_talk_recording.mp4 s3://videos.niteo.co/IRL4/ upload: 'IRL_talk_recording.mp4' -> 's3://videos.niteo.co/IRL4/IRL_talk_recording.mp4' [part 1 of 211, 5MB] [1 of 1] 5242880 of 5242880 100% in 45s 111.64 kB/s done upload: 'IRL_talk_recording.mp4' -> 's3://videos.niteo.co/IRL4/IRL_talk_recording.mp4' [part 2 of 211, 5MB] [1 of 1] 5242880 of 5242880 100% in 37s 136.04 kB/s done upload: 'IRL_talk_recording.mp4' -> 's3://videos.niteo.co/IRL4/IRL_talk_recording.mp4' [part 3 of 211, 5MB] [1 of 1] 5242880 of 5242880 100% in 25s 204.51 kB/s done upload: 'IRL_talk_recording.mp4' -> 's3://videos.niteo.co/IRL4/IRL_talk_recording.mp4' [part 4 of 211, 5MB] [1 of 1] 524288 of 5242880 10% in 3s 155.49 kB/s^CERROR: Upload of 'IRL_talk_recording.mp4' part 4 failed. Use bin/s3cmd abortmp s3://videos.niteo.co/IRL4/IRL_talk_recording.mp4 zOidfC...5RHKek_DS to abort the upload, or bin/s3cmd --upload-id zOidfC...5RHKek_DS put ... to continue the upload. See ya! ~$ s3cmd --upload-id zOidfC...5RHKek_DS put --multipart-chunk-size-mb=5 IRL_talk_recording.mp4 s3://videos.niteo.co/multi/ WARNING: MultiPart: size and md5sum match for s3://videos.niteo.co/IRL4/IRL_talk_recording.mp4 part 1, skipping. WARNING: MultiPart: size and md5sum match for s3://videos.niteo.co/IRL4/IRL_talk_recording.mp4 part 2, skipping. WARNING: MultiPart: size and md5sum match for s3://videos.niteo.co/IRL4/IRL_talk_recording.mp4 part 3, skipping. upload: 'IRL_talk_recording.mp4' -> 's3://videos.niteo.co/IRL4/IRL_talk_recording.mp4' [part 4 of 211, 5MB] [1 of 1] 5242880 of 5242880 100% in 230s 22.17 kB/s done upload: 'IRL_talk_recording.mp4' -> 's3://videos.niteo.co/IRL4/IRL_talk_recording.mp4' [part 5 of 211, 5MB] [1 of 1] 2097152 of 5242880 40% in 60s 33.59 kB/s
Success! Glad I finally found this. Today is not the first time I got failed S3 uploads on flaky internet connections. Now I know how to resume them!
