阅读背景:

aws s3 - s3cmd:“警告:MD5签名不匹配:” - 怎么办?

来源:互联网 

When I use s3cmd to pull down files (of not unreasonable size - less than 100 megabytes) I occasionally see this error: WARNING: MD5 signatures do not match: computed=BLAH, received="NOT-BLAH"

当我使用s3cmd下拉文件(不合理的大小 - 小于100兆字节)时,偶尔会看到此错误:警告:MD5签名不匹配:computed = BLAH,received =“NOT-BLAH”

Googling suggests that this may be caused by the way S3 segments files. Others have said to ignore it.

谷歌搜索表明这可能是由S3分段文件的方式引起的。其他人则说要忽略它。

Does anybody know why this happens and what the right thing to do is?

有谁知道为什么会这样,而正确的做法是什么?

Thank you for your time, -- Henry

谢谢你的时间, - 亨利

5 个解决方案

#1


28  

Looking into this deeper, it seems as though s3cmd is reading the wrong md5 sum from Amazon. It looks as though s3cmd is getting its sum from the ETAG field. Comparing the actual data of the object that was PUT with the object that was GET'ed the contents are identical and this error can be safely ignored.

更深入地看,似乎s3cmd正在从亚马逊读取错误的md5总和。看起来s3cmd似乎从ETAG字段得到它的总和。将PUT对象的实际数据与GET的对象进行比较,内容相同,可以安全地忽略此错误。

#2


7  

The ETag of a file in S3 will not match the MD5 if the file was uploaded as "Multipart". When a file is marked multipart AWS will hash each part, concatenate the results and then hash that value.

如果文件上传为“Multipart”,则S3中文件的ETag将与MD5不匹配。当文件被标记为多部分时,AWS将对每个部分进行哈希处理,连接结果,然后对该值进行哈希处理。

If the file does not actually have multiple parts the result will be a hash of a hash with -1 added to the end. Try disabling multipart in the tool you use to upload files to S3. For s3cmd, the option is --disable-multipart.

如果文件实际上没有多个部分,则结果将是散列的散列,并在末尾添加-1。尝试在用于将文件上载到S3的工具中禁用multipart。对于s3cmd,选项为--disable-multipart。

#3


3  

ETags with a '-' in them are expected, if the file was uploaded using the S3 Multipart Upload feature (typically used for files >15MB or files read from stdin). s3cmd 1.5.2 knows this and ignores such ETags. If your s3cmd is older than 1.5.2, please upgrade.

如果使用S3 Multipart Upload功能(通常用于> 15MB的文件或从stdin读取的文件)上载文件,则预期带有' - '的ETag。 s3cmd 1.5.2知道这一点而忽略了这样的ETag。如果您的s3cmd早于1.5.2,请升级。

#4


2  

This is a bigger problem is you are using s3cmd sync, because it causes it to re-download previously-synced files. To solve this, add the --no-check-md5 option, which causes s3cmd to only check file sizes to determine changed files (this is good for my purposes, but probably not for everyone, depending on the application).

这是一个更大的问题,你使用s3cmd同步,因为它导致它重新下载以前同步的文件。要解决此问题,请添加--no-check-md5选项,这会导致s3cmd仅检查文件大小以确定更改的文件(这对我的目的很好,但可能不适合所有人,具体取决于应用程序)。

#5


1  

I saw reports about an hour ago that S3 is currently having exactly this problem, e.g. this tweet:

我在大约一小时前看到有关S3目前确实存在此问题的报告,例如这条推文:

RT @drags: @ylastic S3 returning incorrect md5s to s3cmd as well. Never seen an md5 with a '-' in it, until AWS. #AWS #S3

RT @drags:@ylastic S3也将错误的md5s返回到s3cmd。从未见过带有' - '的md5,直到AWS。 #AWS#S3

Though the AWS Status Page reports no issue, I expect this is a transient problem. Try again soon :-)

虽然AWS Status Page报告没有问题,但我认为这是一个短暂的问题。请尽快再试一次:-)


分享到: