阅读背景:

使用HTTP POST上传AWS S3浏览器会产生无效签名

来源:互联网 

I'm working on a website where the users should be able to upload video files to AWS. In order to avoid unnecessary traffic I would like the user to upload directly to AWS (and not through the API server). In order to not expose my secret key in the JavaScript I'm trying to generate a signature in the API. It does, however, tell me when I try to upload, that the signature does not match.

我正在一个用户应该能够将视频文件上传到AWS的网站上工作。为了避免不必要的流量,我希望用户直接上传到AWS(而不是通过API服务器)。为了不在JavaScript中公开我的秘密密钥,我试图在API中生成签名。但是,它确实告诉我,当我尝试上传时,签名不匹配。

For signature generation I have been using https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html

对于签名生成,我一直在使用https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html

On the backend I'm running C#.

在后端我正在运行C#。

I generate the signature using

我使用生成签名

string policy = $@"{{""expiration"":""{expiration}"",""conditions"":[{{""bucket"":""dennisjakobsentestbucket""}},[""starts-with"",""$key"",""""],{{""acl"":""private""}},[""starts-with"",""$Content-Type"",""""],{{""x-amz-algorithm"":""AWS4-HMAC-SHA256""}}]}}";

which generates the following

产生以下内容

{"expiration":"2016-11-27T13:59:32Z","conditions":[{"bucket":"dennisjakobsentestbucket"},["starts-with","$key",""],{"acl":"private"},["starts-with","$Content-Type",""],{"x-amz-algorithm":"AWS4-HMAC-SHA256"}]}

based on https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html (I base64 encode the policy). I have tried to keep it very simple, just as a starting point.

基于https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html(我对base64编码策略)。我试图保持它非常简单,只是作为一个起点。

For generating the signature, I use code found on the AWS site.

为了生成签名,我使用AWS站点上的代码。

static byte[] HmacSHA256(String data, byte[] key)
{
    String algorithm = "HmacSHA256";
    KeyedHashAlgorithm kha = KeyedHashAlgorithm.Create(algorithm);
    kha.Key = key;

    return kha.ComputeHash(Encoding.UTF8.GetBytes(data));
}

static byte[] GetSignatureKey(String key, String dateStamp, String regionName, String serviceName)
{
    byte[] kSecret = Encoding.UTF8.GetBytes(("AWS4" + key).ToCharArray());
    byte[] kDate = HmacSHA256(dateStamp, kSecret);
    byte[] kRegion = HmacSHA256(regionName, kDate);
    byte[] kService = HmacSHA256(serviceName, kRegion);
    byte[] kSigning = HmacSHA256("aws4_request", kService);

    return kSigning;
}

Which I use like this:

我用的是这样的:

byte[] signingKey = GetSignatureKey(appSettings["aws:SecretKey"], dateString, appSettings["aws:Region"], "s3");
byte[] signature = HmacSHA256(encodedPolicy, signingKey);

where dateString is on the format yyyymmdd

其中dateString的格式为yyyymmdd

I POST information from JavaScript using

我用JavaScript发布信息

let xmlHttpRequest = new XMLHttpRequest();
let formData = new FormData();
formData.append("key", "<path-to-upload-location>");
formData.append("acl", signature.acl); // private
formData.append("Content-Type", "$Content-Type");
formData.append("AWSAccessKeyId", signature.accessKey);
formData.append("policy", signature.policy); //base64 of policy
formData.append("x-amz-credential", signature.credentials); // <accesskey>/20161126/eu-west-1/s3/aws4_request
formData.append("x-amz-date", signature.date);
formData.append("x-amz-algorithm", "AWS4-HMAC-SHA256");
formData.append("Signature", signature.signature);
formData.append("file", file);

xmlHttpRequest.open("post", "https://<bucketname>.s3-eu-west-1.amazonaws.com/");
xmlHttpRequest.send(formData);

I have been using UTF8 everywhere as prescribed by AWS. In their examples the signature is on a hex format, which I have tried as well. No matter what I try I get an error 403

我一直按照AWS的规定使用UTF8。在他们的例子中,签名是十六进制格式,我也尝试过。无论我尝试什么,我都会收到错误403

The request signature we calculated does not match the signature you provided. Check your key and signing method.

My policy on AWS has "s3:Get*", "s3:Put*"

我在AWS上的政策有“s3:Get *”,“s3:Put *”

Am I missing something or does it just work completely different than what I expect?

我是否遗漏了某些东西,或者它的工作方式与我的期望完全不同?

Edit: The answer below is one of the steps. The other is that AWS distinguish between upper and lowercase hex strings. 0xFF != 0xff in the eyes of AWS. They want the signature in all lowercase.

编辑:下面的答案是其中一个步骤。另一个是AWS区分大写和小写六角形字符串。在AWS眼中,0xFF!= 0xff。他们想要全部小写的签名。

2 个解决方案

#1


4  

You are generating the signature using Signature Version 4, but you are constructing the form as though you were using Signature Version 2... well, sort of.

您正在使用签名版本4生成签名,但是您正在构建表单,就好像您使用的是签名版本2 ...好吧,等等。

formData.append("AWSAccessKeyId", signature.accessKey);

That's V2. It shouldn't be here at all.

那是V2。它根本不应该在这里。

formData.append("x-amz-credential", signature.credentials); // <accesskey>/20161126/eu-west-1/s3/aws4_request

This is V4. Note the redundant submission of the AWS Access Key ID here and above. This one is probably correct, although the examples have capitalization like X-Amz-Credential.

这是V4。请注意此处及以上的AWS Access Key ID的冗余提交。这个可能是正确的,虽然这些例子像X-Amz-Credential一样大写。

formData.append("x-amz-algorithm", "AWS4-HMAC-SHA256");

That is also correct, except it may need to be X-Amz-Algorithm. (The example seems to imply that capitalization is ignored).

这也是正确的,除了它可能需要是X-Amz算法。 (这个例子似乎暗示大写被忽略了)。

formData.append("Signature", signature.signature);

This one is incorrect. This should be X-Amz-Signature. V4 signatures are hex, so that is what you should have here. V2 signatures are base64.

这个是不正确的。这应该是X-Amz-Signature。 V4签名是十六进制的,所以这就是你应该拥有的。 V2签名是base64。

There's a full V4 example here, which even provides you with an example aws key and secret, date, region, bucket name, etc., that you can use with your code to verify that you indeed get the same response. The form won't actually work but the important question is whether your code can generate the same form, policy, and signature.

这里有一个完整的V4示例,它甚至为您提供了示例aws密钥和秘密,日期,区域,存储桶名称等,您可以将其与代码一起使用以验证您确实获得了相同的响应。表单实际上不起作用,但重要的问题是您的代码是否可以生成相同的表单,策略和签名。

For any given request, there is only ever exactly one correct signature; however, for any given policy, there may be more than one valid JSON encoding (due to JSON's flexibility with whitespace) -- but for any given JSON encoding there is only one possible valid base64-encoding of the policy. This means that your code, using the example data, is certified as working correctly if it generates exactly the same form and signature as shown in the example -- and it means that your code is proven invalid if it generates the same form and policy with a different signature -- but there is a third possibility: the test actually proves nothing conclusive about your code if your code generates a different base64 encoding of the policy, because that will necessarily change the signature to not match, yet might still be a valid policy.

对于任何给定的请求,只有一个正确的签名;但是,对于任何给定的策略,可能存在多个有效的JSON编码(由于JSON的空白灵活性) - 但对于任何给定的JSON编码,只有一个可能的有效base64编码的策略。这意味着,如果代码生成与示例中显示的完全相同的表单和签名,则使用示例数据的代码被认证为正常工作 - 这意味着如果代码生成相同的表单和策略,则证明代码无效一个不同的签名 - 但还有第三种可能性:如果您的代码生成不同的base64编码,那么测试实际上证明您的代码没有任何结论,因为这必然会将签名更改为不匹配,但可能仍然有效政策。

Note that Signature V2 is only suported on older S3 regions, while Signature V4 is supported by all S3 regions, so, even though you could alternately fix this by making your entire signing process use V2, that wouldn't be recommended.

请注意,Signature V2仅支持较旧的S3区域,而所有S3区域都支持Signature V4,因此,即使您可以通过使整个签名过程使用V2来交替修复此问题,也不建议这样做。

Note also that The request signature we calculated does not match the signature you provided. Check your key and signing method does not tell you anything about whether the bucket policy or any users policies allow or deny the request. This error is not a permissions error. It will be thrown prior to the permissions checks, based solely on the validity of the signature, not whether the AWS Access Key id is authorized to perform the requested operation, which is something that is only tested after the signature is validated.

另请注意,我们计算的请求签名与您提供的签名不匹配。检查密钥和签名方法不会告诉您有关存储桶策略或任何用户策略是允许还是拒绝请求的任何信息。此错误不是权限错误。它将在权限检查之前抛出,仅基于签名的有效性,而不是AWS Access Key id是否被授权执行所请求的操作,这仅在签名验证后进行测试。

#2


0  

I suggest you to create a pair auth token with permission to POST only, and send an http request like this:

我建议你创建一个只有POST权限的一对身份验证令牌,并发送一个这样的http请求:

require 'rest-client'

class S3Uploader
  def initialize
    @options = {
      aws_access_key_id: "ACCESS_KEY",
      aws_secret_access_key: "ACCESS_SECRET",
      bucket: "BUCKET",
      acl: "private",
      expiration: 3.hours.from_now.utc,
      max_file_size: 524288000
    }
  end

  def fields
    {
      :key => key,
      :acl => @options[:acl],
      :policy => policy,
      :signature => signature,
      "AWSAccessKeyId" => @options[:aws_access_key_id],
      :success_action_status => "201"
    }
  end

  def key
    @key ||= "temp/${filename}"
  end

  def url
    "https://#{@options[:bucket]}.s3.amazonaws.com/"
  end

  def policy
    Base64.encode64(policy_data.to_json).delete("\n")
  end

  def policy_data
    {
      expiration: @options[:expiration],
      conditions: [
        ["starts-with", "$key", ""],
        ["content-length-range", 0, @options[:max_file_size]],
        { bucket: @options[:bucket] },
        { acl: @options[:acl] },
        { success_action_status: "201" }
      ]
    }
  end

  def signature
    Base64.encode64(
      OpenSSL::HMAC.digest(
        OpenSSL::Digest.new("sha1"),
        @options[:aws_secret_access_key], policy
      )
    ).delete("\n")
  end
end

uploader = S3Uploader.new
puts uploader.fields
puts uploader.url

begin
  RestClient.post(uploader.url, uploader.fields.merge(file: File.new('51bb26652134e98eae931fbaa10dc3a1.jpeg'), :multipart => true))
rescue RestClient::ExceptionWithResponse => e
  puts e.response
end

分享到: