使用AWS Lambda中存储在S3上的pickle文件

I'm currently working on an Amazon Web Services Lambda function, where I need to utilize sklearn files that I've created locally but are now stored in S3. I've been stumped on how to actually do this, I've tried a few different methods from boto3's docs but this results in a timeout error:

我目前正在开发一个Amazon Web Services Lambda函数，我需要使用我在本地创建但现在存储在S3中的sklearn文件。我一直难以理解如何实际做到这一点，我尝试了boto3的文档中的一些不同的方法，但这会导致超时错误：

import boto3
import pickle
s3 = boto3.client("s3", region_name="us-west-1")
response = s3.get_object(Bucket="mybucket", Key="bin_files/scaler.pkl")
loaded_file = pickle.load(response["Body"])

While doing this on the other hand results in a file not found error:

另一方面，这样做会导致找不到文件错误：

 with open(key, 'wb') as data:
     s3.Bucket(event['bucket']).download_fileobj(key, data)


"errorMessage": "[Errno 2] No such file or directory: 
'bin_files/scaler.pkl'"

Does anyone have any thoughts or tips on how to do this?

有没有人对如何做到这一点有任何想法或提示？

1 个解决方案

#1

Correct code:

正确的代码：

 with open('scaler.pkl', 'wb') as data:
     s3.Bucket('bucket').download_fileobj(key, data)

If you want to download the s3 file to current directory, then you should just pass the Filename to the file open() method and NOT the entire key. In your case you are passing bin_files/scaler.pkl as the Filename.

如果要将s3文件下载到当前目录，则只需将文件名传递给文件open（）方法而不是整个密钥。在您的情况下，您将传递bin_files / scaler.pkl作为文件名。

In your case, the python code will look for bin_files directory in the current working directory(directory where the boto script is getting executed) and try to write scaler.pkl inside the bin_files directory. Since the directory doesn't exist, it throws the [Errno 2] No such file or directory error.

在您的情况下，python代码将在当前工作目录（执行boto脚本的目录）中查找bin_files目录，并尝试在bin_files目录中编写scaler.pkl。由于目录不存在，它会抛出[Errno 2]没有这样的文件或目录错误。

If you add a PATH to the Filename, then you should ensure the entire PATH exists with appropriate directory permissions.

如果将PATH添加到文件名，则应确保整个PATH存在且具有适当的目录权限。