阅读背景:

Nodejs在文件系统中存储数十亿个图像

来源:互联网 

I'm new to nodejs and currently I'm working on a Node Web API with mongodb database.

我是nodejs的新手,目前我正在开发一个带有mongodb数据库的Node Web API。

  • According to project requirements users should be able to upload their profile pictures and cover photos.
  • 根据项目要求,用户应该能够上传他们的个人资料图片和封面照片。
  • And There can be billions of user accounts
  • 并且可能有数十亿的用户帐户

After doing some research, I have found that "storing images in a file system is better than storing in database"

经过一些研究,我发现“将文件存储在文件系统中比存储在数据库中更好”

So now my plan is to save images in a file system. But, - There is a limited number of files can be placed in a one folder (based on OS) - should be able to access files quickly

所以现在我的计划是将图像保存在文件系统中。但是, - 可以在一个文件夹中放置有限数量的文件(基于操作系统) - 应该能够快速访问文件

because of above requirements, In order to store large number of images, We need to have a good folder structure and a naming policy.

由于上述要求,为了存储大量图像,我们需要有一个好的文件夹结构和命名策略。

Can anyone suggest any good naming algorithm and a folder structure to store billions of profile pics ?

任何人都可以建议任何良好的命名算法和文件夹结构来存储数十亿的个人资料照片吗?

(currently I'm accessing users by object Id's generated by mongodb)

(目前我通过mongodb生成的对象Id访问用户)

1 个解决方案

#1


0  

You could create a folder for each user (if the OS supports so many folders in one place) and store the user information that isn't stored in the database in there.

您可以为每个用户创建一个文件夹(如果操作系统在一个位置支持这么多文件夹)并存储那些未存储在数据库中的用户信息。

The database entry for user should contain an additional entry which is called "data_folder_id" which contains the number of the folder on the filesystem. Or you can name the folders according to the user ID directly.

用户的数据库条目应包含一个名为“data_folder_id”的附加条目,其中包含文件系统上文件夹的编号。或者,您可以直接根据用户ID命名文件夹。

An example folder structure:

示例文件夹结构:

[user_data] // The folder which will contain the data from your all your users
    [1] // The data folder from user 1
        [profile_photo.png] // Profile photo from user 1
        [cover_photo.png] // Cover photo from user 1

    [2]
        [profile_photo.png]
        [cover_photo.png]

    [3]
        [profile_photo.png]
        [cover_photo.png]

    [...]

If you can't store so many folders in one place you have to split them up in subfolders like:

如果您无法在一个地方存储这么多文件夹,则必须将它们拆分为子文件夹,例如:

[user_data] // The folder which will contain the data from all your users
    [1-50] // Contains user folders 1 to 50
        [1] // The data folder from user 1
            [profile_photo.png] // Profile photo from user 1
            [cover_photo.png] // Cover photo from user 1

        [2]
            [profile_photo.png]
            [cover_photo.png]

        [...]

    [51-100]
        [51]
            [profile_photo.png]
            [cover_photo.png]

        [52]
            [profile_photo.png]
            [cover_photo.png]

    [...]

分享到: