AWS S3-Compatible API service on BTFS

AWS S3 is widely adopted as a second-generation centralized cloud storage solution, allowing multitudinous applications to access it through its API protocol or SDKs. In contrast, BTFS as a third-generation decentralized storage protocol has yet to gain popularity, which means developers must redesign and develop their applications to be compatible with BTFS's API protocol. However, with a set of AWS S3-compatible API services from BTFS, it would be far easier for applications to be connected to BTFS or use it as an optional transparent storage layer.

Add configuration and start command options

In the configuration file, you can now find a new section called S3CompatibleAPI which includes three options:

Enable: This option determines whether to enable the S3-compatible API. Please note that its priority is lower than the daemon start command option.
Address: Use this option to configure the service address for the S3-compatible API. Keep in mind that it can be accessed externally when the address is non-local.
HTTPHeaders: Use this option to configure the response headers for the S3-compatible API service. It can also be used to set up cross-domain rules.

  {
     ...
     "S3CompatibleAPI": {
       "Enable": false,
       "Address": "127.0.0.1:6001",
       "HTTPHeaders": null
     },
     ...
   }

Additionally, we have introduced a new bool-type option 's3-compatible-api' in the start command. This option allows you to enable or disable the S3API service. Set it to 'true' to start the service or 'false' to prevent it from starting. By default, it is not set, meaning that the value will be determined by the 'EnableS3API' field in the configuration file.

  # enable the s3-compatible-api server
  btfs daemon --s3-compatible-api
  btfs daemon --s3-compatible-api=true
  
  # disable the s3-compatible-api server
  btfs daemon --se-compatible-api=false
  
  # enable or disable s3-compatible-api server according to the S3CompatibleAPI.Enable field in the configuration file.
  btfs daemon

Access Keys related commands.

Access Keys APIs are here.

Authentication

The BTFS S3-compatible API only supports AWS v4 signatures (AWS4-HMAC-SHA256) for authentication and does not support AWS v2 signatures at this time.

Access Control Lists (ACLs)

The BTFS S3-compatible API provides limited support for access control lists (ACLs), with no current support for object-level ACLs.
The methods GetObjectAcl and GetBucketAcl will work as expected, but GetObjectAcl will return the ACL of the bucket it is in.
A bucket's owner is the access key used to create the bucket (access keys cannot be changed at present), and only the owner can change the ACL of a bucket or delete the bucket.
Supported predefined ACLs include private, public-read, and public-read-write; see AWS ACL for detailed definitions; permissions, by default, are set to public-read.
- private: only the bucket owner can upload, delete, and read objects in the bucket, as well as read the list of the objects inside the bucket.
- public-read: only the bucket owner can upload and delete objects in the bucket; however, public users, including anonymous ones, can read objects and the list of the objects inside the bucket.
- public-read-write: public users, including anonymous ones, can upload, delete, and read objects in the bucket, as well as read the list of the objects inside the bucket.

Supported API methods

1. Supported Bucket Calls:

CreateBuket
HeadBucket
ListBuckets
DeleteBucket
PutBucketAcl
GetBucketAcl

2. Supported Object Calls:

ListObjects
ListObjectsV2
HeadObject
PutObject
CopyObject
DeleteObject
DeleteObjects
GetObjectAcl

3. Supported Multipart Calls:

CreateMultipartUpload
AbortMultipartUpload
CompleteMultipartUpload
UploadPart

For more information on s3api, see here.

Methods for obtaining BTFS Hash

You can get btfs hash from object metadata and response header.

Object metadata key:

{
   "Metadata": {
      "Cid": "<BTFS cid>"
   }
}

Response header

Cid=<BTFS Cid>