6.21.1. Request Headers

Content-MD5

Vault Extension to the S3 API

The Vault system supports the following Request Headers as extensions to the "PUT Bucket lifecycle" operation:

Parameter

Description

Required

x-gmt-tieringinfo

This extension header enables you to configure a bucket for schedule-based automatic transitioning (also known as "auto-tiering") of objects from Vault storage to Amazon S3 storage or Amazon Glacier storage. You can also auto-tier objects from one Vault region to another, or to a third party Vault system.

The x-gmt-tieringinfo header is formatted as follows:

x-gmt-tieringinfo: url-encode(S3/S3GLACIER|EndPoint:url-encode(s3-endpoint),
[Action:stream/nostream/redirect])

S3 or S3GLACIER — Specify S3 if you want to transition the objects to Amazon S3 storage, a different Vault service region, or a third party Vault service. Specify S3GLACIER if you want to transition the objects to Amazon Glacier. See example x-gmt-tieringinfo headers further below in this table.

EndPoint — Endpoint for the transition:

For transitioning to Amazon S3 or Glacier, specify an Amazon S3 endpoint suitable for your location (for example: s3-us-west-1.amazonaws.com). Note that even if the destination is Glacier, you will specify an Amazon S3 endpoint, not a Glacier endpoint. (If you are tiering objects to Glacier, the Vault system will first transition the objects to your specified Amazon S3 endpoint and then they will be immediately transitioned to the corresponding Glacier location.)

By default auto-tiering to Amazon uses a special Amazon account that Nicman sets up for this purpose. If instead you want to auto-tier to an existing Amazon account, you must provide the Vault system with the S3 access credentials for the account. You can do this by using the Vault Admin API’s POST Amazon Tiering Credentials method. The CMC also supports supplying S3 access credentials to use for auto-tiering to an existing account.

For transitioning to a different Vault service region, or to a third party Vault service, you must specify a Vault service endpoint that administrators of your Vault service have added to the Vault S3 configuration file tiering-regions.xml. If you specify an endpoint that is not in this file, your PUT Bucket Lifecycle request will be rejected. Also, the destination Vault domain must be resolvable in your DNS configuration.
As indicated in the x-gmt-tieringinfo format specification above, you must use nested URL encoding. First URL encode the Endpoint value (the endpoint itself), and then URL encode the whole x-gmt-tieringinfo value.

[Action:] — This option specifies how the Vault system will handle GET requests for tiered objects. This option is only applicable for objects that have been transitioned to Amazon S3, a different Vault region, or a third party Vault Service. The supported methods for handling GETs are:

stream — The Vault system GETs the object from the tiered storage system (whether Amazon S3 or Nicman Vault) and streams it through to the client. This is the default method which will be used if you do not specify the Action: option.
nostream — Streaming of tiered objects is not allowed. Instead, the GET is rejected and clients will need to execute a POST Object Restore request in order to temporarily restore a copy of the object in local Vault storage.

redirect — When a user does a GET on a tiered object the response from the Vault system will be an HTTP 307 with a signed redirect URL to the object’s location in the tiered storage system.

For objects tiered to Glacier, using the POST Object Restore operation is the only supported object retrieval method. Streaming and redirects are not supported.

# Example 1 (before URL encoding) - Tiering to Amazon S3

x-gmt-tieringinfo: S3|EndPoint:http://s3.amazonaws.com.

# Example 2 (before URL encoding) - Tiering to a different Vault region

x-gmt-tieringinfo: S3|EndPoint:http://s3-west.my-organization.com.

# Example 3 (before URL encoding) - Tiering to a third party
# Vault service

x-gmt-tieringinfo: S3|EndPoint:http://s3.other-organization.com.

# Example 4 (before URL encoding) - Tiering to Glacier

x-gmt-tieringinfo: S3GLACIER|EndPoint:http://s3.amazonaws.com.

# URL encoding of Example 4

x-gmt-tieringinfo: S3GLACIER%7CEndPoint%3Ahttp%253A%252F%252Fs3.amazonaws.com.

x-gmt-compare

If you include this extension header in your "PUT Bucket lifecycle" request and set the header value to "LAT", then in lifecycle rules that you configure with the "Days" comparator the rule will be implemented as number of days since the object’s Last Access Time.

If you do not use this extension header, or if you include the header but assign it no value or any value other than "LAT", then "Days" based lifecycle rules will be implemented as number of days since the object’s creation (the default Amazon S3 behavior).

You can use this header to create:

Last Access Time based auto-tiering rules (use this header and also the x-gmt-tierinfo header).
Last Access Time based expiration rules (use this header but not the x-gmt-tierinfo header).

An object’s Last Access Time is updated if the object is accessed either for retrieval (GET or HEAD) or modification (PUT/POST/Copy).

If an object is created and then never accessed, its Last Access Time will be its Creation Time.

If you are using the x-gmt-tieringinfo request header, then for the request element StorageClass you must specify "GLACIER". This is true regardless of whether you want to transition the objects to Amazon Glacier, Amazon S3, a different Vault service region, or a third party Vault service.

A sample PUT Bucket Lifecycle request/response pair is below. This rule transitions objects in the user’s Vault S3 storage bucket to Amazon S3 90 days after Last Access Time. Note that the StorageClass element specifies GLACIER even though the tiering target is actually Amazon S3.

# Request

PUT /?lifecycle HTTP/1.1.
Host: bucket1.nicmanlab.com:80.
Accept-Encoding: identity.
Content-Length: 216.
User-Agent: Boto/2.24.0 Python/2.6.6 Linux/2.6.32-358.23.2.el6.x86_64.
x-gmt-tieringinfo: S3%7CEndPoint%3Ahttp%253A%252F%252Fs3.amazonaws.com.
x-gmt-compare: LAT
Date: Sun, 16 Nov 2014 17:54:16 GMT.
Content-MD5: Nip5xNP0P41djj608bRHNQ==.
Content-Type: text/xml.
Authorization: AWS a-key:NfPRnsSbTcxBZ2vN2MX4ZsArJAQ=.
.

<?xml version="1.0" encoding="UTF-8"?>
<LifecycleConfiguration>
  <Rule>
    <ID>Transition to Amazon S3</ID>
    <Prefix></Prefix>
    <Status>Enabled</Status>
    <Transition>
      <StorageClass>GLACIER</StorageClass>
      <Days>90</Days>
    </Transition>
  </Rule>
</LifecycleConfiguration>


# Response

HTTP/1.1 200 OK.
Date: Sun, 16 Nov 2014 17:54:16 GMT.
x-amz-request-id: AF5C7C2098C511E3.
x-gmt-usage: 0,1,623,89,0.
Content-Length: 0.
Server: Nicmanlab.

A second sample PUT Bucket Lifecycle request/response pair is below. This rule transitions all objects in the user’s Vault S3 storage bucket to Amazon Glacier one year after object creation, and then expires (deletes) the objects five years after object creation. Note that the EndPoint is an Amazon S3 endpoint even though the ultimate tiering target is Glacier.

# Request

PUT /?lifecycle HTTP/1.1.
Host: bucket1.nicmanlab.com:80.
Accept-Encoding: identity.
Content-Length: 235.
User-Agent: Boto/2.24.0 Python/2.6.6 Linux/2.6.32-358.23.2.el6.x86_64.
x-gmt-tieringinfo: S3GLACIER%7CEndPoint%3Ahttp%253A%252F%252Fs3.amazonaws.com.
Date: Tue, 18 Feb 2014 17:54:16 GMT.
Content-MD5: Nip5xNP0P41djj608bRQHN==.
Content-Type: text/xml.
Authorization: AWS a-key:NfPRnsSbTcxBZ2vN2MX4ZsArJAQ=.
.

<?xml version="1.0" encoding="UTF-8"?>
<LifecycleConfiguration>
  <Rule>
    <ID>Transition to Glacier and later delete</ID>
    <Prefix></Prefix>
    <Status>Enabled</Status>
    <Transition>
      <StorageClass>GLACIER</StorageClass>
      <Days>365</Days>
    </Transition>
    <Expiration>
      <Days>1825</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>


# Response

HTTP/1.1 200 OK.
Date: Tue, 18 Feb 2014 17:58:16 GMT.
x-amz-request-id: AF5C7C2098C522F4.
x-gmt-usage: 0,1,623,89,0.
Content-Length: 0.
Server: nicmanlab.

Vault Extension to the S3 API

Connect with Us on LinkedIn

Main links

Services

Infrastructure