AWS S3 Usage Logs
Log Type: AWS S3 Usage
Template Description: Parsing the common fields in your AWS S3 Usage log.
Sample Log:
6ec976a42247d687d5d1c87bb53e87c60c925765f87415f472d240c5d18337a7 stag-bloomfilter-000000000000141d [2017-08-16] 209.118.103.150 user/stag-bloomfilter CC487A9D90678CAB REST.GET.OBJECT 636C271B3F171BB8-000000000138CE3D-1405616382510-v1 "GET /636C271B3F171BB8-000000000138CE3D-1405616382510-v1 HTTP/1.1" 200 - 26828 369 89 11 "-" "aws-sdk-java/1.7.11 Linux/3.2.0-57-virtual OpenJDK_64-Bit_Server_VM/23.2-b09/1.7.0_09" -
Parsing Rule:
parse "* * [*] * * * * * \"* HTTP/1.1\" * * * * * * * \"*\" *" as bucket_owner, bucket, time, remoteIP, requester, request_ID, operation, key, request_URI, status_code, error_code, bytes_sent, object_size, total_time, turn_time, referrer, user_agent, version_ID
Resulting Fields:
| Field | Description | Example |
|---|---|---|
| bucket_owner | The canonical user ID of the owner of the source bucket. | 6ec976a42247d687d5d1c87bb53e87c60c925765f87415f472d240c5d18337a7 |
| bucket | The name of the bucket that the request was processed against | stag-bloomfilter-000000000000141d |
| time | The time at which the request was received | 2017-07-13 |
| remoteIP | The apparent Internet address of the requester | 62.118.225.244 |
| requester | The canonical user ID of the requester, or a - for unauthenticated requests | user/stag-bloomfilter |
| request_ID | The request ID is a string generated by Amazon S3 to uniquely identify each request | B5C788A74FDFA7E7 |
| operation | The operation listed here is declared as SOAP.operation, REST.HTTP_method.resource_type,WEBSITE.HTTP_method.resource_type, or BATCH.DELETE.OBJECT | REST.PUT.OBJECT |
| key | The "key" part of the request, URL encoded, or "-" if the operation does not take a key parameter | 636C271B3F171BB8-000000000138CE3D-1405616382510-v1 |
| request_URI | The Request-URI part of the HTTP request | GET /636C271B3F171BB8-000000000138CE3D-1405616382510-v1 |
| status_code | The numeric HTTP status code of the response | 200 |
| error_code | The Amazon S3 Error Code, or "-" if no error occurred | NoSuchKey |
| bytes_sent | The number of response bytes sent, excluding HTTP protocol overhead, or "-" if zero | 5982 |
| object_size | The total size of the object in question | 50768 |
| total_time | The number of milliseconds the request was in flight from the server's perspective | 27 |
| turn_time | The number of milliseconds that Amazon S3 spent processing your request | 24 |
| referrer | The value of the HTTP Referrer header, if present | "http://www.amazon.com/webservice" |
| user_agent | The value of the HTTP User-Agent header | aws-sdk-java/1.7.11 Linux/3.2.0-57-virtual OpenJDK_64-Bit_Server_VM/23.2-b09/1.7.0_09 |
| version_ID | The version ID in the request, or "-" if the operation does not take a versionId parameter | 3HL4kqtJvjVBH40Nrjfkd |