S3 Server Access Logs: How to Enable Them (Console, CLI, Terraform), the Log Format Field by Field, Athena and CloudWatch Logs Queries, Cost, vs CloudTrail, and Why Logs Go Missing
S3 server access logs record the requests made to a bucket: who asked (IP address and identity), what they did (REST.GET.OBJECT, REST.PUT.OBJECT, a delete), which object, the HTTP status and error code, and how long it took. To enable S3 access logs, open the bucket in the S3 console, go to Properties, then Server access logging, choose Edit, and pick a different bucket in the same Region and account as the destination. Or deliver them to Amazon CloudWatch Logs instead, which became available in every AWS Region on October 21, 2026. Delivery to a bucket is free (you pay only for storage); CloudWatch Logs charges $0.50 per GB ingested. What surprises almost everyone: these logs are best-effort. S3 delivers most records within a few hours, but a record can arrive much later, twice, or not at all, so they are not a complete audit trail. For that, you need CloudTrail data events.
Jake's product photos live in an S3 bucket, and the shop's website loads them straight from it. In September the S3 bill doubled, with no new products and no more orders. "Somebody is downloading my photos," Jake said. "Find out who." Ethan switched on server access logging, refreshed the log bucket every ten minutes for an hour, saw nothing, and was about to decide it was broken. It wasn't. It was just S3 being S3.
This guide covers AWS S3 server access logging from switching it on to finding the answer in the logs: the console, CLI and Terraform setup, the new CloudWatch Logs delivery, the log format field by field, ready-made Athena and Logs Insights queries, the cost, how it compares with CloudTrail, and the checklist for when no logs arrive. If S3 itself is new to you, what Amazon S3 is explains buckets and objects first.
What S3 server access logs are, and what they are not
Server access logging is a per-bucket setting. Once it is on, S3 writes a record for (almost) every request made to that bucket and periodically delivers the records to a destination you choose. Each record is one request: a GET of a photo, a PUT of an upload, a DELETE, a HEAD from a monitoring tool, a listing, or a console page that reads the bucket's settings. A request that is refused is logged too, with its 403 or 404 and the error code, which makes the logs the quickest way to see who is being denied.
The logs answer four kinds of question well:
- Who is reading this bucket? Source IP addresses, IAM identities, user agents and referrers, which is how you spot hotlinking or a forgotten script.
- What is driving the bill? Request counts per operation and bytes sent per object or per IP address.
- Why are requests failing? Status and error codes such as
AccessDenied,NoSuchKeyandSlowDown, per key. - How slow are requests? Total time and turnaround time in milliseconds, per request.
And they have three limits you should know before you rely on them:
- Best-effort delivery. Completeness and timeliness are not guaranteed: most records arrive within a few hours, a record can arrive long after the request, it can be duplicated, or it can be missing. The purpose is to give you "an idea of the nature of traffic", not a complete accounting. Your bill can include requests that never appear in a log.
- Changes take effect over time. For about an hour after you turn logging on, or change the destination, some requests are logged and some are not, and some may still go to the old destination.
- Only new requests. There is no history. Logging starts when you enable it; nothing from yesterday will ever appear.
That first limit is why Ethan saw nothing for an hour. Two hours after he enabled logging, the first files appeared, and with them the answer: three IP addresses fetching the same 40 product photos thousands of times a day, all with the Referer of a site that was not Jake's.
How to enable S3 server access logging (console and CLI)
You need two buckets in the same AWS Region and the same account: the source bucket you want to log, and a destination bucket that receives the log files. They can technically be the same bucket, but do not do that: every log file S3 writes into the bucket is itself a request that gets logged, so you get logs about logs, a growing storage bill, and logs that are hard to read. Create a separate bucket such as jakes-shop-access-logs first.
In the S3 console:
- Open the source bucket and choose the Properties tab.
- Scroll to Server access logging and choose Edit.
- Choose Enable, then Browse S3 and pick the destination bucket. Add a prefix ending in a slash, such as
access/, so the logs sit in their own folder. - Under Log object key format, choose the date-based option,
[DestinationPrefix][SourceAccountId]/[SourceRegion]/[SourceBucket]/[YYYY]/[MM]/[DD]/..., with S3 event time. It makes the logs far easier to query later. - Choose Save changes. The console also adds a statement to the destination bucket's policy so the S3 logging service can write there.
- Wait. The first files usually appear within a few hours, not minutes.
With the AWS CLI, you do the policy yourself. First, allow the S3 logging service principal, logging.s3.amazonaws.com, to write into the destination bucket, but only for your source bucket and account:
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "S3ServerAccessLogsPolicy",
"Effect": "Allow",
"Principal": { "Service": "logging.s3.amazonaws.com" },
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::jakes-shop-access-logs/access/*",
"Condition": {
"ArnLike": { "aws:SourceArn": "arn:aws:s3:::jakes-shop-photos" },
"StringEquals": { "aws:SourceAccount": "111122223333" }
}
}]
}
aws s3api put-bucket-policy --bucket jakes-shop-access-logs --policy file://log-policy.json
Then turn logging on for the source bucket, with the date-based key format:
aws s3api put-bucket-logging --bucket jakes-shop-photos --bucket-logging-status '{
"LoggingEnabled": {
"TargetBucket": "jakes-shop-access-logs",
"TargetPrefix": "access/",
"TargetObjectKeyFormat": {
"PartitionedPrefix": { "PartitionDateSource": "EventTime" }
}
}
}'
# check it
aws s3api get-bucket-logging --bucket jakes-shop-photos
# turn it off again
aws s3api put-bucket-logging --bucket jakes-shop-photos --bucket-logging-status '{}'
The two key formats produce files named like this:
| Key format | Example log file key | Use it when |
|---|---|---|
| Simple prefix (default) | access/2026-10-23-06-15-42-E568B2907131C0C0 | You only look at logs now and then, for one bucket |
| Date-based (partitioned) | access/111122223333/us-east-1/jakes-shop-photos/2026/10/23/2026-10-23-06-15-42-E568B2907131C0C0 | You will query with Athena, keep months of logs, or log several buckets into one place |
With date-based keys you choose the date source: EventTime files a record under the day the request happened, DeliveryTime under the day the file was delivered. EventTime is the one you want for "what happened on October 12", because delivery can lag by hours and cross midnight.
Mind the slash in the prefix. With access/ the files go into an access folder; with access (no slash) the prefix is glued to the file name, giving keys like access2026-10-23-06-15-42-... scattered across the top of the bucket.
If you would rather use an ACL than a bucket policy, that only works when the destination bucket still has ACLs enabled. New buckets have the Bucket owner enforced Object Ownership setting, which disables ACLs entirely; for those, the bucket policy above is the only way, and a logging configuration that includes target grants is rejected. How to give access to an S3 bucket explains Object Ownership and bucket policies in more depth.
S3 server access logs in Terraform
In the AWS provider for Terraform, logging is its own resource, aws_s3_bucket_logging, separate from the bucket. A complete setup has four parts: the two buckets, the destination bucket's policy, the logging configuration, and a lifecycle rule so the logs do not grow forever. This configuration was applied with Terraform 1.16.5 and AWS provider 6.70.0, and all five resources created cleanly:
data "aws_caller_identity" "current" {}
resource "aws_s3_bucket" "site" {
bucket = "jakes-shop-photos"
}
resource "aws_s3_bucket" "logs" {
bucket = "jakes-shop-access-logs"
}
# Let the S3 logging service write into the log bucket, only for our source bucket and account
resource "aws_s3_bucket_policy" "logs" {
bucket = aws_s3_bucket.logs.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Sid = "S3ServerAccessLogsPolicy"
Effect = "Allow"
Principal = { Service = "logging.s3.amazonaws.com" }
Action = "s3:PutObject"
Resource = "${aws_s3_bucket.logs.arn}/access/*"
Condition = {
ArnLike = { "aws:SourceArn" = aws_s3_bucket.site.arn }
StringEquals = { "aws:SourceAccount" = data.aws_caller_identity.current.account_id }
}
}]
})
}
resource "aws_s3_bucket_logging" "site" {
bucket = aws_s3_bucket.site.id
target_bucket = aws_s3_bucket.logs.id
target_prefix = "access/"
target_object_key_format {
partitioned_prefix {
partition_date_source = "EventTime"
}
}
depends_on = [aws_s3_bucket_policy.logs]
}
# Logs are evidence, not treasure: expire them so storage does not grow forever
resource "aws_s3_bucket_lifecycle_configuration" "logs" {
bucket = aws_s3_bucket.logs.id
rule {
id = "expire-access-logs"
status = "Enabled"
filter {
prefix = "access/"
}
expiration {
days = 90
}
}
}
aws_s3_bucket_policy.logs: Creation complete after 1s [id=jakes-shop-access-logs]
aws_s3_bucket_logging.site: Creation complete after 1s [id=jakes-shop-photos]
aws_s3_bucket_lifecycle_configuration.logs: Creation complete after 57s [id=jakes-shop-access-logs]
Apply complete! Resources: 5 added, 0 changed, 0 destroyed.
Three details matter. The depends_on makes Terraform write the policy before it turns logging on, so S3 is never configured to deliver into a bucket that refuses it. For a simple prefix, replace the partitioned_prefix block with an empty simple_prefix {}. And to log many buckets, create one aws_s3_bucket_logging per source bucket, all pointing at the same log bucket, with one policy statement whose aws:SourceArn lists every source bucket. The old inline logging block inside aws_s3_bucket is deprecated; if both exist for the same bucket, they fight on every apply.
S3 server access logs to CloudWatch Logs (new in every Region)
Since October 21, 2026, S3 server access logs can be delivered to Amazon CloudWatch Logs in every AWS Region, including GovCloud, China and the European Sovereign Cloud. This path is different in nearly every way that matters: the records arrive as structured JSON with named fields, you can query them in CloudWatch Logs Insights without writing a regex, set alarms on error rates, encrypt them with your own KMS key, and collect them across accounts and Regions. You can use it instead of, or as well as, delivery to a bucket.
It is configured with CloudWatch Logs vended log delivery APIs, not with S3's PutBucketLogging. The log group must be in the same Region as the source bucket. Three commands:
# 1. register the bucket as a log source
aws logs put-delivery-source \
--name jakes-photos-sal \
--resource-arn arn:aws:s3:::jakes-shop-photos \
--log-type S3_SERVER_ACCESS_LOGS
# 2. say where the logs go (an existing log group in the same Region)
aws logs put-delivery-destination \
--name jakes-photos-sal-dest \
--delivery-destination-configuration '{"destinationResourceArn":"arn:aws:logs:us-east-1:111122223333:log-group:/aws/vendedlogs/s3/jakes-shop-photos"}'
# 3. connect them
aws logs create-delivery \
--delivery-source-name jakes-photos-sal \
--delivery-destination-arn arn:aws:logs:us-east-1:111122223333:delivery-destination:jakes-photos-sal-dest
The caller needs logs:PutDeliverySource, logs:PutDeliveryDestination, logs:CreateDelivery, and s3:AllowVendedLogDeliveryForResource on the source bucket; a missing last one is the usual cause of AccessDenied on step 1. The configuration takes 15 to 20 minutes to propagate, and then delivery typically starts within a few hours. As with bucket delivery, there is no backfill.
Set a retention period on the log group, because the default is to keep logs forever:
aws logs put-retention-policy \
--log-group-name /aws/vendedlogs/s3/jakes-shop-photos \
--retention-in-days 90
Each record is a JSON object with fields such as request_time, remote_ip, requester, operation, key_name, http_status, error_code, bytes_sent_size, object_size, total_duration, user_agent, tls_version and access_point_arn, plus a schema_version_id so you can detect new fields later. Unknown values are JSON null instead of the text format's -. From the log group you can also send the logs on to S3 as JSON or Parquet, to Amazon Data Firehose, or mirror them into an S3 Tables (Apache Iceberg) table for SQL in Athena, with no extra charge for the table's storage.
If nothing arrives, aws logs describe-deliveries should show your delivery as ACTIVE, and aws logs describe-delivery-sources should show the right bucket ARN with the log type S3_SERVER_ACCESS_LOGS. If the source bucket or the log group is deleted, delivery stops, and recreating either one needs a new delivery.
Bucket delivery vs CloudWatch Logs delivery
| S3 general purpose bucket | CloudWatch Logs | |
|---|---|---|
| Format | Space-separated text lines | Structured JSON (or JSON/Parquet onward to S3, Iceberg in S3 Tables) |
| Set up with | PutBucketLogging or the console | CloudWatch Logs delivery APIs or the S3 console |
| Query with | Athena (with a regex table) | Logs Insights, or Athena on the S3 Tables mirror |
| Cross-account / cross-Region | No: same account and Region only | Yes, through the S3 and Firehose destinations |
| Encryption | SSE-S3 only | Your KMS key |
| Alarms | Not directly | Yes, with metric filters on the log group |
| Delivery speed | Within a few hours | Within a few hours |
| Cost | Free delivery; S3 storage only | $0.50 per GB ingested (first 10 TB, US East), plus $0.03 per GB-month stored |
The short version: deliver to a bucket if you want the logs cheaply "just in case" and will query them rarely; deliver to CloudWatch Logs if you want to watch errors as they happen, alarm on them, or collect logs from many accounts in one place. Ethan kept bucket delivery for Jake's shop: one bucket, one account, and a question that needed answering once.
The S3 access log format, field by field
Delivered to a bucket, each log file is plain text with one request per line and fields separated by spaces. A field that does not apply, or is unknown, is a single -. Here is a standard example of an upload record (owner IDs shortened):
79a59df9...7ef2be amzn-s3-demo-bucket1 [06/Feb/2019:00:01:57 +0000] 192.0.2.3 79a59df9...7ef2be
DD6CC733AEXAMPLE REST.PUT.OBJECT s3-dg.pdf "PUT /amzn-s3-demo-bucket1/s3-dg.pdf HTTP/1.1" 200 - - 4406583
41754 28 "-" "S3Console/0.4" - 10S62Zv81kBW7BB6SX4XJ48o6kpcl6LPwEoizZQQxJd5qDSCTLX0TgS37kYUBKQW3+bPdrg1234=
SigV4 ECDHE-RSA-AES128-SHA AuthHeader amzn-s3-demo-bucket1.s3.us-west-1.amazonaws.com TLSV1.2 - Yes us-east-1
(It is one line in the file; it is wrapped here to fit.) In order, the fields are:
| # | Field | In the example | What it tells you |
|---|---|---|---|
| 1 | Bucket owner | 79a59df9... | Canonical user ID of the source bucket's owner |
| 2 | Bucket | amzn-s3-demo-bucket1 | The bucket the request was for |
| 3 | Time | [06/Feb/2019:00:01:57 +0000] | When the request was received, in UTC, in square brackets |
| 4 | Remote IP | 192.0.2.3 | The apparent client address (a proxy or NAT shows up as itself) |
| 5 | Requester | 79a59df9... | Canonical ID or IAM ARN of the caller; - for anonymous (public) requests |
| 6 | Request ID | DD6CC733AEXAMPLE | The ID AWS Support asks for |
| 7 | Operation | REST.PUT.OBJECT | What was done: REST.GET.OBJECT, REST.HEAD.OBJECT, REST.DELETE.OBJECT, REST.GET.BUCKET (a listing), S3.EXPIRE.OBJECT (lifecycle) and so on |
| 8 | Key | s3-dg.pdf | The object key (URL-encoded), or - |
| 9 | Request-URI | "PUT /amzn-s3-demo-bucket1/s3-dg.pdf HTTP/1.1" | The HTTP request line, in quotes |
| 10 | HTTP status | 200 | The status returned |
| 11 | Error code | - | The S3 error code, such as AccessDenied or NoSuchKey |
| 12 | Bytes sent | - | Response bytes sent, without HTTP overhead (the number that drives data transfer cost) |
| 13 | Object size | 4406583 | Total size of the object |
| 14 | Total time | 41754 | Milliseconds from receiving the request to sending the last byte, including the client's network |
| 15 | Turn-around time | 28 | Milliseconds S3 itself spent processing; compare with total time to tell a slow S3 from a slow client |
| 16 | Referer | "-" | The HTTP Referer header, in quotes: the hotlinking clue |
| 17 | User-Agent | "S3Console/0.4" | The client software, in quotes |
| 18 | Version ID | - | The object version in the request |
| 19 | Host ID | 10S62Zv8... | The extended request ID (x-amz-id-2), also for AWS Support |
| 20 | Signature version | SigV4 | SigV2, SigV4, or - for unauthenticated requests |
| 21 | Cipher suite | ECDHE-RSA-AES128-SHA | The TLS cipher, or - for plain HTTP |
| 22 | Authentication type | AuthHeader | AuthHeader (signed header), QueryString (a presigned URL) or - |
| 23 | Host header | amzn-s3-demo-bucket1.s3.us-west-1.amazonaws.com | The endpoint the client connected to |
| 24 | TLS version | TLSV1.2 | The TLS version, useful before you enforce a minimum |
| 25 | Access point ARN | - | Set when the request came through an access point |
| 26 | ACL required | Yes | Yes if an ACL was needed to authorize the request: check this before you disable ACLs |
| 27 | Source Region | us-east-1 | The Region the request came from, when known |
Two parsing traps. First, three fields (Request-URI, Referer, User-Agent) are in quotes and contain spaces, so splitting a line on spaces is wrong: the record above splits into 30 pieces, not 27, and every field after the request line lands in the wrong column. Second, S3 may add new fields at the end of the line in the future, so a parser should read the fields it knows and ignore anything after them. The regular expression in the Athena table below does both. Tested on the five standard example records for this format, it matched all five, mapped them to 27 columns, and still matched when two unknown fields were added to the end of a line.
A short Python version of the same parser, for scripts:
import re
LINE = re.compile(r'([^ ]*) ([^ ]*) \[(.*?)\] ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ("[^"]*"|-) (-|[0-9]*) '
r'([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ("[^"]*"|-) ([^ ]*)'
r'(?: ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*))?.*$')
COLS = ("bucketowner bucket_name requestdatetime remoteip requester requestid operation key request_uri "
"httpstatus errorcode bytessent objectsize totaltime turnaroundtime referrer useragent versionid "
"hostid sigv ciphersuite authtype endpoint tlsversion accesspointarn aclrequired sourceregion").split()
def parse(line):
m = LINE.match(line)
return dict(zip(COLS, m.groups())) if m else None
On the five standard example records it prints:
REST.GET.VERSIONING status=200 error=- bytes=113 size=- total_ms=7
REST.GET.LOGGING_STATUS status=200 error=- bytes=242 size=- total_ms=11
REST.GET.BUCKETPOLICY status=404 error=NoSuchBucketPolicy bytes=297 size=- total_ms=38
REST.GET.VERSIONING status=200 error=- bytes=113 size=- total_ms=33
REST.PUT.OBJECT status=200 error=- bytes=- size=4406583 total_ms=41754
That third line is worth knowing about: opening a bucket in the S3 console reads its settings, and a bucket with no policy answers 404 NoSuchBucketPolicy. Those lines are normal. Do not chase every 404 in the logs before filtering out the console's own requests (user agent S3Console).
Query S3 access logs with Athena
For logs delivered to a bucket, Amazon Athena is the standard way to search them with SQL. Create a database, then a table that points at the log prefix and uses the regex above. For date-based keys, partition projection lets Athena read only the days you ask about, which keeps queries fast and cheap:
CREATE DATABASE IF NOT EXISTS s3_access_logs_db;
CREATE EXTERNAL TABLE s3_access_logs_db.jakes_photos_logs (
`bucketowner` STRING, `bucket_name` STRING, `requestdatetime` STRING, `remoteip` STRING,
`requester` STRING, `requestid` STRING, `operation` STRING, `key` STRING, `request_uri` STRING,
`httpstatus` STRING, `errorcode` STRING, `bytessent` BIGINT, `objectsize` BIGINT,
`totaltime` STRING, `turnaroundtime` STRING, `referrer` STRING, `useragent` STRING,
`versionid` STRING, `hostid` STRING, `sigv` STRING, `ciphersuite` STRING, `authtype` STRING,
`endpoint` STRING, `tlsversion` STRING, `accesspointarn` STRING, `aclrequired` STRING,
`sourceregion` STRING)
PARTITIONED BY (`timestamp` STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
'input.regex'='([^ ]*) ([^ ]*) \\[(.*?)\\] ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\"[^\"]*\"|-) (-|[0-9]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\"[^\"]*\"|-) ([^ ]*)(?: ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*))?.*$')
STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://jakes-shop-access-logs/access/111122223333/us-east-1/jakes-shop-photos/'
TBLPROPERTIES (
'projection.enabled'='true',
'projection.timestamp.format'='yyyy/MM/dd',
'projection.timestamp.interval'='1',
'projection.timestamp.interval.unit'='DAYS',
'projection.timestamp.range'='2026/01/01,NOW',
'projection.timestamp.type'='date',
'storage.location.template'='s3://jakes-shop-access-logs/access/111122223333/us-east-1/jakes-shop-photos/${timestamp}');
For simple-prefix logs, leave out PARTITIONED BY and the TBLPROPERTIES, and point LOCATION at the prefix; every query then reads every log file, so add a date filter on requestdatetime and expect it to get slower as logs pile up. Queries that answer the common questions:
-- Who downloads the most, and from which site? (the hotlinking question)
SELECT remoteip, referrer, COUNT(*) AS requests, SUM(bytessent) / 1e9 AS gb_sent
FROM s3_access_logs_db.jakes_photos_logs
WHERE timestamp BETWEEN '2026/10/01' AND '2026/10/22' AND operation = 'REST.GET.OBJECT'
GROUP BY remoteip, referrer ORDER BY gb_sent DESC LIMIT 20;
-- Who deleted this object?
SELECT requestdatetime, remoteip, requester, operation
FROM s3_access_logs_db.jakes_photos_logs
WHERE key = 'products/kettle-blue.jpg' AND operation LIKE '%DELETE%';
-- Every request that was denied, by identity and key
SELECT requester, key, COUNT(*) AS denied
FROM s3_access_logs_db.jakes_photos_logs
WHERE timestamp >= '2026/10/20' AND httpstatus = '403'
GROUP BY requester, key ORDER BY denied DESC;
-- 5xx errors with the IDs AWS Support asks for
SELECT requestdatetime, key, httpstatus, errorcode, requestid, hostid
FROM s3_access_logs_db.jakes_photos_logs
WHERE httpstatus LIKE '5%' AND timestamp BETWEEN '2026/10/20' AND '2026/10/22';
-- Requests that still need an ACL (check before disabling ACLs)
SELECT requester, operation, key, COUNT(*) AS n
FROM s3_access_logs_db.jakes_photos_logs
WHERE aclrequired = 'Yes' GROUP BY requester, operation, key ORDER BY n DESC;
The time column is text in the log's own format, so to filter by the hour, convert it: parse_datetime(requestdatetime, 'dd/MMM/yyyy:HH:mm:ss Z'). Athena charges by data scanned, which is why the partitioned table matters on a busy bucket. Query S3 with Athena step by step covers the Athena basics, workgroups and result locations.
Jake's question took the first query. Three IP addresses, one Referer from a site that sold the same kettles and had simply linked to Jake's photos, and 61 percent of the month's bytes sent. "So my bill is paying for their shop," Jake said. Ethan put the photos behind CloudFront with a referrer check the same afternoon.
Query them with CloudWatch Logs Insights
If you deliver to CloudWatch Logs, there is no table to create: choose the log group in Logs Insights and query the JSON fields by name. Queries that cover most needs:
# Errors by status, code and operation
filter http_status >= 400
| stats count(*) as error_count by http_status, error_code, operation
| sort error_count desc
# Who is being denied, and on which keys
filter http_status = 403
| stats count(*) as denied_count by key_name, remote_ip, requester
| sort denied_count desc
# Throttling (503 SlowDown) by key
filter http_status = 503
| stats count(*) as throttle_count by key_name, remote_ip
| sort throttle_count desc
# Traffic and errors by source IP
stats count(*) as total_requests, sum(http_status >= 400) as errors,
sum(bytes_sent_size) as bytes_transferred by remote_ip
| sort total_requests desc
# Latency per operation (p50 and p95, in ms)
filter operation like /REST\.(GET|PUT)\.OBJECT/
| stats avg(total_duration) as avg_ms, pct(total_duration, 50) as p50_ms,
pct(total_duration, 95) as p95_ms, count(*) as requests by operation
Logs Insights charges $0.005 per GB scanned (US East), so narrow the time range before you run a query over months of logs. To be told about problems instead of looking for them, a metric filter on the log group can count, for example, http_status = 403 and drive a CloudWatch alarm; why a metric filter never counts old logs covers the traps.
S3 access logs vs CloudTrail data events
CloudTrail can also record S3 object activity, through data events. The two overlap, but they are built for different jobs, and the differences decide which one you need:
| S3 server access logs | CloudTrail data events | |
|---|---|---|
| Completeness | Best-effort; records can be late, duplicated or missing | Designed as an audit record, with log file integrity validation |
| Delivery | Within a few hours | Data events about every 5 minutes |
| Price | Free to a bucket (storage only); CloudWatch ingestion rates otherwise | $0.10 per 100,000 events (US East), plus storage |
| Requests with bad credentials | Logged | Not logged (authentication failures and 301 redirects are skipped; AccessDenied is logged) |
| Lifecycle expirations, transitions, restores | Logged | Not logged |
| Only some objects (a prefix) | No, the whole bucket | Yes, with advanced event selectors |
| Object size, total time, turnaround, referrer | Yes | Yes |
| Format | Text lines (bucket) or JSON (CloudWatch Logs) | JSON |
A practical split: use CloudTrail data events for anything you must prove later (who read the customer files, who deleted an object, compliance), scoped to the sensitive buckets or prefixes to control cost; use server access logs for traffic analysis, cost hunting, latency, and the requests CloudTrail does not see, such as failed authentication and lifecycle actions. Many teams run both. What CloudTrail is explains trails, management events and data events from the start.
Ethan's rule for Jake: "If you'd ever show it to a lawyer, CloudTrail. If you just want to know who's hammering the bucket, access logs."
How much S3 server access logging costs
Turning logging on costs nothing. What costs money is where the logs go, how long you keep them, and how you query them. A log line is about 524 bytes on average (the five standard example records range from 508 to 564), so a million requests make roughly half a gigabyte of text. The CloudWatch Logs JSON form of the same record is larger: rebuilding the documented upload record with the CloudWatch field names gives about 1,000 bytes, roughly twice the text line. Prices below are from the AWS price list for US East (N. Virginia), October 2026:
| Requests to the bucket per month | Access logs to S3 (storage per month kept) | Access logs to CloudWatch Logs (ingestion) | CloudTrail data events |
|---|---|---|---|
| 1 million | about 0.5 GB: $0.01 | about 1 GB: $0.50 | $1.00 |
| 10 million | about 5.2 GB: $0.12 | about 10 GB: $5 | $10 |
| 100 million | about 52 GB: $1.20 | about 100 GB: $50 | $100 |
The S3 column uses S3 Standard at $0.023 per GB-month and grows with every month you keep; a 90-day lifecycle rule caps it at about three months' worth. Delivery into the bucket has no request or transfer charge, but reading the logs later is billed like any other S3 read. The CloudWatch column is ingestion at $0.50 per GB for the first 10 TB a month ($0.25 in the Infrequent Access log class); storage is extra at $0.03 per GB-month on the compressed data, and Logs Insights adds $0.005 per GB scanned. CloudTrail is $0.10 per 100,000 data events, plus the storage of its own log files. Your own numbers depend on the length of your keys, user agents and requester ARNs, so treat the table as a scale, not a quote.
So for Jake's bucket, about 4 million requests a month, bucket delivery costs pennies. The money is almost never in the logging; it is in what the logs reveal. S3 request pricing explains why a bucket with tiny storage can still have a big bill.
S3 access logs not showing up? The checklist
"I enabled logging and nothing is there" is the most common complaint, and most of the time the answer is the first item. Work through these in order:
- Wait a few hours. Delivery is periodic and best-effort; the first files usually appear within a few hours. Make a few requests to the source bucket so there is something to log.
- Check that logging is really on for the source bucket:
aws s3api get-bucket-logging --bucket SOURCEmust return aLoggingEnabledblock. An empty{}means off. - Look in the right place. With date-based keys, the files are deep under
prefix/ACCOUNT/REGION/BUCKET/YYYY/MM/DD/. With a prefix that has no trailing slash, they are at the top of the bucket with the prefix glued to the name. - Check the destination bucket policy. It must allow
s3:PutObjectforlogging.s3.amazonaws.comon the prefix, and itsaws:SourceArnandaws:SourceAccountmust match the source bucket and account. Search the policy for any"Effect": "Deny"that could block the logging service, such as a "deny everything not from our VPC" or "deny unencrypted uploads" statement. - Check the destination's default encryption. It must be SSE-S3. With SSE-KMS default encryption, log files can be written with a key you cannot use, or not usefully at all.
- Check Object Lock and default retention. A destination bucket with S3 Object Lock enabled cannot receive access logs, and it must not have a default retention period.
- Check Requester Pays. A Requester Pays bucket cannot be a log destination.
- Check AWS Organizations policies. A service control policy or resource control policy with a broad Deny on S3 can stop delivery.
- Give a destination change an hour. After switching destinations, logs can keep arriving at the old one for a while.
- Still nothing after a few hours? Turn on S3 request metrics in CloudWatch for the destination bucket and look at
4xxErrors, which can show delivery attempts being refused.
For CloudWatch Logs delivery, the checklist is shorter: the delivery must be ACTIVE in aws logs describe-deliveries, the log group must be in the source bucket's Region, the source bucket must have traffic, and if the log group uses a KMS key, the key policy must allow the CloudWatch Logs service principal.
S3 logging error messages and fixes
These are the messages returned when put-bucket-logging is refused, with the cause and the fix:
| Error | What happened | Fix |
|---|---|---|
Cross S3 location logging not allowed | The destination bucket is in a different Region | Use a destination in the source bucket's Region, or CloudWatch Logs delivery with an S3 or Firehose destination for cross-Region |
The owner for the bucket to be logged and the target bucket must be the same | The destination belongs to another account | Use a bucket in the same account, or CloudWatch Logs delivery for cross-account |
The target bucket for logging does not exist | A typo, or the bucket was not created yet | Create the destination first and check the spelling |
Target grants not allowed for bucket owner enforced buckets | The configuration includes TargetGrants, but the destination has ACLs disabled | Remove the grants and use the bucket policy |
AccessDenied on PutDeliverySource | The caller lacks a CloudWatch Logs delivery permission | Add logs:PutDeliverySource and s3:AllowVendedLogDeliveryForResource |
Note what is not in that table: the delivery problems from the checklist, such as a Deny statement, Object Lock, Requester Pays or SSE-KMS on the destination. They are not setup errors. Saving the setting can succeed, and then the logs never arrive, or with KMS they arrive encrypted with a key you cannot use. That is why the checklist above matters more than the error messages.
S3 access logging best practices
- Use a dedicated log bucket per Region and account, never the source bucket itself, with Block Public Access on and SSE-S3 encryption.
- Grant access with a bucket policy scoped by
aws:SourceArnandaws:SourceAccount, not with ACLs. - Choose date-based keys with EventTime when you will query the logs, so Athena can skip the days you do not need.
- Expire old logs with a lifecycle rule (30 to 365 days, as your needs require); see why lifecycle rules seem not to work if they do not appear to run.
- Do not treat access logs as an audit trail. For must-have records, add CloudTrail data events on the sensitive buckets.
- Prefer CloudWatch Logs delivery when you want alarms, KMS encryption, or logs from many accounts in one place.
- Read the
aclrequiredfield before switching a bucket to Bucket owner enforced: anyYesmeans a client still depends on an ACL. - Do not confuse settings.
access_logs.s3.enabledandconnection_logs.s3.enabledare load balancer attributes for ALB logs written to S3, not S3 server access logging.
Troubleshooting by symptom
I enabled logging an hour ago and the bucket is empty
Normal. Delivery is periodic and best-effort; give it a few hours and make some requests to the source bucket.
Logs arrive, but some requests are missing
Expected occasionally: access logs are best-effort. If every request must be recorded, use CloudTrail data events.
The same request appears twice
Rare but possible. De-duplicate on the request ID before counting.
The log bucket keeps growing
Add a lifecycle expiration rule, and check that you are not logging the log bucket into itself.
Log files are at the top of the bucket with odd names
The prefix has no trailing slash. Use access/ instead of access.
Athena returns empty columns or shifted values
The table splits on spaces or uses an old regex. Use the regex SerDe pattern above, which handles quoted fields and new trailing fields.
Athena returns nothing on a partitioned table
The LOCATION or storage.location.template does not match the real key path, including account ID, Region and bucket.
Lots of 404 NoSuchBucketPolicy lines
The S3 console reading the bucket's settings. Filter out the user agent S3Console.
Frequently asked questions
What are S3 server access logs?
Records of the requests made to an S3 bucket: requester, IP address, operation, key, status, error code, bytes and timing, delivered to a bucket or CloudWatch Logs.
How do I enable S3 server access logging?
Open the bucket, choose Properties, Server access logging, Edit, Enable, then choose a destination bucket in the same Region and account and save.
How do I enable S3 access logs with the AWS CLI?
Allow logging.s3.amazonaws.com in the destination bucket policy, then run aws s3api put-bucket-logging with a LoggingEnabled block naming the target bucket and prefix.
How long do S3 access logs take to appear?
Usually within a few hours. Delivery is best-effort, so a record can also arrive much later, twice, or not at all.
Why are my S3 access logs not showing up?
Usually too early. Otherwise check the destination policy, SSE-S3 default encryption, Object Lock, Requester Pays, Deny statements and SCPs.
Can the destination bucket be in another account or Region?
Not for delivery to a bucket: it must be the same account and Region. CloudWatch Logs delivery supports cross-account and cross-Region destinations.
Can I send S3 server access logs to CloudWatch Logs?
Yes, in every Region since October 21, 2026, using put-delivery-source with log type S3_SERVER_ACCESS_LOGS, put-delivery-destination and create-delivery.
What is the S3 access log format?
One request per line, space-separated fields, with the request line, referer and user agent in quotes and a dash for empty values. CloudWatch Logs delivery uses JSON.
How much does S3 server access logging cost?
Delivery to a bucket is free; you pay S3 storage, about 0.5 GB per million requests. CloudWatch Logs delivery costs $0.50 per GB ingested.
What is the difference between S3 access logs and CloudTrail?
Access logs are best-effort and cheap, and include failed authentication and lifecycle actions. CloudTrail data events are a reliable audit record at $0.10 per 100,000 events.
How do I query S3 access logs with Athena?
Create an external table with the RegexSerDe pattern shown in this guide, pointed at the log prefix, ideally with partition projection on the date path.
Should I log a bucket into itself?
No. Every log file written becomes a new logged request, creating logs about logs and a growing bill. Use a separate bucket.
Does S3 access logging work with SSE-KMS?
Not on the destination bucket: it must use SSE-S3. For KMS-encrypted access logs, deliver to CloudWatch Logs with a KMS key.
What does a dash mean in an S3 access log?
That field is unknown, unavailable or does not apply to that request, such as the key on a bucket-level request.
What is the S3 log delivery group?
The old ACL grantee used to let S3 write logs. On buckets with ACLs disabled, grant the logging.s3.amazonaws.com service principal in a bucket policy instead.
Can I enable access logs for only one prefix?
No. Server access logging covers the whole bucket. CloudTrail data events can be limited to prefixes with advanced event selectors.
How do I set up S3 server access logs in Terraform?
Use the aws_s3_bucket_logging resource with target_bucket, target_prefix and target_object_key_format, plus an aws_s3_bucket_policy for the logging service.
Do S3 access logs show who deleted an object?
Usually: look for DELETE operations on the key with the requester and IP. For a guaranteed record, use CloudTrail data events.
How do I turn off S3 server access logging?
In the console, set Server access logging to Disable, or run aws s3api put-bucket-logging with an empty status, '{}'.
Does CloudTrail record authentication failures for S3?
No. CloudTrail skips requests that fail authentication, such as invalid credentials. S3 server access logs do record them, which is why audits often ask for both.
Why does my access log bucket grow by itself?
Usually the log bucket is logging into itself, or another logging bucket delivers into it, so every log file creates more log records. Send logs to a separate bucket that has logging off, and expire old logs with a lifecycle rule.
How do I set up S3 server access logging for multiple buckets?
Point every source bucket at one log bucket in the same Region and account, with a date-based prefix per source, and list every source bucket ARN in the log bucket's policy. For many accounts or Regions, deliver to CloudWatch Logs instead.
Is access_logs.s3.enabled an S3 setting?
No. It is a load balancer attribute that turns on ALB access logs written to S3, a different feature from S3 server access logging.
For readers preparing for an AWS certification
We have been covering AWS certifications alongside our fixes, so here is a short extra for readers studying for one. If you only came for the fix, you already have it; feel free to skip this part.
Exams that test this topic: Solutions Architect – Associate (SAA-C03) • CloudOps Engineer – Associate (SOA-C03) • Security – Specialty (SCS-C03) • Data Engineer – Associate.
Here are the exam-style questions this topic tends to raise, with the reasoning you need to answer them:
1. A security team needs a reliable, tamper-evident record of every read of objects in one sensitive S3 bucket. Server access logging is already on. What should they add?
CloudTrail data events for that bucket (scoped with advanced event selectors), with log file integrity validation; server access logs are best-effort and not a complete record.
2. S3 server access logging was enabled, but no log files arrive after several hours. The destination bucket uses SSE-KMS default encryption. What is the fix?
Change the destination bucket's default encryption to SSE-S3; access log delivery to a bucket supports only SSE-S3.
3. A company wants S3 access logs from 40 accounts collected in one central account and encrypted with a customer managed KMS key. Which delivery option fits?
Deliver server access logs to CloudWatch Logs (vended log delivery), which supports cross-account and cross-Region destinations and KMS encryption; bucket delivery is same-account, same-Region, SSE-S3 only.
4. Which requests appear in S3 server access logs but not in CloudTrail data events?
Requests that fail authentication, 301 redirects, and lifecycle actions such as expirations, transitions and restores.
5. A destination bucket uses the Bucket owner enforced Object Ownership setting. How do you grant S3 permission to deliver access logs?
Add a bucket policy allowing s3:PutObject for the logging.s3.amazonaws.com service principal, scoped with aws:SourceArn and aws:SourceAccount; ACLs and target grants are not supported.
6. Analysts query months of S3 access logs with Athena and every query scans all files. What change reduces cost and time?
Use date-based (partitioned) log object keys with EventTime and an Athena table with partition projection, then filter on the date partition.
Exam codes and dates change often. Our AWS certification changes page keeps track of what is current.
Jake now has a habit: on the first of each month, Ethan runs the "who downloads the most" query and sends him the top five. Last month the top five were Jake's own website, CloudFront, and three of his customers. "Boring," Jake said. Ethan told him boring was the goal.
📌 If you keep one line from this page
Log into a separate bucket with a bucket policy for logging.s3.amazonaws.com and SSE-S3, wait a few hours, and remember the logs are best-effort: use CloudTrail data events for anything you must prove.
Facts from the Amazon S3 User Guide and the AWS price list, October 2026; Terraform example applied with Terraform 1.16.5 and AWS provider 6.70.0; log parser checked against the documented example records.
Revision note. Written October 23, 2026, two days after S3 server access log delivery to CloudWatch Logs reached every AWS Region. "Trust, but verify," the proverb says; with access logs, trust them for patterns and verify anything important with CloudTrail.