Is S3 a Data Lake? Query S3 with Athena Step by Step (Access Logs, UNLOAD, S3 Tables)

Logeshwaran
—

Is S3 a data lake? Not on its own. Amazon S3 is the storage layer of a data lake on AWS: the place where all the raw files live, cheaply and almost without limit. It becomes a data lake when you add three more things: a catalog that records what is in the files (the AWS Glue Data Catalog), a query engine that can read them (most often Amazon Athena, which runs SQL directly on S3), and permissions that decide who sees what (IAM, and AWS Lake Formation for finer control). Here is the part that surprises people: you never load anything into Athena. It reads your files where they already sit in S3 and charges for the data it reads, $5 per terabyte, so the same question can cost $15 or $1.25 depending only on how the files are stored. Below, we start with the basics, then query S3 with Athena step by step: a CSV table, your S3 access logs ("who deleted this file?"), exporting results with UNLOAD, cheaper Parquet files, and the newer S3 Tables.

Jake's phone repair shop had been quietly filling an S3 bucket for three years: nightly CSV exports from the shop's point-of-sale system, scanned invoices, and backups of the website. Two questions finally made him look inside. His accountant wanted revenue by phone model for the last three years, and was going to charge $60 an hour to stitch 36 monthly spreadsheets together. And a customer insisted an invoice PDF had been "deleted by the shop" during a dispute, and Jake had no idea how to prove what had happened. A forum post said "build a data warehouse." Ethan said, "You already have most of a data lake. You just haven't introduced it to Athena."

⚡ Quick Answer

• Is S3 a data lake? It is the storage part of one. The four pieces.

• Query a CSV in S3 with SQL → Athena, step by step.

• Search S3 access logs ("who deleted it?") → the access-log table.

• Export results to S3 as JSON or Parquet → UNLOAD.

Want to pay less per query? Convert to Parquet.

The basics: database, warehouse, data lake

Three kinds of "place to keep data" get mixed up constantly. A shop analogy keeps them apart.

  • A database is the till. It records each sale as it happens, quickly and reliably, for the app that runs the business. It is great at "save this order" and "look up this customer," and poor at "summarize three years."
  • A data warehouse is the accountant's ledger. Data is cleaned, shaped and loaded into it on a schedule, so reports run fast. You decide the structure before the data arrives.
  • A data lake is the stock room. Everything goes in as it is: CSV exports, logs, JSON, images, PDFs. You decide how to read it when you ask a question. That idea has a name, schema-on-read, and it is what makes a lake cheap and flexible.

Two more words matter for everything below. An object is a file in S3, and a prefix is the folder-like part of its name, such as raw/repairs/2026/. And file format matters more than people expect: CSV and JSON are easy to read for humans, while Parquet is a compressed, column-by-column format that query engines can read a fraction of.

"So the stock room already exists," Jake said. "It's just a mess."

"A stock room with a good list on the door isn't a mess," Ethan said. "Athena is the person who reads the list and fetches exactly the box you asked for. You pay them by how much they have to carry."

Is S3 a data lake? The four pieces

So, data lake vs S3: S3 is one piece, the most important one. A working data lake on AWS has four:

PieceAWS serviceIts job
StorageAmazon S3 (general purpose buckets, or S3 Tables)Holds the files, cheaply and durably
CatalogAWS Glue Data CatalogRecords tables: which files, which columns, which types
Query enginesAmazon Athena, Redshift, EMR, SageMakerRead the files to answer questions
PermissionsIAM, AWS Lake FormationDecide who can read which tables, columns and rows

A bucket full of files without a catalog is just a bucket; nobody can query it with SQL. The same bucket with a few catalog entries becomes a data lake that Athena, Redshift and other tools can all read, without copying the data. That "one copy, many tools" idea is the real reason companies build data lakes on S3: storage is cheap, nothing is locked into one product, and new tools can be pointed at old data.

What an S3 data lake architecture looks like

Most S3 data lakes, from a small shop to a large company, organize files into zones by prefix, so raw data is never lost and cleaned data is easy to find:

s3://jakes-shop-data/
  raw/        files exactly as they arrived (CSV exports, logs, JSON)
  clean/      fixed types, removed duplicates, same format
  curated/    Parquet, partitioned, ready for reports
  results/    query outputs and exports (optional)

The Glue Data Catalog holds a table for each dataset you want to query, pointing at a prefix. Athena runs SQL on those tables. The same tables can later be used by Redshift for heavy reporting, by Amazon QuickSight for dashboards, or by SageMaker for machine learning, which is exactly the "one copy, many tools" promise. You do not need all of it on day one. Jake started with one table over his raw/ folder.

Step 1: query a CSV file in S3 with Athena

Here is the shortest path from "files in a bucket" to "SQL results," using Jake's sales exports. His CSV files look like this, one per month, all in the same prefix:

repair_id,repair_date,phone_model,repair_type,amount,came_back
R-10231,2026-09-02,Galaxy S24,screen,129.00,no
R-10232,2026-09-02,iPhone 15,battery,79.00,yes
  1. Put the files under one prefix, for example s3://jakes-shop-data/raw/repairs/. Athena reads every file under the location you give it, so keep only files with the same columns there.
  2. Open the Athena console in the same Region as the bucket, and open the Query editor.
  3. Choose where results go. Since June 2025, Athena workgroups can use managed query results, where Athena stores and cleans up results for you at no extra charge. Otherwise, set an S3 location for results in the workgroup settings.
  4. Create a database (a named group of tables in the Glue Data Catalog): CREATE DATABASE shop;
  5. Create a table that describes the files:
CREATE EXTERNAL TABLE shop.repairs (
  repair_id   string,
  repair_date date,
  phone_model string,
  repair_type string,
  amount      double,
  came_back   string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 's3://jakes-shop-data/raw/repairs/'
TBLPROPERTIES ('skip.header.line.count'='1');

Nothing is copied. This statement only writes a description into the catalog: "the files under this prefix have these columns." Then ask a real question:

SELECT phone_model,
       count(*)              AS repairs,
       round(sum(amount), 2) AS revenue
FROM shop.repairs
WHERE repair_date >= DATE '2024-01-01'
GROUP BY phone_model
ORDER BY revenue DESC
LIMIT 10;

That one query replaced the accountant's 36-spreadsheet job. Over Jake's three years of exports, about 2 GB of CSV, it scanned the whole table and cost about one cent.

Tips for CSV tables that save an hour of confusion

  • Point LOCATION at a prefix, not a file. A LOCATION ending in .csv does not work as people expect; use the folder.
  • Dates must be in YYYY-MM-DD to load into a date column with this format. If yours look like 02/09/2026, declare the column as string and convert it in the query.
  • Commas inside quoted values ("Smith, John") break a simple comma-separated table. Use ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' instead, which understands quotes but reads every column as text.
  • Let a crawler do it if you do not want to write the table yourself. An AWS Glue crawler can look at the files and create the table automatically; it is billed for the time it runs, and the Glue Data Catalog itself is free for the first million objects stored.

Querying JSON files in S3

Many apps export JSON rather than CSV, and Athena reads it just as easily, with one condition that trips almost everyone: each record must be on its own line. A file that is one big pretty-printed JSON array will not work; a file with one JSON object per line (often called JSON Lines) will. The table uses a JSON SerDe, and the column names match the JSON keys:

CREATE EXTERNAL TABLE shop.web_orders (
  order_id    string,
  created_at  string,
  customer    struct<name:string, city:string>,
  items       array<struct<sku:string, qty:int, price:double>>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://jakes-shop-data/raw/web-orders/';

Nested objects become struct columns, read with a dot (customer.city), and lists become array columns, which you flatten with CROSS JOIN UNNEST(items) when you need one row per item. If some records are missing a key, Athena simply returns an empty value for that column, which makes JSON forgiving for data whose shape changes over time.

Let a Glue crawler build the table for you

Writing table definitions by hand is fine for a few datasets. For many, or for data whose columns change, an AWS Glue crawler can inspect the files and create the tables itself:

  1. Open the AWS Glue console and choose Crawlers → Create crawler.
  2. Add a data source: the S3 path of one dataset, such as s3://jakes-shop-data/raw/repairs/.
  3. Choose or create an IAM role that can read that path; the console offers to create one with the right Glue permissions.
  4. Pick the target database (for example shop) and leave the schedule as On demand while you learn.
  5. Run the crawler. When it finishes, the new table appears in Athena under that database.

Two cautions. A crawler pointed at a folder with mixed file layouts may create several odd tables instead of one; give each dataset its own prefix. And a crawler on a frequent schedule bills every time it runs, so for stable data, run it once, or only when the layout changes.

Query S3 server access logs with Athena: "who deleted this file?"

This was Jake's second question, and it is one of the most searched Athena uses: reading S3's own access logs with SQL. S3 can write a log line for every request made to a bucket: who asked, from which IP address, for which object, which operation, and whether it succeeded.

⚠️ The catch: access logs only record requests made after logging is turned on, and delivery is best-effort, usually within a few hours. They cannot answer questions about the past. If you might ever need to prove what happened to your files, turn logging on now. For security investigations, AWS recommends CloudTrail data events, which are easier to set up and record more detail; server access logs are the low-cost option.

Turn on server access logging:

  1. Create a separate bucket for logs, such as jakes-shop-logs, in the same Region. Never log a bucket into itself.
  2. Open the source bucket, go to Properties → Server access logging → Edit, and choose Enable.
  3. Pick the log bucket and a prefix, and choose the date-based log object key format, which organizes logs by account, Region, bucket and date. The table below is built for that layout.
  4. Save. Logs start arriving within a few hours.

Create the table. This is AWS's recommended definition for date-based access logs. It uses partition projection, so Athena works out the date folders by itself and you never have to load partitions. Replace the database, table and the two S3 paths with yours:

CREATE DATABASE s3_access_logs_db;

CREATE EXTERNAL TABLE s3_access_logs_db.shop_bucket_logs(
  `bucketowner` STRING,
  `bucket_name` STRING,
  `requestdatetime` STRING,
  `remoteip` STRING,
  `requester` STRING,
  `requestid` STRING,
  `operation` STRING,
  `key` STRING,
  `request_uri` STRING,
  `httpstatus` STRING,
  `errorcode` STRING,
  `bytessent` BIGINT,
  `objectsize` BIGINT,
  `totaltime` STRING,
  `turnaroundtime` STRING,
  `referrer` STRING,
  `useragent` STRING,
  `versionid` STRING,
  `hostid` STRING,
  `sigv` STRING,
  `ciphersuite` STRING,
  `authtype` STRING,
  `endpoint` STRING,
  `tlsversion` STRING,
  `accesspointarn` STRING,
  `aclrequired` STRING,
  `sourceregion` STRING)
PARTITIONED BY (
  `timestamp` string)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
  'input.regex'='([^ ]*) ([^ ]*) \\[(.*?)\\] ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\"[^\"]*\"|-) (-|[0-9]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\"[^\"]*\"|-) ([^ ]*)(?: ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*))?.*$')
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  's3://jakes-shop-logs/logs/111122223333/us-east-1/jakes-shop-data/'
TBLPROPERTIES (
  'projection.enabled'='true',
  'projection.timestamp.format'='yyyy/MM/dd',
  'projection.timestamp.interval'='1',
  'projection.timestamp.interval.unit'='DAYS',
  'projection.timestamp.range'='2024/01/01,NOW',
  'projection.timestamp.type'='date',
  'storage.location.template'='s3://jakes-shop-logs/logs/111122223333/us-east-1/jakes-shop-data/${timestamp}');

The path follows the date-based layout: your log prefix, then the account ID, Region and the source bucket's name. Copy it exactly from what you see in the log bucket. Then preview the table to check that columns are filled in.

Queries that answer real questions:

-- Who deleted a file, and when, from where
SELECT requestdatetime, remoteip, requester, key
FROM s3_access_logs_db.shop_bucket_logs
WHERE key = 'invoices/2026/INV-4471.pdf'
  AND operation LIKE '%DELETE%';

-- Every request refused with 403 Access Denied this week
SELECT requestdatetime, remoteip, requester, operation, key, errorcode
FROM s3_access_logs_db.shop_bucket_logs
WHERE httpstatus = '403'
  AND timestamp BETWEEN '2026/09/28' AND '2026/10/05';

-- Which IP addresses download the most data
SELECT remoteip, count(*) AS requests, sum(bytessent) AS bytes
FROM s3_access_logs_db.shop_bucket_logs
WHERE operation = 'REST.GET.OBJECT'
  AND timestamp >= '2026/09/01'
GROUP BY remoteip
ORDER BY bytes DESC
LIMIT 20;

Always filter on timestamp, the date partition, as in the last two queries. It tells Athena to read only those days' folders, which keeps the scan, and the bill, small. Jake's invoice turned out never to have been deleted at all: the log showed it had been renamed by his own invoicing app, and the customer had been looking at an old link. A two-minute query settled a week-long argument.

Partition projection, in plain English

The access-log table above uses a feature called partition projection, and it is worth understanding because it solves the most annoying Athena problem: partitions that "exist" in S3 but not in the catalog. Normally, every date folder has to be registered as a partition before Athena will read it, which means running a command or a crawler every day. With projection, you describe the pattern instead (dates in yyyy/MM/dd folders, from 2024 until now), and Athena calculates which folders to read from the WHERE clause of each query. New days work automatically. For any data that arrives in date folders, projection is usually the right choice.

CloudTrail logs: the audit-grade version

If you need stronger evidence than server access logs, AWS CloudTrail records API activity with the full identity of the caller, and with data events turned on for a bucket, it records object-level actions such as reads and deletes too. CloudTrail delivers its logs to S3 as JSON, and Athena can query them like any other data. The CloudTrail console can create the matching Athena table for you from its event history page, which saves writing a long definition by hand. A query such as "every DeleteObject on this bucket in the last week, with the IAM user who made it" then takes seconds. Data events cost extra per event, so most people enable them only for the buckets that matter, such as invoices or customer records. Our CloudTrail guide explains what it records and what it costs.

Step 2: make queries cheaper and faster with Parquet and partitions

Athena charges $5 per terabyte scanned, so the cheapest query is the one that reads the least. Two changes do most of the work:

  • Parquet instead of CSV. Parquet stores data column by column and compresses it. A query that needs three columns reads only those three, compressed. In AWS's own example, a 3 TB uncompressed text dataset becomes about 0.25 TB of compressed Parquet, and the same query drops from $15 to $1.25.
  • Partitions. Splitting files into folders by date (or another column you filter on) lets Athena skip whole folders. A query for one month reads one month.

You can convert inside Athena with one CREATE TABLE AS SELECT (CTAS) statement, which reads the CSV table and writes a new, partitioned Parquet copy:

CREATE TABLE shop.repairs_parquet
WITH (
  format            = 'PARQUET',
  write_compression = 'SNAPPY',
  external_location = 's3://jakes-shop-data/curated/repairs/',
  partitioned_by    = ARRAY['repair_year']
) AS
SELECT repair_id, repair_date, phone_model, repair_type, amount, came_back,
       year(repair_date) AS repair_year
FROM shop.repairs;

Two rules for CTAS: the partition column must be the last one in the SELECT list, and the external_location folder must be empty. Afterward, query shop.repairs_parquet with WHERE repair_year = 2026 and Athena reads only that year. For the deeper cost math, including the 10 MB minimum per query and how partition design changes bills, see our Athena costs guide.

Export query results to S3 with UNLOAD

Athena's normal results are CSV. When another program needs the output in a different format, UNLOAD writes the results of a SELECT straight to S3 as Parquet, ORC, Avro, JSON or text, without creating a table:

UNLOAD (
  SELECT phone_model, count(*) AS repairs, sum(amount) AS revenue
  FROM shop.repairs_parquet
  WHERE repair_year = 2026
  GROUP BY phone_model
)
TO 's3://jakes-shop-data/exports/2026-by-model/'
WITH (format = 'JSON');

Things to know before you rely on it:

  • The destination must be empty. UNLOAD checks first and refuses to overwrite existing data (except when writing partitions). Use a new folder each time, or empty it first.
  • Results are split into several files written in parallel. If your SELECT has an ORDER BY, each file is sorted, but the files are not in order relative to each other.
  • Compression is set with compression = 'SNAPPY' (or ZSTD and others) for Parquet and ORC; JSON and text are written gzip-compressed.
  • Partitioned output uses partitioned_by = ARRAY['column'], with the partition column last in the SELECT, and allows up to 100 partitions.

UNLOAD vs CTAS: both write files to S3. CTAS also creates a table in the catalog, ready to query again; UNLOAD just writes files for something else to pick up. Jake uses UNLOAD to hand his accountant a tidy monthly JSON file that her bookkeeping tool imports directly.

S3 Tables and Athena

S3 Tables are a newer kind of S3 storage built for analytics: table buckets, which store data as Apache Iceberg tables and maintain them automatically. Iceberg is an open table format that brings database-like features to files in S3: updating and deleting rows, adding columns safely, and reading the table as it was at an earlier point in time. With ordinary files, you manage all of that yourself.

To query S3 Tables from Athena:

  1. In the S3 console, open Table buckets and create one, leaving Enable integration checked. That connects table buckets in the Region to the AWS Glue Data Catalog through a catalog named s3tablescatalog.
  2. Create a namespace (a group of tables) and a table, using all-lowercase names for the table and its columns.
  3. In the Athena query editor, choose the s3tablescatalog catalog for your table bucket, pick the namespace as the database, and query the table like any other.

That lowercase rule is not cosmetic: a table or column name with capital letters is not visible to Athena through the catalog, and a query fails with "Unsupported Federation Resource - Invalid table or column names." The integration also uses the Glue Data Catalog, which can add small Glue charges, and Athena's normal per-scan pricing applies.

When to use which: plain files in a general purpose bucket are perfect for data that only grows, such as logs, exports and archives. S3 Tables suit tables that change, where rows are corrected, updated or deleted, or where many writers add data at once. Jake's access logs and monthly exports stay as files; if he ever keeps a live customer table in the lake, S3 Tables is where it would go.

S3 Select vs Athena

Older tutorials suggest S3 Select for running a quick SQL filter on a single object. Do not plan around it: AWS closed S3 Select to new customers on July 25, 2024. Accounts that used it before then can keep using it; new accounts cannot. AWS points people to Athena instead, and for a single small file, simply downloading it and filtering on your own computer is often the easiest answer. Athena handles one file just as well as ten thousand, and costs a tiny fraction of a cent for a small one.

What an S3 data lake with Athena costs

For small and medium data, the costs are pleasantly low, and they come from four meters:

MeterHow it works
Athena queries$5 per TB scanned, rounded up to the nearest MB, with a 10 MB minimum per query. Failed queries and statements such as CREATE TABLE are not charged.
S3 storage and requestsNormal S3 prices for the files, plus small request charges when Athena reads them
Glue Data Catalog and crawlersThe first million catalog objects are free; crawlers bill for the time they run
Query resultsFree with managed query results; otherwise normal S3 storage for the result files

Jake's numbers: 2 GB of CSV, queried perhaps 20 times a month, comes to well under a dollar in Athena charges even before converting to Parquet. Teams that run Athena all day can instead buy capacity reservations at $0.30 per DPU-hour, which makes costs predictable at high volume. Whatever your size, set up workgroups with a per-query data limit, so one accidental SELECT * on a huge table cannot run up a bill.

Athena workgroups: the safety net for your bill

A workgroup is a named group of Athena settings that queries run under. Every account starts with one called primary, and creating your own takes a minute in the Athena console under Workgroups. Each workgroup can set where results go (or use managed results), whether results are encrypted, and, most usefully, a per-query data usage control: a limit on how much data a single query may scan before Athena cancels it. Set it to, say, 10 GB for a small team, and a mistaken query on a huge table stops at a few cents instead of running to dollars. Workgroups can also carry tags, so each team's queries show up separately in Cost Explorer, and you can give people permission to use only their own workgroup. For a one-person setup, one workgroup with a sensible per-query limit is the simplest insurance you can buy, and it costs nothing.

A worked monthly example

Here is Jake's real pattern, priced. His 2 GB of CSV exports, queried about 20 times a month with full scans, is 40 GB, or 0.04 TB, times $5: about $0.20 a month. After converting to partitioned Parquet, a typical query that reads one year of three columns scans a few megabytes, so most queries hit the 10 MB minimum and the monthly Athena charge drops to a few cents. Storage for the files is a separate S3 line of well under a dollar at this size. The access-log queries add a few cents more, because each one reads only the days it filters on. The total is less than a single coffee, and less than one minute of the accountant's time.

Querying Athena from Python, Excel or a BI tool

The console is only one way in. Athena offers JDBC and ODBC drivers, so desktop tools that can connect to a database, including Excel through ODBC and most business-intelligence tools, can run Athena queries and pull results directly. From Python, the AWS SDK (boto3) can start a query, wait for it, and fetch the results, and the open-source AWS SDK for pandas wraps all of that into a single call that returns a DataFrame. Whatever the tool, the same rules apply: the identity running the query needs permission to read the data and the catalog, and every query is billed by the data it scans, so filters and partitions matter just as much from code as from the console.

The data lake mistakes that cost the most

  • Mixing layouts in one prefix. A table reads every file under its location. One file with an extra column, or an old export format, produces errors or silently wrong results. Give each dataset its own folder.
  • Thousands of tiny files. Athena spends time opening every file, so a million 1 KB files query far slower than a few hundred large ones holding the same data. When a process writes small files constantly, compact them into larger files (tens to hundreds of megabytes each) on a schedule, for example with a CTAS or UNLOAD job.
  • SELECT * on big tables. It reads every column and, without a partition filter, every file. Name the columns you need and filter on the partition.
  • Raw data you can overwrite. Keep raw files read-only, and turn on S3 Versioning so a bad job can be undone.
  • Public buckets. A data lake bucket should never be public. Keep S3 Block Public Access on, and share results through permissions, not public links.

Common Athena errors on S3, and what they mean

  • "Zero records returned" when you know there is data: LOCATION points at a file instead of a folder, the prefix is wrong (check for a missing or extra slash), or, for partitioned tables without projection, the partitions were never loaded. Run MSCK REPAIR TABLE tablename or add partitions, or use partition projection as in the access-log table.
  • HIVE_BAD_DATA: a value does not match its column type, such as text in a number column or a date in the wrong format. Declare the column as string and convert it in the query, or clean the source file.
  • Access Denied: the person or role running the query needs permission to read the source bucket and to write to the results location (unless you use managed results). If the bucket is encrypted with a KMS key, they need permission on that key too. Our S3 AccessDenied guide walks through each layer.
  • HIVE_PARTITION_SCHEMA_MISMATCH: a partition's files have different columns from the table, usually because the export format changed one month. Keep each table's files consistent, or create the table again from the newer layout.
  • Query exhausted resources: a very large ORDER BY or a join across huge tables. Add a LIMIT, filter on partitions first, or break the query into steps.

Permissions: IAM first, Lake Formation when it grows

For one person or a small team, IAM permissions are enough: allow reading the data prefixes, writing results, and using the Glue catalog and Athena. As more people share the lake, AWS Lake Formation adds finer control on top of the catalog: grant one team access to certain tables, hide sensitive columns such as phone numbers, or limit rows by region, without creating a separate copy of the data for each audience. It is the difference between handing out keys to the whole stock room and giving each person access to the shelves they need.

When to move beyond Athena: Redshift, QuickSight and EMR

Athena is the easiest way into an S3 data lake, but it is not the only tool that reads it, and that is the point of the lake. The same catalog tables can be used by:

  • Amazon Redshift, the data warehouse, which can query S3 tables directly alongside its own tables. Teams move heavy, repeated reporting there when they need consistently fast dashboards for many users.
  • Amazon QuickSight, AWS's dashboard tool, which can use Athena as a data source, so a chart of "revenue by phone model" updates from the lake without anyone exporting a spreadsheet.
  • Amazon EMR and AWS Glue jobs, which run Apache Spark for large transformations: cleaning billions of rows, joining big datasets, or building the curated Parquet zone on a schedule.
  • SageMaker AI, which can train machine-learning models on data read from the lake.

A sensible rule: start with Athena for questions and ad-hoc reports. Add QuickSight when people want dashboards. Add Glue or EMR jobs when data preparation becomes a regular task. Consider Redshift when the same heavy reports run all day for many users. None of these steps requires moving the data out of S3.

Is a data lake worth it for a small business?

Honestly, sometimes not. If your data fits comfortably in one spreadsheet and nobody asks questions across years of it, a spreadsheet is the right tool, and adding AWS services adds nothing but complexity.

An S3 data lake starts paying off when three things are true. Files accumulate on a schedule (monthly exports, daily logs, nightly backups), so they already live in S3. Someone asks questions that cross many of those files, which is painful by hand. And the answers are worth more than an hour of setup. Jake met all three: three years of exports, an accountant charging by the hour, and a dispute that needed evidence. His total setup took an afternoon, his monthly Athena bill is under a dollar, and the skills carry over directly to larger businesses, which is part of why "S3 plus Athena" shows up so often in AWS certification exams and job descriptions.

For IT admins: running a shared S3 data lake

  • Separate raw from curated, and protect raw data with S3 Versioning and, where required, Object Lock, so nothing is lost to a bad job or a mistake.
  • Encrypt everything with SSE-S3 or SSE-KMS, and make sure query roles can use the KMS key.
  • Use Athena workgroups per team, with per-query scan limits and cost allocation tags, so each department sees its own query bill in Cost Explorer.
  • Turn on CloudTrail data events for sensitive buckets, for audit-grade records of who read and deleted what.
  • Govern with Lake Formation once more than a few teams share tables, rather than growing ever more complex bucket policies.
  • Lifecycle old raw data to cheaper S3 storage classes once it has been converted to Parquet.

S3 data lake and Athena: frequently asked questions

Is S3 a data lake?

S3 is the storage layer of a data lake. It becomes a data lake when you add a catalog such as the AWS Glue Data Catalog, a query engine such as Athena, and permissions.

What is the difference between a data lake and S3?

S3 stores files. A data lake is the whole system around those files: storage, a catalog describing them, engines that query them, and access control.

How do I use Athena to query S3?

Create a database and an external table in Athena that points to an S3 prefix and describes the columns, then run SQL on the table. Athena reads the files where they are.

How do I create an Athena table from a CSV in S3?

Use CREATE EXTERNAL TABLE with the column names and types, ROW FORMAT DELIMITED FIELDS TERMINATED BY ',', the S3 folder as LOCATION, and skip.header.line.count set to 1 if the file has a header.

How do I query S3 access logs with Athena?

Enable server access logging with the date-based format, create the access-log table with the RegexSerDe and partition projection, then query it with SQL, filtering on the timestamp partition.

Can Athena show who deleted a file in S3?

Yes, if server access logging or CloudTrail data events were on before the deletion. Query the log table for the object key with an operation like DELETE.

What does Athena UNLOAD do?

UNLOAD writes the results of a SELECT query to S3 as Parquet, ORC, Avro, JSON or text, without creating a table. The destination folder must be empty.

What is the difference between UNLOAD and CTAS in Athena?

Both write query results to S3. CTAS also creates a table in the catalog that you can query; UNLOAD only writes files for other tools to use.

How much does Athena cost?

$5 per terabyte scanned, rounded up to the nearest megabyte, with a 10 MB minimum per query. Failed queries and table definition statements are not charged. S3 storage is billed separately.

Why is my Athena query returning zero results?

Usually LOCATION points at a file instead of a folder, the S3 path is wrong, or partitions were not loaded. Fix the path, or run MSCK REPAIR TABLE, or use partition projection.

Do I need an S3 bucket for Athena query results?

Not anymore. Since June 2025, Athena managed query results can store and clean up results for you at no extra charge. You can still choose your own S3 location.

What are S3 Tables?

Table buckets that store data as Apache Iceberg tables and maintain them automatically. They suit tables that change, and Athena queries them through the s3tablescatalog catalog.

How do I query S3 Tables with Athena?

Create the table bucket with integration enabled, use lowercase table and column names, then in Athena choose the s3tablescatalog catalog for your bucket and query the table.

Is S3 Select still available?

Only for accounts that used it before July 25, 2024. AWS closed it to new customers and recommends Athena instead.

Why convert CSV to Parquet for Athena?

Parquet is compressed and stored by column, so Athena reads far less data. In AWS's example, the same query dropped from $15 to $1.25 after converting to Parquet.

What is the Glue Data Catalog?

The central list of databases and tables for your data lake. It records where each table's files are and what columns they have, so Athena and other services can read them.

Do I need Lake Formation for a data lake?

Not to start. IAM is enough for a small team. Lake Formation helps when many teams share tables and you need table, column or row level permissions.

What is partition projection in Athena?

A table setting that describes how partitions are laid out, for example dates in yyyy/MM/dd folders, so Athena works out which folders to read from your WHERE clause. New partitions work automatically, with no MSCK REPAIR TABLE or crawler runs.

Can Athena query JSON files in S3?

Yes, using a JSON SerDe such as org.openx.data.jsonserde.JsonSerDe. Each JSON record must be on its own line; one large pretty-printed array will not load. Nested objects become struct columns and lists become array columns.

How do I limit how much an Athena query can cost?

Create a workgroup and set a per-query data usage control. Athena cancels any query that tries to scan more than the limit, so a mistake costs cents instead of dollars.

What is an S3 data lake architecture?

Raw, clean and curated zones as S3 prefixes, a Glue Data Catalog describing the tables, query engines such as Athena and Redshift reading them, and IAM or Lake Formation controlling access.

Jake never built a data warehouse. His three years of exports now answer the accountant's questions for about a cent a query, the monthly report writes itself to a JSON file, and his access logs settled a customer dispute in two minutes. If your S3 bucket has been quietly filling for years and the word "data lake" sounds like a project you cannot afford, take heart: you may already be most of the way there, one table definition at a time.

📌 If you keep one line from this page

S3 stores the files, the catalog describes them, and Athena reads them where they sit.

You pay for what Athena reads, so store it as Parquet and filter by partition.

Revision note. Written October 5, 2026. If your bucket has felt like a cluttered stock room, it was only waiting for a list on the door; may your first query be the one that saves you a weekend.

Related