How to Upload a Folder (or Files) to S3 on Windows, Mac and Linux: Console, aws s3 sync, PowerShell, Python, Terraform, Videos and Errors

Logeshwaran
—

To upload a folder to S3, either drag it into the bucket in the S3 console (fine for up to a few thousand files, and single files up to 160 GB), or use the AWS CLI: aws s3 cp "C:\Users\you\Photos" s3://your-bucket/photos/ --recursive on Windows, or aws s3 sync ./photos s3://your-bucket/photos/ on Mac and Linux. sync uploads only new or changed files and can be re-run to resume. The CLI splits big files (videos, backups) into parallel multipart uploads automatically, up to S3's 50 TB object limit. Other routes: PowerShell's Write-S3Object -Folder ... -Recurse, Python's boto3 upload_file in a loop, and Terraform's aws_s3_object with fileset. With versioning on, uploading a file with the same name keeps the old one as a previous version instead of overwriting it. Below: every method with copy-paste commands, filters and storage classes, large video uploads, speed tips, Content-Type for websites, versioning, and the errors Windows users hit most (credentials, Access Denied, KMS, and the PC clock).

Jake had 40 GB of product photos and videos on the shop's Windows PC to get into S3 before the new website went live. The browser upload stalled at 60% overnight with no way to resume. One CLI command finished the job in an hour, and when the Wi-Fi dropped halfway, running the same command again carried on from where it stopped.

⚡ Quick Answer

• Small job: drag the folder into the S3 console. Steps.

• Anything bigger: aws s3 sync, which resumes. Commands.

• Windows PowerShell: Write-S3Object -Folder ... -Recurse. PowerShell.

• Upload failed? Credentials, Access Denied, KMS or the PC clock. Fixes.

Python · Terraform · Large videos · Speed · Content-Type · New versions.

New to S3 or the AWS CLI? The cards below cover installing the CLI on Windows, bucket access, sync behavior and Python uploads.

🧭 NEW HERE? READ THESE FIRST

New to S3 uploads? These five pages give you the background this one builds on:

📌 Bookmark this; the command table and the error list are the parts you will come back for.

Upload with the S3 console (drag and drop)

First, which method suits your job:

MethodBest forResumes?Limits
S3 consoleA few files or folders, no installNo160 GB per file; tab must stay open
AWS CLI syncAny folder, any size, repeated uploadsYes, by re-running50 TB per object
PowerShell Write-S3ObjectWindows scripts and adminsNo (uploads all)Same S3 limits
Python boto3Uploads inside your own app or with custom logicIf you code itSame S3 limits
TerraformWebsite assets deployed with infrastructureUploads only changed filesSlow for tens of thousands of files

How to upload to S3 without installing anything:

  1. Open the S3 console, choose your bucket, and go into the "folder" (prefix) you want, if any.
  2. Choose Upload, then Add folder (or drag the folder from File Explorer or Finder onto the page).
  3. Optionally open Properties to pick a storage class or encryption key for these files.
  4. Choose Upload and keep the tab open until it shows "Upload succeeded".

The console is fine for a few hundred or a few thousand files. It has limits that push you to the CLI for real jobs: a single file can be up to 160 GB through the console, the upload stops if the tab closes or the PC sleeps, and there is no "resume" after a failure. S3 has no real folders; the console shows the path (photos/2026/jacket.jpg) as folders for your convenience.

A few console details worth knowing. "Create folder" makes an empty placeholder object ending in / so the folder shows before it has files; uploading files with a path creates the folders automatically, so you rarely need it. The upload page's Properties panel lets you set storage class, encryption, tags and metadata for everything in that upload at once, which is quicker than changing them afterwards. After the upload, the summary lists any files that failed; retry just those rather than the whole set. And if the browser seems stuck on thousands of small files, it usually is still working: the console uploads files one batch at a time, which is why the CLI finishes the same job much faster.

Jake: "I dragged 6,000 photos into the console and it's been an hour."

Ethan: "Let it finish this time. Next time, one sync command does it in minutes and resumes if the laptop sleeps."

The command to upload a single file to S3

For one file, the shortest routes:

  • AWS CLI: aws s3 cp report.pdf s3://shop-media/reports/report.pdf. Leave the file name off the destination (s3://shop-media/reports/) to keep the same name.
  • Low-level CLI: aws s3api put-object --bucket shop-media --key reports/report.pdf --body report.pdf, a single request limited to 5 GB, useful when you need exact control of headers.
  • PowerShell: Write-S3Object -BucketName shop-media -File .\report.pdf -Key reports/report.pdf.
  • Python: boto3.client("s3").upload_file("report.pdf", "shop-media", "reports/report.pdf"), which switches to multipart automatically for large files.
  • Someone without AWS access: generate a presigned PUT URL (below) and they upload with curl -X PUT --upload-file report.pdf "URL" or a web form.

Choosing between upload_file and put_object in Python comes down to file size and where the data is; our upload_file vs put_object guide covers it, including the Content-Type fix for files that download instead of opening.

Upload multiple files at once (not the whole folder)

To upload a selection of files in one command, combine --exclude and --include: exclude everything, then include the patterns you want. The order matters, since later filters win:

# only the PDFs and spreadsheets from a folder
aws s3 cp ./exports s3://shop-data/exports/ --recursive --exclude "*" --include "*.pdf" --include "*.xlsx"

# a hand-picked list of files (Linux and Mac)
for f in invoice-101.pdf invoice-102.pdf invoice-110.pdf; do aws s3 cp "$f" s3://shop-data/invoices/; done

# the same in PowerShell
"invoice-101.pdf","invoice-102.pdf" | ForEach-Object { aws s3 cp $_ s3://shop-data/invoices/ }

In the console, select several files in your file manager and drag them onto the bucket page together; the console queues them in one upload. To check what a filter will do before uploading anything, add --dryrun: the CLI prints every upload it would make without sending a byte, which is the quickest way to get the include and exclude order right. Wildcards are not supported by the CLI on the source path itself (aws s3 cp *.pdf ... is expanded by your shell, not the CLI, and fails for more than one file), which is why the filter flags exist.

Key naming mistakes that bite after the upload

S3 accepts almost any key, which makes it easy to upload files under names that cause trouble later:

  • A leading slash: uploading to s3://shop-media//photos/ (two slashes) or giving a key that starts with / creates a "folder" with an empty name, which looks invisible in the console and breaks paths in your code.
  • Missing trailing slash: aws s3 cp report.pdf s3://shop-media/reports (no slash) creates an object called reports, not a file inside a reports folder.
  • Spaces and special characters: they work, but they must be URL-encoded in links and presigned URLs, and characters such as #, % and + are a common source of "NoSuchKey" errors in web apps. Plain letters, numbers, hyphens, underscores and dots save trouble.
  • Case: keys are case-sensitive, so Photos/ and photos/ are two folders; pick one convention.
  • Windows-only names: keys with : or ? upload fine from Linux but cannot be downloaded to Windows later.

If you already uploaded under bad names, fix them with a copy to the new key and a delete of the old one (aws s3 mv does both), then update anything that links to them. Our NoSuchKey guide covers the errors these mistakes cause.

Upload a folder with the AWS CLI (Windows, Mac, Linux)

Install the CLI once (our Windows 11 install guide covers it), run aws configure or sign in with SSO, then:

# Windows (Command Prompt or PowerShell): quote paths with spaces
aws s3 sync "C:\Users\Jake\Pictures\Shop Photos" s3://shop-media/photos/

# Mac and Linux
aws s3 sync ~/Pictures/shop-photos s3://shop-media/photos/

# one file
aws s3 cp "C:\Users\Jake\Videos\launch.mp4" s3://shop-media/videos/launch.mp4

# everything in a folder, every time (no comparison)
aws s3 cp "C:\Users\Jake\Pictures\Shop Photos" s3://shop-media/photos/ --recursive

Which command for which job:

You want toCommand
Upload a folder, resumable, only changes on re-runaws s3 sync LOCAL s3://bucket/prefix/
Upload one fileaws s3 cp FILE s3://bucket/key
Upload only some file types--exclude "*" --include "*.jpg" --include "*.png"
Skip junk files--exclude "*.tmp" --exclude "Thumbs.db" --exclude ".git/*"
See what would happen first--dryrun
Make S3 match the folder exactlyaws s3 sync ... --delete (removes files not in the folder; careful)
Cheaper storage for backups--storage-class INTELLIGENT_TIERING or DEEP_ARCHIVE
Move (upload then delete local copies)aws s3 mv LOCAL s3://bucket/prefix/ --recursive

The trailing slash on the destination matters: s3://shop-media/photos/ puts the folder's contents under photos/. Our sync guide explains how sync decides what to copy and when --delete is safe. For which storage class fits your files, see our storage class guide.

Jake: "The Wi-Fi dropped at 23 GB. Do I start again?"

Ethan: "Run the same sync. It skips everything already there and carries on."

Uploading from Windows: paths, OneDrive and long names

Most upload problems on Windows come from the file system rather than S3:

  • Spaces in paths: wrap the local path in quotes, aws s3 sync "C:\Users\Jake\Pictures\Shop Photos" s3://shop-media/photos/. The CLI converts backslashes to forward slashes in the keys for you.
  • OneDrive "online-only" files: with Files On-Demand, files that show a cloud icon are not really on the disk. Reading them for upload forces OneDrive to download each one first, which makes the upload crawl and can fill the disk. Right-click the folder and choose "Always keep on this device" before a large upload, or upload from a folder outside OneDrive.
  • Long paths: deep folder trees can pass Windows' 260-character limit; turn on long path support or move the folder closer to the drive root before uploading.
  • Locked files: files open in another program (Outlook data files, running databases) may fail to read; close the program or upload a copy.
  • Hidden system files: desktop.ini and Thumbs.db get uploaded too; add --exclude "desktop.ini" --exclude "Thumbs.db".

On a Mac, the equivalent junk is .DS_Store (--exclude "*.DS_Store"). On Mac and Linux, the CLI follows symbolic links by default and uploads what they point to; add --no-follow-symlinks if a link points somewhere you did not mean to upload, such as a huge shared folder.

Jake: "The upload of the photo folder was taking hours for only 8 GB."

Ethan: "Check for the cloud icons. OneDrive was downloading every photo before the CLI could send it."

Upload a folder with PowerShell (AWS Tools for PowerShell)

If you live in PowerShell, the AWS Tools module has a native cmdlet:

Install-Module AWS.Tools.S3 -Scope CurrentUser
Write-S3Object -BucketName shop-media -Folder "C:\Users\Jake\Pictures\Shop Photos" -KeyPrefix photos -Recurse
Write-S3Object -BucketName shop-media -Folder "C:\Users\Jake\Pictures\Shop Photos" -KeyPrefix photos -Recurse -SearchPattern "*.jpg"

It uses the same credentials as the CLI (your profile, or Set-AWSCredential). The CLI's sync is still the better choice for repeated uploads because it skips unchanged files; Write-S3Object uploads everything each time. Where PowerShell shines is scripting around the upload: picking yesterday's files with Get-ChildItem | Where-Object LastWriteTime, renaming on the way, or running from Task Scheduler on a Windows server every night. Add -StorageClass INTELLIGENT_TIERING for backups, and -Region eu-west-2 if your default profile points elsewhere.

For an upload of only new files from a Windows folder, pipe a filtered list into the cmdlet: Get-ChildItem C:\Exports -File | Where-Object { $_.LastWriteTime -gt (Get-Date).AddDays(-1) } | ForEach-Object { Write-S3Object -BucketName shop-data -File $_.FullName -Key ("exports/" + $_.Name) }. Wrap the script in a try/catch and write failures to a log file, so a scheduled task that fails at 2 am leaves a trace you can read in the morning.

Upload a folder with Python (boto3)

import os, mimetypes
import boto3

s3 = boto3.client("s3")
local_root = r"C:\Users\Jake\Pictures\Shop Photos"
bucket, prefix = "shop-media", "photos/"

for folder, _, files in os.walk(local_root):
    for name in files:
        path = os.path.join(folder, name)
        key = prefix + os.path.relpath(path, local_root).replace("\\", "/")
        content_type, _ = mimetypes.guess_type(name)
        s3.upload_file(path, bucket, key,
                       ExtraArgs={"ContentType": content_type or "application/octet-stream"})
        print("uploaded", key)

Two details make this script safe on Windows. The replace("\\", "/") converts Windows backslashes to the forward slashes S3 keys use; forgetting it gives you object names like photos/2026\jacket.jpg, which the console shows as one strange file instead of a folder. And setting ContentType from the extension means images and videos open in a browser instead of downloading. upload_file handles multipart for large files on its own; our upload_file vs put_object guide covers progress bars, threads and the other options.

Three improvements turn this into a dependable tool. Skip what is already there: call head_object for each key and compare its ContentLength with the local file size, so a rerun after an interruption only sends the missing files, like sync. Upload in parallel: wrap the call in a function and run it through concurrent.futures.ThreadPoolExecutor with eight or so workers, since the client is thread-safe and small-file uploads spend most of their time waiting on the network. Show progress: upload_file accepts a Callback that receives the bytes sent so far, which is enough for a simple progress line on large files. For exact behavior on large files, pass a TransferConfig with your preferred part size and concurrency.

Upload a folder with Terraform

For website files and other assets you deploy with infrastructure code:

resource "aws_s3_object" "site" {
  for_each     = fileset("${path.module}/site", "**")
  bucket       = aws_s3_bucket.site.id
  key          = each.value
  source       = "${path.module}/site/${each.value}"
  etag         = filemd5("${path.module}/site/${each.value}")
  content_type = lookup(local.mime_types, regex("\\.[^.]+$", each.value), "application/octet-stream")
}

locals {
  mime_types = { ".html" = "text/html", ".css" = "text/css", ".js" = "application/javascript",
                 ".png" = "image/png", ".jpg" = "image/jpeg", ".svg" = "image/svg+xml" }
}

The etag line makes Terraform re-upload a file only when its content changes. Terraform tracks every file as a resource, which is fine for a website of hundreds of files and slow for tens of thousands; for big media folders, upload with the CLI in your pipeline instead.

Two more Terraform details catch people out. The etag trick does not work for objects encrypted with a customer managed KMS key, because S3 then returns an ETag that is not the file's MD5, so Terraform sees a change on every plan; use source_hash with the same filemd5() value instead in that case. And deleting a file from the local folder removes it from the fileset, so Terraform deletes the object from S3 on the next apply, which is right for a website and dangerous for anything else. Keep Terraform for files that belong to the infrastructure (website builds, config files, small assets), and keep data uploads out of state entirely.

Jake: "Every terraform plan says all 300 site files will change."

Ethan: "The bucket uses your KMS key, so the ETags aren't MD5s. Switch etag to source_hash."

Uploading large files and videos

Videos and backups are where browser uploads give up. What to know for big files:

  • Size limits: one S3 object can be up to 50 TB; one upload request (a single PUT) is limited to 5 GB; the console accepts files up to 160 GB. The CLI and SDKs use multipart upload automatically for anything over 8 MB, sending parts in parallel and retrying failed parts.
  • Resuming: sync skips files already uploaded, but a single huge file interrupted mid-way restarts that file. Unfinished multipart uploads leave invisible parts that you pay for until removed; add a lifecycle rule to abort incomplete multipart uploads after 7 days.
  • Content-Type for playback: the CLI sets video/mp4 from the extension; check it if the browser downloads the video instead of playing it.
  • Errors: our EntityTooLarge guide covers the size errors and how to switch to multipart in your own code.

If you write your own multipart code for very large files over unreliable links, it can resume rather than restart: keep the upload ID, call list_parts after a failure to see which parts S3 already has, upload only the missing ones, and then complete the upload. The CLI and upload_file do not resume across separate runs, so for a 500 GB file on a shaky connection, either use code that resumes, or upload from a machine with a stable link, such as an EC2 instance that received the file another way.

S3 file upload size limits, explained

LimitValueWhat it means for you
Largest object50 TBBigger than almost any single file you will have
Single PUT request5 GBAbove this, multipart upload is required
Console upload160 GB per fileUse the CLI or SDK for anything larger
Parts per multipart upload1 to 10,000Part size must grow with file size
Part size5 MiB to 5 GiB (last part can be smaller)Tiny parts fail with EntityTooSmall
CLI multipart threshold8 MB by defaultThe CLI handles parts for you

The part limits explain the arithmetic behind big uploads: with at most 10,000 parts, a 1 TB file needs parts of at least about 105 MB, and the tools pick a part size for you when they know the file size in advance. The case where they cannot is a stream, which is the next section. If your own code hits a 5 GB error, it is calling a single PUT for a file that needs multipart; our EntityTooLarge guide shows the fix.

Streaming uploads and the --expected-size option

The CLI can upload from standard input, which lets you send a backup or a database export straight to S3 without writing it to disk first:

# compress a folder on the fly and upload it as one object
tar czf - /var/www | aws s3 cp - s3://shop-backups/www-2026-10-20.tgz

# a stream larger than 50 GB needs its approximate size in bytes
tar cf - /data | aws s3 cp - s3://shop-backups/data.tar --expected-size 120000000000

Why the extra option: with a stream, the CLI cannot see the total size in advance, so it uses a default part size. For streams over 50 GB, that default would need more than the 10,000-part maximum and the upload fails partway with too many parts. --expected-size tells the CLI roughly how big the stream will be, so it picks parts large enough. An estimate on the high side is fine; too low can still fail. For streams under 50 GB, leave it out.

Jake: "The nightly backup upload died at 52 GB every time."

Ethan: "It was a stream. Add --expected-size with a generous number and it picks bigger parts."

Browser and app uploads with presigned URLs

When users upload from a web page or a mobile app, the file should go straight to S3, not through your servers. Your backend signs permission for one upload, and the browser sends the file to S3 directly:

import boto3

s3 = boto3.client("s3", region_name="eu-west-2")
url = s3.generate_presigned_url(
    "put_object",
    Params={"Bucket": "shop-uploads", "Key": "user-123/avatar.png", "ContentType": "image/png"},
    ExpiresIn=600,
)
# the browser then sends: fetch(url, {method: "PUT", body: file, headers: {"Content-Type": "image/png"}})

Three things make this work in practice. The browser must send the same Content-Type that was signed, or S3 rejects the request with a signature error. The bucket needs a CORS rule allowing PUT from your site's origin. And to limit file size, use a presigned POST instead (generate_presigned_post) with a content-length-range condition, since a presigned PUT cannot cap the size.

Large files from the browser (videos of several GB) need multipart: your backend calls create_multipart_upload, returns a presigned URL for each part (upload_part with a PartNumber), the browser uploads the parts in parallel and collects each response's ETag header, and your backend calls complete_multipart_upload with the list of parts. Add ETag to the CORS rule's ExposeHeaders, or the browser cannot read the ETags and the final step fails. Our S3 CORS guide has a working rule.

Every way to get data into S3

RouteBest for
ConsoleA few files by hand
AWS CLI cp / syncFolders of any size, scripts, repeated uploads
SDKs (Python, JavaScript, Java and others)Uploads from your own application
Presigned URLsUsers uploading from browsers and apps
AWS DataSyncLarge or recurring copies from office NFS or SMB storage
AWS Transfer FamilyPartners who can only send by SFTP or FTPS
S3 File GatewayOffice users saving to a network share that lands in S3
Offline transfer devicesTens of terabytes over a slow link, where available

A quick rule for big one-off moves: if uploading over your connection would take more than a week, ask about offline transfer; if it is recurring, DataSync or a gateway beats any script you would have to babysit.

Make uploads faster

  1. Raise parallelism: aws configure set default.s3.max_concurrent_requests 32 (the default is 10). On a fast connection with many small files, this is the biggest win.
  2. Larger parts for huge files: aws configure set default.s3.multipart_chunksize 64MB.
  3. Upload from the cloud when you can: if the files are already on an EC2 instance or another cloud, upload from there instead of your office line.
  4. S3 Transfer Acceleration for long distances (for example Australia to a US bucket): enable it on the bucket and add --endpoint-url https://shop-media.s3-accelerate.amazonaws.com. It adds a per-GB fee, so test whether it helps your route.
  5. Zip thousands of tiny files into archives if you only need them as a backup; per-file overhead dominates with very small files.

What uploading to S3 costs

Uploading itself is cheap: data transfer into S3 is free, from anywhere. What you pay for:

  • PUT requests: about $0.005 per 1,000 for S3 Standard in US East. Each file is at least one request, and each part of a multipart upload is one more, so 100,000 small files cost about 50 cents in requests, while a 50 GB video sent in 100 MB parts adds about 500 part requests, a fraction of a cent.
  • Storage from the moment the object arrives, at the rate of its storage class, plus the 128 KB and 30-day minimums for infrequent-access classes.
  • Incomplete multipart uploads: parts of uploads that never finished are stored and billed until aborted, which is why the lifecycle rule above matters.
  • Extras you opt into: Transfer Acceleration charges per GB uploaded through it; KMS charges per request when a customer managed key encrypts each object; NAT gateways charge per GB if a private server uploads without an S3 endpoint.

For most people the upload bill is pennies; the storage that follows is the real cost, so choosing the storage class at upload time (next section) matters more than any upload setting.

Choosing storage class and encryption at upload

You can set how an object is stored and encrypted as you upload it, which saves a second pass later:

  • Storage class: --storage-class STANDARD_IA for files read rarely, INTELLIGENT_TIERING when you cannot predict access, GLACIER_IR for archives you may still need quickly, or DEEP_ARCHIVE for long-term backups. Our storage classes guide helps you choose.
  • Encryption: every new object is encrypted at rest by default with S3-managed keys; add --sse aws:kms --sse-kms-key-id alias/shop-data for a customer managed KMS key, which also needs kms:GenerateDataKey permission for the uploader.
  • Checksums: --checksum-algorithm SHA256 asks S3 to verify the data with that checksum on arrival and store it, so later downloads can be checked against it.
  • Metadata and tags: --metadata project=winter-sale and, through s3api put-object, --tagging for cost allocation and lifecycle rules by tag.

Small files are a trap with infrequent-access classes: objects in Standard-IA are billed as at least 128 KB each and for at least 30 days, so thousands of tiny files can cost more there than in Standard.

Check that an upload is complete

Before deleting the local copy, compare both sides. aws s3 ls s3://shop-media/photos/2025/ --recursive --summarize ends with the object count and total size; compare those with the local folder (its Properties on Windows, or find and du on Linux and Mac). Then run the same aws s3 sync once more: when nothing is left to upload, it prints nothing. For important backups, upload with --checksum-algorithm SHA256 so S3 itself checks every byte as it arrives.

Clean up afterwards too. An interrupted multipart upload leaves hidden parts that are billed as storage; aws s3api list-multipart-uploads --bucket shop-media shows any that are left, and a lifecycle rule that aborts incomplete multipart uploads after 7 days removes them automatically.

Content-Type and caching for website files

When the files are a website or images served through CloudFront, two headers decide how browsers treat them:

aws s3 sync ./build s3://shop-site/ --delete --cache-control "public, max-age=31536000" --exclude "*.html"
aws s3 sync ./build s3://shop-site/ --delete --cache-control "no-cache" --exclude "*" --include "*.html"
  • Content-Type: the CLI guesses it from the extension; a wrong one makes browsers download files instead of showing them. Override with --content-type for unusual extensions.
  • Cache-Control: long caching for versioned assets (CSS, JS, images with hashes in their names), no caching for HTML, as above.
  • Order of upload: upload new assets first and the HTML that references them last, so no visitor gets a page pointing at files that are not there yet.
  • After uploading, CloudFront may keep serving the old copy until its cache expires; our CloudFront cache guide covers invalidations.

Uploading to a bucket in another AWS account

Uploading into someone else's bucket, such as a client's or a partner's, needs permission on both sides: their bucket policy must allow your role or user s3:PutObject on the right prefix, and your own IAM policy must allow the same action on their bucket's ARN. If their bucket uses a customer managed KMS key, its key policy must also let you use it for encryption.

Ownership used to be the trap. With older bucket settings, objects you uploaded stayed owned by your account, and the bucket owner could not read them unless you added --acl bucket-owner-full-control. Buckets created since 2023 default to "bucket owner enforced", where ACLs are disabled and the bucket owner automatically owns every object, so the flag is no longer needed (and is rejected if it conflicts with the setting). If the owner reports they cannot read your uploads, ask which object ownership setting the bucket uses. Our cross-account access guide covers both policies.

Automate a nightly upload or backup

The usual small-business backup is a scheduled aws s3 sync from a folder or server to a bucket. Done carefully, it is reliable and cheap:

  1. Use a dedicated IAM user or role with permission only to list and write to the backup bucket, not to delete, so a compromised machine cannot wipe the backups.
  2. Turn on versioning on the bucket, so an overwritten or encrypted-by-ransomware file still has its earlier versions.
  3. Schedule the sync: Task Scheduler on Windows (daily trigger, "Run whether user is logged on or not"), or cron or a systemd timer on Linux.
  4. Leave out --delete for backups, so files deleted locally by mistake are not deleted in S3 as well; our sync and delete guide explains how that flag behaves.
  5. Log the output to a file and check it, or send a notification when the job fails; a backup nobody checks is a hope, not a backup.
  6. Add lifecycle rules: move older backups to a cheaper storage class and expire noncurrent versions after a set number of days.

Keep at least one copy of truly critical data outside the same AWS account too, for example in a second account with its own credentials, so one compromised login cannot reach every copy. Test a restore from the bucket once after setting it up, and again every few months; the day you need the backup is the wrong day to learn the restore steps.

Uploading from EC2, Lambda and containers

Code running in AWS should upload with an IAM role, never with access keys copied onto the machine. Give the instance profile, Lambda execution role or ECS task role s3:PutObject on the target prefix (plus kms:GenerateDataKey if the bucket uses a customer managed key), and the SDKs and CLI pick up the credentials automatically. Keep the compute in the bucket's Region, where transfer into S3 is free, and if the server sits in a private subnet, add a gateway VPC endpoint for S3 so uploads do not pass through a NAT gateway, which charges per GB processed.

In Lambda, files you create live in /tmp (up to 10 GB if you raise the setting), and the function's timeout limits how long an upload can take, so very large uploads belong in a container task or on an instance. For files arriving from users, presigned URLs (above) keep the data from passing through your function at all.

Uploading files as new versions

Uploading a file to a key that already exists replaces it, unless the bucket has versioning enabled. With versioning on, each upload of the same name becomes a new version, the latest one is what people get by default, and earlier ones stay available:

aws s3api put-bucket-versioning --bucket shop-media --versioning-configuration Status=Enabled
aws s3api list-object-versions --bucket shop-media --prefix photos/jacket.jpg \
  --query "Versions[].[VersionId,LastModified,IsLatest,Size]" --output table

To put an earlier version back as the current one, copy it onto itself: aws s3api copy-object --bucket shop-media --key photos/jacket.jpg --copy-source "shop-media/photos/jacket.jpg?versionId=VERSION_ID". The old content becomes the newest version, and nothing is lost along the way.

Every version is stored and billed, so pair versioning with a lifecycle rule that expires noncurrent versions after, say, 30 or 90 days. It is the cheapest protection against an accidental overwrite of an important file.

Jake: "I overwrote the good logo with the draft."

Ethan: "Versioning is on. The good one is still there as the previous version. Copy it back on top."

Upload errors and fixes

  • "Unable to locate credentials": the CLI has no credentials; run aws configure or aws sso login. Our credentials guide has every case.
  • AccessDenied when calling PutObject: your user or role lacks s3:PutObject on bucket/* (and s3:ListBucket for sync); see our bucket access guide and 403 guide.
  • AccessDenied on a bucket with SSE-KMS: you also need kms:GenerateDataKey on the bucket's KMS key.
  • RequestTimeTooSkewed ("the difference between the request time and the current time is too large"): your PC's clock is off by more than 15 minutes. Sync the clock in Windows Settings, Time and language, Date and time, "Sync now".
  • "The user-provided path ... does not exist": a typo or an unquoted path with spaces; wrap the path in double quotes.
  • "Could not connect to the endpoint URL": a wrong Region name, a proxy or firewall blocking HTTPS, or no internet on that machine.
  • EntityTooLarge: a single PUT over 5 GB from your own code; use multipart (the CLI does).
  • Uploads stall at the same file: the file is locked by another program (open video editor, OneDrive syncing); close it and re-run sync.

Frequently asked questions

Can an interrupted S3 upload resume?

aws s3 sync resumes at the file level by skipping finished files; within one huge file, only custom multipart code that keeps the upload ID and checks list_parts can resume.

What is the maximum number of parts in a multipart upload?

10,000 parts, each 5 MiB to 5 GiB except the last, which is why very large files need larger parts.

How do I upload to an S3 bucket in another account?

The owner's bucket policy must allow your role s3:PutObject, and your IAM policy must allow it too; buckets with "bucket owner enforced" ownership own your uploads automatically.

Why is there an empty-named folder in my bucket?

A key started with a slash or the destination had a double slash; move the objects to keys without the leading slash.

How do I test an S3 upload command without uploading?

Add --dryrun to aws s3 cp or sync; it lists every file it would upload without sending anything.

Does it cost money to upload to S3?

Transfer into S3 is free; you pay about $0.005 per 1,000 PUT requests in S3 Standard, plus storage from the moment the files arrive.

Why does Terraform re-upload my S3 files every time?

With a customer managed KMS key the ETag is not an MD5, so etag = filemd5() always differs; use source_hash instead.

Why is uploading from OneDrive so slow?

Online-only files are downloaded by OneDrive before they can be read; mark the folder "Always keep on this device" first, or upload from a folder outside OneDrive.

How do I schedule a daily upload to S3?

Run aws s3 sync from Task Scheduler on Windows or cron on Linux, without --delete for backups, with logging and a versioned bucket.

How do I restore a previous version after an upload overwrote a file?

Copy the older version onto the same key with aws s3api copy-object and a copy source that includes ?versionId=.

How should a server upload to S3 without access keys?

Attach an IAM role (instance profile, Lambda execution role or ECS task role) with s3:PutObject on the target prefix.

What is the command to upload a file to S3?

aws s3 cp file-name s3://bucket-name/path/ with the AWS CLI, or Write-S3Object in PowerShell, or upload_file in boto3.

What is the S3 file upload size limit?

An object can be up to 50 TB; a single PUT is limited to 5 GB, so larger files use multipart upload, and the console accepts up to 160 GB per file.

What does --expected-size do in aws s3 cp?

It tells the CLI the approximate size of a stream from standard input, needed above 50 GB so it chooses parts large enough to stay under 10,000 parts.

How do I upload multiple files to S3 at once?

Use aws s3 cp with --recursive --exclude "*" --include patterns, loop over a list of files, or drag several files into the console together.

How do I upload large files to S3 from a browser?

Use presigned multipart upload: the backend creates the upload and presigns each part, the browser uploads parts and returns their ETags, and the backend completes it.

How do I limit upload size with a presigned URL?

Use a presigned POST with a content-length-range condition; a presigned PUT cannot cap the file size.

What are the ways to upload data to S3?

The console, AWS CLI, SDKs, presigned URLs, DataSync, Transfer Family for SFTP, S3 File Gateway and offline transfer devices for very large moves.

How do I check an S3 upload is complete?

Compare aws s3 ls --recursive --summarize with the local folder, rerun sync to confirm nothing is left, and use --checksum-algorithm for integrity.

How do I upload a folder to S3?

Drag it into the bucket in the S3 console, or run aws s3 sync with the folder path and s3://bucket/prefix/.

What is the S3 command to upload a file?

aws s3 cp FILE s3://bucket/key, for example aws s3 cp report.pdf s3://my-bucket/reports/report.pdf.

How do I upload multiple files to S3 at once?

Use aws s3 sync or aws s3 cp --recursive on a folder, with --include and --exclude to pick file types.

How do I upload files to S3 from Windows?

Install the AWS CLI and run aws s3 sync "C:\path\to\folder" s3://bucket/prefix/, quoting paths with spaces, or use Write-S3Object in PowerShell.

What is the largest file I can upload to S3?

An object can be up to 50 TB; a single PUT is limited to 5 GB, and the console accepts files up to 160 GB.

How do I upload large video files to S3?

Use the AWS CLI or an SDK, which upload big files in parallel parts automatically; the console works up to 160 GB but cannot resume.

Can I resume an interrupted S3 upload?

Re-run aws s3 sync; it skips files already uploaded. A single interrupted large file restarts.

What does uploading as new versions mean in S3?

With versioning enabled, uploading to an existing key creates a new version and keeps the previous one.

How do I upload a folder to S3 with Python?

Walk the folder with os.walk and call boto3 upload_file for each file, converting Windows backslashes to forward slashes in the key.

How do I upload a folder to S3 with Terraform?

Use aws_s3_object with for_each over fileset(), and set etag and content_type for each file.

Why does my S3 upload say RequestTimeTooSkewed?

Your computer's clock is more than 15 minutes off; sync the system time and retry.

Why do I get Access Denied uploading to S3?

Missing s3:PutObject on the bucket's objects, s3:ListBucket for sync, or kms:GenerateDataKey for SSE-KMS buckets.

How can I make S3 uploads faster?

Raise max_concurrent_requests, use larger multipart chunks for huge files, upload from inside AWS, or try Transfer Acceleration for long distances.

Does aws s3 sync delete files in S3?

Only with --delete, which removes S3 objects that are not in your local folder.

Does S3 have folders?

Not really; keys like photos/2026/jacket.jpg are shown as folders by the console and tools.

How do I set Content-Type when uploading to S3?

The CLI guesses it from the extension; override with --content-type, or ContentType in boto3 ExtraArgs.

What happens to failed multipart uploads?

Their uploaded parts stay and are billed until aborted; add a lifecycle rule to abort incomplete multipart uploads.

Can I upload to S3 with PowerShell?

Yes, with Write-S3Object from the AWS.Tools.S3 module, using -Folder and -Recurse for folders.

For readers preparing for an AWS certification

We have been covering AWS certifications alongside our fixes, so here is a short extra for readers studying for one. If you only came for the fix, you already have it; feel free to skip this part.

Exams that test this topic: Cloud Practitioner • CloudOps Engineer – Associate (SOA-C03) • Solutions Architect – Associate (SAA-C03).

Here are the exam-style questions this topic tends to raise, with the reasoning you need to answer them:

1. What is the largest single object you can store in Amazon S3?

50 TB (since December 2025).

2. Which upload method should be used for objects larger than 5 GB?

Multipart upload (the CLI and SDKs use it automatically).

3. Which AWS CLI command uploads only new or changed files from a local folder to S3?

aws s3 sync.

4. How do you protect S3 objects against accidental overwrites and deletions?

Enable versioning (with lifecycle rules to expire old versions).

5. Which feature speeds up uploads to S3 over long distances using edge locations?

S3 Transfer Acceleration.

6. What cleans up the parts of multipart uploads that never completed?

A lifecycle rule that aborts incomplete multipart uploads.

Exam codes and dates change often. Our AWS certification changes page keeps track of what is current.

πŸ“š ALSO READ

Where to go next, from upload errors to storage classes:

📌 Bookmark this; the command table and the error list are the parts you will come back for.

The 40 GB of photos and videos went up with one aws s3 sync and a re-run after the Wi-Fi drop. The website build now uploads in the deployment pipeline with the two-step Cache-Control sync, the bucket has versioning with a 30-day cleanup rule, and the PC's clock syncs automatically after that one confusing error.

📌 If you keep one line from this page

Upload folders with aws s3 sync "LOCAL PATH" s3://bucket/prefix/ (quote Windows paths, re-run to resume); the CLI handles big videos with multipart up to 50 TB, and most failures are credentials, Access Denied, KMS or a wrong PC clock.

Console for small jobs · Write-S3Object in PowerShell · versioning against overwrites.

Revision note. Written October 20, 2026, with S3's 50 TB object limit in effect.

Related