SQS Lambda Batch Size: 1 vs 10 vs 10,000, the Batching Window Rule, Visibility Timeout x6, Partial Batch Failures (and the Setting That Silently Ignores Them), and Scaling to 1,250

The batch size of an SQS trigger is the maximum number of messages Lambda passes to your function in one invocation. It defaults to 10, can go up to 10,000 for standard queues (but only with a batching window of at least 1 second), and is capped at 10 for FIFO queues, which do not support a batching window at all. A batch can also be smaller than your setting, because the whole batch must fit in Lambda's 6 MB invocation payload. The setting that decides whether batches are safe is not the size, though, it is error handling. If one message in a batch fails and your function throws, the whole batch comes back, including the messages that succeeded. The fix is partial batch responses: turn on ReportBatchItemFailures and return the IDs of the failed messages. Two traps sit right next to it: without that setting, a returned batchItemFailures list is simply ignored and the "failed" messages are deleted with the rest; and with it, a single misspelled key or wrong ID makes Lambda treat the entire batch as failed. This guide covers choosing a batch size, the batching window, visibility timeout, scaling from 5 to 1,250 concurrent invocations, partial failures, and two scripts we tested that check your setup.

Jake's order queue feeds a Lambda function that updates stock levels. Ethan raised the batch size from 1 to 100 to cut costs, and the next morning some orders had been processed three times and one not at all. "Bigger batches were supposed to be cheaper," Jake said. "They are," Ethan said. "As long as one bad order doesn't send the other ninety-nine around again."

New to this setup? Amazon SQS is a message queue, and when you add a queue as a trigger to a Lambda function, Lambda runs an event source mapping: a poller that reads messages from the queue, hands them to your function in batches, and deletes them when the function succeeds. Our plain-English SQS guide covers queues and visibility timeouts, and the SQS limits guide covers the queue-side numbers.

SQS trigger batch size limits

You set these on the trigger (the event source mapping), not on the queue. Console: the function's Configuration, Triggers, select the SQS trigger, Edit. CLI:

aws lambda update-event-source-mapping --uuid 1a2b3c4d-1111-2222-3333-444455556666 \
    --batch-size 100 --maximum-batching-window-in-seconds 5 \
    --function-response-types ReportBatchItemFailures

Choosing a batch size: 1, 10, 100 or more

Bigger batches give you fewer invocations, fewer SQS receive calls and less per-invocation overhead, so your bill drops and quick, simple messages often move faster. The price you pay is shared fate: one invocation holds many messages, so a timeout or crash in your code affects all of them. A practical guide:

Batch size 1 is the simplest to reason about, because each message succeeds or fails alone, and it is a fine choice for slow work. For fast work it is wasteful: ten times the invocations of a batch of 10. With partial batch responses, you get the same "each message fails alone" behavior at batch size 10 or more, which is why the next section matters more than the number you pick.

Whatever size you choose, set the function timeout for the whole batch: if one message takes 200 ms, a batch of 100 needs 20 seconds plus a margin. Our timeout guide covers the default 3-second limit that catches new functions.

Partial batch failures: return only the messages that failed

By default, Lambda deletes a batch only when the invocation succeeds. If your function throws an exception, the whole batch becomes visible again after the visibility timeout, and messages that were processed successfully get processed again. With a batch of 100, one bad order means 99 duplicates. That is what happened to Jake's stock levels.

Partial batch responses fix it. To set them up:

1. Turn on ReportBatchItemFailures on your trigger: in the console, edit the SQS trigger and check Report batch item failures; with the CLI, add --function-response-types ReportBatchItemFailures.
2. Change your handler to catch errors per message instead of letting one exception end the invocation.
3. Return the messageId of each failed message in batchItemFailures. Lambda deletes the rest.
4. Test with a batch that contains one bad message, and confirm only that one comes back.

A handler that does this in Python:

import json

def handler(event, context):
    failures = []
    for record in event['Records']:
        try:
            process(json.loads(record['body']))
        except Exception as e:
            print(f"failed {record['messageId']}: {e}")
            failures.append({'itemIdentifier': record['messageId']})
    return {'batchItemFailures': failures}

Lambda then reads your response by strict rules. We wrote a small function that applies them, and ran it on six responses to a batch of three messages:

correct                        -> partial: retry ['m-2'], delete the rest
nothing failed                 -> complete success (no failures listed)
typo in key                    -> complete failure (bad key name)
receipt handle instead of ID   -> complete failure (unknown message ID 'AQEB...handle')
empty ID                       -> complete failure (empty itemIdentifier)
returned None                  -> complete success (null or empty response)

Those outcomes follow Lambda's documented conditions. An empty or null batchItemFailures list, or an empty response, means complete success. Invalid JSON, an empty or null itemIdentifier, a misspelled key (itemIdentifer) or an ID that is not in the batch means complete failure, and the entire batch is retried. Use record['messageId'], never the receipt handle, and if your function throws instead of returning, the whole batch fails as before.

And the trap on the other side: without ReportBatchItemFailures on the trigger, your returned list is ignored. The invocation succeeded, so Lambda deletes the whole batch, including the messages you meant to retry. Code that returns batchItemFailures and a trigger that is not configured for it lose messages silently. Check both together. For FIFO queues, stop processing at the first failure and return that message and every one after it, so the order is preserved.

Powertools for AWS Lambda includes a batch processor that implements this pattern for Python, TypeScript, Java and .NET, if you prefer not to write it yourself.

The batching window: waiting to fill a batch

MaximumBatchingWindowInSeconds tells the poller how long it may wait to gather a fuller batch before it invokes your function. With a window of 0 (the default), Lambda invokes as soon as it has messages, so batches on a quiet queue are often small even with a large batch size. With a window of, say, 5 seconds, Lambda waits up to 5 seconds for more messages, up to the batch size or the 6 MB payload, whichever comes first.

• A window of at least 1 second is required for batch sizes above 10 on standard queues.
• FIFO queues do not support a window.
• The window adds up to that many seconds of delay before a message is processed, so keep it short for user-facing work.
• Add the window to your visibility timeout calculation.

Visibility timeout: 6 times the function timeout

When the poller receives a batch, the messages become invisible for the queue's visibility timeout. If they are not deleted in that time, they reappear and can be delivered again. Lambda requires the visibility timeout to be at least the function timeout, and recommends six times the function timeout, plus the batching window. The extra margin covers retries when your function is throttled: a batch can wait for capacity before it even starts.

For a function with a 30-second timeout and a 5-second window, that is a visibility timeout of at least 185 seconds. Set it on the queue:

aws sqs set-queue-attributes --queue-url https://sqs.us-east-1.amazonaws.com/111122223333/orders \
    --attributes VisibilityTimeout=185

A visibility timeout that is too short is a classic cause of "my function processed the same message twice" with no error anywhere. Our duplicate messages guide covers the other causes and how to make processing idempotent, which you need anyway with at-least-once delivery.

Why batches are smaller than the batch size

Three reasons your function can receive fewer messages than BatchSize:

1. There were not enough messages when the poller read the queue, and the batching window was 0 or ran out.
2. The 6 MB invocation payload limit. Lambda passes the whole batch in one synchronous invocation, which cannot exceed 6 MB including the metadata Lambda and SQS add to each record. With messages near SQS's maximum size, a batch can hold only a few. Our 6 MB guide covers the payload limit.
3. Several pollers share the messages. As Lambda scales up, multiple pollers read the queue at once, and each gets a share.

So treat BatchSize as a maximum, and write your function so any batch size from 1 up works.

How Lambda scales SQS triggers: 5, then +300 a minute, up to 1,250

For a standard queue, the poller starts with five concurrent batches (five concurrent invocations). While messages remain, Lambda adds up to 300 concurrent invocations per minute, up to 1,250 concurrent invocations per event source mapping. When traffic is low it scales back to five, and can drop to as few as two to reduce SQS calls. FIFO queues scale differently: messages in the same message group are processed in order, one batch at a time per group, so concurrency depends on how many groups have messages.

Two controls limit how far it goes:

• Maximum concurrency on the trigger (2 to 1,000): the most invocations this queue may run at once. It is the right tool to protect a database or a downstream API from a sudden flood. It also turns off the low-traffic optimization down to two.
• Reserved concurrency on the function: a hard cap shared by every trigger of the function. If it is lower than the trigger's maximum concurrency, the function is throttled before the trigger limit is reached, and messages wait or are retried. If you use reserved concurrency, set it to at least 5, because the poller starts with five batches.

For spiky, high-volume queues there is also provisioned mode, which keeps a minimum number of dedicated event pollers (2 to 200) ready and scales them up to your maximum. Each poller handles up to 1 MB/s, 10 concurrent invocations or 10 SQS polling calls per second. Provisioned mode and maximum concurrency cannot be used together, and provisioned mode adds its own charges.

Lab: check every SQS trigger of a function

This read-only script reads a function's timeout and reserved concurrency, each SQS trigger's settings and each queue's visibility timeout, and flags the rules on this page:

#!/usr/bin/env python3
"""sqs_esm_check.py FUNCTION_NAME [--region R]: check every SQS trigger of a Lambda function against the documented rules."""
import sys, boto3
args = sys.argv[1:]; region = None
if '--region' in args: i = args.index('--region'); region = args[i + 1]; args = args[:i] + args[i + 2:]
fn = args[0]
lam = boto3.client('lambda', region_name=region); sqs = boto3.client('sqs', region_name=region)

def check(fn):
    cfg = lam.get_function_configuration(FunctionName=fn)
    timeout = cfg['Timeout']
    reserved = lam.get_function_concurrency(FunctionName=fn).get('ReservedConcurrentExecutions')
    maps = [m for p in lam.get_paginator('list_event_source_mappings').paginate(FunctionName=fn)
            for m in p['EventSourceMappings'] if ':sqs:' in m['EventSourceArn']]
    if not maps: print('no SQS triggers'); return
    for m in maps:
        arn = m['EventSourceArn']; name = arn.split(':')[-1]; fifo = name.endswith('.fifo')
        url = sqs.get_queue_url(QueueName=name)['QueueUrl']
        vis = int(sqs.get_queue_attributes(QueueUrl=url, AttributeNames=['VisibilityTimeout'])['Attributes']['VisibilityTimeout'])
        bs, win = m.get('BatchSize', 10), m.get('MaximumBatchingWindowInSeconds', 0)
        maxc = (m.get('ScalingConfig') or {}).get('MaximumConcurrency')
        print(f"== {name}  batch={bs} window={win}s  function timeout={timeout}s  visibility={vis}s  state={m.get('State')}")
        if fifo and bs > 10: print('  ERROR FIFO queues allow a batch size of at most 10')
        if fifo and win: print('  ERROR batching window is not supported for FIFO queues')
        if not fifo and bs > 10 and win < 1: print('  ERROR batch size over 10 needs MaximumBatchingWindowInSeconds of at least 1')
        need = 6 * timeout + win
        if vis < timeout: print(f'  ERROR visibility timeout {vis}s is shorter than the function timeout {timeout}s')
        elif vis < need: print(f'  WARN  visibility timeout {vis}s < 6 x timeout + window = {need}s; throttled retries can make messages visible again mid-flight')
        if 'ReportBatchItemFailures' not in m.get('FunctionResponseTypes', []):
            print('  WARN  ReportBatchItemFailures is off: one failed message retries the whole batch, and a returned batchItemFailures list is ignored')
        if reserved is not None and maxc and reserved < maxc:
            print(f'  WARN  reserved concurrency {reserved} < MaximumConcurrency {maxc}: Lambda will be throttled before the mapping limit')
        if reserved is not None and reserved < 5:
            print(f'  WARN  reserved concurrency {reserved} is below 5; the SQS poller starts with 5 concurrent batches')
        if not maxc and reserved is None:
            print('  INFO  no MaximumConcurrency and no reserved concurrency: this queue can scale to 1,250 concurrent invokes')

check(fn)

We tested it in moto, a library that mocks AWS locally, with a function (30-second timeout, reserved concurrency 3) and two triggers: a standard queue with batch size 100, no window and maximum concurrency 10, and a FIFO queue with batch size 10 and partial responses on. Its findings:

== orders  batch=100 window=0s  function timeout=30s  visibility=30s  state=Enabled
  ERROR batch size over 10 needs MaximumBatchingWindowInSeconds of at least 1
  WARN  visibility timeout 30s < 6 x timeout + window = 180s; throttled retries can make messages visible again mid-flight
  WARN  ReportBatchItemFailures is off: one failed message retries the whole batch, and a returned batchItemFailures list is ignored
  WARN  reserved concurrency 3 < MaximumConcurrency 10: Lambda will be throttled before the mapping limit
  WARN  reserved concurrency 3 is below 5; the SQS poller starts with 5 concurrent batches
== orders.fifo  batch=10 window=0s  function timeout=30s  visibility=200s  state=Enabled
  WARN  reserved concurrency 3 is below 5; the SQS poller starts with 5 concurrent batches

One honest caveat about the test: moto let us create the batch-size-100 trigger with no batching window, which real Lambda refuses when you create or update the mapping. The script flags it either way, which is useful for templates and older configurations you are reviewing before deployment.

Monitoring: knowing when batches fail

Lambda does not publish a metric for "messages returned in batchItemFailures", so you have to watch other signals:

• ApproximateAgeOfOldestMessage (SQS): a steady climb means messages are not being deleted, the classic sign of a whole batch failing again and again.
• NumberOfMessagesDeleted (SQS): if it drops to zero while messages arrive, your function's responses are not being accepted.
• A dead-letter queue on the source queue with a sensible maxReceiveCount, and an alarm on its ApproximateNumberOfMessagesVisible. Our DLQ and redrive guide walks through it.
• Your own metric: log the count of failures per invocation (with the CloudWatch embedded metric format, it becomes a metric without extra API calls) and alarm on it.
• Lambda's Throttles and ConcurrentExecutions, to spot reserved concurrency limiting the trigger.

What it costs, and why bigger batches are cheaper

You pay for Lambda invocations and duration, and for the SQS API requests the poller makes. A batch of 10 messages processed in one invocation costs one invocation instead of ten, and the poller's receive and delete calls also handle up to 10 messages per call. The poller keeps polling even when the queue is empty (with long polling), and those requests count as SQS requests, which is why an idle queue with a trigger still shows a small, steady request count. Lambda reduces its pollers on quiet queues to keep this low, unless maximum concurrency is set. For most workloads your request cost is tiny next to your function's own duration cost; your batch size and your function's speed matter far more.

Common problems with SQS batches

Messages are processed more than once

A failed batch without partial responses, a visibility timeout shorter than six times the function timeout, or normal at-least-once delivery. Turn on ReportBatchItemFailures, raise the visibility timeout, and make processing idempotent.

Some messages were never processed and there was no error

The function returned batchItemFailures, but the trigger does not have ReportBatchItemFailures turned on, so Lambda deleted the whole batch. Turn the setting on.

The same batch fails forever

The partial response is malformed (a misspelled key, an empty ID, a receipt handle instead of the message ID), so Lambda treats every response as complete failure. Check the response shape, and add a dead-letter queue so poison messages stop eventually.

"Batch size over 10" is rejected

Set MaximumBatchingWindowInSeconds to at least 1 second, and check that the queue is not FIFO, where the maximum is 10.

The function waits for the timeout before picking up another batch

Usually the invocation is not finishing: a background task, open connection or unawaited promise keeps it running until the timeout. Also check the batching window, which deliberately waits to fill a batch.

Throttling errors with an SQS trigger

Reserved concurrency is lower than the trigger needs. Raise it to at least 5, or use the trigger's maximum concurrency to limit load instead.

SQS Lambda batch size: frequently asked questions

With partial batch responses on, a visibility timeout of 185 seconds and maximum concurrency set to 20 to protect the stock database, Jake's orders went back to being processed exactly once, a hundred at a time. "So we kept the cheap batches," Jake said. "And stopped paying for the same order three times," Ethan said. If your results differ from what this page describes, please tell us. Guides like this stay right because readers write in.

Revision note. Written October 24, 2026, with a trigger checker and a partial-response validator tested against mocked queues. "Don't put all your eggs in one basket," the old saying goes; partial batch responses let you, safely.