What Is AWS X-Ray? Pricing, Free Tier, Tracing Lambda, ECS and API Gateway, OpenTelemetry Migration and Missing Traces

AWS X-Ray is AWS's distributed tracing service: it follows each request as it travels through your application (an API Gateway call, a Lambda function, a database query, a call to a payment provider) and shows you exactly where the time went and where errors happened, as a trace map of your services and a timeline for every request. Pricing is simple: the first 100,000 traces recorded and 1 million traces retrieved or scanned each month are free, then $5 per million traces recorded and $0.50 per million retrieved or scanned, with X-Ray Insights at $1 per million traces processed; because X-Ray samples requests by default, most apps pay little or nothing. One big change to know in 2026: the X-Ray SDKs and daemon entered maintenance on February 25, 2026 and reach end of support on February 25, 2027, so new instrumentation should use OpenTelemetry (AWS Distro for OpenTelemetry), which still sends traces to X-Ray; the X-Ray service itself continues and gains features such as CloudWatch Transaction Search. Use X-Ray when you need to know which hop in a request is slow, and turn on Lambda active tracing as the quickest way to start. Below: what X-Ray is and how it works, traces, segments and sampling, every price with worked examples, setup for Lambda, API Gateway, ECS and EC2, the OpenTelemetry migration, Transaction Search, X-Ray vs CloudWatch and Datadog, limits, and fixes for missing traces.

The furniture store's checkout had a problem nobody could pin down: most orders completed in under a second, but about one in fifty took four seconds or more, and a few timed out. The request went through API Gateway, a Lambda function, DocumentDB for the cart, a shipping-rate Lambda, and the payment provider's API. The logs showed every step finishing, eventually. Jake asked which step was the slow one. Ethan said that was exactly the question logs cannot answer, and opened the X-Ray console.

Debugging slow serverless apps? The cards below cover Lambda, Lambda timeouts, CloudWatch Logs costs, ECS and billing alerts, the pieces you will trace with X-Ray.

What is AWS X-Ray?

In a modern application, one click can touch a dozen components: an API gateway, several functions or containers, a database, a cache, a queue and an external API. When that click is slow, metrics tell you the average got worse and logs tell you what each component did, but neither tells you how the pieces of one specific request fit together. Distributed tracing does: each request gets a trace ID that every component passes along, and each component records how long its part took. Put together, you see the request's whole journey as a timeline.

AWS X-Ray is AWS's managed tracing service. It collects those timing records from your code and from AWS services, stitches them into traces, and gives you:

• A trace map (formerly service map): every service and dependency as a node, with average latency, request rate and error rates on each connection.
• Trace timelines: for each request, a waterfall of every call, how long it took, and any error or exception.
• Filtering and analytics: find the slowest requests, all requests that failed with a 5xx error, or every request for a particular customer, using filter expressions on annotations you add.
• X-Ray Insights: automatic detection of anomalies such as a jump in faults, with notifications.
• Native integrations with Lambda, API Gateway, Step Functions, SNS, SQS, App Runner, Elastic Beanstalk, and containers through OpenTelemetry.

Jake: "So it's a stopwatch that follows one order through every system."

Ethan: "Every order we sample, through every system, with the lap times written down. Then you just look for the long lap."

What X-Ray is used for: finding which call makes an API slow (a database query, a third-party API, a cold start), seeing why some requests fail while others succeed, understanding how a microservices or serverless system actually connects (the map often surprises its own builders), measuring the effect of a change on each hop, spotting retries and duplicate calls, and checking that queues and asynchronous steps are not adding hidden delay. It is most valuable in event-driven and serverless systems, where a single request crosses many short-lived components and there is no server to log in to.

How X-Ray works

1. A request arrives at the first traced component, for example an API Gateway stage or a Lambda function, which starts a trace and gives it a trace ID.
2. The trace ID travels with the request in the X-Amzn-Trace-Id header (or the W3C traceparent header with OpenTelemetry), from service to service and through SQS and SNS messages.
3. Each component records its part: a segment for the service's own work, and subsegments for each downstream call it makes (a DynamoDB query, an HTTP request, a database call), with start and end times, status and errors.
4. The data is sent to X-Ray: by the AWS service itself (Lambda, API Gateway), by an SDK or OpenTelemetry library in your code, through an agent or collector running next to your app (the X-Ray daemon, the CloudWatch agent or the ADOT Collector) that batches it to the X-Ray API.
5. X-Ray assembles the trace from all segments with the same ID, builds the trace map and statistics, and keeps the data for 30 days.
6. You explore it in the CloudWatch console (where X-Ray now lives, under Application Signals and Traces): the map, trace lists filtered by latency or errors, and each trace's timeline.

X-Ray does not record every request by default: sampling keeps costs and overhead low while still capturing enough requests to show patterns. When you need every request, Transaction Search (below) can capture all spans as logs.

Traces, segments, annotations and sampling

• Trace: the full record of one request, made of all segments sharing a trace ID.
• Segment: one service's work for that request (for example your Lambda function), with timing, HTTP status, errors and resource details.
• Subsegment: a timed block inside a segment, usually a downstream call (an AWS SDK call, an HTTP request, a SQL query) or a piece of your own code you want to time.
• Annotations: key-value pairs you add that X-Ray indexes, so you can filter on them, for example customer_tier = "trade" or order_id = "10442". Keep them for things you will search by.
• Metadata: extra data stored with the segment but not indexed, such as a request payload summary.
• Sampling rules: decide which requests are traced. The default rule records the first request each second plus 5 percent of additional requests, per service. Custom rules can sample more for a critical path (100 percent of checkout calls), less for health checks (0 percent), or by URL, method, host or service name.
• Groups: saved filter expressions (for example all traces with errors from the checkout service) with their own trace map, metrics and Insights notifications.
• Trace map: the graph of services and their connections, colored by health, built from all sampled traces.

Annotations are what turn X-Ray from "a picture of a slow request" into "every slow request for trade customers yesterday afternoon". The store added the order ID, the customer type and the shipping method as annotations, which later found the culprit in two minutes.

AWS X-Ray pricing

What you use	Free every month	Price after that
Traces recorded	100,000	$5.00 per million ($0.000005 each)
Traces retrieved or scanned	1,000,000	$0.50 per million
X-Ray Insights (traces processed)	None	$1.00 per million
Transaction Search: spans ingested to CloudWatch Logs	100 GB and 1 million indexed spans, for 3 months	$0.35 per GB (first 10 TB), $0.20 (next 20 TB), $0.15 (next 42 TB)
Transaction Search: span indexing	1% of spans included with ingestion	$0.75 per million indexed spans beyond that
Application Signals (APM)	100 million signals for 3 months	$1.50 per million signals (first 100M), $0.75 (next 900M), $0.30 beyond

• A trace counts once, however many segments it has; recording means sending it to X-Ray, retrieval means loading full traces, and scanning happens when you filter or search across traces.
• Sampling controls the bill: with the default rule, a service handling 100 requests a second records about 6 traces a second, roughly 15 million a month, about $75; at 10 requests a second it records about 1.5 a second, 3.8 million a month, about $19.
• Other costs: Lambda functions run slightly longer while sending trace data (milliseconds), collectors and agents use some CPU and memory, and Transaction Search spans are stored as CloudWatch Logs (see our CloudWatch Logs cost guide).

Is AWS X-Ray free?

For many small and medium applications, it effectively is. The X-Ray free tier is available every month, not just for your first year: 100,000 traces recorded and 1,000,000 traces retrieved or scanned. With the default sampling rule (one request a second plus 5 percent of the rest), a service averaging under about one request a second never leaves the free tier, and several small services together often stay close to it. What is not free: X-Ray Insights, Transaction Search beyond its three-month trial allowance, Application Signals beyond its trial, and the small extra compute your functions and collectors use. Raising sampling rates or turning on Transaction Search is where bills appear, so make those changes deliberately.

Worked examples: what X-Ray really costs

Example 1: the store's checkout. About 5 requests a second at peak, averaging 1.5 a second over the month, with a custom rule tracing 100 percent of checkout calls: about 3.9 million traces recorded a month. 3.9 million minus the free 100,000 is 3.8 million x $5 per million = $19 a month, plus a few cents of retrieval when Ethan investigates.

Example 2: a small app on the default rule. 148,800 traces a month: 48,800 above the free tier x $0.000005 = $0.24 a month, or $0.39 with Insights.

Ethan: "Nineteen dollars a month to see every checkout's lap times."

Jake: "Cheaper than one lost order a month."

Example 3: a busy API. 1,000 requests a second on the default rule records about 51 traces a second, about 132 million a month: 131.9 million x $5 per million = about $660 a month. Lowering the rate to 1 percent for healthy, high-volume paths cuts that to about $140, while a separate rule keeps 100 percent of errors and slow requests visible through Transaction Search or error-focused sampling.

Example 4: Transaction Search for full coverage. 2 TB of spans a month ingested to CloudWatch Logs: 2,048 GB x $0.35 = $716.80 a month, with 1 percent of spans indexed as X-Ray traces included. It costs more than sampled tracing, but every request becomes searchable.

Example 5: the accidental 100 percent. A team sets the sampling rate to 100 percent on a 2,000-request-per-second service "just for a day" and forgets: 5.2 billion traces in a month, about $26,000. Sampling rules deserve the same review as any other cost setting, and a billing alert catches the mistake in days.

Set up tracing: Lambda, API Gateway, Step Functions, ECS and EC2

1. Lambda: in the function's configuration, open Monitoring and operations tools and turn on Active tracing (or set TracingConfig: Mode: Active in your template). The execution role needs X-Ray write permissions (the AWS managed AWSXRayDaemonWriteAccess policy covers them). Lambda then records a segment for every sampled invocation. To trace calls your code makes, add OpenTelemetry instrumentation (the ADOT Lambda layer or Application Signals) rather than the legacy X-Ray SDK.
2. API Gateway (REST APIs): on the stage settings, turn on X-Ray tracing; API Gateway starts the trace and passes the header to your backend. HTTP APIs do not support X-Ray tracing, a common surprise.
3. Step Functions: turn on tracing for the state machine to see each state and the services it calls in one trace (see our Step Functions explainer).
4. ECS and EKS: run the ADOT Collector or the CloudWatch agent as a sidecar or DaemonSet, give the task role X-Ray permissions, and instrument your app with OpenTelemetry (automatic instrumentation agents exist for Java, Python, Node.js and .NET).
5. EC2 and on-premises servers: install the CloudWatch agent with its OpenTelemetry receiver enabled (the modern replacement for the X-Ray daemon), and instrument the app with OpenTelemetry.
6. Check the trace map in the CloudWatch console after a few requests; you should see each traced service and the calls between them.

# Turn on active tracing for a Lambda function from the CLI
aws lambda update-function-configuration --function-name store-checkout --tracing-config Mode=Active

# A custom sampling rule: trace every checkout request, capped by a reservoir of 10 per second
aws xray create-sampling-rule --sampling-rule '{
  "RuleName": "checkout-all", "Priority": 100, "FixedRate": 1.0, "ReservoirSize": 10,
  "ServiceName": "store-checkout", "ServiceType": "*", "Host": "*", "HTTPMethod": "POST",
  "URLPath": "/checkout*", "ResourceARN": "*", "Version": 1}'

# Find the slowest traces from the last hour with an annotation filter
aws xray get-trace-summaries --start-time $(date -d '-1 hour' +%s) --end-time $(date +%s) \
  --filter-expression 'service("store-checkout") AND duration > 3 AND annotation.shipping = "white-glove"'

Sampling rules apply across every service that uses them, so changing a rule in the console takes effect within about a minute without redeploying code. Lower numbers in Priority are evaluated first.

Tutorial: trace a serverless API in 15 minutes

1. Pick one request path that matters, such as checkout or sign-up, with an API Gateway REST API in front of a Lambda function.
2. Turn on tracing on the API stage and active tracing on the function, and attach X-Ray write permissions to the function's role.
3. Add the ADOT Lambda layer (or turn on Application Signals for the function) so calls to DynamoDB, S3 and HTTP APIs appear as subsegments.
4. Add two or three span attributes you will want to filter by, such as an order ID and a customer type.
5. Create a sampling rule that traces 100 percent of that path while you investigate, with a modest reservoir.
6. Send some traffic, open the trace map in the CloudWatch console, then list traces sorted by duration and open the slowest.
7. Turn the sampling back down once you have what you need, so the bill stays small.

Fifteen minutes is genuinely enough to see your first useful trace, and most teams find at least one surprise (an extra call, a slow dependency, a cold start pattern) the same afternoon.

X-Ray for Lambda in depth

With active tracing on, every sampled invocation produces a segment for the Lambda service and one for your function, with subsegments that are surprisingly informative on their own:

• Initialization: appears only on cold starts and shows how long your code's setup (imports, SDK clients, configuration) took, which is the first place to look when only some requests are slow. Our guide to Lambda timeouts covers what to do about long ones.
• Invocation: your handler's run time, with your instrumented downstream calls nested inside it.
• Overhead: time Lambda spent after your handler returned, such as running extensions.

Active vs PassThrough: in Active mode Lambda samples and records traces itself; in PassThrough mode (the default for functions called by an already-traced service) it only passes the trace header along, so you see the function inside upstream traces only if the upstream sampled the request. Asynchronous invocations and functions triggered by SQS keep the trace context, so a request that goes API Gateway, Lambda, SQS, Lambda shows the queue wait as part of one trace. Libraries such as Powertools for AWS Lambda wrap tracing in a few decorators; check that the version you use supports OpenTelemetry before you build on it for the long term.

Reading a trace: the store's slow checkout

Here is what the store's slow trace looked like, from the top of the timeline down:

1. API Gateway, POST /checkout: 4.12 seconds in total, status 200.
2. Lambda store-checkout, Invocation: 4.05 seconds, no Initialization subsegment, so not a cold start.
3. DocumentDB find cart: 38 milliseconds; the database was never the problem.
4. Lambda Invoke shipping-quote: 3.61 seconds, and inside it, two HTTP calls to the carrier's quote API of about 1.75 seconds each, the second identical to the first.
5. Payment provider call: 290 milliseconds.
6. Annotations: shipping = white-glove, customer_tier = retail.

Filtering all traces over 3 seconds showed the same shape every time, always with white-glove delivery: the carrier's white-glove quotes were slow, and a retry wrapper in the store's code called the API twice even when the first call succeeded. Nobody would have found the duplicate call in logs, because each call logged a normal success. That is the kind of problem tracing exists for: the time is not lost in one broken component, but in how healthy components are combined.

Tracing asynchronous and event-driven flows

• SQS: the trace header travels as a message system attribute, so a consumer Lambda continues the producer's trace and the time a message waited in the queue becomes visible.
• SNS: active tracing on a topic passes the trace context to subscribers such as Lambda and SQS.
• Step Functions: with tracing on, each state, retry and service integration appears in one trace, which makes long workflows readable.
• EventBridge and custom queues: pass the trace context in the event or message (OpenTelemetry propagators can do this for you) and start a linked span in the consumer.
• Batch processing: when one consumer handles many messages from different requests, OpenTelemetry span links connect the batch span to each original trace instead of forcing them into one.

The rule of thumb: every place a request crosses a boundary (HTTP, a queue, a topic, a workflow), the trace context must cross with it, or the trace breaks into pieces. Most "missing" services on a trace map are missing at exactly such a boundary.

Tracing an ECS service, step by step

1. Add a collector container to your task definition: the CloudWatch agent (or the ADOT Collector image) as a sidecar, configured to receive OTLP traces on ports 4317 and 4318 and export them to X-Ray.
2. Give the task role permissions to write traces (xray:PutTraceSegments, xray:PutTelemetryRecords) and to read sampling rules (xray:GetSamplingRules, xray:GetSamplingTargets).
3. Instrument the application container with OpenTelemetry automatic instrumentation for its language, pointing the exporter at localhost:4318 (containers in the same task share a network namespace in awsvpc mode).
4. Set a service name (OTEL_SERVICE_NAME) so the trace map shows a meaningful node rather than a container ID.
5. Deploy and send a few requests, then check the trace map; the service appears with its calls to databases, other services and AWS APIs.

For many services at once, Application Signals can enable this for ECS and EKS with less per-service setup, and on EKS a DaemonSet collector per node replaces per-task sidecars. Our ECS explainer covers task definitions and roles.

The X-Ray daemon and SDK, if you still run them

The X-Ray daemon is a small program that runs next to your application (on an EC2 instance, as an ECS sidecar, or built into Elastic Beanstalk and Lambda), listens for segment data from the X-Ray SDK on UDP port 2000, buffers it, and sends it in batches to the X-Ray API, so your application never waits on the network to record a trace. It also fetches sampling rules. The X-Ray SDK is the library in your code that creates segments and subsegments and patches common libraries (the AWS SDK, HTTP clients, SQL drivers) to record calls automatically.

Existing code keeps working until you migrate. A typical legacy Python setup looks like this:

# Legacy X-Ray SDK for Python (maintenance mode since Feb 25, 2026; plan to replace with OpenTelemetry)
from aws_xray_sdk.core import xray_recorder, patch_all

patch_all()   # traces boto3, requests, sqlite3 and other supported libraries automatically

@xray_recorder.capture("quote_shipping")
def quote_shipping(order):
    xray_recorder.put_annotation("order_id", order["id"])
    return call_rates_api(order)

If you inherit a system like this, inventory where the SDK and daemon run, and migrate the busiest or most important services first. The trace map keeps working during the transition, because both kinds of instrumentation report to the same X-Ray service.

The 2026 change: X-Ray SDKs and daemon end of support, and OpenTelemetry

AWS has moved X-Ray's instrumentation to OpenTelemetry, the open, vendor-neutral standard for traces, metrics and logs. The timeline:

• February 25, 2026: the X-Ray SDKs (for Java, Python, Node.js, .NET, Go and Ruby) and the X-Ray daemon entered maintenance mode: only critical security fixes, no new features.
• February 25, 2027: end of support. No more releases; existing versions stay downloadable and keep working, and X-Ray keeps accepting their traces, but you are on unsupported code.
• The X-Ray service is not retiring. It keeps receiving traces, now natively in OpenTelemetry format, and gains features such as CloudWatch Transaction Search.

You use today	Replace it with	Notes
X-Ray daemon (UDP 2000)	CloudWatch agent with OTLP receiver, or the ADOT Collector	Listens on 4317/4318; also forwards metrics and logs
X-Ray SDK for Java	ADOT Java agent (automatic instrumentation)	Attach with -javaagent; no code changes for common libraries
X-Ray SDK for Python	aws-opentelemetry-distro + opentelemetry-instrument	Replaces patch_all and xray_recorder
X-Ray SDK for Node.js	ADOT JavaScript auto-instrumentation	Loaded with a require or import hook at startup
X-Ray SDK for .NET and Go	OpenTelemetry SDKs with AWS components	Some manual setup of exporters and propagators
X-Ray SDK in Lambda	ADOT Lambda layer or Application Signals for Lambda	Keep active tracing on for the invocation segment
Annotations and metadata	Span attributes	Mapped to X-Ray annotations for filtering
X-Ray sampling rules	The X-Ray remote sampler in OpenTelemetry	Same rules, managed centrally

What to use instead: AWS Distro for OpenTelemetry (ADOT), AWS's supported distribution of the OpenTelemetry SDKs, agents and collector, or upstream OpenTelemetry. Both can send traces to X-Ray. In practice:

1. Replace the X-Ray daemon with the CloudWatch agent (with its OpenTelemetry receiver) or the ADOT Collector, which accept OTLP data from your apps and forward it to X-Ray.
2. Replace SDK instrumentation with OpenTelemetry: for many languages, zero-code automatic instrumentation (a Java agent, opentelemetry-instrument for Python, a Node.js require hook) traces HTTP servers, clients, AWS SDK calls and databases without changing your code.
3. Move annotations to OpenTelemetry span attributes; the AWS exporters map them so they stay searchable.
4. Switch propagation to W3C Trace Context where your services allow, keeping X-Ray propagation where AWS services still use the X-Amzn-Trace-Id header.
5. For Lambda, use the ADOT Lambda layer or Application Signals for Lambda instead of bundling the X-Ray SDK.

# Python app on ECS or EC2: automatic OpenTelemetry instrumentation, exported over OTLP
pip install aws-opentelemetry-distro
export OTEL_SERVICE_NAME=store-checkout
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces   # the CloudWatch agent or ADOT Collector
export OTEL_TRACES_SAMPLER=xray                                              # use X-Ray sampling rules
opentelemetry-instrument python app.py

Adding your own span attributes (the OpenTelemetry equivalent of annotations) is a couple of lines:

from opentelemetry import trace

tracer = trace.get_tracer("store.checkout")

def quote_shipping(order):
    with tracer.start_as_current_span("quote_shipping") as span:
        span.set_attribute("order_id", order["id"])
        span.set_attribute("shipping", order["shipping_method"])
        return call_rates_api(order)

Plan the migration before February 2027, service by service; the trace map keeps working during the mix because both old and new instrumentation send traces to the same place. Our list of AWS services and features retiring covers other deadlines.

CloudWatch Transaction Search and Application Signals

• Transaction Search sends every span (not just sampled traces) to CloudWatch Logs in a dedicated log group, where you can search and analyze all of them, for example "every checkout over 3 seconds for trade customers in the last week", and jump from a span to its full trace. One percent of spans are indexed as X-Ray traces at no extra cost; you can index more for a fee. It is the answer when sampling hides the one request you need.
• Application Signals is CloudWatch's application performance monitoring (APM): it automatically instruments your services, builds dashboards of latency, errors and availability per service and operation, and lets you set service level objectives (SLOs) with alerts when you burn through your error budget. It uses the same traces, billed per signal.
• How they fit: X-Ray traces for request-level detail, Transaction Search for complete searchable coverage, Application Signals for service health and SLOs, CloudWatch Logs and metrics for everything else, all in the CloudWatch console.

X-Ray vs CloudWatch, Datadog, New Relic and open-source tracing

• X-Ray vs CloudWatch Logs and metrics: metrics show that latency rose; logs show what one component did; X-Ray shows how one request moved through every component and which hop was slow. You need all three, and they now share one console.
• X-Ray vs Datadog, New Relic, Dynatrace and other APM tools: commercial platforms offer polished APM, profiling, real-user monitoring and broad integrations across clouds, usually at a higher price per host or per GB. X-Ray and Application Signals are AWS-native, cheap at low to moderate volume, and need no third-party agent. Many teams use OpenTelemetry so they can send the same data to either.
• X-Ray vs Jaeger, Zipkin and Grafana Tempo: open-source tracing back ends you host yourself; free per trace, but you run the storage and UI. Amazon Managed Grafana can display X-Ray traces alongside Tempo or Prometheus data.
• X-Ray vs CloudTrail: CloudTrail records who called which AWS API (an audit log); X-Ray records how your application's requests performed. Different questions entirely.
• X-Ray vs Amazon Inspector: despite the similar-sounding names, Inspector scans EC2 instances, container images and Lambda functions for software vulnerabilities; it has nothing to do with request tracing.
• X-Ray vs Dynatrace and New Relic: full-stack observability platforms with automatic discovery, code-level profiling, AI-assisted root cause analysis and real-user monitoring across clouds and on-premises. They are powerful and priced accordingly; X-Ray covers request tracing on AWS at a fraction of the cost, and Application Signals adds the APM dashboards and SLOs many teams need.

The trace map, explained: each node is a service (your function, a container service, a DynamoDB table, an external API), each edge is a call between them, and the colors show health: green for successful, yellow for client errors (4xx), red for faults (5xx), purple for throttling. Select a node to see its latency distribution and jump to the traces behind a spike. The map is built only from sampled traces, so rarely used paths may appear and disappear.

The X-Ray API in one list

• Sending data: PutTraceSegments and PutTelemetryRecords (used by the daemon, agents and SDKs).
• Finding traces: GetTraceSummaries (filter by time and expression), BatchGetTraces (full trace documents), GetTraceGraph.
• Maps and statistics: GetServiceGraph and GetTimeSeriesServiceStatistics.
• Sampling: CreateSamplingRule, UpdateSamplingRule, GetSamplingRules, GetSamplingTargets.
• Groups and Insights: CreateGroup, GetInsightSummaries, GetInsightEvents.
• Encryption: PutEncryptionConfig to encrypt trace data with your own KMS key.

These APIs are how dashboards, CI checks ("did this deployment make the checkout slower?") and incident tools pull trace data automatically. Remember that every trace you retrieve or scan through them counts toward the retrieval charge after the first million a month, so a dashboard that polls every few seconds over wide time ranges can cost more than the tracing itself; cache results or query narrower windows.

Security and privacy of trace data

• Access: anyone with xray:BatchGetTraces or console access can read traces, including URLs, annotations and metadata; grant it like access to logs.
• Personal data: keep emails, names, card numbers and tokens out of annotations, metadata and span attributes; record IDs you can look up elsewhere instead.
• URLs and headers: query strings can contain tokens or personal data; OpenTelemetry instrumentation can be configured to drop or redact them.
• Encryption: traces are encrypted at rest, optionally with your own KMS key, and sent over TLS.
• Retention: X-Ray keeps traces for 30 days; spans in Transaction Search follow your log group's retention, so set it consciously.

A useful habit is to review a handful of real traces from production after you instrument a new service, specifically looking for anything you would not want in a support ticket: full URLs with tokens, request bodies in metadata, or customer details in attributes. Automatic instrumentation records more than people expect, and it is far easier to add a redaction rule on day one than to explain later why personal data sat in trace storage for a month. For regulated workloads, check that X-Ray is in scope for your compliance program before sending it sensitive traffic.

Limits and quotas

• Retention: traces are kept for 30 days; Transaction Search spans follow your CloudWatch Logs retention.
• Segment size: up to 64 KB per segment document; very large metadata gets dropped, so keep payloads out of segments.
• Annotations: up to 50 per trace, with keys of letters, numbers and underscores.
• Sampling rules and groups: a limited number per account and Region (adjustable), evaluated by priority.
• Throughput: default quotas on segments per second per Region apply, adjustable through Service Quotas for very busy accounts.
• Encryption: trace data is encrypted at rest by default; you can choose your own KMS key.

If you hit the segment-size or throughput limits, it is usually a sign that the instrumentation records too much: large metadata, a subsegment for every loop iteration, or 100 percent sampling on a very busy path. Trimming those makes traces easier to read as well as cheaper.

Missing traces and other common problems

• No traces at all: tracing is not turned on (Lambda active tracing, API Gateway stage tracing), the role lacks xray:PutTraceSegments and xray:PutTelemetryRecords, or the agent or collector is not running or not reachable.
• Traces from Lambda but not from your downstream calls: active tracing only records the invocation; add OpenTelemetry instrumentation (or the legacy SDK's patching) for SDK, HTTP and database calls.
• Broken traces (services appear disconnected): the trace header is not passed along, for example through an HTTP client that is not instrumented, a message queue consumer that ignores the trace context, or a proxy that strips headers.
• Daemon "connection refused" or nothing sent from containers: the X-Ray daemon listens on UDP port 2000 (OTLP collectors on 4317 and 4318); make sure the app sends to the right host and port inside the task or pod.
• Only some requests are traced: that is sampling working as designed. Add a rule with a higher rate for the path you are investigating, or use Transaction Search.
• Lambda shows "PassThrough" mode: the function only forwards trace headers from upstream services; set the mode to Active to record segments.
• HTTP API in API Gateway not traced: HTTP APIs do not support X-Ray; trace from the backend Lambda, or use a REST API stage.
• Odd timelines or negative durations: clock skew on self-managed hosts; keep instance clocks synchronized with the Amazon Time Sync Service.

Best practices

• Start with Lambda and API Gateway tracing; it costs little and shows most bottlenecks.
• Use OpenTelemetry (ADOT) for all new instrumentation, and plan the move off the X-Ray SDKs before February 25, 2027.
• Write sampling rules on purpose: high rates for critical, low-volume paths; low rates for busy, healthy ones; zero for health checks.
• Add a few useful annotations (order or customer ID, tier, feature flag) so you can find the traces you care about.
• Use groups and Insights for the paths that matter, with notifications on fault spikes.
• Keep payloads and personal data out of traces; they are visible to anyone with trace access.
• Review tracing costs monthly alongside CloudWatch, and set a billing alert.

Frequently asked questions

The answer took two minutes once the traces were in. Filtering for checkouts over 3 seconds showed the same pattern every time: a 3.5-second subsegment in the shipping-rate Lambda, calling the carrier's quote API, and only when the annotation said white-glove delivery. The carrier's white-glove quote endpoint was slow, and the store's code called it twice. Caching the quote for ten minutes and removing the duplicate call brought the slowest checkouts under a second. The X-Ray bill was $19. Jake asked whether the stopwatch could stay on. "It already is," Ethan said. "That's the point. Next time, we'll know before the customers do."

Revision note. Written October 21, 2026. Prices are US East (N. Virginia) list prices on that date. We will update this page when X-Ray prices or features change.