Amazon Bedrock Streaming Responses: ConverseStream in Python and JavaScript, Streamlit Chat, Lambda Function URLs, WebSocket, Agents, and Every Streaming Error

To get a streaming response from Amazon Bedrock, use ConverseStream instead of Converse (converse_stream() in boto3, ConverseStreamCommand in JavaScript) and read the events as they arrive: text comes in contentBlockDelta events, the reason the model stopped in messageStop, and token usage and latency in a final metadata event. For each model's native request format, the older InvokeModelWithResponseStream returns raw JSON chunk events instead. Your IAM policy needs bedrock:InvokeModelWithResponseStream (which also covers ConverseStream). Getting the stream to a browser is a separate step: a Lambda function URL in RESPONSE_STREAM mode (Node.js, or Python through the Lambda Web Adapter), an API Gateway REST API with response streaming, a WebSocket API, or a small container serving server-sent events. Below: complete Python and JavaScript examples, a Streamlit chat app with history, streaming with reasoning and tool calls, streaming from Bedrock Agents, knowledge bases and AgentCore, and the timeouts and errors that break streams.

Ethan's warranty assistant for Jake's computer shop worked, but it felt slow. The front desk typed a question and stared at a blank box for six or seven seconds while the model wrote its full answer, then the whole answer appeared at once. The model was not slow at all: the first words were ready in under a second. The app just waited for the last word before showing the first. Streaming fixes that, and it changed how the assistant felt more than any model upgrade could have. This page is how to do it, end to end.

New to Bedrock? Our plain-English Bedrock explainer covers the service, and the cards below cover the rest.

Why streaming changes how your app feels

A language model writes its answer one small piece (a token, roughly three-quarters of a word) at a time. Without streaming, your code waits until the very last token, then shows everything. With streaming, you show each piece as it is written.

Two numbers explain the difference:

• Time to first token: how long until the first words appear. Typically well under a second or two for a short prompt.
• Total time: how long until the answer is complete. For a few paragraphs, often five to fifteen seconds, longer for reasoning models.

Without streaming, the user waits the total time staring at nothing. With streaming, they wait only the time to first token, then read along as the answer is written, which feels several times faster even though the total time is identical. Streaming also lets your user stop a wrong answer early, and it keeps long-running requests from hitting timeouts in some setups.

Streaming does not make the model cheaper or smarter: you pay the same tokens either way. It is purely about how the answer is delivered, which is why it belongs in every chat-style interface and usually nowhere else. A script that summarizes 500 documents overnight gains nothing from it.

Jake: "So the answer was ready all along, we just weren't showing it?"

Ethan: "The first sentence was ready in half a second. We were waiting for the last one before showing any of it."

Add streaming to an existing Bedrock app, step by step

If your app already calls Converse and works, streaming is a contained change. In order:

1. Allow the streaming action in IAM: add bedrock:InvokeModelWithResponseStream next to bedrock:InvokeModel for the same models and inference profiles.
2. Switch the call from converse() to converse_stream(); the request stays the same.
3. Replace "read the answer" with a loop over the stream that passes each text delta onward as soon as it arrives.
4. Move your error handling inside the loop, because errors can now arrive after the answer has started.
5. Stream to the user, through a function URL in RESPONSE_STREAM mode, a streaming REST API, a WebSocket, or your web framework's streaming response.
6. Test layer by layer: script to Bedrock, then function, then URL or API, then browser, until the words arrive piece by piece at the screen.

The first four steps take minutes. The fifth is where the real work is, and the sections below cover each delivery route with code.

Streaming with ConverseStream in Python

ConverseStream uses the same request as Converse, so if you already call converse(), switching is one word plus a loop:

import boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")

resp = brt.converse_stream(
    modelId="us.amazon.nova-pro-v1:0",
    system=[{"text": "You answer questions for a computer repair shop. Be brief."}],
    messages=[{"role": "user", "content": [{"text": "Why would a laptop battery swell?"}]}],
    inferenceConfig={"maxTokens": 600, "temperature": 0.3},
)

for event in resp["stream"]:
    if "contentBlockDelta" in event:
        delta = event["contentBlockDelta"]["delta"]
        if "text" in delta:
            print(delta["text"], end="", flush=True)
    elif "messageStop" in event:
        print("\n[stop reason:", event["messageStop"]["stopReason"] + "]")
    elif "metadata" in event:
        meta = event["metadata"]
        print("[tokens:", meta["usage"], "| latency ms:", meta["metrics"]["latencyMs"], "]")

What happens here: converse_stream() sends the request and returns as soon as Bedrock starts responding, well before the answer is written. resp["stream"] is an event stream you can loop over once; each pass of the loop waits for the next event. If you need the complete text afterwards, for your chat history or a log, collect the pieces in a list as you print them and join them at the end, because there is no second chance to read the stream.

flush=True matters in a terminal: without it, Python may hold the text in a buffer and print it in bursts, which looks like streaming is not working. The same modelId rules apply as for Converse: many current models need an inference profile ID such as us. or global., covered in our model IDs guide.

The ConverseStream events, explained

Errors can also arrive inside the stream, after it has started: modelStreamErrorException, throttlingException, validationException, serviceUnavailableException and internalServerException. In boto3 these surface as exceptions raised from the loop, so wrap the loop, not just the initial call, in your error handling. A stream that dies halfway leaves your user with half an answer; tell them so and offer a retry rather than showing it as complete.

The stopReason deserves a glance in every app. max_tokens means the answer was cut off by your limit, which users experience as an answer that ends mid-sentence. Raise maxTokens or ask for shorter answers.

Streaming with reasoning and tool calls

Reasoning models and tool calls add two more kinds of delta. A fuller loop collects all three:

import json

answer, thinking, tool_calls, current = [], [], [], None

for event in resp["stream"]:
    if "contentBlockStart" in event:
        start = event["contentBlockStart"]["start"]
        if "toolUse" in start:
            current = {"name": start["toolUse"]["name"],
                       "id": start["toolUse"]["toolUseId"], "input": ""}
    elif "contentBlockDelta" in event:
        d = event["contentBlockDelta"]["delta"]
        if "text" in d:
            answer.append(d["text"]); print(d["text"], end="", flush=True)
        elif "reasoningContent" in d and "text" in d["reasoningContent"]:
            thinking.append(d["reasoningContent"]["text"])
        elif "toolUse" in d and current:
            current["input"] += d["toolUse"]["input"]          # partial JSON, piece by piece
    elif "contentBlockStop" in event and current:
        current["input"] = json.loads(current["input"] or "{}")
        tool_calls.append(current); current = None
    elif "messageStop" in event:
        stop = event["messageStop"]["stopReason"]

Two points about tools. The tool's arguments arrive as fragments of JSON text; only when the block stops is the string complete enough to parse. And when stopReason is tool_use, the model is waiting for you: run the tools, add their results to the conversation, and call ConverseStream again for the final answer. Showing the user "Checking stock..." while the tool runs keeps the stream feeling alive.

Most apps keep reasoning text out of the main answer. Show it in a collapsible "thinking" panel if your users find it useful, or simply discard it.

InvokeModelWithResponseStream and the anthropic_version question

Before Converse existed, streaming meant InvokeModelWithResponseStream, which takes each model's native request body and returns raw chunk events, each holding bytes of that model's own JSON. It still works, and it is the only route for features a provider exposes only in its native format:

import json
encode = json.JSONEncoder().encode

body = {
    "schemaVersion": "messages-v1",
    "messages": [{"role": "user", "content": [{"text": "Why would a laptop battery swell?"}]}],
    "inferenceConfig": {"maxTokens": 600},
}
resp = brt.invoke_model_with_response_stream(modelId="us.amazon.nova-pro-v1:0", body=encode(body))

for event in resp["body"]:
    chunk = json.loads(event["chunk"]["bytes"])
    text = chunk.get("contentBlockDelta", {}).get("delta", {}).get("text")
    if text:
        print(text, end="", flush=True)

The shape of each chunk differs by provider, which is exactly the pain Converse removes. For new code, use ConverseStream unless you need something only the native format offers.

"What anthropic_version do I use?" When you call Anthropic's models through InvokeModel or InvokeModelWithResponseStream, the body must include "anthropic_version": "bedrock-2023-05-31". Despite the date, that string is current: it names the version of the request format on Bedrock, not the model, and it does not change when you move to a newer model. Converse and ConverseStream need no such field.

Streaming in JavaScript (Node.js)

import { BedrockRuntimeClient, ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";

const client = new BedrockRuntimeClient({ region: "us-east-1" });

const res = await client.send(new ConverseStreamCommand({
  modelId: "us.amazon.nova-pro-v1:0",
  messages: [{ role: "user", content: [{ text: "Why would a laptop battery swell?" }] }],
  inferenceConfig: { maxTokens: 600 },
}));

for await (const ev of res.stream) {
  const text = ev.contentBlockDelta?.delta?.text;
  if (text) process.stdout.write(text);
  if (ev.metadata) console.log("\n", ev.metadata.usage);
}

As in Python, errors can surface after the first text has arrived, so put the whole for await loop inside a try/catch, and tell the user when an answer was cut short rather than leaving half a reply looking finished. The stopReason arrives on the messageStop event as ev.messageStop.stopReason.

The package is @aws-sdk/client-bedrock-runtime (version 3 of the SDK). A common mistake from older examples is importing from @aws-sdk/client-bedrock, the control-plane client for managing models and jobs, which has no streaming commands at all.

In the browser, reading a streamed HTTP response from your backend takes a few lines with fetch:

async function ask(question, output) {
  const res = await fetch(FUNCTION_URL, {
    method: "POST",
    body: JSON.stringify({ question }),
  });
  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  output.textContent = "";
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    output.textContent += decoder.decode(value, { stream: true });
  }
}

Each read() returns whatever bytes have arrived so far, and { stream: true } keeps characters that span two pieces, such as accented letters or emoji, from breaking. If your page is on a different domain from the function URL, enable CORS on the URL for your site's origin.

A Streamlit chat app with streaming and history

Streamlit is the quickest way to put a streaming chat in a browser for internal use, and it is how Ethan first gave the front desk the warranty assistant. Save this as app.py, then pip install streamlit boto3 and streamlit run app.py:

import boto3
import streamlit as st

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
MODEL = "us.amazon.nova-pro-v1:0"

st.title("Shop warranty assistant")
if "history" not in st.session_state:
    st.session_state.history = []          # Converse-format messages

for m in st.session_state.history:
    with st.chat_message(m["role"]):
        st.write(m["content"][0]["text"])

if question := st.chat_input("Ask about a warranty or repair"):
    st.session_state.history.append({"role": "user", "content": [{"text": question}]})
    with st.chat_message("user"):
        st.write(question)

    def tokens():
        resp = brt.converse_stream(
            modelId=MODEL,
            system=[{"text": "You answer warranty questions for a computer repair shop."}],
            messages=st.session_state.history,
            inferenceConfig={"maxTokens": 600},
        )
        for ev in resp["stream"]:
            text = ev.get("contentBlockDelta", {}).get("delta", {}).get("text")
            if text:
                yield text

    with st.chat_message("assistant"):
        full = st.write_stream(tokens())
    st.session_state.history.append({"role": "assistant", "content": [{"text": full}]})

st.write_stream displays each piece as the generator yields it and returns the full text at the end, which goes back into the history. Sending the whole history with every request is what "keeps the conversation going": the model has no memory between calls, so it knows only what you send. For long chats, add prompt caching on the history (a cachePoint after the last earlier message) so earlier turns cost less with each new message.

Streamlit suits internal tools. For customer-facing apps, you will want your own front end and one of the delivery options below.

Getting the stream to a browser: your options

Bedrock streams to your code. Your code then has to stream to the user's browser, and that is where most "streaming doesn't work" problems really live. The main options on AWS:

Our REST vs WebSocket explainer helps choose between the API Gateway options. The serverless route most people start with is the function URL.

A streaming Lambda function (Node.js). Lambda response streaming works with the Node.js managed runtimes and with custom runtimes. Wrap the handler in awslambda.streamifyResponse and write to the stream:

import { BedrockRuntimeClient, ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";

const client = new BedrockRuntimeClient({});

export const handler = awslambda.streamifyResponse(async (event, responseStream) => {
  const { question } = JSON.parse(event.body || "{}");
  const res = await client.send(new ConverseStreamCommand({
    modelId: "us.amazon.nova-pro-v1:0",
    messages: [{ role: "user", content: [{ text: question || "Hello" }] }],
    inferenceConfig: { maxTokens: 600 },
  }));
  for await (const ev of res.stream) {
    const text = ev.contentBlockDelta?.delta?.text;
    if (text) responseStream.write(text);
  }
  responseStream.end();
});

Then create a function URL with the invoke mode set to RESPONSE_STREAM (the default, BUFFERED, waits for the whole response, which silently undoes the streaming). Give the function's role bedrock:InvokeModelWithResponseStream, and raise its timeout well above the default 3 seconds; our Lambda timeout guide shows where. Streamed responses can be up to 200 MB by default, far more than any chat answer, and bytes streamed beyond the first 6 MB carry an extra network charge.

Python on Lambda. Python's managed runtime does not stream responses directly. The usual route is the Lambda Web Adapter, which runs an ordinary web framework (FastAPI, Flask) inside Lambda and passes its streamed responses through a function URL. Alternatively, run the Python app in a container on App Runner or ECS, where streaming works like on any server.

In the browser, read the response body as a stream with fetch and a reader, appending each piece to the page as it arrives. And if anything sits between the browser and your app (a reverse proxy such as nginx, a load balancer, a CDN), make sure it does not buffer responses, or it will collect the whole stream and deliver it at once, which looks exactly like streaming is broken.

Jake: "So it was streaming from Amazon, but not to our screen?"

Ethan: "On the first try, yes. The function URL was set to buffered. One setting, and the words started appearing as they were written."

Streaming from a Python web app (FastAPI, server-sent events)

If your backend is Python, a small FastAPI app streams Bedrock to the browser with server-sent events (SSE), a simple standard where the server sends lines starting with data: and the browser's EventSource (or a fetch reader) receives them:

import boto3
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()
brt = boto3.client("bedrock-runtime", region_name="us-east-1")

@app.get("/ask")
def ask(q: str):
    def events():
        resp = brt.converse_stream(
            modelId="us.amazon.nova-pro-v1:0",
            messages=[{"role": "user", "content": [{"text": q}]}],
            inferenceConfig={"maxTokens": 600},
        )
        for ev in resp["stream"]:
            text = ev.get("contentBlockDelta", {}).get("delta", {}).get("text")
            if text:
                yield "data: " + text.replace("\n", "\ndata: ") + "\n\n"
        yield "event: done\ndata: \n\n"
    return StreamingResponse(events(), media_type="text/event-stream")

Run it with uvicorn app:app. Because boto3 is synchronous, the generator is a plain function and FastAPI runs it in a worker thread, which is fine for modest traffic. Multi-line text is split into several data: lines, as the SSE format requires, and a final done event tells the browser the answer is complete.

To host it on AWS, run the app in a container on App Runner or ECS, where streaming works like any web server, or package it for Lambda with the Lambda Web Adapter and a RESPONSE_STREAM function URL. Put authentication in front before anyone outside your network can reach it: every request spends tokens.

Streaming over an API Gateway WebSocket

A WebSocket keeps one connection open in both directions, which suits long chat sessions: the browser sends questions, and your function sends back each piece of the answer as its own message. The setup:

1. Create a WebSocket API in API Gateway with the routes $connect, $disconnect and a custom route such as ask, chosen by a field in the incoming message.
2. Integrate the ask route with a Lambda function that calls ConverseStream.
3. Give the function permission to send messages back: execute-api:ManageConnections on the API, plus the Bedrock streaming action.
4. Deploy the API to a stage and note its wss:// URL and its management endpoint.
5. Connect from the browser with new WebSocket(url), send {"action": "ask", "question": "..."}, and append each incoming message to the answer.

import json, os
import boto3

brt = boto3.client("bedrock-runtime")
encode = json.JSONEncoder().encode

def handler(event, context):
    ctx = event["requestContext"]
    mgmt = boto3.client("apigatewaymanagementapi",
                        endpoint_url=f"https://{ctx['domainName']}/{ctx['stage']}")
    question = json.loads(event["body"]).get("question", "")
    resp = brt.converse_stream(
        modelId="us.amazon.nova-pro-v1:0",
        messages=[{"role": "user", "content": [{"text": question}]}],
        inferenceConfig={"maxTokens": 600},
    )
    for ev in resp["stream"]:
        text = ev.get("contentBlockDelta", {}).get("delta", {}).get("text")
        if text:
            mgmt.post_to_connection(ConnectionId=ctx["connectionId"],
                                    Data=encode({"type": "delta", "text": text}))
    mgmt.post_to_connection(ConnectionId=ctx["connectionId"], Data=encode({"type": "done"}))
    return {"statusCode": 200}

Sending one WebSocket message per delta is simple but chatty; batching a few deltas (say, every 100 milliseconds) cuts the number of messages without the user noticing. If the connection fails before any of this runs, our WebSocket connection guide covers the usual route and permission problems.

Streaming with Bedrock Guardrails

If you apply a guardrail to filter harmful content or personal information, streaming adds one choice. ConverseStream's guardrail settings include a stream processing mode:

• sync: the guardrail checks the response in chunks before they reach you. Safer, with a little extra delay as text is held for checking.
• async: text is sent to you immediately while the guardrail checks in the background. Faster, but content the guardrail later flags may already have been shown, so your app must be ready to replace or retract it.

For customer-facing apps where a leaked phone number or offensive phrase on screen is unacceptable even briefly, use sync. For internal tools where speed matters more, async is often fine. A blocked answer arrives with the stop reason guardrail_intervened, which your loop should handle by showing the guardrail's configured message instead of a half-finished answer.

Streaming from Bedrock Agents, knowledge bases and AgentCore

Knowledge bases. If you use a Bedrock knowledge base for RAG, RetrieveAndGenerateStream streams the generated answer the same way, with citations arriving alongside the text, so you can show sources as the answer forms.

Bedrock Agents. InvokeAgent returns its result as an event stream of chunks. By default an agent may deliver its final answer in larger pieces once it has finished its steps; to stream the final answer token by token, set streamingConfigurations={"streamFinalResponse": True} on the call. Agents that run several tool steps first still take time before the final answer begins; showing progress from the trace events keeps users informed during those steps.

AgentCore. Agents you host on Amazon Bedrock AgentCore Runtime can stream too: write the entrypoint as an asynchronous generator that yields pieces as your framework produces them, and AgentCore streams them to the caller as server-sent events. Frameworks such as LangGraph and Strands provide streaming methods that slot straight into that pattern; our LangChain with Bedrock guide shows LangChain's stream and astream, which call ConverseStream under the hood.

Common streaming errors and fixes

When a stream misbehaves, test in layers: first a plain Python script straight to Bedrock (if that streams, Bedrock is fine), then your function on its own, then the function behind its URL or API, then the browser. The layer where the text stops arriving piece by piece is the one to fix.

Measuring time to first token and total time

The metadata event reports latencyMs for the whole response, but the number your users feel most is time to first token, and you measure that yourself in two lines:

import time

start = time.perf_counter()
first = None
resp = brt.converse_stream(modelId="us.amazon.nova-pro-v1:0",
                           messages=[{"role": "user", "content": [{"text": "Hello"}]}])
for ev in resp["stream"]:
    if first is None and "contentBlockDelta" in ev:
        first = time.perf_counter() - start
total = time.perf_counter() - start
print(f"first token after {first:.2f}s, finished after {total:.2f}s")

Log both numbers for real traffic, not just one test. Time to first token rises with prompt length (the model reads everything first), with reasoning effort on thinking models, and at busy times; total time rises with answer length. If first tokens are slow, shorten or cache the prompt; if totals are slow, ask for shorter answers or a faster model.

Streaming from async Python apps

boto3 is synchronous: each loop step blocks while waiting for the next event. In a plain script or a threaded web server that is fine. In an asyncio application (an async FastAPI endpoint, a Discord or Slack bot, an AgentCore entrypoint), a blocking loop stalls every other task while it waits. Two common fixes:

• Run the blocking loop in a thread and pass pieces back through an asyncio.Queue, using asyncio.to_thread or a thread pool. Simple, and it uses the official SDK.
• Use an async AWS SDK wrapper such as aioboto3 or aiobotocore, community libraries that offer async for over the same Bedrock streams.

LangChain users get this for free: llm.astream(...) on ChatBedrockConverse handles the async plumbing, which is one reason many teams use it for streaming chat backends.

Making streamed chat feel fast

• Keep the first token quick. Long prompts delay the first word, because the model reads the whole prompt first. Shorter system prompts, fewer retrieved passages and prompt caching all cut time to first token.
• Show activity immediately. Display a "thinking..." indicator the instant the user sends a message, before the first token arrives.
• Render Markdown as you go, so lists and bold text form during the stream rather than appearing as raw symbols.
• Let users stop. A stop button that closes the stream lets people abandon an answer that is clearly going the wrong way and ask again, instead of waiting for it to finish.
• Log latency. The metadata event gives you latency for every answer; track it, and you will notice slowdowns before your users do.

Frequently asked questions

The day the streaming version went live, nobody at the front desk mentioned speed. They just started using the assistant more. Answers began within a second, and people read along instead of glancing away and losing their place. Ethan's metrics showed the total time per answer had not changed at all. "Same model, same seconds," he told Jake. "It just stopped making people stare at an empty box."

Revision note. Written October 19, 2026. Streaming does not make a model faster; it makes the wait visible as progress, which is most of what users mean by fast.