智能体
aws.amazon.com ·
@Salman Ahmed
使用Amazon Bedrock AgentCore与Nova 2部署多模态WhatsApp订餐助手
AWS博客介绍了如何通过Amazon Bedrock AgentCore和Amazon Nova 2，在单一业务号码上构建支持文本、语音消息和实时语音通话的多模态点餐助手。该系统将渠道与订单处理层解耦，并利用共享记忆来优化多轮交互体验。
AI推荐理由 该方案展示了如何在单一WhatsApp业务号上融合文本、语音消息与实时通话，通过解耦渠道与订单处理层并共享记忆，降低多模态客服系统的集成复杂度。对实践者而言，这意味着多模态交互的落地正从自建基础设施转向云端Agent编排，成本与准入门槛可能显著下降。
79
正文
本文展示了如何部署一个使用Amazon Bedrock AgentCore和Amazon Nova 2构建的多模态WhatsApp点餐助手。许多快餐店将点餐分散在应用程序、网站、电话线和柜台。每一个都是需要构建和运行的独立系统。每一个也分割了客户的历史记录，使同一个人在每个渠道上都像陌生人。客户已经生活在他们的消息应用程序中。WhatsApp覆盖超过二十亿人。能够从同一对话中发送文本、语音笔记或拨打电话的客户无需安装任何东西或登录。一个WhatsApp Business号码托管该助手。客户可以给餐厅发短信、发送语音笔记或拨打语音电话。AI代理端到端地接单，从问候到确认。所有三个渠道共享一个后端和一个跨渠道记忆。今天发短信、明天打电话的客户被识别为同一个人。该解决方案使用Meta WhatsApp Business Platform作为客户前门。Amazon Bedrock AgentCore托管代理。Amazon Nova 2 Lite通过Amazon Bedrock Converse API处理文本，Amazon Nova 2 Sonic处理语音笔记和通话上的实时语音。代理通过模型上下文协议（MCP）到达餐厅后端。您使用AWS Cloud Development Kit（AWS CDK）部署整个系统。渠道和点餐逻辑保持分离，因此当您添加或删除渠道时，后端不会改变。解决方案概述该设计将三件事分开：(1) WhatsApp层处理对话，(2) 三个代理运行时为其渠道运行对话，(3) 后端持有菜单、购物车、订单和位置。入站流量到达单个HTTPS webhook，立即以200确认，然后异步处理，因此没有请求阻塞响应。这种分离使每一层都可以独立部署，并且易于推理。客户前门是Meta WhatsApp Business Platform。它公开了Cloud API webhook、Messages API、Media API和Calling API。Meta管理此服务。您将其设置并连接为先决条件，而不是此解决方案部署的内容。AWS CDK在AWS端配置所有内容。它发出您向Meta注册的webhook URL。您使用AWS CDK将以下AWS资源作为一组依赖顺序的堆栈部署，按功能分组。Amazon API Gateway提供两个REST API。第一个是AWS托管证书上的区域HTTPS webhook。它是唯一的公共端点。第二个是点餐逻辑前面的IAM授权后端API。AWS Lambda运行webhook摄取、webhook工作线程、消息发送器和点餐业务逻辑。Amazon Simple Queue Service（Amazon SQS）提供入站队列，带有死信队列，将快速确认与其余处理解耦。AgentCore运行时（Amazon Bedrock AgentCore的一项功能）托管三个代理。每个对话在其自己的微VM中运行。会话保持隔离。Amazon Nova 2 Lite（通过Converse API的文本）和Amazon Nova 2 Sonic（用于语音的语音到语音）通过Amazon Bedrock调用。AgentCore Gateway（Amazon Bedrock AgentCore的一项功能）是托管的MCP服务器，将后端REST API公开为代理按名称调用的MCP工具。AgentCore memory（Amazon Bedrock AgentCore的一项功能）是一个共享的跨渠道记录，以哈希客户ID为键。Amazon DynamoDB存储客户资料、订单、菜单项、购物车和位置。它还保存WhatsApp层的最后入站窗口表。Amazon Location Service处理地理编码和最近位置查找。Amazon Kinesis Video Streams（Amazon KVS）提供信令通道，语音通话运行时使用该通道为托管的围绕NAT的中继（TURN）中继铸造凭据，该中继承载通话媒体。Amazon Virtual Private Cloud（Amazon VPC）带有单个网络地址转换（NAT）网关是语音通话运行时的出站路径。这是唯一需要VPC的运行时。AWS Secrets Manager保存Meta访问令牌、应用密钥和验证令牌，创建为空容器，您带外填充。AWS Systems Manager Parameter Store保存客户ID胡椒。Amazon Elastic Container Registry（Amazon ECR）、AWS CodeBuild和Amazon Simple Storage Service（Amazon S3）构建并存储ARM64代理容器镜像。Amazon CloudWatch捕获日志和指标，AWS Key Management Service（AWS KMS）加密静态数据。架构图图1显示了完整架构。该图将解决方案组织为标记为A到G的组，这些组承载请求路径。两个支持组位于该路径之外，处理在部署时运行一次以构建代理镜像的构建流水线，以及在运行时支持每个渠道的安全和监控服务。图1：AWS上的多模态WhatsApp点餐架构A. WhatsApp入口和投递：webhook API Gateway、摄取、工作线程和发送器Lambda，以及Amazon SQS队列。摄取验证Meta签名并将消息入队。工作线程处理剩余处理。发送器Lambda投递回复。B. 代理运行时：AgentCore运行时上的三个代理，每个都是ARM64容器，每个渠道一个：聊天（Amazon Nova 2 Lite）、语音笔记（Amazon Nova 2 Sonic）和语音通话（Amazon Nova 2 Sonic over Web Real-Time Communication，或WebRTC）。只有语音通话运行时在VPC中运行。C. 人工智能和机器学习（AI/ML）：通过Amazon Bedrock调用的Amazon Nova 2 Lite和Amazon Nova 2 Sonic，以及以哈希customer_id为键的共享AgentCore memory，用于跨渠道连续性。D. MCP服务器（托管）：AgentCore Gateway将后端REST API公开为可发现的MCP工具（GetMenu、AddToCart、Pla
ceOrder等），每个运行时都按名称调用。E. API和计算：经IAM授权的后端API网关和持有业务逻辑的订单Lambda。F.数据存储：Amazon DynamoDB存储客户资料、订单、菜单项、购物车和位置。G.网络和位置：Amazon Location Service用于地理编码和最近位置查找，Amazon KVS托管的TURN中继用于语音通话媒体，以及带有NAT网关的VPC作为语音通话运行时的出站路径。两个支持组位于请求路径之外。构建管道（AWS CDK、AWS CodeBuild、Amazon ECR和Amazon S3）在部署时运行一次，用于构建和存储ARM64代理镜像。它不在任何请求的路径中。安全与监控（AWS Secrets Manager、AWS Systems Manager Parameter Store、Amazon CloudWatch和AWS KMS）在运行时支持每个通道。以下步骤跟踪单个请求在架构中的端到端流程：Meta将入站webhook（文本、语音笔记或通话事件）传送到Webhook API网关和Webhook摄取Lambda。摄取功能验证Meta签名，将其排入Amazon SQS队列，并在Meta的时间窗口内返回200。Webhook Worker使用AWS Systems Manager Parameter Store中的pepper派生一个假名customer_id。Worker从Meta Media API获取媒体，并调用Amazon Bedrock AgentCore运行时（聊天、语音笔记或语音通话）上匹配的代理，其中session_id = customer_id。运行时在会话开始时从AgentCore内存中读取客户的长期洞察。它通过Amazon Bedrock使用Amazon Nova 2 Lite（文本）或Amazon Nova 2 Sonic（语音）进行对话。AgentCore Gateway是托管的MCP服务器。它将后端REST API作为MCP工具暴露给代理按名称调用。工具通过后端API网关路由到AWS Lambda、Amazon DynamoDB和Amazon Location Service。语音通话媒体使用Amazon KVS TURN中继（在VPC中运行）。回复通过发送者Lambda（文本）或Worker（语音）发出。事件在会话结束时写回内存。AWS CDK通过AWS CodeBuild将ARM64镜像构建到Amazon ECR中。Amazon CloudWatch记录组件，AWS KMS加密静态数据。简而言之，每个请求从Meta的webhook流经摄取、队列、Worker和代理运行时到后端工具，然后通过WhatsApp返回给客户。通道流程所有三个通道共享同一个前门、后端工具和内存。不同之处在于线路上的媒体和处理它的运行时。文本消息：文本消息到达webhook。Worker派生customer_id并调用聊天运行时，该运行时读取内存，通过Converse API流式传输Amazon Nova 2 Lite，并在需要时通过MCP网关调用后端工具。回复通过发送者Lambda发送，事件在会话结束时写入内存。图2显示了文本流程，从入站webhook通过Converse API上的Amazon Nova 2 Lite到发送者Lambda传递的回复。图2：使用Amazon Nova 2 Lite的文本流程语音笔记（语音到语音）：语音笔记作为音频消息到达。Worker下载OGG Opus字节并调用语音笔记运行时。读取内存后，音频被解码为16 kHz脉冲编码调制（PCM），并输入到有界的Amazon Nova 2 Sonic语音到语音会话中。工具可通过同一网关使用。口语回复作为WhatsApp语音消息返回。路径中没有转录服务。这是真正的语音进、语音出。图3显示了语音笔记流程，这是一个有界的Amazon Nova 2 Sonic语音到语音会话，返回口语回复，路径中没有转录。图3：使用Amazon Nova 2 Sonic的语音笔记流程语音通话（WebRTC）：客户选择“呼叫”，Meta的Calling API传递一个带有WebRTC会话描述协议（SDP）offer的连接webhook。Worker将其以turnOnly模式中继到语音通话运行时，因为它没有公共IP。TURN凭据来自Amazon KVS。Meta不提供trickle交互式连接建立（ICE）路径。aiortc应答器等待ICE收集并返回单次SDP应答。Worker将该应答传递给Meta。然后媒体通过数据报传输层安全和安全实时传输协议（DTLS/SRTP）流经KVS托管的TURN中继。Amazon Nova 2 Sonic驱动对话。图4显示了语音通话流程，其中WebRTC媒体通过Amazon KVS托管的TURN中继进行中继，Amazon Nova 2 Sonic驱动对话。图4：使用Amazon Nova 2 Sonic的语音通话流程先决条件此解决方案在两个方面有先决条件：您的AWS账户和您的Meta WhatsApp设置。在运行部署之前完成两者，因为它会要求您提供特定的WhatsApp值，并且代理在这些值就位之前无法回复。AWS先决条件您需要一个有效的AWS账户，并在您的部署区域中为Amazon Nova 2 Lite（amazon.nova-2-lite-v1:0）和Amazon Nova 2 Sonic（amazon.nova-2-sonic-v1:0）启用Amazon Bedrock模型访问。您的IAM用户或角色必须具有部署AWS CDK堆栈和创建此解决方案使用的资源的权限，包括AgentCore运行时、网关和内存。在您的本地机器上，安装Node.js 24.x或更高版本、配置了凭据的AWS CLI 2.x以及git。最后，在您的目标账户和区域中引导AWS CDK（npx cdk bootstrap aws:///）。代理容器在AWS CodeBuild中在ARM64上构建，因此您不需要在本地安装Python、Docker或音频工具链。在Amazon Nova 2 Lite、Amazon Nova 2 Sonic以及AgentCore运行时、网关和内存都可用的AWS区域中部署。美国东部（弗吉尼亚北部）区域（us-east-1）是一个不错的起点。有关按区域划分的模型可用性，请参阅Amazon Bedrock中按AWS区域划分的支持模型。WhatsApp Business Platform（Meta）先决条件WhatsApp方面在Meta控制台中设置一次，作为本演练的先决条件，而不是步骤。没有AWS API可以为您创建Meta应用，因此请先完成它，并在部署前准备好值。对于演示，Meta沙盒测试号码（无需额外费用）就足够了。您不需要业务验证或生产号码。完整程序位于Meta的文档中，链接在下一节。在部署前准备好以下项目。一个Meta开发者应用，已添加WhatsApp产品，并链接到业务组合。添加产品会预置一个WhatsApp Business账户（WABA）和一个沙盒测试号码，无需额外费用。请参阅Cloud API入门。应用ID和应用密钥。应用密钥是Meta用于签署每个webhook的密钥，webhook Lambda会重新计算该签名。一个访问令牌。临时令牌可用于快速测试，大约24小时后过期。对于更长期的，请创建一个具有whatsapp_business_messaging和whatsapp_business_management范围的系统用户令牌。业务组合ID（设置CLI可以自动发现电话号码ID和WABA ID）。一个您自己发明的验证令牌。这是一个难以猜测的字符串，Meta在一次性的webhook验证握手中会回显它。对于语音通话，请在号码上启用WhatsApp Calling API。不要将其中任何一个存储在源代码控制中。访问令牌、应用密钥和验证令牌是秘密，在部署期间进入AWS Secrets Manager，而不是进入CDK模板。使用AWS CDK部署解决方案完整解决方案位于GitHub上的示例存储库中。该存储库包含三个代理容器、AWS CDK基础设施代码以及将WhatsApp webhook连接到您的账户的设置脚本。克隆它，运行预检检查，然后使用前缀部署。前缀添加到每个资源名称中，因此您可以在每个账户中部署多次。git clone https://github.com/aws-samples/sample-multimodal-whatsapp-restaurant-agent.git cd sample-multimodal-whatsapp-restaurant-agent ./scripts/preflight-check.sh ./scripts/deploy-all.sh --deploymentPrefix qsr-wa 该脚本按依赖顺序预置每个堆栈。它将每个堆栈的输出传递给下一个。首先，它部署共享VPC。然后部署后端：Amazon DynamoDB、Amazon Location Service、订单Lambda和后端REST API。接下来，它部署AgentCore Gateway和共享的AgentCore内存。然后，它使用AWS CodeBuild构建每个ARM64容器，推送到Amazon ECR，并部署三个运行时。之后，它部署WhatsApp webhook和订单通知器。然后，它填充示例菜单和位置数据。每个容器的首次构建大约需要8-12分钟。如需引导式浏览器体验，请使用./scripts/deploy-all.sh --interactive-web-ui。WhatsApp值如何到达部署对安全性很重要。CDK为三个秘密创建空的Secrets Manager容器。它不会将秘密作为CDK参数，这会将秘密烘焙到合成的模板中。您在带外填充秘密值，而非秘密标识符（电话号码ID、WABA ID、应用ID）作为参数传递。scripts/whatsapp-setup/ CLI在两种流程中处理此问题。部署前流程验证访问令牌并自动发现WABA和电话号码ID。如果需要，它会生成验证令牌并填充秘密。部署发出webhook URL后，部署后流程在Meta中设置回调URL、验证令牌和订阅字段。然后，它订阅WABA并完成验证握手。cd scripts/whatsapp-setup npm start # 选择“Pre-deploy”，然后在部署后选择“Post-deploy” node whatsapp-setup.mjs --doctor # 只读端到端检查一旦webhook被订阅且秘密被填充，号码即生效，代理在所有三个通道上回复。快速确认并异步处理Meta期望立即收到HTTP响应，并返回200 OK
先决条件，而不是本演练中的步骤。没有AWS API可以为您创建Meta应用，因此请先完成它，并在部署前准备好值。对于演示，Meta沙盒测试号码（无需额外费用）就足够了。您不需要业务验证或生产号码。完整程序位于Meta的文档中，链接在下一节。在部署前准备好以下项目。一个Meta开发者应用，已添加WhatsApp产品，并链接到业务组合。添加产品会预置一个WhatsApp Business账户（WABA）和一个沙盒测试号码，无需额外费用。请参阅Cloud API入门。应用ID和应用密钥。应用密钥是Meta用于签署每个webhook的密钥，webhook Lambda会重新计算该签名。一个访问令牌。临时令牌可用于快速测试，大约24小时后过期。对于更长期的，请创建一个具有whatsapp_business_messaging和whatsapp_business_management范围的系统用户令牌。业务组合ID（设置CLI可以自动发现电话号码ID和WABA ID）。一个您自己发明的验证令牌。这是一个难以猜测的字符串，Meta在一次性的webhook验证握手中会回显它。对于语音通话，请在号码上启用WhatsApp Calling API。不要将其中任何一个存储在源代码控制中。访问令牌、应用密钥和验证令牌是秘密，在部署期间进入AWS Secrets Manager，而不是进入CDK模板。使用AWS CDK部署解决方案完整解决方案位于GitHub上的示例存储库中。该存储库包含三个代理容器、AWS CDK基础设施代码以及将WhatsApp webhook连接到您的账户的设置脚本。克隆它，运行预检检查，然后使用前缀部署。前缀添加到每个资源名称中，因此您可以在每个账户中部署多次。git clone https://github.com/aws-samples/sample-multimodal-whatsapp-restaurant-agent.git cd sample-multimodal-whatsapp-restaurant-agent ./scripts/preflight-check.sh ./scripts/deploy-all.sh --deploymentPrefix qsr-wa 该脚本按依赖顺序预置每个堆栈。它将每个堆栈的输出传递给下一个。首先，它部署共享VPC。然后部署后端：Amazon DynamoDB、Amazon Location Service、订单Lambda和后端REST API。接下来，它部署AgentCore Gateway和共享的AgentCore内存。然后，它使用AWS CodeBuild构建每个ARM64容器，推送到Amazon ECR，并部署三个运行时。之后，它部署WhatsApp webhook和订单通知器。然后，它填充示例菜单和位置数据。每个容器的首次构建大约需要8-12分钟。如需引导式浏览器体验，请使用./scripts/deploy-all.sh --interactive-web-ui。WhatsApp值如何到达部署对安全性很重要。CDK为三个秘密创建空的Secrets Manager容器。它不会将秘密作为CDK参数，这会将秘密烘焙到合成的模板中。您在带外填充秘密值，而非秘密标识符（电话号码ID、WABA ID、应用ID）作为参数传递。scripts/whatsapp-setup/ CLI在两种流程中处理此问题。部署前流程验证访问令牌并自动发现WABA和电话号码ID。如果需要，它会生成验证令牌并填充秘密。部署发出webhook URL后，部署后流程在Meta中设置回调URL、验证令牌和订阅字段。然后，它订阅WABA并完成验证握手。cd scripts/whatsapp-setup npm start # 选择“Pre-deploy”，然后在部署后选择“Post-deploy” node whatsapp-setup.mjs --doctor # 只读端到端检查一旦webhook被订阅且秘密被填充，号码即生效，代理在所有三个通道上回复。快速确认并异步处理Meta期望立即收到HTTP响应，并返回200 OK
查看原文（中文正文为 AI 翻译）
This post shows how to deploy a multimodal WhatsApp ordering assistant built with Amazon Bedrock AgentCore and Amazon Nova 2. Many quick-service restaurants spread ordering across an app, a website, a phone line, and the counter. Each of those is a separate system to build and run. Each one also fragments the customer’s history, making the same person look like a stranger on every channel. Customers already live in their messaging apps. WhatsApp reaches more than two billion people. A customer who can text, send a voice note, or place a call from the same conversation does not need to install anything or sign in. A single WhatsApp Business number hosts the assistant. A customer can text the restaurant, send a voice note, or place a voice call. An AI agent takes the order end to end, from greeting to confirmation. All three channels share one backend and one cross-channel memory. A customer who texts today and calls tomorrow is recognized as the same person. The solution uses the Meta WhatsApp Business Platform as the customer front door. Amazon Bedrock AgentCore hosts the agents. Amazon Nova 2 Lite handles text through the Amazon Bedrock Converse API, and Amazon Nova 2 Sonic handles real-time speech on voice notes and calls. The agents reach the restaurant backend through the Model Context Protocol (MCP). You deploy the whole system with the AWS Cloud Development Kit (AWS CDK). The channel and the ordering logic stay separate, so the backend doesn’t change when you add or remove a channel. Solution overview The design keeps three things apart: (1) the WhatsApp layer handles the conversation, (2) three agent runtimes run the conversations for their channels, and (3) the backend holds the menu, carts, orders, and locations. Inbound traffic arrives on a single HTTPS webhook, is acknowledged with a 200 immediately, and then processed asynchronously so no request blocks the response. This separation keeps each layer independently deployable and straightforward to reason about. The customer front door is the Meta WhatsApp Business Platform. It exposes the Cloud API webhook, Messages API, Media API, and Calling API. Meta manages this service. You set it up and connect it as a prerequisite, not something this solution deploys. The AWS CDK provisions everything on the AWS side. It emits the webhook URL you register with Meta. You use the AWS CDK to deploy the following AWS resources as a set of stacks in dependency order, grouped here by function. Amazon API Gateway provides two REST APIs. The first is a regional HTTPS webhook on an AWS managed certificate. It’s the only public endpoint. The second is an AWS Identity and Access Management (IAM)-authorized backend API in front of the ordering logic. AWS Lambda runs the webhook ingest, the webhook worker, the message sender, and the ordering business logic. Amazon Simple Queue Service (Amazon SQS) provides the inbound queue, with a dead-letter queue, that decouples the quick acknowledgment from the rest of the processing. AgentCore runtime, a capability of Amazon Bedrock AgentCore, hosts the three agents. Each conversation runs in its own microVM. Sessions stay isolated. Amazon Nova 2 Lite (text through the Converse API) and Amazon Nova 2 Sonic (speech-to-speech for voice) are invoked through Amazon Bedrock. AgentCore Gateway, a capability of Amazon Bedrock AgentCore, is a managed MCP server that exposes the backend REST API as MCP tools the agents call by name. AgentCore memory, a capability of Amazon Bedrock AgentCore, is one shared, cross-channel record keyed by a hashed customer ID. Amazon DynamoDB stores customer profiles, orders, menu items, carts, and locations. It also holds a last inbound window table for the WhatsApp layer. Amazon Location Service handles geocoding and nearest-location lookups. Amazon Kinesis Video Streams (Amazon KVS) provides a signaling channel that the voice-call runtime uses to mint credentials for the managed Traversal Using Relays around NAT (TURN) relay that carries call media. Amazon Virtual Private Cloud (Amazon VPC) with a single network address translation (NAT) gateway is the outbound path for the voice-call runtime. This is the only runtime that requires a VPC. AWS Secrets Manager holds the Meta Access Token, App Secret, and Verify Token, created as empty containers you populate out-of-band. AWS Systems Manager Parameter Store holds the customer-id pepper. Amazon Elastic Container Registry (Amazon ECR), AWS CodeBuild, and Amazon Simple Storage Service (Amazon S3) build and store the ARM64 agent container images. Amazon CloudWatch captures logs and metrics, and AWS Key Management Service (AWS KMS) encrypts data at rest. Architecture diagram Figure 1 shows the full architecture. The diagram organizes the solution into labeled groups A through G that carry the request path. Two supporting groups sit outside that path, handling a build pipeline that runs once at deploy time to build the agent images and security and monitoring services that back every channel at runtime. Figure 1: Multimodal WhatsApp ordering architecture on AWS A. WhatsApp ingress and delivery: The webhook API Gateway, the ingest, worker, and sender Lambdas, and the Amazon SQS queue. The ingest verifies the Meta signature and enqueues the message. The worker handles the remaining processing. The Sender Lambda delivers replies. B. Agent runtimes: Three agents on AgentCore runtime, each an ARM64 container, one per channel: chat (Amazon Nova 2 Lite), voice note (Amazon Nova 2 Sonic), and voice call (Amazon Nova 2 Sonic over Web Real-Time Communication, or WebRTC). Only the voice-call runtime runs in a VPC. C. Artificial intelligence and machine learning (AI/ML): Amazon Nova 2 Lite and Amazon Nova 2 Sonic invoked through Amazon Bedrock, plus the shared AgentCore memory keyed by a hashed customer_id for cross-channel continuity. D. MCP server (managed): AgentCore Gateway exposes the backend REST API as discoverable MCP tools (GetMenu, AddToCart, PlaceOrder, and more) that every runtime calls by name. E. API and compute: The IAM-authorized backend API Gateway and the ordering Lambdas that hold the business logic. F. Data storage: Amazon DynamoDB stores customer profiles, orders, menu items, carts, and locations. G. Networking and location: Amazon Location Service for geocoding and nearest-location lookups, the Amazon KVS managed TURN relay for voice-call media, and the VPC with a NAT gateway as the outbound path for the voice-call runtime. Two supporting groups sit outside the request path. The build pipeline (AWS CDK, AWS CodeBuild, Amazon ECR, and Amazon S3) runs once at deploy time to build and store the ARM64 agent images. It is not in the path of any request. Security and monitoring (AWS Secrets Manager, AWS Systems Manager Parameter Store, Amazon CloudWatch, and AWS KMS) back every channel at runtime. The following steps trace a single request end to end through the architecture: Meta delivers an inbound webhook (text, voice note, or call event) to the Webhook API Gateway and the Webhook Ingest Lambda. The ingest verifies the Meta signature, enqueues to Amazon SQS, and returns 200 within Meta’s window. The Webhook Worker derives a pseudonymous customer_id using the pepper in AWS Systems Manager Parameter Store. The worker fetches media from the Meta Media API and invokes the matching agent on Amazon Bedrock AgentCore runtime (chat, voice note, or voice call) with session_id = customer_id. The runtime reads the customer’s long-term insights from AgentCore memory at session start. It runs the conversation with Amazon Nova 2 Lite (text) or Amazon Nova 2 Sonic (voice) through Amazon Bedrock. AgentCore Gateway is the managed MCP server. It exposes the backend REST API as MCP tools the agent calls by name. Tools route through the backend API Gateway to AWS Lambda, Amazon DynamoDB, and Amazon Location Service. Voice-call media uses the Amazon KVS TURN relay (runtime in a VPC). Replies go out through the Sender Lambda (text) or the worker (voice). Events are written back to memory at session end. AWS CDK builds ARM64 images through AWS CodeBuild into Amazon ECR. Amazon CloudWatch logs components and AWS KMS encrypts data at rest. In short, every request flows from Meta’s webhook through the ingest, queue, worker, and agent runtime to the backend tools, then back to the customer on WhatsApp. Channel flows All three channels share the same front door, backend tools, and memory. What differs is the media on the wire and the runtime that handles it. Text message: A text message arrives on the webhook. The worker derives customer_id and invokes the chat runtime, which reads memory, streams Amazon Nova 2 Lite through the Converse API, and calls backend tools through the MCP gateway as needed. Replies go through the Sender Lambda, and events are written to memory at session end. Figure 2 shows the text flow, from the inbound webhook through Amazon Nova 2 Lite on the Converse API to the reply delivered by the Sender Lambda. Figure 2: Text flow with Amazon Nova 2 Lite Voice note (speech-to-speech): A voice note arrives as an audio message. The worker downloads the OGG Opus bytes and invokes the voice-note runtime. After reading memory, the audio is decoded to 16 kHz pulse-code modulation (PCM) and fed into a bounded Amazon Nova 2 Sonic speech-to-speech session. Tools are available through the same gateway. The spoken reply is returned as a WhatsApp voice message. There’s no transcription service in the path. It’s true voice-in, voice-out. Figure 3 shows the voice-note flow, a bounded Amazon Nova 2 Sonic speech-to-speech session that returns a spoken reply with no transcription in the path. Figure 3: Voice note flow with Amazon Nova 2 Sonic Voice call (WebRTC): The customer selects Call, and Meta’s Calling API delivers a connect webhook with the WebRTC Session Description Protocol (SDP) offer. The worker relays it to the voice-call runtime in turnOnly mode, because it has no public IP. TURN credentials come from Amazon KVS. Meta provides no trickle Interactive Connectivity Establishment (ICE) path. The aiortc answerer waits for ICE gathering and returns a single-shot SDP answer. The worker delivers that answer to Meta. Media then flows over Datagram Transport Layer Security and Secure Real-time Transport Protocol (DTLS/SRTP) through the KVS managed TURN relay. Amazon Nova 2 Sonic drives the conversation. Figure 4 shows the voice-call flow, where WebRTC media is relayed through the Amazon KVS managed TURN relay and Amazon Nova 2 Sonic drives the conversation. Figure 4: Voice call flow with Amazon Nova 2 Sonic Prerequisites This solution has prerequisites in two areas: your AWS account and your Meta WhatsApp setup. Complete both prior to running the deployment, because it asks you for specific WhatsApp values and the agent cannot reply until they are in place. AWS prerequisites You need an active AWS account with Amazon Bedrock model access enabled for Amazon Nova 2 Lite (amazon.nova-2-lite-v1:0) and Amazon Nova 2 Sonic (amazon.nova-2-sonic-v1:0) in your deployment Region. Your IAM user or role must have permissions to deploy AWS CDK stacks and create the resources this solution uses, including AgentCore runtime, Gateway, and memory. On your local machine, install Node.js 24.x or later, the AWS CLI 2.x configured with credentials, and git. Finally, bootstrap AWS CDK in your target account and Region (npx cdk bootstrap aws:///). The agent containers build inside AWS CodeBuild on ARM64, so you do not need Python, Docker, or audio toolchains locally. Deploy in an AWS Region where Amazon Nova 2 Lite, Amazon Nova 2 Sonic, and AgentCore runtime, Gateway, and memory are all available. The US East (N. Virginia) Region (us-east-1) is a good place to start. For model availability by Region, refer to Supported models by AWS Region in Amazon Bedrock. WhatsApp Business Platform (Meta) prerequisites The WhatsApp side is set up once in the Meta consoles, as a prerequisite rather than a step in this walkthrough. No AWS API can create a Meta app for you, so complete it first and have the values on hand before you deploy. For a demo, the Meta sandbox test number, which is available at no additional cost, is enough. You do not need business verification or a production number. The full procedure lives in Meta’s documentation, linked in the following section. Have these items ready before you deploy. A Meta Developer App with the WhatsApp product added, linked to a Business portfolio. Adding the product provisions a WhatsApp Business Account (WABA) and a sandbox test number at no additional cost. Refer to Get started with the Cloud API. The App ID and App Secret. The App Secret is the key Meta uses to sign each webhook, and the webhook Lambda recomputes that signature. An Access Token. The temporary token works for a quick test and expires in about 24 hours. For something longer-lived, create a System User token with the whatsapp_business_messaging and whatsapp_business_management scopes. The Business portfolio ID (the setup CLI can auto-discover the Phone Number ID and WABA ID). A Verify Token that you invent. It is a hard-to-guess string Meta echoes back during the one-time webhook verification handshake. For voice calls, the WhatsApp Calling API enabled on the number. Store none of these in source control. The Access Token, App Secret, and Verify Token are secrets and go into AWS Secrets Manager during deployment, not into the CDK templates. Deploy the solution with AWS CDK The full solution is in the sample repository on GitHub. The repository contains three agent containers, the AWS CDK infrastructure code, and setup scripts that wire the WhatsApp webhook to your account. Clone it, run the preflight check, then deploy with a prefix. The prefix is added to every resource name, so you can deploy more than once per account. git clone https://github.com/aws-samples/sample-multimodal-whatsapp-restaurant-agent.git cd sample-multimodal-whatsapp-restaurant-agent ./scripts/preflight-check.sh ./scripts/deploy-all.sh --deploymentPrefix qsr-wa The script provisions every stack in dependency order. It passes each stack’s outputs to the next. First it deploys the shared VPC. Then it deploys the backend: Amazon DynamoDB, Amazon Location Service, the ordering Lambdas, and the backend REST API. Next it deploys AgentCore Gateway and the shared AgentCore memory. It then builds each ARM64 container with AWS CodeBuild, pushes to Amazon ECR, and deploys the three runtimes. After that it deploys the WhatsApp webhook and the order-notifier. It then seeds sample menu and location data. The first build of each container takes roughly 8–12 minutes. For a guided, browser-based experience, use ./scripts/deploy-all.sh --interactive-web-ui. How the WhatsApp values reach the deployment matters for security. The CDK creates empty Secrets Manager containers for the three secrets. It does not take a secret as a CDK parameter, which would bake it into the synthesized template. You populate the secret values out-of-band, while the non-secret identifiers (Phone Number ID, WABA ID, App ID) ride along as parameters. The scripts/whatsapp-setup/ CLI handles this in two flows. The pre-deploy flow validates the Access Token and auto-discovers the WABA and Phone Number ID. It generates a Verify Token if needed and populates the secrets. After the deploy emits the webhook URL, the post-deploy flow sets the callback URL, Verify Token, and subscribed fields in Meta. It then subscribes the WABA and completes the verification handshake. cd scripts/whatsapp-setup npm start # choose "Pre-deploy", then "Post-deploy" after deploy node whatsapp-setup.mjs --doctor # read-only end-to-end check Once the webhook is subscribed and the secrets are populated, the number is live and the agent replies across all three channels. Acknowledging fast and processing asynchronously Meta expects a prompt HTTP response with a 200 OK