面向 AI 原生平台的架构：RAG、LLM 编排与 Agent 模式

AI adoption in an enterprise SaaS platform is rarely about adding an LLM API and calling it done. The difficult part is integrating AI into an existing platform without weakening the properties that made the platform trustworthy in the first place. A mature SaaS platform already has: Domain models Authentication and authorization Tenant isolation Data governance Audit trails APIs Event-driven workflows Operational controls Reliability mechanisms AI should not create a parallel architecture that bypasses these capabilities. It should inherit them. That's the architectural principle I use when thinking about evolving a mature SaaS platform toward AI-native capabilities. AI Should Be an Augmentation Layer, Not a Parallel Platform The first architectural decision is where AI belongs. A tempting approach looks like this: Existing Platform │ └──────► AI Platform │ ├── Own data ├── Own permissions ├── Own workflows └── Own state Enter fullscreen mode Exit fullscreen mode This creates a dangerous divergence. Now there are effectively two systems that understand the business. The better model is: ┌─────────────────────┐ │ AI Capabilities │ │ │ │ RAG / LLM / Agents │ └──────────┬──────────┘ │ Platform APIs │ ┌──────────▼──────────┐ │ Canonical Domain │ │ Model │ └──────────┬──────────┘ │ ┌──────────▼──────────┐ │ Core Platform │ │ │ │ Auth / Tenancy / │ │ Data / Audit / APIs │ └─────────────────────┘ Enter fullscreen mode Exit fullscreen mode The core platform remains the source of truth. AI becomes another consumer and orchestrator of platform capabilities. This distinction becomes increasingly important as AI moves from simply generating answers to taking actions. Pattern 1: RAG — Give the Model the Right Context Large language models are powerful, but they don't automatically know your organization's current data. For enterprise applications, the challenge is therefore often less: "Which model should we use?" and more: "How do we reliably provide the right context to the model?" That's where Retrieval-Augmented Generation (RAG) becomes useful. A simplified RAG pipeline looks like: Documents / Domain Data │ ▼ Chunking │ ▼ Embedding │ ▼ Vector Index │ │ ┌───▼────┐ │ Query │ └───┬────┘ │ ▼ Retrieval │ ▼ Relevant Context │ ▼ LLM │ ▼ Response Enter fullscreen mode Exit fullscreen mode The model isn't expected to remember everything. The application retrieves relevant information and supplies it as context. The Vector Pipeline Is the Foundation If you're building multiple AI features, one of the first reusable platform capabilities should be the vector pipeline: Ingest ↓ Normalize ↓ Chunk ↓ Embed ↓ Index ↓ Retrieve ↓ Rerank / Filter ↓ Generate Enter fullscreen mode Exit fullscreen mode The specific vector technology can change. For example, depending on the architecture and requirements, this could involve: Amazon OpenSearch Pinecone Another vector-capable datastore The important architectural decision is to avoid coupling every AI feature directly to the indexing implementation. Instead: AI Features / | \ / | \ Search Assistant Agent \ | / \ | / ▼ ▼ ▼ Retrieval API │ ▼ Vector Pipeline │ ▼ Domain Data Enter fullscreen mode Exit fullscreen mode Once retrieval becomes a platform capability, multiple AI features can reuse it. RAG Is More Than Semantic Search One common mistake is to think of RAG as: Question ↓ Vector search ↓ Top 5 documents ↓ LLM Enter fullscreen mode Exit fullscreen mode Production systems usually need more controls. The retrieval layer may need to consider: Tenant User permissions Document type Data freshness Metadata Access policies Relevance Source authority Temporal constraints For example: User Query │ ▼ Authorization Context │ ▼ Tenant / Scope Filter │ ▼ Semantic Retrieval │ ▼ Metadata / Permission Filtering │ ▼ Relevant Context │ ▼ LLM Enter fullscreen mode Exit fullscreen mode This is critical. Retrieving information that the user isn't authorized to access is still a security vulnerability—even if the LLM never intentionally exposes it. RAG Needs Freshness and Provenance Enterprise data changes. A vector index can therefore become stale. Consider: Source updated │ ▼ Database = current │ └──────► Vector index = old Enter fullscreen mode Exit fullscreen mode The system now has two versions of reality. That's why a production RAG architecture should think about: Incremental indexing Deletes Updates Re-indexing Document versioning Embedding version changes Source timestamps Provenance A useful principle is: The vector index is a derived representation, not the source of truth. That makes lifecycle management much clearer. Pattern 2: LLM Orchestration Once AI workflows become more sophisticated, a single model invocation isn't enough. A real enterprise workflow might look like: Request │ ▼ Authorize │ ▼ Retrieve │ ▼ Enrich │ ▼ Generate │ ▼ Validate │ ▼ Persist │ ▼ Audit Enter fullscreen mode Exit fullscreen mode This is where orchestration becomes important. For AWS-based architectures, workflow services such as Step Functions can provide explicit state management around multi-step operations. The key architectural idea is: Don't hide a distributed workflow inside one giant prompt or Lambda function. Model the workflow explicitly. Make AI Workflows Observable Traditional distributed systems already taught us that asynchronous workflows need state. AI workflows need the same discipline. Instead of: AI Request → ??? → Response Enter fullscreen mode Exit fullscreen mode think: AI Workflow │ ┌──────────────┼──────────────┐ ▼ ▼ ▼ Retrieve Generate Validate │ │ │ └──────────────┼──────────────┘ ▼ Store │ ▼ Audit Enter fullscreen mode Exit fullscreen mode Each stage should have enough metadata to understand: What was requested? Which tenant initiated it? Which data sources were consulted? Which model/version was used? What workflow executed? Which tools were invoked? What failed? What was retried? What was ultimately returned? AI systems need observability at both the application and model layers. Model Selection Should Be an Architecture Decision Not every request needs the largest or most expensive model. A mature AI platform can route workloads according to their requirements. For example: Request │ ▼ Classify Task │ ┌───────────┼───────────┐ ▼ ▼ ▼ Simple Complex Batch │ │ │ ▼ ▼ ▼ Fast/cheap Capable LLM Offline model model inference Enter fullscreen mode Exit fullscreen mode This introduces another important architectural metric: Cost per successful business outcome rather than simply: Cost per LLM request. The cheapest model isn't useful if it produces an answer that requires repeated retries or human correction. Where SageMaker Fits Not every AI workload is an LLM workflow. Traditional machine-learning workloads still matter. For use cases involving: Model training Feature engineering Batch inference Model evaluation MLOps Model deployment a platform such as Amazon SageMaker can provide a different execution model. A useful architectural separation is: AI Platform │ ┌────────────┴────────────┐ │ │ ▼ ▼ Generative AI Predictive ML │ │ LLM / RAG / Agents Training / Inference │ │ ▼ ▼ Bedrock / LLM stack SageMaker Enter fullscreen mode Exit fullscreen mode The goal isn't to force every AI capability through the same technology. Pattern 3: Agentic Workflows Agents introduce a fundamentally different capability. A traditional application does this: User ↓ API ↓ Business Logic ↓ Result Enter fullscreen mode Exit fullscreen mode An agent can potentially do: User ↓ Agent ↓ Decide next action ↓ Call tool ↓ Observe result ↓ Decide next action ↓ Call another tool ↓ Return result Enter fullscreen mode Exit fullscreen mode That introduces a new architectural concern: bounded autonomy. Agents Need Explicit Boundaries An agent should not automatically receive unrestricted access to the platform. Instead, define: Scope What business problem is the agent allowed to solve? Tools Which APIs or actions can it invoke? Permissions What can it read? What can it modify? Limits How many actions can it perform? How much can it spend? Approval Which actions require human confirmation? A useful mental model is: ┌───────────────┐ │ Agent │ └───────┬───────┘ │ Policy / IAM │ ┌────────────┼────────────┐ ▼ ▼ ▼ Tool A Tool B Tool C Read Read Write Enter fullscreen mode Exit fullscreen mode The agent doesn't get "platform access." It gets specific capabilities. Human-in-the-Loop Is a Control, Not a Failure For high-impact actions, autonomy shouldn't necessarily mean zero human involvement. Consider: Agent proposes action │ ▼ Policy evaluation │ ├── Low risk ──→ Execute │ └── High risk ─→ Human approval │ ▼ Execute Enter fullscreen mode Exit fullscreen mode The important question isn't: "Can the agent do this automatically?" It's: "What level of autonomy is appropriate for this action?" This is the same risk-based thinking we already use in distributed systems and security architecture. Audit Every Agent Action Once an AI system can take actions, logging the final response isn't enough. You need to understand the chain of execution. For example: Request ↓ Agent decision ↓ Tool selected ↓ Tool parameters ↓ Authorization check ↓ Tool result ↓ Next decision ↓ Final action Enter fullscreen mode Exit fullscreen mode The exact implementation will depend on the platform and privacy requirements, but the architectural principle is straightforward: An action taken by an agent should be as auditable as an action taken by a human or traditional service. This becomes particularly important for regulated or enterprise environments. Security Must Flow Through the AI Stack One of the most dangerous architectural mistakes is treating AI as a separate security domain. Your existing platform may already have: Authentication RBAC ABAC Tenant isolation IAM API authorization Audit logging The AI layer should inherit these controls. Consider a multi-tenant SaaS application: User │ ▼ Authentication │ ▼ Tenant Context │ ▼ Authorization │ ┌─────┴─────┐ ▼ ▼ RAG Agent │ │ ▼ ▼ Retrieval Tools │ │ └─────┬─────┘ ▼ Domain APIs Enter fullscreen mode Exit fullscreen mode The AI system should not create a backdoor around the authorization model. This is especially important for RAG. Tenant isolation must exist in retrieval itself, not merely in the user interface. Guardrails Are Architecture AI guardrails shouldn't be an afterthought added after the first production incident. They belong in the architecture. Examples include: Input controls Prompt validation Input size limits Abuse detection Sensitive-data handling Retrieval controls Tenant filtering Authorization checks Source validation Freshness requirements Output controls Schema validation Content validation Confidence or quality checks Business-rule validation Operational controls Token budgets Rate limits Model routing Timeout limits Retry limits A useful pipeline looks like: Input ↓ Validate ↓ Authorize ↓ Retrieve ↓ Generate ↓ Validate Output ↓ Business Rules ↓ Persist / Act ↓ Audit Enter fullscreen mode Exit fullscreen mode The LLM is one component inside the workflow. It shouldn't become the workflow itself. Don't Let the LLM Become the Source of Truth This is perhaps the most important design principle. An LLM should generally reason over authoritative data, not replace it. For example: ┌───────────────┐ │ Source of │ │ Truth │ └───────┬───────┘ │ ▼ AI Context │ ▼ LLM │ ▼ Proposed Answer / Action │ ▼ Domain Validation │ ▼ Platform Enter fullscreen mode Exit fullscreen mode The model generates a result. The platform determines whether that result is valid. This distinction becomes critical when AI starts taking actions rather than simply answering questions. What I Tell Engineering Teams I use a simple rule when reviewing AI architecture: The AI layer should inherit the trust properties of the core platform. If your platform has strong: Multitenancy Authorization Auditability Observability Idempotency Resilience then AI should inherit those properties. If those properties are weak, introducing AI doesn't hide the weakness. It can amplify it. An AI system that can access ten times more data or execute ten times more actions can turn a small authorization mistake into a much larger incident. A Practical AI-Native Evolution Path You don't need to build an autonomous agent platform on day one. A pragmatic evolution can look like this: Phase 1 Canonical data + APIs │ ▼ Phase 2 RAG / Retrieval │ ▼ Phase 3 LLM-powered workflows │ ▼ Phase 4 Tool-enabled assistants │ ▼ Phase 5 Bounded agentic workflows │ ▼ Phase 6 Selective autonomous actions Enter fullscreen mode Exit fullscreen mode Each phase builds on the previous one. This is important because the hardest part of AI adoption isn't usually the model. It's building the platform capabilities around the model. The AI Platform Capabilities I'd Build First Before investing heavily in autonomous agents, establish the foundations. 1. Canonical data access AI should consume well-defined domain APIs and data products. 2. Retrieval platform Build reusable ingestion, chunking, embedding, indexing, retrieval, and authorization capabilities. 3. Model gateway Centralize model access where practical so applications don't each implement their own: Authentication Model selection Rate limiting Cost tracking Observability Safety controls 4. Workflow orchestration Use explicit workflows for multi-step AI operations. 5. Tool layer Expose controlled business capabilities as tools rather than giving agents unrestricted database or infrastructure access. 6. Evaluation Build repeatable evaluation datasets and quality metrics. An AI feature isn't production-ready simply because it works for ten manually tested prompts. You need to know: Does it work? How often does it fail? When does it fail? Which tenants/data types are affected? Did a model or prompt change make it worse? Enter fullscreen mode Exit fullscreen mode Metrics That Matter Traditional application metrics aren't enough. An AI-native platform should track several dimensions. Quality Retrieval precision / relevance Groundedness Task success rate Validation failure rate Human correction rate Performance End-to-end latency Retrieval latency Model latency Token usage Reliability Workflow failures Tool failures Retry rates Timeout rates Security Authorization failures Cross-tenant retrieval attempts Policy violations Blocked tool calls Economics Cost per request Cost per successful task Cost per tenant Cost by model Cost by workflow The goal is to optimize business outcomes, not simply model metrics. What Changes When AI Becomes Agentic? RAG primarily changes how applications retrieve information. LLM orchestration changes how applications coordinate AI-powered workflows. Agents change how applications take actions. That means the risk profile evolves: RAG │ └── Information risk LLM workflows │ └── Information + workflow risk Agents │ └── Information + workflow + action risk Enter fullscreen mode Exit fullscreen mode The more autonomy you introduce, the stronger your controls need to become. The Real Lesson AI-native architecture isn't about putting an LLM at the center of everything. It's about creating a platform where AI capabilities can evolve without bypassing the engineering disciplines that already protect the business. The architecture I want looks like: AI Applications │ ┌──────────────┼──────────────┐ ▼ ▼ ▼ RAG Workflows Agents │ │ │ └──────────────┼──────────────┘ ▼ AI Platform │ ┌─────────────┼─────────────┐ ▼ ▼ ▼ Retrieval Models Tools │ │ │ └─────────────┼─────────────┘ ▼ Core Platform │ ┌────────────────┼────────────────┐ ▼ ▼ ▼ Identity Domain Data Audit │ │ │ └────────────────┼────────────────┘ ▼ Source of Truth Enter fullscreen mode Exit fullscreen mode The goal isn't to make the platform "AI-powered." The goal is to make AI a first-class capability of the platform without making it a special exception to the platform's rules. That's the difference between adding AI features and building an AI-native platform. AI should inherit your platform's trust model—not replace it.