使用 Amazon SageMaker JumpStart 部署与评估模型

Choosing the right model for a specific application requires a practical way to test it with representative inputs. Amazon SageMaker JumpStart provides access to a broad catalog of already pretrained models. In this post, we show how to deploy an available model from SageMaker JumpStart, run a lightweight and repeatable performance test using the SageMaker Python SDK, and delete the endpoint when testing is complete. For supported models and available capacity, this workflow provides a repeatable evaluation path, although deployment time varies by model, instance type, quota, and AWS Region. In 2026, SageMaker JumpStart introduced two improvements that simplify model discovery and deployment. In the Model Hub section of the Amazon SageMaker AI console, you can browse the SageMaker JumpStart catalog, view sample notebooks, and open them directly in SageMaker Studio. Pre-configured endpoint deployment options provide named configurations optimized for priorities such as low cost, low latency, or general-purpose workloads across a range of generative AI models, including Neuron-optimized configurations. Additionally, you can now go from Hugging Face model discovery to SageMaker AI deployment in a single click using the new deep link integration. Solution overview Our approach uses a reusable Python script that performs three steps: Discover and deploy a model from SageMaker JumpStart. Run model-specific sample prompts and measure latency, approximate output length, and single-request generation rate. Clean up the endpoint. This pattern works for many compatible SageMaker JumpStart models, although payload formats, required instance types, end-user license agreement (EULA) requirements, and response schemas may require model-specific adjustments. Prerequisites An AWS account with SageMaker AI permissions. Python 3+ with SageMaker Python SDK v3 installed (pip install "sagemaker>=3, 0 else 0, } ) print("Input:\n", payload.body, end="\n\n") print("Output:\n", output, end="\n\n") print( f"Latency: {latency:.2f} seconds\n" f"Estimated output words: {output_words}\n" f"Estimated single-request output generation rate: " f"{output_words / latency:.1f} words/second\n" ) print("-" * 80) print("\nSummary:") for result in results: print(result) Result: Input: {"max_tokens": 256, "messages": [{"content": "What is deep learning?", "role": "user"}], "model": "/opt/ml/model"} Output: Here's a thinking process that leads to the explanation of deep learning: 1. **Deconstruct the Request:** * **Question:** "What is deep learning?" * **Intent:** The user wants a clear, comprehensive, yet accessible definition and overview of deep learning. * **Target Audience:** Likely someone looking for a general understanding, possibly a student, tech enthusiast, or professional exploring the field. The tone should be informative, structured, and not overly jargon-heavy (though key terms need explanation). 2. **Initial Brainstorming & Key Concepts:** * What is it? A subset of machine learning, which is a subset of AI. * Core mechanism: Artificial Neural Networks (ANNs). * "Deep" means many layers (hidden layers). * How does it work? Learning representations from data (feature learning). * Key applications: Image recognition, NLP, speech, self-driving cars. * Why is it popular now? Big data, GPU power, better algorithms. * Pros/Cons: High accuracy vs. black box, data hunger. 3. **Structuring the Answer:** Latency: 9.07 seconds Estimated output words: 153 Estimated single-request output generation rate: 16.9 words/second -------------------------------------------------------------------------------- Summary: {'request_preview': '{"max_tokens": 256, "messages": {"content": "What', 'latency_s': 9.07, 'estimated_output_words': 153, 'words_per_second': 16.9} Step 3: Cleanup print(f"Deleting endpoint: {endpoint.endpoint_name}") endpoint.delete() endpoint.wait_for_delete() print("Endpoint deleted") Conclusion This pattern works for many available SageMaker JumpStart models, but the payload format, required instance type, EULA, and response parsing may require model-specific adjustments. By using consistent test prompts and measuring latency, throughput, and output quality across models, you can compare options and select the right fit for your workload. Try this approach with different models from Amazon SageMaker JumpStart to find the right balance of quality, speed, and cost for your applications. References SageMaker Python SDK Documentation Benchmark and Optimize SageMaker JumpStart Deployment (blog post) From Hugging Face to SageMaker Studio in One Click (blog post) Amazon SageMaker JumpStart About the authors Vamsi Goparaju Vamsi leads Go-to-Market (GTM) and Revenue Acceleration initiatives at AWS, enabling Inference and ModelOps workloads for our customers, and driving OSS Use Cases and Product features for Amazon SageMaker AI, SageMaker JumpStart, and Generative AI. With over 16 years of experience spanning AWS, Dell, and Infosys, he brings a unique blend of technical depth and business knowledge across cloud, high-performance computing (HPC), artificial intelligence (AI), machine learning (ML), and analytics. He holds an MBA from Texas A&M University’s Mays Business School. Gula Nurmatova Gula is a Software Development Manager at AWS based in New York, USA. She leads the engineering teams behind Amazon SageMaker JumpStart, making foundation models from providers like NVIDIA, Meta, and Mistral easy to discover, evaluate, and deploy across SageMaker and Amazon Bedrock. Prior to AWS, she held engineering and engineering management roles at RTX and GE, and holds a master’s degree in Computer Science. Naidile Murali Naidile is a Product Manager at AWS based in Bellevue, WA. She focuses on enhancing the AI/ML developer experience on Amazon SageMaker AI, including onboarding, IDE connectivity, and GPU capacity management. Prior to AWS, she worked as a software engineer at HSBC. Naidile holds an MBA from Georgetown University. Dan Ferguson Dan is a Solutions Architect at AWS, based in New York, USA. As a machine learning services expert, Dan works to support customers on their journey to integrating ML workflows efficiently, effectively, and sustainably.