Introduction to Amazon SageMaker Serverless Inference | Concepts & Code examples

Amazon SageMaker Serverless Inference is a model hosting feature that lets you deploy endpoints for inference that automatically starts and scales the compute resources based on traffic. With SageMaker Serverless Inference you don’t have to manage instance types and you only pay for prediction requests and time taken to process those requests.

In this video we discuss key concepts, who is it for, how to use it and walkthrough a code example showing how to host a Serverless inference endpoint.

[akashsivasubramanian]

Thanks so much, this is exactly what I needed 😄

[shreyasibanerjee]

Super clear and very easy to understand!

[mildlyoverfitted]

Loved the video! Thank you for making it:)

[BalamurugaMuthumani]

Hi Shashank
Is there a GPU support for AWS serverless inference?

[spyrosavlonitis]

Thank you for your video! I am trying to find a similar example for PyTorch. Do you know any?

[sankarnatarajan]

Should we configure endpoint input and output manually or would it be created automatically?

[tothebit]

Hey, is there a way to schedule jobs of inference using input images from an s3 on a given interval, lets say every night run inference on 100 images stored in s3

[talha_anwar]

Thanks alot. Can you compare aws lambda with sagemaker inference

[nishantkumar-lwce]

I have a question related to processing job. How can I parameterize inputs?

[riazahmad]

Hello sir, please make vidoe that how to insert csv data to dynamodb in serverless framework using lambad in nodejs

[manaskumar]

thanks, Shashank can you please share the notebook if possible.

[jiteshdas]

How much time does the Serverless Sagemaker endpoints to stay active once invoked?

[mrunallveer]

I got an error while invoking the endpoint - ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from model with message "{"error": "Unsupported Media Type: application/x-image"}"