Error "Request has oversize body" When Invoking SageMaker Endpoint with Large Video File

[Manishthakur2503]

Hi,

I am new to using SageMaker and recently deployed a model using a Docker image that I pushed to ECR. I created the model in SageMaker, configured an endpoint for serverless inference, and deployed the endpoint successfully. Before deploying, I tested the Docker image locally with the following code:

import requests

url = 'http://localhost:8080/invocations'
file_path = "C:\\Projects\\ML\\Video-Audio-Face-Emotion-Recognition\\Original_cloned_code\\input_files\\disgust.mp4"

with open(file_path, 'rb') as file:
    files = {'file': file}
    response = requests.post(url, files=files)

print(response.json())

This worked perfectly. However, after deploying the endpoint on SageMaker, I am invoking it like this:

import boto3

# Initialize the SageMaker runtime client
runtime = boto3.client("sagemaker-runtime")

endpoint_name = "<endpoint name>"  
content_type = "application/octet-stream"

# Path to the video file
file_path = "C:\\Projects\\ML\\Video-Audio-Face-Emotion-Recognition\\Original_cloned_code\\input_files\\disgust.mp4"

# Read the video file as binary
with open(file_path, 'rb') as file:
    payload = file.read()

# Invoke the SageMaker endpoint
response = runtime.invoke_endpoint(
    EndpointName=endpoint_name,
    ContentType=content_type,
    Body=payload
)

# Process the response
result = response['Body'].read().decode('utf-8')
print(result)

However, I'm encountering an error when running the code:

Traceback (most recent call last):
  File "c:\Users\manis\Desktop\test3.py", line 18, in <module>
    response = runtime.invoke_endpoint(
  File "C:\Users\manis\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\client.py", line 569, in _api_call    
    return self._make_api_call(operation_name, kwargs)
  File "C:\Users\manis\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.ValidationError: An error occurred (ValidationError) when calling the InvokeEndpoint operation: Request <id> has oversize body.

The input video file I’m using is also stored in S3. I'm unsure how to proceed, and I’m wondering if there's a size limit for the payload or if there's a recommended way to handle large video files when invoking SageMaker endpoints. Could you please provide guidance on how to resolve this issue?

Thank you in advance
Manish Thakur
manish.thakur@manomayaai.com

[Manishthakur2503]

Hi , I am able to solve the issue.
But I am getting one another error:

Traceback (most recent call last):
  File "c:\Users\manis\Desktop\test3.py", line 16, in <module>
    response = runtime.invoke_endpoint(
  File "C:\Users\manis\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "C:\Users\manis\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from model with message "{"error":"[Errno 13] Permission denied: '/app/models/face/face_norm_scalar.pkl'"}

This model folder is part of my docker which I pushed to ECR and while creating the docker image I gave the permission as well
RUN chmod +x /usr/local/bin/serve && chmod 777 /app/input_files && chmod 777 /app/models.

Thanks
Manish Thakur

[rahulmisal27]

@Manishthakur2503 This is because your model is not in /opt/program/model path. SageMaker makes only this path readable so create this folder structure in your container.