The workflow presented here can be used for a number of UGC processing use cases such as:

For workloads dealing with larger media files beyond Lambda’s memory capacity, mounting an Amazon EFS file system should still be considered. UGC is any content created by individuals and is usually characterized by small or capped file sizes. Lambda functions can be configured with up to 10240 MB of memory, which makes the workflow well suited for processing user-generated content (UGC). Instead of copying media files locally for processing, we put together a workflow to use Lambda’s memory space. This approach, however, involves re-architecting the storage workflow (Amazon S3 and/or Amazon EFS), setting up and managing other services like Amazon Virtual Private Cloud (Amazon VPC), and may incur additional cost.

At the same time, Lambda can be configured to mount an Amazon Elastic File System (Amazon EFS). Lambda is equipped with temporary storage space of 512 MB, which is often not sufficient for media processing. In this post, we introduce a workflow to process media files stored in Amazon Simple Storage Service (Amazon S3) using AWS Lambda that doesn’t require copying files to Lambda’s local storage.

Ffmpeg python generate mp4 in memory