Skip to content

Idempotent Workflow Bug: Duplicate IVS Channels Created by IVSCreateChannelsLambdaFunction #424

Open
@NullPointer4096

Description

@NullPointer4096

Hello,

The ivs-create-channels.lambda_handler function is asynchronously invoked by CloudFormation to create, update, or delete IVS channels and maintain the video-to-channel mappings in SSM.

For each video_file_key, the workflow is that if there’s no IVS channel associated with it in SSM, create a channel with a name generated from the current time and record the mapping in SSM. If an error occurs after the channel is created but before the mapping is updated (L156 or L164), the function retries and creates a new channel with a different name in the second execution. This creates duplicate IVS channels, leaving the old channel unused and undeleted. Though unlikely to happen, the triggering error includes any software, hardware, and network errors that would cause the asynchronously invoked function to retry. The expected behavior is that the function should not create duplicate IVS channels at retries after a partial execution (i.e., created channels in the first run but did not update the mapping stored in SSM).

Suggested fix:
First, make the channel_name deterministic across retries. Maybe replace the ever-changing timestamp with the deterministic aws_request_id passed along in the context object. Second, before creating a channel, query whether a channel of this name has been created through IVS’ list_channels API. If it has been created, record the association in SSM. Otherwise, create the channel and record it to SSM. This approach ensures that only one channel is created and associated with the video, preventing the creation of unused channels.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions