SQS → EventBridge Pipe → Step Functions Test

This is a minimal AWS CDK TypeScript project that demonstrates:

• SQS Queue as the source
• EventBridge Pipe for event routing
• Step Functions State Machine as the target (fire-and-forget invocation)
• Proper IAM permissions

Architecture

graph LR
    A[SQS Queue<br/>test-sqs-pipe-source-queue] -->|EventBridge Pipe| B[Step Functions<br/>Express Workflow]
    
    subgraph "Message Flow"
        C[Producer] -->|Send Message| A
        B -->|Process & Log| D[CloudWatch Logs]
    end
    
    style A fill:#FF9900,stroke:#333,stroke-width:2px,color:#fff
    style B fill:#146EB4,stroke:#333,stroke-width:2px,color:#fff
    style C fill:#232F3E,stroke:#333,stroke-width:2px,color:#fff
    style D fill:#759C3E,stroke:#333,stroke-width:2px,color:#fff

Loading

Data Flow Sequence

sequenceDiagram
    participant P as Producer
    participant Q as SQS Queue
    participant E as EventBridge Pipe
    participant S as Step Functions
    participant L as CloudWatch Logs
    
    P->>Q: Send Message
    Note over Q: Message queued<br/>(Visibility: 300s)
    E->>Q: Poll for messages
    Q->>E: Return message batch
    E->>S: StartExecution<br/>(Fire & Forget)
    S->>S: Process message
    S->>L: Log execution details
    E->>Q: Delete message
    Note over S: Express workflow<br/>completes

Loading

Prerequisites

• AWS CLI configured with credentials
• Node.js and npm installed

Setup

1. Install dependencies:

npm install

2. Deploy the stack:

npm run cdk deploy

3. Note the outputs:

• QueueUrl - The SQS queue URL for sending test messages
• StateMachineArn - The Step Functions state machine ARN
• PipeArn - The EventBridge Pipe ARN

Testing

Send a test message to the SQS queue:

export QUEUE_URL=<your-queue-url-from-cdk-output>
npx ts-node test-message.ts

Components

SQS Queue

• Name: test-sqs-pipe-source-queue
• Visibility timeout: 300 seconds
• Retention period: 1 day

Step Functions State Machine

• Type: Express workflow (for fire-and-forget)
• Processes incoming messages from SQS
• Logs messages to CloudWatch Logs

EventBridge Pipe

• Connects SQS to Step Functions
• Batch size: 1 (processes messages individually)
• Invocation type: Fire and forget

IAM Permissions

The pipe role has:

• SQS permissions: ReceiveMessage, DeleteMessage, GetQueueAttributes
• Step Functions permissions: StartExecution, StartSyncExecution

Clean Up

Remove the stack:

npm run cdk destroy