AWS SQS Trigger

Introduction

Amazon Simple Queue Service (AWS SQS) is a fully managed message queuing service offered by Amazon Web Services (AWS) that enables you to decouple and scale microservices, distributed systems, and serverless applications. By integrating AWS SQS with Martini, your packages can seamlessly become part of the AWS ecosystem. This integration allows for the invocation of services and workflows in response to SQS events with minimal coding. Martini leverages the AWS SQS SDK's polling API to efficiently retrieve messages from the queue for processing.

Usage in Workflows

To learn how to setup and use the SQS trigger in workflows, see Martini Workflows Trigger Nodes

Usage in Services

To learn how to setup and use the SQS trigger in services, see Invoking Services via a Trigger

Trigger Configuration

To utilize an AWS SQS trigger in Martini, you must configure a trigger with specific settings. Below is an overview of the necessary configuration options for setting up an AWS SQS trigger:

Trigger Type

• aws-sqs-listener

Trigger Configuration

Configuration Option	Required	Default Value	Description
queueName	Yes	N/A	The name of the AWS SQS queue to be polled.
pollTimeInSeconds	No	10	The polling interval in seconds. Adjusting this value can reduce AWS costs by decreasing the frequency of polling requests.
maxNumberOfMessage	No	10	The maximum number of messages to be polled in a single request.
awsCredentialOption	Yes	N/A	Determines how AWS credentials are supplied: - DEFAULT: Uses the default AWS credential provider chain. - BASIC: Requires specifying AWS credentials directly in the trigger properties.
awsAccessKeyId	No	N/A	Necessary if awsCredentialOption is set to BASIC. This is the AWS Access Key ID.
awsSecretKey	No	N/A	Necessary if awsCredentialOption is set to BASIC. This is the AWS Secret Access Key.
awsSessionToken	No	N/A	Necessary if awsCredentialOption is set to BASIC. This is the AWS session token, applicable for temporary credentials.
messageAttributes	Yes	N/A	Specifies the message attributes to fetch during polling. For multiple attributes, separate them with commas.

Configuration Guide

1. AWS Credentials: Ensure your AWS credentials are correctly configured. For DEFAULT, confirm the AWS credential provider chain is correctly set up. For BASIC, prepare your awsAccessKeyId, awsSecretKey, and, if applicable, awsSessionToken.

2. Trigger Configuration: In Martini, create a new AWS SQS trigger and fill in the configuration details as outlined above, tailoring settings to your specific needs.

3. Define Message Attributes: Determine which message attributes need to be retrieved during polling. Accurate attribute selection is critical for the effective processing of messages within your services and workflows.

4. Initiate Polling: With the trigger configured, Martini will start polling the specified SQS queue at the intervals you've set. Incoming messages will activate the associated services or workflows for automated message handling.

Best Practices

• Principle of Least Privilege: Ensure the IAM role or user has only the permissions necessary for operations on the SQS queue to enhance security.
• Optimize Costs: Modify the pollTimeInSeconds based on your application's requirements. A higher polling interval can reduce costs but might delay message processing.
• Robust Error Handling: Implement comprehensive error handling within your services and workflows to manage any message processing issues effectively.

Troubleshooting

• Access Permissions: If there are issues accessing the SQS queue, double-check that the AWS credentials used have the correct permissions.
• Processing Delays: For delayed message processing, consider decreasing the pollTimeInSeconds or increasing the maxNumberOfMessage to process more messages per polling request.

Trigger Input Properties

When the AWS SQS trigger receives a message, it provides these properties that you can access and use in your workflows or services for message processing.

Name	Type	Description
queueName	String	The name of the AWS SQS queue being polled.
messageId	String	The unique identifier for the message within AWS SQS.
body	String	The content of the message.
message	com.amazonaws.services.sqs.model.Message	The complete message object as received from AWS SQS, providing access to all details of the SQS message.
receiptHandle	String	The message's receipt handle, used for message deletion or change visibility operations.
messageAttributes	Array of Pair Model	An array of key-value pairs containing extra information about each message. To use a specific attribute in your workflow or service, add a Script node or step, and follow the steps in Accessing Custom Message Attributes.

Accessing and Using AWS SQS Message Properties

You can access AWS SQS message properties in Martini workflows and services. This lets you automate business logic, integrate with databases, or trigger downstream processes based on message content or metadata.

Getting Started

• Workflow Trigger Node: Access AWS SQS message properties from the Workflow Node form just like any other Martini workflow trigger node. See Workflow Trigger Node – Accessing and Using Trigger Properties for step-by-step instructions and examples.

• Service Trigger: Access AWS SQS message properties from the Service Trigger form just like any other Martini Service Trigger. See Service Trigger – Accessing and Using Trigger Properties for step-by-step instructions and examples.

Accessing Message Attributes

The messageAttributes property contains AWS SQS Message Attributes - custom key-value pairs that can be added to SQS messages to provide additional metadata.

To get a specific message attribute value, use one of these approaches:

Using Script Steps/Nodes:

Add a Script Step (for services) or Script Node (for workflows) with this script:

// Safely access a message attribute by key and assign to a new variable
def myVal = messageAttributes?.find { it.key == "keyNameHere" }?.value

Using Map Steps/Nodes:

In a Map Step (for services) or Map Node (for workflows), use this expression to map the value directly to a property in the workflow or service:

messageAttributes?.find { it.key == "keyNameHere" }?.value

Expected Result: You can use property values from AWS SQS messages received by the AWS SQS Trigger in your workflow or service.

Troubleshooting

Problem	Detection	Cause	Fix
Property not available in workflow or service	Property can't be accessed using Map or Script node	Property not selected in Properties tab	Select the property in the Properties tab
Message attribute returns null	Script returns null for attribute	Attribute key does not exist in message	Check attribute name and ensure it is sent with the message

Related Resources

Martini Documentation:

• Creating Workflows in Martini – Step-by-step guide to building workflows
• Creating Services – Guide to creating Martini services
• Invoking Services Via Triggers – Service trigger documentation
• Martini Workflow Triggers – Overview of all workflow triggers
• Workflow Trigger Node – Accessing and Using Trigger Properties – Specific trigger property access
• Workflow Script Node – How to use Script nodes
• Script Step – How to use Script steps in services
• Workflow Map Node – How to use Map nodes

AWS Documentation:

• AWS SQS Message API Reference – Official AWS documentation
• AWS SQS Message Attributes – Message attribute details

Usage and Testing for AWS SQS Trigger in Martini

This section provides a step-by-step guide on how to set up, configure, and test an AWS SQS trigger in Martini. This process involves creating an SQS queue in the AWS Console, setting up a new package in Martini with the necessary configuration, and testing the trigger by sending a message to the queue.

1. Set Up an AWS SQS Queue

• AWS Console: Go to the AWS Management Console, navigate to the SQS service, and create a new SQS queue. Configure the queue settings according to your requirements.

2. Configure Martini to Use AWS SQS

Usage in Workflows

To learn how to setup and use the SQS trigger in workflows, see Martini Workflows Trigger Nodes

Usage in Services

To learn how to setup and use the SQS trigger in services, see Invoking Services via a Trigger

Sample XML Configuration

<endpoint type="aws-sqs-listener" name="MyEndpoint" service="Test/TestService" enabled="true" modifiable="true">
  <properties>
    <property name="track">false</property>
    <property name="documentType">AWS Sqs Listener</property>
    <property name="replicated">true</property>
    <property name="queueName">ToroQueue</property>
    <property name="pollTimeInSeconds">10</property>
    <property name="maxNumberOfMessage">10</property>
    <property name="awsCredentialOption">BASIC</property>
    <property name="awsRegion">us-east-1</property>
    <property name="awsAccessKeyId">[Your_Access_Key_ID]</property>
    <property name="awsSecretKey">[Your_Secret_Access_Key]</property>
    <property name="awsSessionToken">[Your_Session_Token]</property>
    <property name="messageAttributes">test1,test2</property>
  </properties>
</endpoint>

Note: Replace [Your_Access_Key_ID], [Your_Secret_Access_Key], and [Your_Session_Token] with your actual AWS credentials.

3. Start the Trigger

• Initiate the Trigger: Start the configured AWS SQS trigger in Martini and ensure it's running properly.

4. Test the Trigger

• Send a Message: Use the AWS Console to send a message to your SQS queue.
• Verify Reception: Check if Martini successfully receives the message. You should see the message's properties printed out, confirming that the trigger is correctly processing incoming messages.

5. Observations

• Message Processing: Upon successful configuration and testing, Martini will automatically receive and process messages sent to the specified AWS SQS queue, demonstrating the trigger's effectiveness in integrating AWS services with Martini's capabilities.