How do we close the gap between the necessity for secure, efficient cloud-native development and the practical challenges faced by teams, especially when access to AWS credentials is limited? As organizations grow and prioritize security, the distribution of AWS credentials becomes highly restricted. This leaves developers, particularly those in junior positions, with two suboptimal choices:

• Rely on inadequate mocks that fail to capture the nuances of AWS services
• Skip local testing altogether, directly impacting QA or staging environments. This approach not only introduces risk but also slows the development cycle.

So, why continue in this direction when a solution exists that directly addresses these challenges?

LocalStack: Bridging the Gap

LocalStack addresses this gap by enabling the local simulation of AWS cloud environments for both development and testing. This innovative tool offers:

Realistic AWS Environment Simulation: LocalStack creates a self-contained, on-premise replica of AWS services, allowing for extensive testing of cloud-specific features without the costs or complexities associated with the actual AWS cloud.

Enhanced Security and Control: By facilitating local testing, LocalStack mitigates the need for widespread sharing of AWS credentials, enhancing security and reducing risks in larger teams.

Cost Efficiency and Accelerated Development: Local testing with LocalStack reduces expenses and speeds up the development cycle, allowing for rapid iteration without the need for remote deployments.

Seamless CI/CD Integration: Incorporating LocalStack into CI/CD pipelines ensures a production-like testing environment that is both safe and representative, without exposing the actual production environment to risk.

Now moving on to the practical part, let us employ a practical, hands-on exercise that will enable one to appreciate and comprehend these theoretical concepts in a practical dynamic. We will testify the use case of real life involving AWS EventScheduler, SQS and Lambda and all these will be done in-project

Illustration: Event Scheduling and Lambda Invocation

To illustrate how to apply the previously discussed concepts in a practical use case, let’s focus on scheduling the expiry of a search listing after 3 days. We’ll create a system that schedules an EventBridge event to trigger a Lambda function after 3 days. This Lambda function is supposed to mark a search listing as expired. If the Lambda invocation fails, it will retry three times before the message is sent to a Dead Letter Queue (DLQ).

Step 1: Set Up Your Environment

To get started with the practical exercises in this article, you’ll need to have Docker and AWS CLI installed on your machine. Additionally, LocalStack is essential for our purposes.

Below are the links to the tools you’ll need:

LocalStack: Visit GitHub

Docker: Visit Docker

AWS CLI: Visit AWS CLI

This article is centered around LocalStack and its functionalities, so we won’t cover the installation processes for Docker and AWS CLI.

Install LocalStack: There are multiple ways to install LocalStack, Please go through the LocalStack Installation Page for alternatives. One of the quickest way with Python is using pip.

pip install localstack

Start LocalStack: You can start LocalStack using the command line:

localstack start

Step 2: Create Your Lambda Function

Write the Lambda Function: Create a simple Lambda function in your preferred language supported by AWS Lambda. For example, here’s a basic Lambda function in Python with file name handler.py:

import json

def lambda_handler(event, context):
    print("Lambda invoked successfully!")
    # Do something here related to listing expiry i.e REST call to some service
    return {
        'statusCode': 200,
        'body': json.dumps('Request sent successfully'),
        'response': response.text
    }

Package Your Lambda: For languages like Python, you can zip your code:

pip install -r requirements.txt -t ./package && cp handler.py ./package/ && cd package &&  zip -r9 ../handler.zip . 

This command creates a ZIP file named handler.zip in the parent directory of package, containing all the contents of the package folder.

Step 3: Deploy Your Lambda Function to LocalStack

Set AWS CLI to Use LocalStack: Configure AWS CLI to interact with LocalStack by setting the endpoint to LocalStack’s endpoint. Replace AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with dummy values (LocalStack doesn’t use them for authentication).

aws configure

Your input will look something like this:

AWS Access Key ID [None]: test
AWS Secret Access Key [None]: test
Default region name [None]: ap-south-1
Default output format [None]: json

These dummy credentials are only used for interacting with LocalStack and won’t impact your actual AWS configurations. Enter dummy credentials and use http://localhost:4566 as the default region endpoint.

Deploy your Lambda function, specifying the DLQ ARN for retry and failure handling:

aws --endpoint-url=http://localhost:4566 lambda create-function --function-name listing-expiry-function \
--zip-file fileb://path/to/lambda_function.zip --handler handler.lambda_handler --runtime python3.8 \
--role arn:aws:iam::000000000000:role/irrelevant --dead-letter-config TargetArn=<DLQ ARN>

How to Get the DLQ ARN?

To get the ARN of your Dead Letter Queue (DLQ), you first need to create an SQS queue that will serve as the DLQ. Here’s how you can create it and retrieve the ARN:

1. Create the SQS Dead Letter Queue:

aws --endpoint-url=http://localhost:4566 sqs create-queue --queue-name ListingExpiryDLQ

2. Retrieve the DLQ ARN:

After creating the queue, you can retrieve its ARN with the following command:

aws --endpoint-url=http://localhost:4566 sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/ListingExpiryDLQ --attribute-names QueueArn

LocalStack includes a user-friendly dashboard that allows users to easily interact with and manage their cloud resources. For this you can install LocalStack Desktop.

With LocalStack Desktop also one can easily get the details required as mentioned in the attached image.

After deploying lambda via console with the above mentioned command of lambda create-function we get something similar to what I have attached below.

Step 4: Create the EventBridge Schedule

Schedule the Lambda to be triggered 3 days after listing creation. The rate() expression in EventBridge Scheduler doesn’t directly support day intervals, so you might consider triggering a Lambda function based on the event timestamp already computed (i.e at(yyyy-MM-ddTHH:mm:ss)) or using a cron expression for specific timing.

aws --endpoint-url=http://localhost:4566 scheduler create-schedule --schedule-expression "at(2024-02-04T16:29:31)" --name expire-listing-rule-02 --target '{"RoleArn": "arn:aws:iam::000000000000:role/irrelevant", "Arn": "arn:aws:lambda:ap-south-1:000000000000:function:listing-expiry-function", "Input": "{}"}' --schedule-expression-timezone "Asia/Kolkata" --flexible-time-window '{ "Mode": "OFF"}'

After creating the rule, one can see the Schedule created successfully.

One can list down schedules by following command

aws scheduler list-schedules --endpoint-url=http://localhost:4566

Just as in the real AWS ecosystem, one can examine the CloudWatch logs associated with Lambda functions to gain insights into their execution and performance. As we can see the event got triggered as expected.

Final thoughts

To conclude our discussion in LocalStack, it is also evident that although LocalStack can make local testing of AWS cloud-native environments easier, one still needs to incept detailed attention to the security policies, roles, as well as network configurations that LocalStack is unable to replicate.

It is crucial to do the thorough testing in a real AWS environment before the transition into the production phase so that the compliance, security, and performance guidelines are met. LocalStack is good at fast cycles of development, but an effective deployment to production must follow a rigorous security and networking process, as well as a detailed approach to parameters that are specific to the cloud platform, which makes a structured approach that takes into consideration the balance between LocalStack efficiency and robust AWS infrastructure the most optimal strategy.