Skip to content

Lambda Function Cache-Aside with DSQL #421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ target

# These are backup files generated by rustfmt
**/*.rs.bk
*.DS_STORE*
*node_modules*
*cdk.out*
44 changes: 44 additions & 0 deletions example/aws/cache-aside-dsql-lambda/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
<img src="https://docs.momentohq.com/img/momento-logo-forest.svg" alt="logo" width="400"/>

[![project status](https://momentohq.github.io/standards-and-practices/badges/project-status-official.svg)](https://github.com/momentohq/standards-and-practices/blob/main/docs/momento-on-github.md)
[![project stability](https://momentohq.github.io/standards-and-practices/badges/project-stability-beta.svg)](https://github.com/momentohq/standards-and-practices/blob/main/docs/momento-on-github.md)


# Momento Rust SDK - AWS Lambda CDK Example

This directory contains an example project defining a Rust-based AWS Lambda function that implements a cache-aside strategy when working with AWS DSQL and uses the Momento Rust SDK to interact with the Cache.

The Lambda function is deployed using the AWS Cloud Development Kit (CDK).

## Example Requirements

- Follow the [installation guide](https://doc.rust-lang.org/cargo/getting-started/installation.html) to install Rust and Cargo.
- You will also need the [cargo-lambda cargo extension](https://www.cargo-lambda.info/)
- The CDK code in this repo is written in TypeScript, so you will need `Node.js` version 16 or later, and a compatible
version of `npm` installed. If you don't have these, we recommend [nodenv](https://github.com/nodenv/nodenv).
- To get started with Momento you will need a Momento API key. You can get one from the [Momento Console](https://console.gomomento.com).

## Building and Deploying the Lambda Function

This solution uses SQLx which requires connecting to the Database to build the `.sqlx` prepared query directory. As of this writing, working with it and DSQL is a challenge so the easiest way to get this up and running is to create a local Postgres instance in Docker, build the table there, point the DATABASE_URL at that local container, and then run the DDL queries against DSQL when done.

To build and deploy the Lambda function, first make sure that your AWS credentials are set up properly (via env vars or ~/.aws/credentials file). Then all you need to do is run the following commands:

```bash
cd infra
export DATABASE_URL=<Local Postgre Connection String>
export CLUSTER_ENDPOINT=<DSQL Cluster Endpoint>
export MOMENTO_API_KEY=<Momento API Key>
cdk deploy
```

After the lambda is deployed, you can use the defined FunctionURL on the Lambda Function and supply and `?id=` to the URL for the Item in your database. Make sure to build a table in DSQL that matches the Rust `CacheableItem` struct definition.

## Interesting Files in this Example

- `infra/bin/lambda-cache-aside.ts` - this is the CDK stack that defines the Lambda function its associated resources. It uses the `RustFunction` construct from the `cargo-lambda-cdk` package.
- `rust/get-lambda/Cargo.toml` - this is the Cargo.toml file for the Lambda function. It includes the `lambda_runtime`crate as a dependency; this makes it easy to write Lambda functions in Rust
- `rust/get-lambda/src/main.rs` - this is the Rust code for the Lambda function. It uses the AWS `lambda_runtime` crate to implement the `main` function in a way that is compatible with AWS's provided Amazon Linux runtimes.

----------------------------------------------------------------------------------------
For more info, visit our website at [https://gomomento.com](https://gomomento.com)!
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env node

import * as cdk from 'aws-cdk-lib';
import { LambdaCacheAsideStack } from '../lib/lambda-cache-aside-stack';

const app = new cdk.App();
new LambdaCacheAsideStack(app, 'LambdaCacheAsideDSQLStack', {
env: {
region: "us-east-1"
}
});
80 changes: 80 additions & 0 deletions example/aws/cache-aside-dsql-lambda/infra/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
{
"app": "npx ts-node --prefer-ts-exts bin/lambda-cache-aside.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
"@aws-cdk/core:checkSecretUsage": true,
"@aws-cdk/core:target-partitions": [
"aws",
"aws-cn"
],
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
"@aws-cdk/aws-iam:minimizePolicies": true,
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
"@aws-cdk/core:enablePartitionLiterals": true,
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
"@aws-cdk/aws-route53-patters:useCertificate": true,
"@aws-cdk/customresources:installLatestAwsSdkDefault": false,
"@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
"@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
"@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
"@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
"@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
"@aws-cdk/aws-redshift:columnId": true,
"@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
"@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
"@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
"@aws-cdk/aws-kms:aliasNameRef": true,
"@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
"@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
"@aws-cdk/aws-efs:denyAnonymousAccess": true,
"@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
"@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
"@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
"@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
"@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
"@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
"@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
"@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
"@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
"@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
"@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
"@aws-cdk/aws-eks:nodegroupNameAttribute": true,
"@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
"@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
"@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
"@aws-cdk/aws-s3:keepNotificationInImportedBucket": false,
"@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true,
"@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true,
"@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true,
"@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true,
"@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true,
"@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true,
"@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true,
"@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true
}
}
8 changes: 8 additions & 0 deletions example/aws/cache-aside-dsql-lambda/infra/jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
module.exports = {
testEnvironment: 'node',
roots: ['<rootDir>/test'],
testMatch: ['**/*.test.ts'],
transform: {
'^.+\\.tsx?$': 'ts-jest'
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import { Effect, PolicyStatement } from "aws-cdk-lib/aws-iam";
import { Architecture, FunctionUrlAuthType, LayerVersion } from "aws-cdk-lib/aws-lambda";
import { RustFunction } from "cargo-lambda-cdk";
import { Construct } from "constructs"

export class LambdaConstruct extends Construct {
constructor(scope: Construct, id: string) {
super(scope, id);

const select = new RustFunction(scope, 'SelectFunction', {
architecture: Architecture.ARM_64,
functionName: "cacheable-table-select-dsql",
manifestPath: '../rust/get-lambda',
memorySize: 256,
environment: {
CLUSTER_ENDPOINT: process.env.CLUSTER_ENDPOINT!,
MOMENTO_API_KEY: process.env.MOMENTO_API_KEY!,
CACHE_NAME: "CacheableTable",
RUST_LOG: 'info',
},
})

select.addToRolePolicy(new PolicyStatement({
effect: Effect.ALLOW,
actions: ["dsql:*"],
resources: ["*"]
}))

select.addFunctionUrl({
authType: FunctionUrlAuthType.NONE,
cors: {
allowedOrigins: ["*"]
}
})


}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { LambdaConstruct } from './constructs/lambda-construct';

export class LambdaCacheAsideStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);

new LambdaConstruct(this, 'LambdaConstruct');

}
}
Loading