From 5881bc139bd06742dec490f16f33fa7b2896ebbe Mon Sep 17 00:00:00 2001 From: atovpeko Date: Mon, 24 Feb 2025 10:08:59 +0200 Subject: [PATCH 1/4] draft --- use-timescale/integrations/cloudflare.md | 101 +++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 use-timescale/integrations/cloudflare.md diff --git a/use-timescale/integrations/cloudflare.md b/use-timescale/integrations/cloudflare.md new file mode 100644 index 0000000000..0397fde83f --- /dev/null +++ b/use-timescale/integrations/cloudflare.md @@ -0,0 +1,101 @@ +--- +title: Integrate Cloudflare Hyperdrive with Timescale Cloud +excerpt: +products: [cloud, mst, self_hosted] +keywords: [Cloudflare Hyperdrive, Timescale Cloud, database integration] +--- + +import IntegrationPrereqs from "versionContent/_partials/_integration-prereqs.mdx"; + +# Integrate Cloudflare Hyperdrive with $CLOUD_LONG + +[Cloudflare Hyperdrive][hyperdrive] is a global caching and query acceleration service for databases. It enables faster access by caching queries at Cloudflare’s edge locations. + +This page explains how to integrate Cloudflare Hyperdrive with $CLOUD_LONG to create a serverless, globally distributed API. + +## Prerequisites + + + +- Create a [Cloudflare account][cloudflare-account] and enable Hyperdrive. +- Install [Wrangler CLI][wrangler-cli]. + +## Connect your $SERVICE_LONG to Cloudflare Hyperdrive + +To connect to $CLOUD_LONG: + + + +1. **Connect to your $SERVICE_LONG using your [connection details][connection-info]** + + Use an [SQL editor][run-queries] in $CONSOLE. For self-hosted $TIMESCALE_DB, use [psql][psql]. + +1. **In your $SERVICE_SHORT, create a hypertable** + + Create a regular table, then convert it to a hypertable. For example: + + ```sql + CREATE TABLE readings( + ts timestamptz DEFAULT now() NOT NULL, + sensor UUID NOT NULL, + metadata jsonb, + value numeric NOT NULL + ); + + SELECT create_hypertable('readings', 'ts'); + ``` + +1. **Create a Worker project** + + 1. Run the following command to create a Worker project: + + + + + + ```shell + npm create cloudflare@latest -- timescale-api + ``` + + + + + + ```shell + pnpm create cloudflare@latest timescale-api + ``` + + + + + + ```shell + yarn create cloudflare timescale-api + ``` + + + + + + Specify the following options for setup: + + - `What would you like to start with?`:`Hello World`. + - `Which template would you like to use?`: `Hello World Worker`. + - `Which language do you want to use?`: `TypeScript`. + - `Do you want to use git for version control?`: `Yes`. + - `Do you want to deploy your application?`: `No`. + +1. **Create a Hyperdrive Configuration** + +1. **Deploy your Worker** + + + +You have successfully integrated Cloudflare Hyperdrive with Timescale Cloud. + +[connection-info]: /use-timescale/:currentVersion:/integrations/find-connection-details/ +[hyperdrive]: https://developers.cloudflare.com/hyperdrive/ +[cloudflare-account]: https://dash.cloudflare.com/sign-up +[wrangler-cli]: https://developers.cloudflare.com/workers/wrangler/get-started/ +[run-queries]: /getting-started/:currentVersion:/run-queries-from-console/ +[psql]: /use-timescale/:currentVersion:/integrations/psql/ From 0979144b4ed0d2fdf5bbeeb9cb89a204a077be2c Mon Sep 17 00:00:00 2001 From: atovpeko Date: Mon, 24 Feb 2025 10:58:07 +0200 Subject: [PATCH 2/4] review --- use-timescale/integrations/cloudflare.md | 166 ++++++++++++++++++++++- use-timescale/page-index/page-index.js | 5 + 2 files changed, 170 insertions(+), 1 deletion(-) diff --git a/use-timescale/integrations/cloudflare.md b/use-timescale/integrations/cloudflare.md index 0397fde83f..96554e2a2b 100644 --- a/use-timescale/integrations/cloudflare.md +++ b/use-timescale/integrations/cloudflare.md @@ -87,11 +87,173 @@ To connect to $CLOUD_LONG: 1. **Create a Hyperdrive Configuration** + 1. Run the `create` command with the `--connection-string` argument to pass the name of your Hyperdrive and your $SERVICE_SHORT URL from your [connection details][connection-info]: + + ```shell + npx wrangler hyperdrive create --connection-string="SERVICEURL" + ``` + + Hyperdrive attempts to connect to your $SERVICE_SHORT with the provided credentials. In case of an error, refer to the [Hyperdrive's troubleshooting documentation][hyperdrive-troubleshoot]. This command outputs your Hyperdrive ID. You can now bind your Hyperdrive configuration to your Worker in your [Wrangler configuration file][wrangler-configuration] by replacing the content with the following: + + + + + + ```json + { + "name": "hyperdrive-example", + "main": "src/index.ts", + "compatibility_date": "2024-08-21", + "compatibility_flags": [ + "nodejs_compat" + ], + "hyperdrive": [ + { + "binding": "HYPERDRIVE", + "id": "" + } + ] + } + ``` + + + + + + ```toml + name = "hyperdrive-example" + main = "src/index.ts" + compatibility_date = "2024-08-21" + compatibility_flags = ["nodejs_compat"] + + # Pasted from the output of `wrangler hyperdrive create --connection-string=[...]` above. + [[hyperdrive]] + binding = "HYPERDRIVE" + id = "" + ``` + + + + + + 1. Install the Postgres driver into your Worker project: + + ```shell + npm install pg + ``` + + 1. Replace the current code in `./src/index.ts` with the following: + + ```typescript + import { Client } from "pg"; + + export interface Env { + HYPERDRIVE: Hyperdrive; + } + + export default { + async fetch(request, env, ctx): Promise { + const client = new Client({ + connectionString: env.HYPERDRIVE.connectionString, + }); + await client.connect(); + + const url = new URL(request.url); + // Create a route for inserting JSON as readings + if (request.method === "POST" && url.pathname === "/readings") { + // Parse the request's JSON payload + const productData = await request.json(); + + // Write the raw query. You are using jsonb_to_recordset to expand the JSON + // to PG INSERT format to insert all items at once, and using coalesce to + // insert with the current timestamp if no ts field exists + const insertQuery = ` + INSERT INTO readings (ts, sensor, metadata, value) + SELECT coalesce(ts, now()), sensor, metadata, value FROM jsonb_to_recordset($1::jsonb) + AS t(ts timestamptz, sensor UUID, metadata jsonb, value numeric) + `; + + const insertResult = await client.query(insertQuery, [ + JSON.stringify(productData), + ]); + + // Collect the raw row count inserted to return + const resp = new Response(JSON.stringify(insertResult.rowCount), { + headers: { "Content-Type": "application/json" }, + }); + + ctx.waitUntil(client.end()); + return resp; + + // Create a route for querying within a time-frame + } else if (request.method === "GET" && url.pathname === "/readings") { + const limit = url.searchParams.get("limit"); + + // Query the readings table using the limit param passed + const result = await client.query( + "SELECT * FROM readings ORDER BY ts DESC LIMIT $1", + [limit], + ); + + // Return the result as JSON + const resp = new Response(JSON.stringify(result.rows), { + headers: { "Content-Type": "application/json" }, + }); + + ctx.waitUntil(client.end()); + return resp; + } + }, + } satisfies ExportedHandler; + ``` + + This code: + + - Uses Hyperdrive to connect to $CLOUD_LONG using the connection string. + - Creates a POST route which accepts an array of JSON readings to insert into $CLOUD_LONG in one transaction. + - Creates a GET route which takes a limit parameter and returns the most recent readings. This could be adapted to filter by ID or by timestamp. + 1. **Deploy your Worker** + Run the following command: + + ```shell + npx wrangler deploy + ``` + + The output shows the exact URI of your application that in the following format:` timescale-api..workers.dev`. + +1. **Interact with your $SERVICE_LONG** + + You can now interact with the IoT readings in your $SERVICE_SHORT using your Cloudflare Worker. For example: + + - Insert new rows into the `readings` hypertable you have created earlier. To do this, send a `POST` request to your Worker’s URL with the `/readings` path, along with a JSON payload containing the new product data. For example: + + ```curl + curl --request POST --data @- 'https://timescale-api..workers.dev/readings' <` with the deploy command output from the previous step. This command omits the `ts` (the timestamp) and `metadata` (the JSON blob) so they will be set to `now()` and `NULL`, respectively. + + - Query the `readings` hypertable by sending a `GET` request to your Worker’s URL with the `/readings` path. Set the `limit` parameter to control the amount of returned records: + + ```curl + curl "https://timescale-api..workers.dev/readings?limit=10" + ``` + -You have successfully integrated Cloudflare Hyperdrive with Timescale Cloud. +You have successfully integrated Cloudflare Hyperdrive with $CLOUD_LONG. [connection-info]: /use-timescale/:currentVersion:/integrations/find-connection-details/ [hyperdrive]: https://developers.cloudflare.com/hyperdrive/ @@ -99,3 +261,5 @@ You have successfully integrated Cloudflare Hyperdrive with Timescale Cloud. [wrangler-cli]: https://developers.cloudflare.com/workers/wrangler/get-started/ [run-queries]: /getting-started/:currentVersion:/run-queries-from-console/ [psql]: /use-timescale/:currentVersion:/integrations/psql/ +[hyperdrive-troubleshoot]: https://developers.cloudflare.com/hyperdrive/observability/troubleshooting/ +[wrangler-configuration]: https://developers.cloudflare.com/workers/wrangler/configuration/ \ No newline at end of file diff --git a/use-timescale/page-index/page-index.js b/use-timescale/page-index/page-index.js index 21e3991a98..6c6c3cf2cc 100644 --- a/use-timescale/page-index/page-index.js +++ b/use-timescale/page-index/page-index.js @@ -803,6 +803,11 @@ module.exports = [ href: "azure-data-studio", excerpt: "Integrate Azure Data Studio with Timescale products", }, + { + title: "Cloudflare Hyperdrive", + href: "cloudflare", + excerpt: "Integrate Cloudflare Hyperdrive with Timescale products", + }, { title: "Datadog", href: "datadog", From 0bac4962dfcd7e567ceb5f6716392aa3e4a72b49 Mon Sep 17 00:00:00 2001 From: atovpeko Date: Mon, 24 Feb 2025 11:33:13 +0200 Subject: [PATCH 3/4] draft --- use-timescale/integrations/cloudflare.md | 52 ++++++++++++------------ use-timescale/integrations/index.md | 10 +++-- 2 files changed, 33 insertions(+), 29 deletions(-) diff --git a/use-timescale/integrations/cloudflare.md b/use-timescale/integrations/cloudflare.md index 96554e2a2b..5b826f59a0 100644 --- a/use-timescale/integrations/cloudflare.md +++ b/use-timescale/integrations/cloudflare.md @@ -47,39 +47,39 @@ To connect to $CLOUD_LONG: 1. **Create a Worker project** - 1. Run the following command to create a Worker project: + Run the following command to create a Worker project: - + - + - ```shell - npm create cloudflare@latest -- timescale-api - ``` - - + ```shell + npm create cloudflare@latest -- timescale-api + ``` - + + + - ```shell - pnpm create cloudflare@latest timescale-api - ``` + ```shell + pnpm create cloudflare@latest timescale-api + ``` - + - + - ```shell - yarn create cloudflare timescale-api - ``` + ```shell + yarn create cloudflare timescale-api + ``` - + - + - Specify the following options for setup: + Specify the following options for setup: - - `What would you like to start with?`:`Hello World`. + - `What would you like to start with?`:`Hello World example`. - `Which template would you like to use?`: `Hello World Worker`. - `Which language do you want to use?`: `TypeScript`. - `Do you want to use git for version control?`: `Yes`. @@ -93,7 +93,9 @@ To connect to $CLOUD_LONG: npx wrangler hyperdrive create --connection-string="SERVICEURL" ``` - Hyperdrive attempts to connect to your $SERVICE_SHORT with the provided credentials. In case of an error, refer to the [Hyperdrive's troubleshooting documentation][hyperdrive-troubleshoot]. This command outputs your Hyperdrive ID. You can now bind your Hyperdrive configuration to your Worker in your [Wrangler configuration file][wrangler-configuration] by replacing the content with the following: + Hyperdrive attempts to connect to your $SERVICE_SHORT with the provided credentials. In case of an error, refer to the [Hyperdrive's troubleshooting documentation][hyperdrive-troubleshoot]. This command outputs your Hyperdrive ID. + + 1. Bind your Hyperdrive configuration to your Worker in your [Wrangler configuration file][wrangler-configuration] by replacing the content with the following: @@ -221,13 +223,13 @@ To connect to $CLOUD_LONG: npx wrangler deploy ``` - The output shows the exact URI of your application that in the following format:` timescale-api..workers.dev`. + The output shows the exact URI of your running application in the following format:` timescale-api..workers.dev`. 1. **Interact with your $SERVICE_LONG** You can now interact with the IoT readings in your $SERVICE_SHORT using your Cloudflare Worker. For example: - - Insert new rows into the `readings` hypertable you have created earlier. To do this, send a `POST` request to your Worker’s URL with the `/readings` path, along with a JSON payload containing the new product data. For example: + - Insert new rows into the `readings` hypertable you have created earlier. To do this, send a `POST` request to your Worker’s URL with the `/readings` path, along with a JSON payload containing the new product data. Replace `` with the deploy command output from the previous step. For example: ```curl curl --request POST --data @- 'https://timescale-api..workers.dev/readings' <` with the deploy command output from the previous step. This command omits the `ts` (the timestamp) and `metadata` (the JSON blob) so they will be set to `now()` and `NULL`, respectively. + This command omits the `ts` (the timestamp) and `metadata` (the JSON blob) so they will be set to `now()` and `NULL`, respectively. - Query the `readings` hypertable by sending a `GET` request to your Worker’s URL with the `/readings` path. Set the `limit` parameter to control the amount of returned records: diff --git a/use-timescale/integrations/index.md b/use-timescale/integrations/index.md index 8035336ba3..c151aefbef 100644 --- a/use-timescale/integrations/index.md +++ b/use-timescale/integrations/index.md @@ -40,9 +40,10 @@ Some of the most in-demand integrations for $CLOUD_LONG are listed below, with l ## Configuration and deployment -| Name | Description | -|:---------------------------:|--------------------------------------------------------------------------| -| [Terraform][terraform] | Safely and predictably provision and manage infrastructure in any cloud. | +| Name | Description | +|:-----------------------------------:|------------------------------------------------------------------------------------------------| +| [Terraform][terraform] | Safely and predictably provision and manage infrastructure in any cloud. | +| [Cloudflare Hyperdrive][cloudflare] | Distribute data storage and accelerate access across to your data from multiple cloud regions. | ## Data engineering and extract, transform, load @@ -69,4 +70,5 @@ Some of the most in-demand integrations for $CLOUD_LONG are listed below, with l [postgresql-integrations]: https://slashdot.org/software/p/PostgreSQL/integrations/ [prometheus]: /use-timescale/:currentVersion:/integrations/prometheus [amazon-sagemaker]: /use-timescale/:currentVersion:/integrations/amazon-sagemaker -[postgresql]: /use-timescale/:currentVersion:/integrations/postgresql \ No newline at end of file +[postgresql]: /use-timescale/:currentVersion:/integrations/postgresql +[cloudflare]: /use-timescale/:currentVersion:/integrations/cloudflare \ No newline at end of file From 13e502aedb3508686926fa90137079d74bba95d7 Mon Sep 17 00:00:00 2001 From: Anagha Mittal Date: Tue, 18 Mar 2025 06:44:01 +0530 Subject: [PATCH 4/4] updated cloudflare based on available resources --- use-timescale/integrations/cloudflare.md | 47 +++++++++++++----------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/use-timescale/integrations/cloudflare.md b/use-timescale/integrations/cloudflare.md index 5b826f59a0..9a44dbf5fc 100644 --- a/use-timescale/integrations/cloudflare.md +++ b/use-timescale/integrations/cloudflare.md @@ -28,7 +28,7 @@ To connect to $CLOUD_LONG: 1. **Connect to your $SERVICE_LONG using your [connection details][connection-info]** - Use an [SQL editor][run-queries] in $CONSOLE. For self-hosted $TIMESCALE_DB, use [psql][psql]. + Use an [SQL editor][run-queries] in $CONSOLE. For self-hosted $TIMESCALE_DB, use [psql][psql]. 1. **In your $SERVICE_SHORT, create a hypertable** @@ -85,12 +85,18 @@ To connect to $CLOUD_LONG: - `Do you want to use git for version control?`: `Yes`. - `Do you want to deploy your application?`: `No`. + You should see the following: + + ```bash + SUCCESS Application created successfully! + ``` + 1. **Create a Hyperdrive Configuration** 1. Run the `create` command with the `--connection-string` argument to pass the name of your Hyperdrive and your $SERVICE_SHORT URL from your [connection details][connection-info]: ```shell - npx wrangler hyperdrive create --connection-string="SERVICEURL" + npx wrangler hyperdrive create hyperdrive --connection-string="SERVICEURL" ``` Hyperdrive attempts to connect to your $SERVICE_SHORT with the provided credentials. In case of an error, refer to the [Hyperdrive's troubleshooting documentation][hyperdrive-troubleshoot]. This command outputs your Hyperdrive ID. @@ -103,19 +109,19 @@ To connect to $CLOUD_LONG: ```json { - "name": "hyperdrive-example", - "main": "src/index.ts", - "compatibility_date": "2024-08-21", - "compatibility_flags": [ - "nodejs_compat" - ], - "hyperdrive": [ - { - "binding": "HYPERDRIVE", - "id": "" - } - ] - } + "name": "timescale-api", + "main": "src/index.ts", + "compatibility_date": "2024-09-23", + "compatibility_flags": [ + "nodejs_compat" + ], + "hyperdrive": [ + { + "binding": "HYPERDRIVE", + "id": "YOUR_HYPERDRIVE_ID" + } + ] + } ``` @@ -123,15 +129,14 @@ To connect to $CLOUD_LONG: ```toml - name = "hyperdrive-example" + name = "timescale-api" main = "src/index.ts" - compatibility_date = "2024-08-21" - compatibility_flags = ["nodejs_compat"] - - # Pasted from the output of `wrangler hyperdrive create --connection-string=[...]` above. + compatibility_date = "2024-09-23" + compatibility_flags = [ "nodejs_compat"] + [[hyperdrive]] binding = "HYPERDRIVE" - id = "" + id = "YOUR_HYPERDRIVE_ID" ```