---
title: Create a data lake of clickstream data · Cloudflare Pipelines Docs
description: In this tutorial, you will learn how to build a data lake of website interaction events (clickstream data), using Pipelines.
lastUpdated: 2025-04-30T09:59:18.000Z
source_url:
  html: https://77261bbd.preview.developers.cloudflare.com/pipelines/tutorials/send-data-from-client/index.md
  md: https://77261bbd.preview.developers.cloudflare.com/pipelines/tutorials/send-data-from-client/
---

In this tutorial, you will learn how to build a data lake of website interaction events (clickstream data), using Pipelines.

Data lakes are a way to store large volumes of raw data in an object storage service such as [R2](https://developers.cloudflare.com/r2). You can run queries over a data lake to analyze the raw events.

For this tutorial, you will build a landing page for an e-commerce website. Users can click on the website, to view products or add them to the cart. When a user clicks on buttons on the page, events will be sent to a pipeline. These events are sent from the client-side (directly from the user's browser). Your pipeline will automatically batch the ingested data, build output files, and deliver them to an [R2 bucket](https://developers.cloudflare.com/r2) to build your data lake.

## Prerequisites

1. Install [`Node.js`](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm).

## 1. Create a new Workers project

You will create a new Worker project that will use [Static Assets](https://developers.cloudflare.com/workers/static-assets/) to serve the HTML file. While you can use any front-end framework, this tutorial uses plain HTML and JavaScript to keep things simple. If you are interested in learning how to build and deploy a web application on Workers with Static Assets, you can refer to the [Frameworks](https://developers.cloudflare.com/workers/frameworks/) documentation.

Create a new Worker project by running the following commands:

* npm

  ```sh
  npm create cloudflare@latest -- e-commerce-pipelines-client-side
  ```

* pnpm

  ```sh
  pnpm create cloudflare@latest e-commerce-pipelines-client-side
  ```

* yarn

  ```sh
  yarn create cloudflare e-commerce-pipelines-client-side
  ```

For setup, select the following options:

* For *What would you like to start with?*, choose `Hello World Starter`.
* For *Which template would you like to use?*, choose `SSR / full-stack app`.
* For *Which language do you want to use?*, choose `TypeScript`.
* For *Do you want to use git for version control?*, choose `Yes`.
* For *Do you want to deploy your application?*, choose `No` (we will be making some changes before deploying).

Navigate to the `e-commerce-pipelines-client-side` directory:

```sh
cd e-commerce-pipelines-client-side
```

## 2. Update the website frontend

Using Static Assets, you can serve the frontend of your application from your Worker. The above step creates a new Worker project with a default `public/index.html` file. Update the `public/index.html` file with the following HTML code:

The above code does the following:

* Uses Tailwind CSS to style the page.
* Renders a list of products.
* Adds a button to view the details of a product.
* Adds a button to add a product to the cart.
* Contains a `handleClick` function to handle the click events. This function logs the action and the product ID. In the next steps, you will create a pipeline and add the logic to send the click events to this pipeline.

## 3. Create an R2 Bucket

We'll create a new R2 bucket to use as the sink for our pipeline. Create a new r2 bucket `clickstream-bucket` using the [Wrangler CLI](https://developers.cloudflare.com/workers/wrangler/). Open a terminal window, and run the following command:

```sh
npx wrangler r2 bucket create clickstream-bucket
```

## 4. Create a pipeline

You need to create a new pipeline and connect it to your R2 bucket.

Create a new pipeline `clickstream-pipeline-client` using the [Wrangler CLI](https://developers.cloudflare.com/workers/wrangler/). Open a terminal window, and run the following command:

```sh
npx wrangler pipelines create clickstream-pipeline-client --r2-bucket clickstream-bucket --compression none --batch-max-seconds 5
```

When you run the command, you will be prompted to authorize Cloudflare Workers Pipelines to create R2 API tokens on your behalf. These tokens are required by your Pipeline. Your Pipeline uses these tokens when loading data into your bucket. You can approve the request through the browser link which will open automatically.

Note

The above command creates a pipeline using two optional flags: `--compression none`, and `--batch-max-seconds 5`.

With these flags, your pipeline will deliver an uncompressed file of data to your R2 bucket every 5 seconds.

These flags are useful for testing, but we recommend keeping the default settings in a production environment.

```txt
✅ Successfully created Pipeline "clickstream-pipeline-client" with ID <PIPELINE_ID>


Id:    <PIPELINE_ID>
Name:  clickstream-pipeline-client
Sources:
  HTTP:
    Endpoint:        https://<PIPELINE_ID>.pipelines.cloudflare.com
    Authentication:  off
    Format:          JSON
  Worker:
    Format:  JSON
Destination:
  Type:         R2
  Bucket:       clickstream-bucket
  Format:       newline-delimited JSON
  Compression:  NONE
  Batch hints:
    Max bytes:     100 MB
    Max duration:  300 seconds
    Max records:   10,000,000


🎉 You can now send data to your Pipeline!


Send data to your Pipeline's HTTP endpoint:


curl "https://<PIPELINE_ID>.pipelines.cloudflare.com" -d '[{"foo": "bar"}]'
```

Make a note of the URL of the pipeline. You will use this URL to send the clickstream data from the client-side.

## 5. Generate clickstream data

You need to send clickstream data like the `timestamp`, `user_id`, `session_id`, and `device_info` to your pipeline. You can generate this data on the client side. Add the following function in the `<script>` tag in your `public/index.html`. This function gets the device information:

```js
function extractDeviceInfo(userAgent) {
  let browser = "Unknown";
  let os = "Unknown";
  let device = "Unknown";


  // Extract browser
  if (userAgent.includes("Firefox")) {
    browser = "Firefox";
  } else if (userAgent.includes("Chrome")) {
    browser = "Chrome";
  } else if (userAgent.includes("Safari")) {
    browser = "Safari";
  } else if (userAgent.includes("Opera") || userAgent.includes("OPR")) {
    browser = "Opera";
  } else if (userAgent.includes("Edge")) {
    browser = "Edge";
  } else if (userAgent.includes("MSIE") || userAgent.includes("Trident/")) {
    browser = "Internet Explorer";
  }


  // Extract OS
  if (userAgent.includes("Win")) {
    os = "Windows";
  } else if (userAgent.includes("Mac")) {
    os = "MacOS";
  } else if (userAgent.includes("Linux")) {
    os = "Linux";
  } else if (userAgent.includes("Android")) {
    os = "Android";
  } else if (userAgent.includes("iOS")) {
    os = "iOS";
  }


  // Extract device
  const mobileKeywords = [
    "Android",
    "webOS",
    "iPhone",
    "iPad",
    "iPod",
    "BlackBerry",
    "Windows Phone",
  ];
  device = mobileKeywords.some((keyword) => userAgent.includes(keyword))
    ? "Mobile"
    : "Desktop";


  return { browser, os, device };
}
```

## 6. Send clickstream data to your pipeline

You will send the clickstream data to the pipline from the client-side. To do that, update the `handleClick` function to make a `POST` request to the pipeline URL with the data. Replace `<PIPELINE_URL>` with the URL of your pipeline.

```js
async function handleClick(action, id) {
  console.log(`Clicked ${action} for product with id ${id}`);


  const userAgent = window.navigator.userAgent;
  const timestamp = new Date().toISOString();
  const { browser, os, device } = extractDeviceInfo(userAgent);


  const data = {
    timestamp,
    session_id: "1234567890abcdef", // For production use a unique session ID
    user_id: "user" + Math.floor(Math.random() * 1000), // For production use a unique user ID
    event_data: {
      event_id: Math.floor(Math.random() * 1000),
      event_type: action,
      page_url: window.location.href,
      timestamp,
      product_id: id,
    },
    device_info: {
      browser,
      os,
      device,
      userAgent,
    },
    referrer: document.referrer,
  };
  try {
    const response = await fetch("<PIPELINE_URL>", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify([data]),
    });
    if (!response.ok) {
      throw new Error("Failed to send data to pipeline");
    }
  } catch (error) {
    console.error("Error sending data to pipeline:", error);
  }
}
```

The `handleClick` function does the following:

* Gets the device information using the `extractDeviceInfo` function.
* Makes a `POST` request to the pipeline with the data.
* Logs any errors that occur.

If you start the development server and open the application in the browser, you can see the `handleClick` function gets executed when you click on the `View Details` or `Add to Cart` button.

```sh
npm run dev
```

However, no data gets sent to the pipeline. Inspect the browser console to view the error message. The error message you see is for [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS). In the next step, you will update the CORS settings to allow the client-side JavaScript to send data to the pipeline.

## 7. Update CORS settings

By default, the HTTP ingestion endpoint for your pipeline does not allow cross-origin requests. You need to update the CORS settings to allow the client-side JavaScript to send data to the pipeline. To update the CORS settings, execute the following command:

```sh
npx wrangler pipelines update clickstream-pipeline-client --cors-origins http://localhost:8787
```

Now when you run the development server locally, and open the website in a browser, clickstream data will be successfully sent to the pipeline. You can learn more about the CORS settings in the [Specifying CORS settings](https://developers.cloudflare.com/pipelines/build-with-pipelines/sources/http/#specifying-cors-settings) documentation.

## 8. Deploy the application

To deploy the application, run the following command:

```sh
npm run deploy
```

This will deploy the application to the Cloudflare Workers platform.

```txt
🌀 Building list of assets...
🌀 Starting asset upload...
🌀 Found 1 new or modified static asset to upload. Proceeding with upload...
+ /index.html
Uploaded 1 of 1 assets
✨ Success! Uploaded 1 file (2.37 sec)


Total Upload: 25.73 KiB / gzip: 6.17 KiB
Worker Startup Time: 15 ms
Uploaded e-commerce-pipelines-client-side (11.79 sec)
Deployed e-commerce-pipelines-client-side triggers (7.60 sec)
  https://<URL>.workers.dev
Current Version ID: <VERSION_ID>
```

We now need to update the pipeline's CORS settings again. This time, we'll include the URL of our newly deployed application. Run the command below, and replace `<URL>` with the URL of the application.

```sh
npx wrangler pipelines update clickstream-pipeline-client --cors-origins http://localhost:8787 https://<URL>.workers.dev
```

Now, you can access the application at the deployed URL. When you click on the `View Details` or `Add to Cart` button, the clickstream data will be sent to your pipeline.

## 9. View the data in R2

You can view the data in the R2 bucket. If you are not signed in to the Cloudflare dashboard, sign in and navigate to the [R2 overview](https://dash.cloudflare.com/?to=/:account/r2/overview) page.

Open the bucket you configured for your pipeline in Step 3. You can see files, representing the clickstream data. These files are newline delimited JSON files. Each row in a file represents one click event. Download one of the files, and open it in your preferred text editor to see the output:

```json
{"timestamp":"2025-04-06T16:24:29.213Z","session_id":"1234567890abcdef","user_id":"user965","event_data":{"event_id":673,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:29.213Z","product_id":2},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""}
{"timestamp":"2025-04-06T16:24:30.436Z","session_id":"1234567890abcdef","user_id":"user998","event_data":{"event_id":787,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:30.436Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""}
{"timestamp":"2025-04-06T16:24:31.330Z","session_id":"1234567890abcdef","user_id":"user22","event_data":{"event_id":529,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:31.330Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""}
{"timestamp":"2025-04-06T16:24:31.879Z","session_id":"1234567890abcdef","user_id":"user750","event_data":{"event_id":756,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:31.879Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""}
{"timestamp":"2025-04-06T16:24:33.978Z","session_id":"1234567890abcdef","user_id":"user333","event_data":{"event_id":467,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:33.978Z","product_id":6},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""}
```

## 10. Optional: Connect a query engine to your R2 bucket and query the data

Once you have collected the raw events in R2, you might want to query the events, to answer questions such as "how many events occurred?". You can connect a query engine, such as MotherDuck, to your R2 bucket.

You can connect the bucket to MotherDuck in several ways, which you can learn about from the [MotherDuck documentation](https://motherduck.com/docs/integrations/cloud-storage/cloudflare-r2/). In this tutorial, you will connect the bucket to MotherDuck using the MotherDuck dashboard.

### Connect your bucket to MotherDuck

Before connecting the bucket to MotherDuck, you need to obtain the Access Key ID and Secret Access Key for the R2 bucket. You can find the instructions to obtain the keys in the [R2 API documentation](https://developers.cloudflare.com/r2/api/tokens/).

1. Log in to the MotherDuck dashboard and select your profile.

2. Navigate to the **Secrets** page.

3. Select the **Add Secret** button and enter the following information:

   * **Secret Name**: `Clickstream pipeline`
   * **Secret Type**: `Cloudflare R2`
   * **Access Key ID**: `ACCESS_KEY_ID` (replace with the Access Key ID)
   * **Secret Access Key**: `SECRET_ACCESS_KEY` (replace with the Secret Access Key)

4. Select the **Add Secret** button to save the secret.

### Query the data

In this step, you will query the data stored in the R2 bucket using MotherDuck.

1. Navigate back to the MotherDuck dashboard and select the **+** icon to add a new Notebook.

2. Select the **Add Cell** button to add a new cell to the notebook.

3. In the cell, enter the following query and select the **Run** button to execute the query:

```sql
SELECT count(*) FROM read_json_auto('r2://clickstream-bucket/**/*');
```

The query will return a count of all the events received.

## Conclusion

You have successfully created a Pipeline and used it to send clickstream data from the client. Through this tutorial, you've gained hands-on experience in:

1. Creating a Workers project, using static assets
2. Generating and capturing clickstream data
3. Setting up a pipeline to ingest data into R2
4. Deploying the application to Workers
5. Using MotherDuck to query the data

You can find the source code of the application in the [GitHub repository](https://github.com/harshil1712/e-commerce-pipelines-client-side).
