Version 2.1 — implementing the search feature

David Ramovich Mandal

7 min read

·

Dec 28, 2025

--

Listen

Share

So in the previous version , we have build a website where the people can navigate and download files . In this development , we shall be implementing the search feature in the website . This shall allow the users to search from a long list of all the files.

First of all , let’s look at the constraints that we have :

• Static JS website
• API Gateway → Lambda → S3
• Lambda returns file metadata
• CloudFront handles downloads
• Want real-time search while typing
• Want to keep it simple + serverless

Which means that :

❌ We should NOT call Lambda on every keystroke
✅ We should NOT re-scan S3 on every search
✅ We SHOULD fetch data once and search in-memory (client-side)

Hence , the best approach is going to be :

• Fetch all file metadata ONCE
• Page load → API Gateway → Lambda → S3
• Store metadata in a JS array
• Implement client-side real-time filtering
• Use debounce to avoid unnecessary work
• Show results instantly as the user types

I.e. implement client-side , debounced search

Architecture (Before vs After)

Before :

User → Website → API Gateway → Lambda → S3 → Metadata → UI

After :

(Page Load)
User → Website → API Gateway → Lambda → S3 → ALL metadata → JS memory

(Search Typing)
User typing → JS filter() → Instant UI update

Since this is a static, serverless website, calling Lambda on every keystroke would increase latency and cost. Instead, we fetch file metadata once on page load and perform real-time search on the client side using JavaScript. This approach provides instant feedback to users, reduces AWS costs, and scales well for read-heavy workloads.

How far does client-side, debounced search actually scale?

When implementing real-time search using JavaScript and debouncing, the first question that usually comes up is: how long will this approach work?

Interestingly, the limiting factors here are not AWS services.

The real constraints are entirely on the client side.

What actually limits this approach?

Client-side search performance depends on a few key factors:

• Browser memory availability
• Size of the initial metadata payload
• Cost of filtering on the client (which is O(n))
• Overall user experience, especially page load time

Since the entire file list is loaded into the browser and searched locally, the browser becomes the bottleneck — not the backend.

Practical scalability limits (realistic numbers)

Works perfectly up to ~5,000 files

At this range, there are no noticeable performance issues. The initial payload is small, filtering is fast, and the UI remains responsive.

Still fine up to ~10,000 files

On modern desktops and laptops, even 10,000 filenames can be filtered smoothly. Filenames are small strings, and the search operation is a simple .includes() check. Debouncing ensures the filtering logic doesn’t run excessively.

Around 20,000–30,000 files, some limitations begin to appear:

• Initial page load becomes slower due to a larger JSON response
• Low-end devices may show slight lag during typing
• Mobile browsers tend to struggle earlier than desktops

The application still works, but the user experience starts to suffer.

Not recommended beyond ~50,000 files

At this point, client-side search is no longer a good idea:

• The metadata payload becomes very large (often several MBs)
• Browser memory usage increases significantly
• Filtering on every search becomes expensive
• Users on slow networks experience poor performance

Now , lets get to work :

Update the lambda function :

import json              # Used to convert Python objects to JSON strings
import boto3             # AWS SDK for Python to interact with AWS services

# Create an S3 client using Lambda's IAM role permissions
s3 = boto3.client('s3')

# Name of the S3 bucket to read files from
BUCKET_NAME = "bucket-name"

def lambda_handler(event, context):
    """
    AWS Lambda entry point.
    This function lists all objects in the given S3 bucket
    and returns their metadata as a JSON response.
    """
    try:
        # Fetch the list of objects present in the S3 bucket
        response = s3.list_objects_v2(Bucket=BUCKET_NAME)

        # This list will store file metadata
        files = []

        # 'Contents' key exists only if the bucket has objects
        if "Contents" in response:
            for obj in response["Contents"]:
                # Append relevant file details to the list
                files.append({
                    # Original file name (S3 object key)
                    "fileName": obj["Key"],

                    # Lowercase version for easier case-insensitive searching
                    "fileNameLower": obj["Key"].lower(),

                    # File size in bytes
                    "size": obj["Size"],

                    # Last modified timestamp in ISO 8601 format
                    "lastModified": obj["LastModified"].isoformat()
                })

        # Successful HTTP response
        return {
            "statusCode": 200,
            "headers": {
                "Content-Type": "application/json",

                # Allows frontend apps (CORS) to access this API
                "Access-Control-Allow-Origin": "*"
            },
            # Convert Python list to JSON string
            "body": json.dumps(files)
        }

    except Exception as e:
        # Error response in case something goes wrong
        return {
            "statusCode": 500,
            "headers": {
                "Content-Type": "application/json",
                "Access-Control-Allow-Origin": "*"
            },
            # Send error message to help debugging
            "body": json.dumps({"error": str(e)})
        }

Test the lambda function :

We shall send this into the test input :

{}

Press enter or click to view image in full size

Update the JS file :

<!DOCTYPE html>
<html lang="en">
<head>
  <!-- Character encoding -->
  <meta charset="UTF-8" />

  <!-- Page title -->
  <title>Public File Downloads</title>

  <style>
    /* Basic page styling */
    body {
      font-family: Arial, sans-serif;
      background: #f4f6f8;
      padding: 40px;
    }

    h1 {
      margin-bottom: 20px;
    }

    /* Search input styling */
    input {
      padding: 10px;
      width: 100%;
      margin-bottom: 20px;
      font-size: 16px;
    }

    /* Individual file row container */
    .file {
      background: white;
      padding: 15px;
      margin-bottom: 10px;
      border-radius: 6px;
      display: flex;
      justify-content: space-between;
      align-items: center;
    }

    /* Download button styling */
    .file button {
      padding: 8px 14px;
      background: #0073bb;
      color: white;
      border: none;
      border-radius: 4px;
      cursor: pointer;
    }

    .file button:hover {
      background: #005fa3;
    }
  </style>
</head>
<body>

  <!-- Page heading -->
  <h1>Available Downloads</h1>

  <!-- Search input for filtering file names -->
  <input
    type="text"
    id="search"
    placeholder="Search files..."
  />

  <!-- Container where file list will be dynamically rendered -->
  <div id="file-list">Loading files...</div>

  <script>
    // API Gateway URL that triggers the Lambda function
    const API_URL = "ur api url";

    // CloudFront distribution URL for downloading files
    const CLOUDFRONT_URL = "https://d2xxxxxxxxxxxx.net";

    // Stores all files fetched from the API
    let allFiles = [];

    /**
     * Debounce function
     * Prevents the search function from running on every keystroke
     * Improves performance for large file lists
     */
    function debounce(fn, delay = 200) {
      let timer;
      return (...args) => {
        clearTimeout(timer);
        timer = setTimeout(() => fn(...args), delay);
      };
    }

    /**
     * Fetch file metadata from the Lambda API
     */
    async function loadFiles() {
      try {
        const response = await fetch(API_URL);
        allFiles = await response.json();

        // Render all files initially
        renderFiles(allFiles);
      } catch (error) {
        // Show error message if API fails
        document.getElementById("file-list").innerHTML =
          "<p>Error loading files.</p>";
        console.error(error);
      }
    }

    /**
     * Render the list of files on the page
     */
    function renderFiles(files) {
      const container = document.getElementById("file-list");
      container.innerHTML = "";

      // If no files match the search query
      if (files.length === 0) {
        container.innerHTML = "<p>No matching files found.</p>";
        return;
      }

      // Create a row for each file
      files.forEach(file => {
        const div = document.createElement("div");
        div.className = "file";

        // Display file name
        const name = document.createElement("span");
        name.textContent = file.fileName;

        // Download button
        const button = document.createElement("button");
        button.textContent = "Download";

        // Redirect to CloudFront URL on click
        button.onclick = () => {
          window.location.href = `${CLOUDFRONT_URL}/${file.fileName}`;
        };

        div.appendChild(name);
        div.appendChild(button);
        container.appendChild(div);
      });
    }

    // Reference to the search input field
    const searchInput = document.getElementById("search");

    /**
     * Handle file search with debounce
     */
    const handleSearch = debounce((query) => {
      // If search box is empty, show all files
      if (!query) {
        renderFiles(allFiles);
        return;
      }

      // Filter files using lowercase comparison
      const filtered = allFiles.filter(file =>
        file.fileNameLower.includes(query)
      );

      renderFiles(filtered);
    });

    // Trigger search whenever user types
    searchInput.addEventListener("input", (e) => {
      handleSearch(e.target.value.toLowerCase());
    });

    // Initial API call on page load
    loadFiles();
  </script>

</body>
</html>

Now, let’s test the website :

Press enter or click to view image in full size

Press enter or click to view image in full size

How the architecture should evolve as files increase

This design works well because it’s intentionally simple — but it also leaves room for clean evolution.

Phase 1: Small to medium datasets (current approach)

For small to medium numbers of files, the architecture looks like this:

S3 → Lambda → Browser memory → JavaScript search

This approach offers:

• Zero backend load during search
• Minimal AWS cost
• Instant, Google-like search experience

It’s an excellent choice for an MVP or early-stage project.

Phase 2: Medium to large datasets (10k–100k files)

As the dataset grows, search should move to the backend.

A common upgrade path is DynamoDB-based search:

• Lambda indexes file metadata into DynamoDB
• The search API accepts a query
• DynamoDB uses indexed attributes (via GSIs) to filter results
• Only matching files are returned to the client

Architecture:

Browser → API Gateway → Lambda → DynamoDB

DynamoDB fits well here because it’s serverless, fast, and cost-effective at scale.

Phase 3: Very large or enterprise-scale datasets (100k+ files)

For large-scale systems that require advanced search features, the natural choice is Amazon OpenSearch.

This becomes necessary when you need:

• Full-text search
• Fuzzy matching
• Ranking and relevance scoring
• Autocomplete and suggestions
• Typo tolerance

Architecture:

Browser → API Gateway → Lambda → OpenSearch