AWS Textract: Building a Document API Platform

Krishnan Sriram

16 min read

·

Oct 12, 2026

--

1

Listen

Share

Press enter or click to view image in full size

This is a follow up from our previous implementation. Previously we did all of document data extraction directly and that sort of makes it difficult for applications to use it. What we want is to build an abstraction layer to help us build more of a document processing platform.

What we are building

Part 1 ended with three lessons from calling Textract by hand:

• The synchronous call rejects multi-page PDFs. Those need the asynchronous calls.
• Textract keeps no copy of a document. It reads the file from S3 on every call.
• The caller needs both Textract permissions and s3:GetObject on the file.

That is fine for one person at a terminal. It is a poor fit for applications, which would each need AWS credentials, Textract permissions and their own polling logic.

Part 2 wraps all of that in a small service with two operations:

• Submit a document. Either POST it to an API, or copy it into an S3 bucket.
• Retrieve the result. GET the document by its id and receive the extracted text as JSON.

The requirements are short:

• One Python Lambda function makes every Textract call. No application talks to Textract directly.
• Both submit paths behave the same, so a file dropped into S3 is processed exactly like one sent through the API.
• Single-page and multi-page documents both work.

High-level architecture

Four AWS services do the work: API Gateway is the front door, Lambda holds the logic, S3 holds documents and state, and Textract does the reading.

Press enter or click to view image in full size

The numbers on the diagram mark the five connections:

1. Submit and retrieve through API Gateway. Applications call two routes over HTTPS: POST /documents and GET /documents/{id}. API Gateway checks the request signature, then invokes the Lambda.
2. Direct deposit. An application that already has AWS access can skip the API and copy a file into the bucket’s incoming/ folder.
3. S3 event. Any object created under incoming/ triggers the Lambda. This happens for both submit paths.
4. Lambda calls Textract. It starts an asynchronous job for each new document and later asks for the results.
5. Textract reads the document. Textract fetches the file from incoming/ using the Lambda role's permissions.

The components

• API Gateway (HTTP API). Two routes, both pointing at the same Lambda. IAM authorization means callers sign requests with AWS credentials.
• Lambda (Python 3.12). One function with three jobs: store submitted files, start Textract jobs, and return results.
• S3 bucket. Three folders. incoming/ holds documents, jobs/ holds one small JSON record per Textract job, and results/ holds extracted content.
• Amazon Textract. The asynchronous API, which handles images, single-page PDFs and multi-page PDFs alike.
• IAM role. The Lambda can call four Textract actions and read and write this one bucket, nothing more.
• CloudWatch Logs. Lambda output, kept for 14 days.

High-level design

A document goes through three stages: submit, process, retrieve. Only the first stage differs between the two submit paths.

Press enter or click to view image in full size

A document goes through three stages: submit, process, retrieve.

Stage 1: submit

The application calls POST /documents in one of two ways.

With the file. The application sends the file as base64 inside a JSON body. The Lambda saves it to the bucket under uploads/, starts a Textract job on it, and returns the job id.

With a key. If the file is already in the bucket, the application sends only its key. The Lambda skips the upload and starts the job on that file.

Either way the response arrives straight away with status IN_PROGRESS. Starting a job takes a moment; the reading itself happens afterwards.

Stage 2: process

Textract reads the document from S3 and works in the background. Nothing in the service runs during this stage, and there is nothing for the application to do except wait.

Stage 3: retrieve

The application calls GET /documents/<job_id>. The Lambda asks Textract for that job’s status and output.

• Still running: the Lambda returns 202, and the application asks again a few seconds later.
• Finished: the Lambda collects every page of output, groups the lines by page, and returns 200 with the content.

Each request goes to Textract afresh. Nothing is saved by the service, so a result can be fetched for as long as Textract keeps it, which is 7 days.

Document ids

The Textract job id is the document id. It is a 64-character string returned by POST, and it is the only thing the application needs to keep.

The response also includes the file’s key in the bucket. For an uploaded file, the Lambda builds it from a random prefix plus the cleaned file name, such as uploads/3114a4f1a88a4e91a132c3fc8ac7a68d-shipping_order_2page.pdf.

Status values

• IN_PROGRESS: Textract is working on the document.
• SUCCEEDED: the content is ready and included in the response.
• FAILED: Textract could not process the file. The response includes the reason.
• PARTIAL_SUCCESS: Textract read some pages but not all. This version reports the status without returning the text.

A file that Textract refuses at the start, such as an unsupported format, never gets a job id. The POST returns 400 with the error message instead.

Design decisions

Asynchronous Textract calls for every document. Part 1 showed the synchronous call rejecting a multi-page PDF. The asynchronous calls accept everything the synchronous ones do, so using them throughout means one code path and no need to count pages first. The cost is that even a one-page image takes a few seconds and a second request.

The API starts the job directly. POST /documents saves the file and calls Textract in the same request, then returns the job id. There is no queue or event in between, so there are fewer moving parts and a failure to start is reported to the caller immediately.

Two ways to submit. A caller can send the file itself as base64, or name a file that is already in the bucket with {"key": ...}. The second form covers files too large to send through the API and files placed in S3 by another process.

The Textract job id is the document id. The service stores no records of its own. The caller keeps the job id returned by POST and presents it to GET, and Textract holds the job’s status and output. This keeps the stack to four services with no database. A table such as DynamoDB becomes worthwhile once you need to list documents, track who submitted them, or keep results longer.

Results are read from Textract on every request. GET asks Textract for the job’s output each time and returns the lines grouped by page. Textract keeps job output for 7 days, so a result must be fetched within that window. Saving a copy on first retrieval would remove that limit and is a natural next step.

Uploaded files get a unique name. Files sent through the API are stored under uploads/ with a random prefix, so two uploads with the same file name never overwrite each other.

IAM authorization on the API. An open endpoint would let anyone with the URL run Textract at the account’s expense and read results. With IAM authorization, callers must hold AWS credentials allowed to invoke the API.

A 4 MB limit on uploads through the API. Lambda accepts request bodies up to 6 MB, and base64 adds about a third. Larger files go into the bucket directly and are submitted by key, where the limits are Textract’s own.

No automatic clean-up yet. Uploaded files stay in the bucket until they are deleted. A lifecycle rule that expires objects after one day is a simple addition; one day is the shortest period S3 allows, and deletion happens after the following midnight UTC, so files would live between 24 and 48 hours.

The Lambda function

The function lives in one file, lambda/handler.py, and uses only boto3, which the Lambda runtime already includes. The excerpts below are shortened; the full file adds input validation and error handling.

One handler, three entry points

An S3 event carries a Records list. An API Gateway request carries a routeKey. The handler uses that difference to decide what to do.

Let’s start by creating a simple Lambda function

Press enter or click to view image in full size

Create the function

1. In the console (region us-east-2), open Lambda and click Create function.
2. Choose Author from scratch, name it textract-demo, and pick the runtime Python 3.13.
3. Under permissions, leave Create a new role with basic Lambda permissions and click Create function.

Give it access to Textract and the bucket

1. Open Configuration > Permissions and click the role name; it opens in IAM.
2. Click Add permissions > Create inline policy, switch to JSON, and paste:

json

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Effect": "Allow",
   "Action": [
    "textract:StartDocumentTextDetection",
    "textract:GetDocumentTextDetection"
   ],
   "Resource": "*"
  },
  {
   "Effect": "Allow",
   "Resource": "arn:aws:s3:::textract-test-oh-dev-01/*",
   "Action": ["s3:GetObject", "s3:PutObject"]
  }
 ]
}

1. Name it textract-demo-access and create it.

Configure it

1. In Configuration > General configuration, click Edit and set Timeout to 30 seconds (the default 3 is too short).
2. In Configuration > Environment variables, add BUCKET = textract-test-oh-dev-01.

Add the code

In the Code tab, replace the contents of lambda_function.py with this and click Deploy:

import base64
import json
import os
import re
import uuid

import boto3
from botocore.exceptions import ClientError

BUCKET = os.environ["BUCKET"]
s3 = boto3.client("s3")
textract = boto3.client("textract")


def upload(filename, content_base64):
    data = base64.b64decode(content_base64, validate=True)
    name = re.sub(r"[^A-Za-z0-9._-]", "_", os.path.basename(filename or "document"))
    key = f"uploads/{uuid.uuid4().hex}-{name}"
    s3.put_object(Bucket=BUCKET, Key=key, Body=data)
    return key


def start(key):
    out = textract.start_document_text_detection(
        DocumentLocation={"S3Object": {"Bucket": BUCKET, "Name": key}}
    )
    return {"job_id": out["JobId"], "status": "IN_PROGRESS", "key": key}


def get(job_id):
    pages, token = {}, None
    while True:
        args = {"JobId": job_id}
        if token:
            args["NextToken"] = token
        out = textract.get_document_text_detection(**args)
        if out["JobStatus"] != "SUCCEEDED":
            return {"job_id": job_id, "status": out["JobStatus"],
                    "message": out.get("StatusMessage", "")}
        for block in out["Blocks"]:
            if block["BlockType"] == "LINE":
                pages.setdefault(str(block.get("Page", 1)), []).append(block["Text"])
        token = out.get("NextToken")
        if not token:
            return {"job_id": job_id, "status": "SUCCEEDED", "pages": pages}


def reply(status_code, body):
    return {"statusCode": status_code,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps(body)}


def lambda_handler(event, context):
    route = event.get("routeKey")

    if route is None:  # console Test events
        if "key" in event:
            return start(event["key"])
        if "job_id" in event:
            return get(event["job_id"])
        return {"error": 'Send {"key": "..."} or {"job_id": "..."}'}

    try:
        if route == "POST /documents":
            raw = event.get("body") or "{}"
            if event.get("isBase64Encoded"):
                raw = base64.b64decode(raw)
            body = json.loads(raw)
            if body.get("content_base64"):
                key = upload(body.get("filename"), body["content_base64"])
            else:
                key = body.get("key")
            if not key:
                return reply(400, {"error": 'Send {"key": ...} or {"filename": ..., "content_base64": ...}'})
            return reply(202, start(key))

        if route == "GET /documents/{job_id}":
            result = get(event["pathParameters"]["job_id"])
            return reply(202 if result["status"] == "IN_PROGRESS" else 200, result)
    except (ClientError, ValueError) as exc:
        return reply(400, {"error": str(exc)})

    return reply(404, {"error": "Not found"})

Test it

In the Test tab, create an event named start with this body and click Test:

{ "key": "textract_sample.pdf" }

This is the results I saw

Status: Succeeded
Test Event Name: start

Response:
{
  "job_id": "639acaa5b75a1ed161cff327d1d848ef51a49b74951e25fa609deb4bd938cf18"
}

The area below shows the last 4 KB of the execution log.

Function Logs:
START RequestId: 3d26ebaa-d753-49ed-a7cd-cbf0ee7f7b9f Version: $LATEST
END RequestId: 3d26ebaa-d753-49ed-a7cd-cbf0ee7f7b9f
REPORT RequestId: 3d26ebaa-d753-49ed-a7cd-cbf0ee7f7b9f  Duration: 404.71 ms Billed Duration: 827 ms Memory Size: 128 MB Max Memory Used: 93 MB  Init Duration: 421.72 ms

Request ID: 3d26ebaa-d753-49ed-a7cd-cbf0ee7f7b9f

1. Create a second event named get, paste that job id, and click Test:

{ "job_id": "639acaa5b75a1ed161cff327d1d848ef51a49b74951e25fa609deb4bd938cf18" }

Result of execution

Status: Succeeded
Test Event Name: get

Response:
{
  "status": "SUCCEEDED",
  "pages": {
    "1": [
      "Northwind Office Supply",
      "Sample Invoice (Textract test document)",
      "This is a single-page test document for Amazon Textract. It contains a paragraph of plain text,",
      "a set of key-value pairs, and a table, so it can be used with both DetectDocumentTe and",
      "AnalyzeDocument.",
      "Invoice Number:",
      "INV-2026-00417",
      "Invoice Date:",
      "October 10, 2026",
      "Due Date:",
      "November 9, 2026",
      "Customer Name:",
      "Jordan Example",
      "Customer ID:",
      "CUST-88213",
      "Payment Terms:",
      "Net 30",
      "Line Items",
      "Item",
      "Description",
      "Qty",
      "Unit Price",
      "Amount",
      "1",
      "Copy paper, letter, case",
      "4",
      "$42.50",
      "$170.00",
      "2",
      "Ballpoint pens, box of 12",
      "10",
      "$6.25",
      "$62.50",
      "3",
      "Desk organizer, black",
      "3",
      "$18.00",
      "$54.00",
      "4",
      "Toner cartridge, HY",
      "2",
      "$89.99",
      "$179.98",
      "Subtotal",
      "$466.48",
      "Tax (7.5%)",
      "$34.99",
      "Total",
      "$501.47",
      "Notes: Please reference the invoice number with your payment. Questions can be sent to",
      "billing@northwind.example."
    ]
  }
}

The area below shows the last 4 KB of the execution log.

Function Logs:
START RequestId: 9e2c178c-cfbe-4f28-a590-269853363dd8 Version: $LATEST
END RequestId: 9e2c178c-cfbe-4f28-a590-269853363dd8
REPORT RequestId: 9e2c178c-cfbe-4f28-a590-269853363dd8  Duration: 408.93 ms Billed Duration: 409 ms Memory Size: 128 MB Max Memory Used: 95 MB

Request ID: 9e2c178c-cfbe-4f28-a590-269853363dd8

Now that we have function working, our next step is to build API Gateway and connect it over to Azure function this way we no longer need to come to console for execution

Step 2: API Gateway in the console

1. Open API Gateway (region us-east-2), click Create API, and under HTTP API click Build.
2. Click Add integration, choose Lambda, and select textract-demo. Name the API textract-demo-api and click Next.
3. On the routes page, replace the pre-filled route so you have these two, both pointing at textract-demo: Method POST, path /documents, Method GET, path /documents/{job_id}
4. Click Next, keep the $default stage with auto-deploy on, then Next and Create.
5. Copy the Invoke URL from the API’s page.

The wizard also grants API Gateway permission to invoke the function, so there’s nothing to add in IAM.

Time to take it for a run

curl -s -X POST $API/documents \
  -H 'Content-Type: application/json' \
  -d '{"key":"textract_sample.pdf"}'
# Execution Results
{"job_id": "6657945fc91ac2d988758286a30c6ba2aade0ecbf93e1583d1600ebdafe1677d", "status": "IN_PROGRESS"}%                                            

# Now lets extract data from processed files
curl -s $API/documents/6657945fc91ac2d988758286a30c6ba2aade0ecbf93e1583d1600ebdafe1677d
# Execution Results
{"job_id": "6657945fc91ac2d988758286a30c6ba2aade0ecbf93e1583d1600ebdafe1677d", "status": "SUCCEEDED", "pages": {"1": ["Northwind Office Supply", "Sample Invoice (Textract test document)", "This is a single-page test document for Amazon Textract. It contains a paragraph of plain text,", "a set of key-value pairs, and a table, so it can be used with both DetectDocumentTe and", "AnalyzeDocument.", "Invoice Number:", "INV-2026-00417", "Invoice Date:", "October 10, 2026", "Due Date:", "November 9, 2026", "Customer Name:", "Jordan Example", "Customer ID:", "CUST-88213", "Payment Terms:", "Net 30", "Line Items", "Item", "Description", "Qty", "Unit Price", "Amount", "1", "Copy paper, letter, case", "4", "$42.50", "$170.00", "2", "Ballpoint pens, box of 12", "10", "$6.25", "$62.50", "3", "Desk organizer, black", "3", "$18.00", "$54.00", "4", "Toner cartridge, HY", "2", "$89.99", "$179.98", "Subtotal", "$466.48", "Tax (7.5%)", "$34.99", "Total", "$501.47", "Notes: Please reference the invoice number with your payment. Questions can be sent to", "billing@northwind.example."]}}% 

We still have a caveat in security. These API’s is open and can be used by anyone. Let’s fix that first

Do this next: lock the API

1. In API Gateway, open Routes, select POST /documents, click Attach authorization, choose IAM, and attach it.
2. Repeat for GET /documents/{job_id}

Let’s test to see if this is really secure

curl -s $API/documents/6657945fc91ac2d988758286a30c6ba2aade0ecbf93e1583d1600ebdafe1677d
# Execution Results
{"message":"Forbidden"}%                         

# Get the credentials
eval "$(aws configure export-credentials --profile aiservices-dev --format env)"
# Use those credentials to read
curl -s $API/documents/6657945fc91ac2d988758286a30c6ba2aade0ecbf93e1583d1600ebdafe1677d \
  --aws-sigv4 "aws:amz:us-east-2:execute-api" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
  -H "x-amz-security-token: $AWS_SESSION_TOKEN"
# Execution results
{"job_id": "6657945fc91ac2d988758286a30c6ba2aade0ecbf93e1583d1600ebdafe1677d", "status": "SUCCEEDED", "pages": {"1": ["Northwind Office Supply", "Sample Invoice (Textract test document)", "This is a single-page test document for Amazon Textract. It contains a paragraph of plain text,", "a set of key-value pairs, and a table, so it can be used with both DetectDocumentTe and", "AnalyzeDocument.", "Invoice Number:", "INV-2026-00417", "Invoice Date:", "October 10, 2026", "Due Date:", "November 9, 2026", "Customer Name:", "Jordan Example", "Customer ID:", "CUST-88213", "Payment Terms:", "Net 30", "Line Items", "Item", "Description", "Qty", "Unit Price", "Amount", "1", "Copy paper, letter, case", "4", "$42.50", "$170.00", "2", "Ballpoint pens, box of 12", "10", "$6.25", "$62.50", "3", "Desk organizer, black", "3", "$18.00", "$54.00", "4", "Toner cartridge, HY", "2", "$89.99", "$179.98", "Subtotal", "$466.48", "Tax (7.5%)", "$34.99", "Total", "$501.47", "Notes: Please reference the invoice number with your payment. Questions can be sent to", "billing@northwind.example."]}}

For upload to work, make sure you see this in your permissions for the service. Without PutObject permission, Lambda will fail to upload files in.

Test Upload file feature

printf '{"filename":"shipping_order_2page.pdf","content_base64":"%s"}' \
  "$(base64 -i shipping_order_2page.pdf)" > shipping_order_2page.json

curl -s -X POST $API/documents "${SIGN[@]}" \
  -H 'Content-Type: application/json' -d @shipping_order_2page.json
# Execution results
{"job_id": "21a930a8680e81edde661a7976647524ed4f2960df340aacac2423e5766681aa", "status": "IN_PROGRESS", "key": "uploads/3114a4f1a88a4e91a132c3fc8ac7a68d-shipping_order_2page.pdf"}%                                                                                                                    

curl -s $API/documents/21a930a8680e81edde661a7976647524ed4f2960df340aacac2423e5766681aa "${SIGN[@]}"
# Execution Results
{"job_id": "21a930a8680e81edde661a7976647524ed4f2960df340aacac2423e5766681aa", "status": "SUCCEEDED", "pages": {"1": ["Harborline Freight Co.", "Shipping Order (Textract test document, page 1 of 2)", "This two-page test document checks that multi-page PDFs are processed. Page one holds", "the shipment details and a packing list. Page two holds delivery instructions and a sign-off", "block.", "Order Number:", "SO-58213", "Order Date:", "October 11, 2026", "Ship By:", "October 15, 2026", "Carrier:", "Bluewater Logistics", "Tracking ID:", "BWL-7740-2291-08", "Ship To:", "Riverside Books, 18 Example Lane, Springfield", "Packing List", "Box", "Contents", "Units", "Weight (lb)", "1", "Hardcover books, assorted", "24", "38.5", "2", "Paperback books, assorted", "60", "41.0", "3", "Bookmarks and gift cards", "500", "6.2", "4", "Display stands, folded", "8", "22.7", "Total", "592", "108.4"], "2": ["Shipping Order SO-58213 (page 2 of 2)", "Delivery Instructions", "1. Deliver to the rear loading dock between 8:00 AM and 2:00 PM on weekdays.", "2. Call the receiving desk at 555-0142 thirty minutes before arrival.", "3. Box 4 is fragile and must be stacked on top.", "4. Obtain a signature from the receiving manager before leaving.", "Charges", "Charge", "Amount", "Freight", "$184.00", "Fuel surcharge", "$21.60", "Insurance", "$12.50", "Total due", "$218.10", "Sign-off", "Prepared By:", "Morgan Sample", "Approved By:", "Casey Placeholder", "Approval Date:", "October 11, 2026", "Status:", "Ready to ship"]}}

• First command converts shipping_order_2page.pdf to base64 text and wraps it in a small JSON document along with the file name.
• Second command posts that JSON to the API with a signed request. API Gateway verifies the signature and passes the request to the Lambda, which decodes the file, saves it to the bucket under uploads/ with a random prefix, and starts an asynchronous Textract job on it. The response returns immediately with the job id, a status of IN_PROGRESS, and the S3 key where the file was stored.
• Third command asks the API for that job’s result.

Limits and what to harden next

This is a working starting point, not a production service. Its known limits:

• Polling. The application has to ask repeatedly for the result. Textract can publish a completion message to SNS, which would let the service notify callers instead.
• File size through the API. 4 MB. Larger files must already be in the bucket and be submitted by key, which means the caller needs S3 access. A common fix is a route that returns a presigned S3 upload URL, so large files go straight to S3 without AWS credentials.
• Large results. A Lambda response is capped at 6 MB and API Gateway waits at most 30 seconds. A document with hundreds of pages could exceed either. Saving the result to S3 and returning a link would avoid both.
• Results last 7 days. The service keeps no copy of the output, so a result can only be fetched while Textract still holds it.
• No record of documents. The job id returned by POST is the only handle on a document. If the caller loses it, the result can’t be found, and there is no way to list or search what has been submitted. Add a table if applications need that.
• No clean-up. Uploaded files stay in the bucket until someone deletes them. A lifecycle rule that expires objects after a day or more would handle it.
• No separation between callers. Any caller allowed to use the API can fetch any result if it has the job id, and can submit any file in the bucket by key. That is acceptable for one team and needs tightening before several teams share the service.
• Repeated submissions. A POST that is retried, for example after a network timeout, starts a second job for the same document and is charged twice. Passing a ClientRequestToken to Textract makes the start call safe to repeat.
• Text only. This version returns lines of text. Tables and form fields need Textract’s analysis calls, which the Lambda doesn’t use yet.
• Partially read documents. If Textract reads some pages but not others, the service reports the status without returning the pages that were read.
• No screening for sensitive data. Whatever is in the document comes back in the response, including personal and card data. The next article adds PII and card-data scanning.
• Encryption and access logs. The bucket uses S3’s default encryption. Regulated documents usually call for a customer-managed KMS key and API access logging.
• Cost. Textract charges per page. This version uses text detection, the lowest rate; tables and forms cost more per page if you add them. Check current pricing before sending volume.

What’s Next

In the next step we’ll try and see how we can handle PII and PCI check for the documents and what are the ways to get there. Until then keep reading and stay safe.