Fastly VCL: Streaming, PDF Caching, and Timeout Tuning (Without Breaking Everything)

John Sourgiadakis

5 min read

·

Mar 13, 2026

--

Listen

Share

Three small configuration tweaks that can dramatically improve reliability when serving large files through Fastly.

If you run a platform that serves both fast web pages and large downloadable files, you’ve probably run into this situation: everything looks perfectly fine in configuration, but production tells a different story.

Recently I was working on a Fastly setup for a ticketing platform — the kind of system that serves lightweight HTML pages alongside heavier PDF ticket downloads. On paper, the configuration looked reasonable. In practice, it kept causing subtle but painful problems:

• PDF downloads occasionally timing out
• streamed responses that wouldn’t cache
• a between_bytes_timeout so strict that larger files could fail mid-transfer

None of these issues were dramatic on their own, but together they created a poor experience — especially when someone is trying to download their ticket minutes before an event.

The good news: fixing it required only three small VCL adjustments.

Here’s what changed, why it matters, and what to watch out for when tuning Fastly for large downloads.

Fastly vs. Varnish: a quick note before we start

If you’ve worked with Varnish Cache, Fastly’s VCL will feel very familiar. Under the hood, Fastly is built on Varnish concepts.

However, there’s one detail that often confuses people when reading documentation or copying examples.

Fastly still uses older Varnish 2/3 style subroutine names. That means:

• Fastly uses vcl_fetch
• Modern Varnish (4+) uses vcl_backend_response

The response object is still called beresp, so most examples still translate easily.

It’s a small difference, but worth remembering when reading examples online.

Understanding the backend timeout configuration

A typical Fastly backend configuration might look like this:

{
  name = "example"
  host = "example.com"
  port = 443
  connect_timeout = 10000
  first_byte_timeout = 200000
  between_bytes_timeout = 10000
  use_ssl = true
  ssl_cert_hostname = "example.com"
  ssl_sni_hostname = "example.com"
}

All timeout values are measured in milliseconds.

Here’s what each one controls:

SettingMeaningconnect_timeoutMaximum time Fastly waits to establish a TCP connectionfirst_byte_timeoutMaximum time Fastly waits for the first byte of the responsebetween_bytes_timeoutMaximum delay allowed between any two bytes of a response

The first two are usually fine.

The third one is where problems often appear.

Problem: between_bytes_timeout is often too aggressive

In the example above:

between_bytes_timeout = 10000

This means Fastly allows only 10 seconds between response bytes.

That might sound reasonable, but in real production environments it can cause issues when:

• PDFs are generated dynamically
• large files are served under load
• the backend temporarily slows down

If more than 10 seconds pass between chunks of the response, Fastly assumes the backend has stalled and returns a 503 error.

For someone downloading a ticket or document, that’s a frustrating failure.

Fix #1: Increase between_bytes_timeout

The easiest improvement is simply making the timeout more forgiving.

For most mixed workloads, 60 seconds is a practical default.

between_bytes_timeout = 60000 # 60 seconds

This gives large downloads enough breathing room without allowing connections to hang forever.

You can override this behavior only for certain content types in VCL, but in most real-world setups raising the global value is the simpler and safer solution.

Fix #2: Explicitly cache PDFs

Fastly — like Varnish — is intentionally cautious about caching.

If the backend sends unclear or restrictive cache headers, Fastly may decide not to cache the response at all and simply forward every request to the backend.

For static PDFs (like ticket files), that’s inefficient.

You can explicitly tell Fastly to cache them using VCL:

sub vcl_fetch {
  if (beresp.http.Content-Type ~ "application/pdf") {
    set beresp.ttl = 1h;
    set beresp.cacheable = true;
  }
  return(deliver);
}

A small but important detail:

Fastly uses beresp.cacheable (a boolean) to control caching.

Vanilla Varnish 4 often uses beresp.uncacheable, which can lead to confusion when copying examples.

Important: backend headers still matter

If your backend sends something like:

Cache-Control: no-store

Fastly will respect that directive and ignore your caching override.

If caching doesn’t behave the way you expect, always check the response headers first:

curl -I https://example.com/file.pdf

Often the problem is simply that the backend is instructing caches not to store the file.

Fix #3: Use streaming carefully

Another Fastly feature worth understanding is response streaming.

set beresp.do_stream = true;

Streaming allows Fastly to start sending the response to the client before the entire response has been downloaded from the backend.

For large files, this improves perceived performance because the download begins immediately.

However, there’s an important tradeoff.

Streamed responses generally cannot be cached.

Fastly needs the complete response before it can store it.

So if you enable streaming for PDFs and also want them cached, those goals conflict.

A practical compromise

In many setups, the best approach is:

• Cache PDFs
• Stream everything else

That configuration looks like this:

sub vcl_fetch {
  if (beresp.http.Content-Type ~ "application/pdf") {
    set beresp.ttl = 1h;
    set beresp.cacheable = true;
  } else {
    set beresp.do_stream = true;
  }
  return(deliver);
}

This allows large reusable files to be cached while still improving performance for other responses.

Final configuration example

Putting everything together:

{
  name = "example"
  host = "example.com"
  port = 443
  connect_timeout = 10000
  first_byte_timeout = 200000
  between_bytes_timeout = 60000
  use_ssl = true
  ssl_cert_hostname = "example.com"
  ssl_sni_hostname = "example.com"
}

sub vcl_fetch {
  if (beresp.http.Content-Type ~ "application/pdf") {
    set beresp.ttl = 1h;
    set beresp.cacheable = true;
  } else {
    set beresp.do_stream = true;
  }
  return(deliver);
}

These are small changes, but they can significantly reduce failed downloads and unnecessary backend load.

A few things still worth monitoring

Even with the configuration above, there are a few edge cases to keep in mind.

Cache key collisions

If PDFs contain personalized content, make sure the URL or cache key is unique per user. Otherwise one user might receive another user’s file.

Fastly’s default cache key is the URL, which usually solves this automatically.

Backend headers overriding your VCL

Always check what headers your backend sends.
Cache-Control directives can override caching behavior even if your VCL says otherwise.

TTL vs. content freshness

A one-hour TTL is a reasonable starting point for static files, but if files can change you may want to implement purging or surrogate keys so specific objects can be invalidated when needed.

Final thoughts

None of these changes are particularly complex.

They’re just a few small adjustments to timeouts, caching behavior, and streaming.

But when your system is delivering files that people actually depend on — tickets, documents, downloads — those small adjustments can make the difference between a smooth experience and a frustrating failure.

Sometimes the biggest improvements come from the simplest configuration tweaks.