Internally Caching Longer Than Externally Caching

We use varnish for a lot of our file caching needs, and recently we figured out how to do something rather important through a combination of technologies. Imagine you have backend servers generating dynamic content based on user input. So your users do something that fits the following categories:

  • is expensive to generate dynamically, and should be served from cache
  • many requests come in for the same objects, bandwidth should be conserved
  • doesnt change very often
  • once changed needs to take effect quickly

Now wish varnish we’ve been using the Expires header for a long time with great success, but for this we were having no luck. If we set the expires header to 3 weeks, then clients also cache the content for 3 weeks (violating requirement #3.) We can kill the Expires header in varnish at vcl_deliver, but then clients don’t cache at all (#2.) We can add Content-Control, overwrite the Age (otherwise reported Age: will be greater than max-age), and kill the Expires headers in the same place, but this isn’t pretty, and seems like a cheap hack. Ideally we could rewrite the Expires header in varnish, but that doesn’t seem doable.

So what we ended up doing, was header rewriting at the load balancer (nginx.) inside our location tag we added the following:

proxy_hide_header Age;
proxy_hide_header Expires;
proxy_hide_header Cache-Control;
add_header Source-Age $upstream_http_Age;
expires  300s;

Now nginx setsa proper Cache-Control: and Expires: headers for us, disregarding what varnish serves out. Web clients dont check back for 5 minutes (reusing the old object) and varnish can cache until judgment dat because we get wild card invalidation

Isn’t technology fun?!

6 thoughts on “Internally Caching Longer Than Externally Caching

  1. It would be great if you could post relevant portions of your nginx and varnish config as examples as there is a dearth of those right now.

  2. Good post. I was doing some test using nginx to proxy another 2 upstream nginx servers and was having problem with the expires header.

  3. Whats "add_header Source-Age $upstream_http_Age;" this line for? It wasn't clear for me. Tried searching info about thin on internet but couldn'd find anything/

    • apokalyptik says:

      This may have simply been an advisory header. It\’s been a while since 2008, I don\’t remember exactly đŸ™‚

Leave a Reply