in Tutorials, Linux, PHP, Tech

Configuring Varnish for Drupal best technique and practices

updated 1.3k Views

Configuring Varnish for Drupal best technique and practices

For best connection between Varnish the Reverse-Proxy and Drupal CMS, and to correctly Configuring Varnish for Drupal, We need to completely differentiate between anonymous and logged-in users traffic.
so We will need to do some extra configuration adjustment in both sides (the Drupal, and Varnish sides).

Before You Begin

  1. You can use this guide to make refining or tuning Varnish Reverse-Proxy VCL and Configuration with Drupal CMS, so you can get the most power of Varnish with Drupal CMS.
    to be familiar with Varnish Installation see Getting Started with Varnish Cache,
    and for Drupal Installation please see, Installing Drupal 7 guide.

  2. We will make a custom Drupal module, so you may want to know more information about creating custom Drupal modules, please see Creating Drupal 7.x modules guide.

note

The Environment used in this Tutorial is LAMP Stack, varnish-3.0.7, and Drupal 7. and we will need root privileges too.

How this Connection between Varnish and Drupal can be done?

When Drupal get a request from Varnish, Drupal will reply with a custom Cookie word send to Varnish, That word can help varnish to detect the current user type, and also can provide Varnish with extra information about current user ( ie: user id, user role, user name, and others ) using same technique.

so Varnish Can read this Cookie word that received from Drupal, and take an predefined VCL action regards to it.

In general Case we will use this technique to differentiate between anonymous and logged-in users,
by creating a custom Drupal module, this module will make Drupal send a Cookie named DRUPAL_UID to varnish and set it’s value to the User ID uid.
but this Cookie will created only for logged-in users.. not for anonymous.

note

reason that we ignored creating DRUPAL_UID cookie with ID “0” or in anonymous users case is that, Drupal creating this cookie using hook_init() method, and this hooked method will not fired when Drupal Cache pages for anonymous users caching performance feature are enabled.

so Varnish will read the Drupal Cookie message and.. if it found the word “DRUPAL_UID” is set, it will understand that its logged-in user.
else it will be anonymous user.

Another Connection adjustment will be needed here, it’s Purging Cache requests That can be fired automatically and regards to content change events from Drupal side to Varnish.
to accomplished this connection we will use the following 2 Contributed modules

  1. purge module

  2. expire module

now let’s go to preparing Drupal side first..

Drupal side

Creating Drupal varnish_helper custom module

we need to create the custom Drupal Modules varnish_helper, and make it responsible to sending Drupal Cookie word “DRUPAL_UID” to Varnish, and as usual we will creating this module 2 files as following

  1. varnish_helper.info file which will located in sites/all/modules/varnish_helper/varnish_helper.info, in Drupal installation directory, and will contain the following code.

file: sites/all/modules/varnish_helper/varnish_helper.info

  name = Varnish Helper
description = Varnish Cookie Based Helper Module
package = other
core = 7.x
version = "7.x-1"
project = "varnish_helper"

 

  1. varnish_helper.module file which will located in sites/all/modules/varnish_helper/varnish_helper.module,in Drupal installation directory, and will contain the following code.

file: sites/all/modules/varnish_helper/varnish_helper.module

<?php

/**
 * Default cookie name.
 */
define('VARNISH_COOKIE', 'DRUPAL_UID');



/**
 * Implements hook_init().
 */
 
function varnish_helper_init()
{
  global $user;
  
  if (strpos($_SERVER['SCRIPT_FILENAME'], 'index.php') === FALSE) {
    return;
  }
  
  $uid = isset($user->uid) ? $user->uid : 0;
  
  if (isset($_COOKIE[VARNISH_COOKIE]) && $uid == 0) {
    varnish_helper_set_cookie($uid, REQUEST_TIME - 86400);
  }
  
  elseif ((!isset($_COOKIE[VARNISH_COOKIE]) || $_COOKIE[VARNISH_COOKIE] == '-1') && $uid != 0) {
    varnish_helper_set_cookie($uid);
  }
  
  elseif (isset($_COOKIE[VARNISH_COOKIE]) && $_COOKIE[VARNISH_COOKIE] == '-1' && ($_SERVER['REQUEST_METHOD'] == 'GET' || $_SERVER['REQUEST_METHOD'] == 'HEAD')) {
    varnish_helper_set_cookie($uid, REQUEST_TIME - 86400);
  }
  
  if ($uid == 0 && $_SERVER['REQUEST_METHOD'] != 'GET' && $_SERVER['REQUEST_METHOD'] != 'HEAD') {
    varnish_helper_set_cookie(-1);
  }
    
}


function varnish_helper_set_cookie($uid, $expires = NULL) {
  if (!$expires) {
    $expires = ini_get('session.cookie_lifetime');
    $expires = (!empty($expires) && is_numeric($expires)) ? REQUEST_TIME + (int)$expires : 0;
    setcookie(VARNISH_COOKIE, strval($uid), $expires, ini_get('session.cookie_path'), ini_get('session.cookie_domain'), ini_get('session.cookie_secure') == '1');
  }
  else {
    setcookie(VARNISH_COOKIE, '0', $expires, ini_get('session.cookie_path'), ini_get('session.cookie_domain'), ini_get('session.cookie_secure') == '1');
  }
}
    ~~~

 

note:

This custom module code block modified and abstracted from the BOOST Drupal module source code.

Installing Purge Cache Modules

  1. Install and enable Purge Module

  2. Install and enable Expire Module

Varnish side

Fixing Connections timeout issues

to avoid connections timeout issues with Drupal POST/PURGE requests we will adjust the Varnish Configuration file varnish whitch located in /etc/sysconfig/varnish with the following timeouts parameters and values.

  • sess_timeout is the Idle timeout for persistent sessions, and If a HTTP request has not been received in this variable values of seconds, the session is closed, so for users who works behind slow internet connections may received connection timeout message while doing POST requests, so we need to increase the sess_timeout value to 90 sec. its default is 5 sec.

  • cli_timeout If varnish is heavily loaded, it might not answer the management thread in a timely fashion, which in turn will kill it off. To avoid that, increase the cli_timeout value to 25. its default is 10 sec.

note:

This configuration file use memory as Varnish Cache storage with memory allocated value 24G .. and its suitable with Linode 48GB plan if you use Apahce and Varnish same server.
also you can change settings to use binary file varnish_storage.bin as alternative storage method if your Linode VPS lower memory.

to switch between Varnish Files and Memory cache storage method, in the Varnish configuration file varnish just uncomment VARNISH_STORAGE variable value line VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" and comment the one above it which using malloc method VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}".

We need to adjustment the variable VARNISH_LISTEN_ADDRESS value with our Varnish Server public IP address.

we can also adjustment total size used for binary file | or for memory size at variable VARNISH_STORAGE_SIZE value.

The full and final Varnish configure file varnish will be as the following

file: /etc/sysconfig/varnish

NFILES=131072
MEMLOCK=82000
NPROCS="unlimited"
RELOAD_VCL=1
VARNISH_VCL_CONF=/etc/varnish/varnish.vcl
#Make change below with your Varnish server IP Address
VARNISH_LISTEN_ADDRESS=YOUR LISTEN SERVER IP
VARNISH_LISTEN_PORT=80
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
VARNISH_SECRET_FILE=/etc/varnish/secret
VARNISH_MIN_THREADS=50
VARNISH_MAX_THREADS=1000
VARNISH_THREAD_TIMEOUT=120
VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin
#Make change here with your suitable value according to your Linode memory size.
VARNISH_STORAGE_SIZE=24G
#Switching between memory and file storage methods.
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
#VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
VARNISH_TTL=120
#Setting VARNISH_SESS_TIMEOUT variable to set sess_timeout
VARNISH_SESS_TIMEOUT="sess_timeout=90"
#Settting VARNISH_CLI_TIMEOUT variable to set cli_timeout
VARNISH_CLI_TIMEOUT="cli_timeout=25"
#Running Varnish Daemons with our options.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
-f ${VARNISH_VCL_CONF} \
-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
-t ${VARNISH_TTL} \
-w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
-u varnish -g varnish \
-p ${VARNISH_CLI_TIMEOUT} \
-p ${VARNISH_SESS_TIMEOUT} \
-S ${VARNISH_SECRET_FILE} \
-s ${VARNISH_STORAGE}"

Backend Timeouts

The backend timeouts decides how long Varnish Cache will wait for backend operations to complete.
connect_timeout (default: 3.5s) is how long we wait for a TCP connection to the backend to come up.
first_byte_timeout (default: 60s) limits how long the processing time of the backend may be. The first byte of the response must come down the TCP connection within this timeout.
between_bytes_timeout (default: 60s) limits how long we will wait between for two subsequent successful reads on the backend connection.

so we will configure backend default in the configuration file varnish.vcl which located in /etc/varnish/varnish.vcl as following

backend default {
        .host = "127.0.0.1";
        .port = "8080";
        .first_byte_timeout     = 600s;     
        .connect_timeout        = 600s;     
        .between_bytes_timeout  = 600s;  
        .max_connections = 800; 
}

 

note:

For more Information about backed timeouts please review this Varnish doc “UNDERSTANDING TIMEOUTS IN VARNISH CACHE”.

Varnish actions for Drupal

for Drupal connection with Varnish we will configure the Varnish VCL configuration file varnish.vcl which located in /etc/varnish/varnish.vcl with the following actions and VCL code.

  • Accept Purging requests fired from Drupal
    and for security reasons we just allowed purging requests from only well known IP address, otherwise Purge request will fail and send error.
    in the file varnish.vcl we will create allowed IP addresses list acl internal and only accept purge requests from them as the following.
acl internal {
  "localhost";  
  "127.0.0.1";
  #"YOUR LOCAL VARNISH / APACHE SERVER IP";
  #"YOUR STATIC IP IF AVAILABLE";
}

and conditional purging request acceptance code will be inserted into vcl_recv subroutine as

if (req.request == "PURGE") {
    if (!client.ip ~ internal) {
        error 405 "This IP is not allowed to send PURGE requests.";
    }
    ban("req.http.host == " + req.http.host + " && req.url == " + req.url);
    return (lookup);
}

 

note

below VCL codes will be inserted into vcl_recv subroutine, but if we say another subroutine.

  • We need to pipe POST requests for logged-in users.
if(req.request == "POST" && req.http.Cookie ~ "DRUPAL_UID=") {
      return(pipe);
}
  • We can as optional Pass or Pipe logged-in users in general.
    we can ignore checking domain condition, but its making good limitation if you have more than one Drupal websites running.
if( req.http.Host == "yourdomain.com"){
    if( req.http.Cookie ~ "DRUPAL_UID=" ) {
      #return (pass); 
      return (pipe); 
    }
}

note

the different between pass and pipe ..

Calling pipe basically tells Varnish to send the request, deliver the result and then stop caring about anything (including logging etc.). Basically some kind of fire-and-forget.

pass also sends out the requests and returns the result to the client, but will ensure that logs are written and Varnish is able to handle it’s own stuff, also the deliver function of your VCL is called, which not happens when you use pipe.

please see this Varnish FAQ for more information.

  • we can safely remove all cookies form non logged-in users paths as well.
if (!(req.http.Cookie ~ "DRUPAL_UID=")) {
  if (!(req.url ~ "(^/(admin|cache|misc|modules|sites|system|openid|themes|node/add|node/.*/edit))|(/(comment/reply|edit|devel|user|user/(login|password|register))$)")) {
    unset req.http.Cookie;
  }
}
  • we need to avoid caching error messages/pages and not to loop POST requests too.
  
   if (beresp.status >= 500 && beresp.status <= 599){
   set beresp.http.Cache-Control = "max-age=0";
    set beresp.ttl = 0s; 
        if (req.request != "POST") {
           return(restart);
       }
   }
 
 if (beresp.status == 404 || beresp.status == 403 ) { 
     set beresp.http.Cache-Control = "max-age=0";
    set beresp.ttl = 0s; 
   if (req.request != "POST"){
           return(restart);
       }
   }
  • Redirect urls requests to non-www domain
    todo redirection to non www-domain with Varnish in we need 2 steps..
  1. firing custom error code
if (req.http.host == "www.yourdomain.com") { 
      set req.http.host = "yourdomain.com"; 
      error 720 "http://" + req.http.host + req.url; 
  }
  1. then create a redirection as action for this error code in vcl_error subroutine.
if (obj.status == 720) {
      set obj.status = 301;
      set obj.http.Location = obj.response;
      return (deliver);
  }
  • Ignore caching headers received from Drupal
    we need to force caching all target objects regardless to caching header that received from Drupal and its modules.
    so we need to add this line at vcl_fetch subroutine,to caching target objects for 1 Hour, and ignoring any cache headers from backend.
set beresp.ttl = 1h;

Varnish General Actions

  • Remove all cookies for static files
if (req.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]|pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff|xml|zip|docx|ttf|html|htm)(\?.*)?$" || req.url ~ "(\?itok=)([a-zA-Z0-9]+)?$") {
    unset req.http.Cookie;
}
  • Only deal with normal request types
if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "PATCH" &&
        req.request != "DELETE") {
    
    return (pipe);
}

if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
      
        remove req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
        set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate") {
        set req.http.Accept-Encoding = "deflate";
    } else {
        remove req.http.Accept-Encoding;
    }
}
  • Allow CORS (Cross-origin resource sharing) Response
    to allow CORS response for webfonts as example we need to add this code into VCL configuration vcl_fetch subroutine.
if(req.url ~ "^[^?]*\.(ttf|otf|eot|woff|svg|swf|css|gz|js)(\?.*)?$"){
    if(!beresp.http.Access-Control-Allow-Origin)
    {
      set beresp.http.Access-Control-Allow-Origin = "*";
    }
}
  • Keep sending X-Forwarded-For with all requests..

to keep send X-Forwarded-For each request we need to add this code at vcl_recv subroutine.

if (req.restarts == 0) {
      if (req.http.X-Forwarded-For) {
          set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
      } else {
          set req.http.X-Forwarded-For = client.ip;
      }
  }

and also at vcl_pipe subroutine

set bereq.http.Connection = "Close";

note

only the first request to the backend will have X-Forwarded-For set. If you use X-Forwarded-For and want to have it set for all requests, make sure to have: set bereq.http.connection = "close"; at vcl_pipe subroutine method It is not set by default as it might break some broken web.

  • Some generic Cookie manipulation
    we can use this below JavaScript cookie manipulation as a template optional.
# Strip hash.
  if (req.url ~ "\#") {
      set req.url = regsub(req.url, "\#.*$", "");
  }
  
  # Strip a trailing ? if it exists.
  if (req.url ~ "\?$") {
      set req.url = regsub(req.url, "\?$", "");
  }

# Get ride of progress.js query params
if (req.url ~ "^/misc/progress\.js\?[0-9]+$") {
  set req.url = "/misc/progress.js";
}

# Remove the "has_js" cookie
  set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
  
# Remove collapsiblock cookies
set req.http.Cookie = regsuball(req.http.Cookie, "collapsiblock=[^;]+(; )?", "");
  
set req.http.Cookie = regsuball(req.http.Cookie, "__utma.=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__auc=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__atuvc=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__asc=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *fbm_.=[^;]+;? *", "\1");
  set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
  
  # Remove DoubleClick offensive cookies
  set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
  
  # Remove the Quant Capital cookies (added by some plugin, all __qca)
  set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
  
  # Remove the AddThis cookies
  set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__atuvc.=[^;]+;? *", "\1");
  
  # Remove a ";" prefix in the cookie if present
  set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");

# Are there cookies left with only spaces or that are empty?
  if (req.http.Cookie ~ "^\s*$") {
      unset req.http.Cookie;
  }

 

  • at least we will check anything else left? to clean it up from cookies!.
if (!req.http.Cookie) {
  unset req.http.Cookie;
}
  • Customize header send to client/user
    we need to modify headers that are sent to the client (ie.. remove PHP, Apache,and Drupal headers) and so, we can Insert Varnish objects Cache status Header.
    will need to add below code at vcl_deliver subroutine as following.

 

    
if (obj.hits > 0) {
    set resp.http.X-Cache = "cached";
} else {
    set resp.http.x-Cache = "uncached";
}
  
# Remove some headers: PHP version
unset resp.http.X-Powered-By;
  
# Remove some headers: Apache version & OS
unset resp.http.Server;
unset resp.http.X-Drupal-Cache;
unset resp.http.X-Varnish;
unset resp.http.Via;
unset resp.http.Link;

Varnish VCL Configuration Full and Final file .

note:

the modified Varnish VCL source code is this VCL Version

  • The full and final VCL configuration file can be as following

file: /etc/varnish/varnish.vcl

#Define the internal network subnet.
acl internal {
  "localhost";  
  "127.0.0.1";
 #"YOUR LOCAL VARNISH / APACHE SERVER IP";
 #"YOUR STATIC IP IF AVAILABLE";
}

#Default backend definition. and timeouts.
backend default {
  .host = "127.0.0.1";
  .port = "8080";

  .first_byte_timeout     = 600s;     # How long to wait before we receive a first byte from our backend?
  .connect_timeout        = 600s;     # How long to wait for a backend connection?
  .between_bytes_timeout  = 600s;     # How long to wait between bytes received from our backend?
  
  .max_connections = 800;
}


#Handle the HTTP request received by the client
sub vcl_recv {
  
  if (req.restarts == 0) {
        if (req.http.X-Forwarded-For) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
  
  #Replace host value with your doman
  if( req.http.Host == "yourdomain.com"){
      if( req.http.Cookie ~ "DRUPAL_UID=" ) {
           return (pipe);
      }
  }
  
    if(req.request == "POST" && req.http.Cookie ~ "DRUPAL_UID=") {
        return(pipe);
    }

    if (req.request == "PURGE") {
        if (!client.ip ~ internal) {
        error 405 "This IP is not allowed to send PURGE requests.";
        }
    ban("req.http.host == " + req.http.host + " && req.url == " + req.url);
        return (lookup);
    }
  
  #Normalize the header, remove the port (in case you're testing this on various TCP ports)
    set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");       

  
  #Do Redirection to non www-domain
  #Replace host value with your doman
  if (req.http.host == "www.yourdomain.com") { 
      set req.http.host = "yourdomain.com"; 
      error 720 "http://" + req.http.host + req.url; 
  }
  
  
  if (req.request != "GET" &&
    req.request != "HEAD" &&
    req.request != "PUT" &&
    req.request != "POST" &&
    req.request != "TRACE" &&
    req.request != "OPTIONS" &&
    req.request != "DELETE") {
      #Non-RFC2616 or CONNECT which is weird.
      return (pipe);
  } 
   

    if (req.backend.healthy) {
        set req.grace = 1h;
    } else {
        unset req.http.Cookie;
        set req.grace = 1h;
    }
  

  if (!(req.http.Cookie ~ "DRUPAL_UID=")) {
    if (!(req.url ~ "(^/(admin|cache|misc|modules|sites|system|openid|themes|node/add|node/.*/edit))|(/(comment/reply|edit|devel|user|user/(login|password|register))$)")) {
      unset req.http.Cookie;
    }
  }
  

  #First remove the Google Analytics added parameters
    if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
        set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");
        set req.url = regsub(req.url, "\?&", "?");
        set req.url = regsub(req.url, "\?$", "");
    }
  
  #Some generic cookie manipulation
  
  #Strip hash.
    if (req.url ~ "\#") {
        set req.url = regsub(req.url, "\#.*$", "");
    }

    #Strip a trailing ? if it exists.
    if (req.url ~ "\?$") {
        set req.url = regsub(req.url, "\?$", "");
    }
  
  #Get ride of progress.js query params
  if (req.url ~ "^/misc/progress\.js\?[0-9]+$") {
    set req.url = "/misc/progress.js";
  }
  
  #Remove the "has_js" cookie
    set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
  #Remove collapsiblock cookies
  set req.http.Cookie = regsuball(req.http.Cookie, "collapsiblock=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "__utma.=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "__auc=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "__atuvc=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "__asc=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *fbm_.=[^;]+;? *", "\1");
    set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");

    #Remove DoubleClick offensive cookies
    set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");

    #Remove the Quant Capital cookies (added by some plugin, all __qca)
    set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");

    #Remove the AddThis cookies
    set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
  set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__atuvc.=[^;]+;? *", "\1");

    #Remove a ";" prefix in the cookie if present
    set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
  
  #Are there cookies left with only spaces or that are empty?
    if (req.http.Cookie ~ "^\s*$") {
        unset req.http.Cookie;
    }
  


    
  
  #Normalize Accept-Encoding header
    #straight from the manual: https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
            #No point in compressing these
            remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            #unkown algorithm
            remove req.http.Accept-Encoding;
        }
    }
  
    #Remove all cookies for static files
    if (req.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]|pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff|xml|zip|docx|ttf|html|htm)(\?.*)?$" || req.url ~ "(\?itok=)([a-zA-Z0-9]+)?$") {
        unset req.http.Cookie;
    }

  #Anything else left?
  if (!req.http.Cookie) {
    unset req.http.Cookie;
  }

    #Try a cache-lookup
  return (lookup);



}

sub vcl_pipe {
    #keep sending X-Forwarded-For each request
    set bereq.http.connection = "close";
    return (pipe);
}

sub vcl_pass {
    return (pass);
}

#The data on which the hashing will take place
#Routine used to determine the cache key if storing/retrieving a cached page.
sub vcl_hash {
    hash_data(req.url);

    if (req.http.Host) {
        hash_data(req.http.Host);
  } else {
        hash_data(server.ip);
    }
    #hash cookies for requests that have them
    if (req.http.Cookie) {
        hash_data(req.http.Cookie);
    }

    return (hash);
}

sub vcl_hit {
    #Allow purges
    if (req.request == "PURGE") {
        purge;
        error 200 "purged";
    }

    return (deliver);
}

sub vcl_miss {
    #Allow purges
    if (req.request == "PURGE") {
        purge;
        error 200 "purged; URL was not in cache";
    }

    return (fetch);
}

#Handle the HTTP request coming from our backend
sub vcl_fetch {

    #Cors Response;
  if(req.url ~ "^[^?]*\.(ttf|otf|eot|woff|svg|swf|css|gz|js)(\?.*)?$")
  {
    if(!beresp.http.Access-Control-Allow-Origin)
    {
      set beresp.http.Access-Control-Allow-Origin = "*";
    }
  }
  
  #gzip content that can be compressed
    if (beresp.http.content-type ~ "text/plain"
          || beresp.http.content-type ~ "text/xml"
          || beresp.http.content-type ~ "text/css"
          || beresp.http.content-type ~ "text/html"
      || beresp.http.content-type ~ "text/javascript"
      || beresp.http.content-type ~ "application/xml"
      || beresp.http.content-type ~ "application/rss\+xml"        
          || beresp.http.content-type ~ "application/(x-)?javascript"
          || beresp.http.content-type ~ "application/(x-)?font-ttf"
          || beresp.http.content-type ~ "application/(x-)?font-opentype"
          || beresp.http.content-type ~ "application/font-woff"
          || beresp.http.content-type ~ "application/vnd\.ms-fontobject"
          || beresp.http.content-type ~ "image/svg\+xml"
       ) {
        set beresp.do_gzip = true;
    }

  #Don't allow static files to set cookies, and enable cache for all static files
    if (req.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]|pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff|xml|zip|docx|ttf|html|htm)(\?.*)?$" || req.url ~ "(\?itok=)([a-zA-Z0-9]+)?$") {
    unset beresp.http.cookie;
    unset beresp.http.set-cookie;
  }

  
  #will not loop POST requests and will not cache errors
    if (beresp.status >= 500 && beresp.status <= 599){
    set beresp.http.Cache-Control = "max-age=0";
        set beresp.ttl = 0s; 
        if (req.request != "POST") {
            return(restart);
        }
    }
  
  if (beresp.status == 404 || beresp.status == 403 ) { 
      set beresp.http.Cache-Control = "max-age=0";
        set beresp.ttl = 0s; 
    if (req.request != "POST"){
            return(restart);
        }
    }
  
    #Anything else left? without Cookie setting, and prevent it from setting cookie.
  if (!req.http.Cookie) {
    unset beresp.http.cookie;
    unset beresp.http.set-cookie;
  }
  
    #To prevent accidental replace, we only filter the 301/302 redirects for now.
    if (beresp.status == 301 || beresp.status == 302) {
        set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", "");
    }
  
  
    #Set 1H cache if unset for static files
    if (beresp.ttl <= 0s || beresp.http.set-cookie || beresp.http.Vary == "*") {
    set beresp.ttl = 1h;
        return (hit_for_pass);
    }
  
  #Allow items to be stale if needed for 1Hour.
  set beresp.grace = 1h;
  #cache for 1Hour, ignoring any cache headers from backend.
  set beresp.ttl = 1h;
  
  
  return (deliver);
}

#The routine when we deliver the HTTP request to the user
sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "cached";
    } else {
        set resp.http.x-Cache = "uncached";
    }

    #Remove some headers: PHP version
    unset resp.http.X-Powered-By;

    #Remove some headers: Apache version & OS
    unset resp.http.Server;
    unset resp.http.X-Drupal-Cache;
    unset resp.http.X-Varnish;
    unset resp.http.Via;
    unset resp.http.Link;

    return (deliver);
}

sub vcl_error {
  if ((obj.status == 404 || obj.status == 500 || obj.status == 503 || obj.status == 403) && req.restarts < 4 )
  {
    return(restart);
    
  } elseif (obj.status >= 500 && obj.status <= 599 && req.restarts < 4) {
        return(restart);        
    } elseif (obj.status == 720) {
        set obj.status = 301;
        set obj.http.Location = obj.response;
        return (deliver);
    } elseif (obj.status == 721) {
        set obj.status = 302;
        set obj.http.Location = obj.response;
        return (deliver);
    } else {
         #The vcl_error() procedure
      set obj.http.Content-Type = "text/html; charset=utf-8";
      set obj.http.Retry-After = "5";

      synthetic {"
        <!DOCTYPE html>
        <html lang="en">
        <head>
        <meta charset="utf-8">
        <title>"} + obj.status + " " + obj.response + {"</title>
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <meta name="description" content="Backend Error page">
        <meta name="author" content="Pascal A.">
        <meta name="generator" content="vim">
        <meta http-equiv="Content-Type" content="text/html;charset=utf-8" >
        <link href="//netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/css/bootstrap-combined.min.css" rel="stylesheet">
        <style>
        body {
          padding-top: 60px; /* 60px to make the container go all the way to the bottom of the topbar */
        }
        </style>
        </head>
        <body>
        <div class="container">
        <div class="page-header">
                <h1 class="pagination-centered">Error "} + obj.status + " " + obj.response + {"</h1>
        </div>
        </div>
        <script src="//netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/js/bootstrap.min.js"></script>
        </body>
        </html>
      "};
    }

    return (deliver);
}

sub vcl_init {
    return (ok);
}

sub vcl_fini {
    return (ok);
}
    ~~~

What do you think?

11 Points
Upvote Downvote

Leave a Reply

Your email address will not be published. Required fields are marked *

Free SSL Certificate

Free SSL Certificate
Borland C++ 5.5 Compiler Windows10

Borland C++ 5.5 Compiler Windows10