Clear Varnish cache via PHP: a Drupal 7 proof of concept

Using Varnish as reverse proxy or proxy is an useful approach to reduce the load of webservers like Apache.

In Drupal 7 I’ve to clear the varnish cache of a specific domain when Drupal caches are globally cleared. Drupal has the right hook invoked when cache are cleared:

function clearcachevarnish_flush_caches() {
  $filename = '/var/www/varnishdomains2cleardir/varnishdomains2clear';
  // each domain on a separate line: append to the end of the file
  $myfile = fopen($filename, "a");
  $h = $_SERVER['HTTP_HOST'];
  $txt = $h . "\n";
  fwrite($myfile, $txt);
  fclose($myfile);
  drupal_set_message('Varnish cache queued to be cleared. Please wait 1 minute before checking.');
  // no cache table should be cleared
  return array();
}

Now this piece of code simply adds the current domain to a ASCII text file on /var/www/varnishdomains2cleardir/varnishdomains2clear.

Preparing the file to the write

On CentOS you have to add /var/www/varnishdomains2cleardir to the httpd-writable directories list using:

mkdir /var/www/varnishdomains2cleardir;
chcon -v --type=httpd_sys_content_t /var/www/varnishdomains2cleardir;
chown myuser:mygroup /var/www/varnishdomains2cleardir;
chmod -R 777 /var/www/varnishdomains2cleardir;
touch /var/www/varnishdomains2cleardir/varnishdomains2clear;

Now the empty file is ready to be written by your hook_flush_caches() implementation. Now enable the clearvarnishcache module and clear the cache to write the current domain name to the file.

The clear varnish cache script

To clear the varnish cache you usually have to be logged as root using the command varnishadm. Here a script that will read the domains file written above, clear the varnish cache for that domain and then remove the domains lines.

#!/bin/bash
callinguser=`whoami`
if [ "root" != "$callinguser" ]
then
 echo "Only root can run this command."
 exit 1
fi
cd /path/to/clear/cache/command/

date=`date +%Y-%m-%d_%H:%M:%S`

# check lock
# prevent the script from being run more than once
if [ -f /tmp/clearcachevarnish-lock ]; then
echo "Script clearcachevarnish is already running. You can rm /tmp/clearcachevarnish-lock to break the lock manually."
exit 1
fi
touch /tmp/clearcachevarnish-lock
dominidapulire=`less /var/www/varnishdomains2cleardir/varnishdomains2clear`
while [[ ! -z $dominidapulire ]]
do
 dominio=$(echo "$dominidapulire" | sed -n '$p')
 echo $dominio
 dominidapulire=$(echo "$dominidapulire" | sed '$d')
 if [ "" != "$dominio" ]
 then
 varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret ban req.http.host == "$dominio"
 echo "varnish cleared on $dominio"
 fi
done
# remove all domains lines
truncate --size 0 /var/www/varnishdomains2cleardir/varnishdomains2clear

# remove lock
rm /tmp/clearcachevarnish-lock

Make this script as executable .sh file using chmod a+x on it. If you run the bash script, varnish cache for files on the domains list will be cleared. It’s not so useful when using the Drupal UI so we should schedule this task periodically, e.g. every minute.

Scheduling the varnish clear cache

Here the crontab entry for execute the script every minute:


* * * * * root /path/to/clear/cache/command/clearcachevarnish.sh

The steps

  1. User clear Drupal cache
  2. hook_flush_caches() is invoked: the domains list file is written
  3. clear varnish cache script is launched by root every minute
  4. for each domain in the list, varnish cache is cleared

This is the end of this proof of concept. The code wasn’t tested against attacks so please comment if you have any suggestion to improve it. I’m not very fond of the idea of a php script writing something read by a bash script but this is the less problematic solution I found for this case.

Cron cannot run on Drupal: the drupal_goto() case

Sometimes you want to redirect a page to another on drupal. You can do this using a simple function called drupal_goto().

On few sites I’ve enabled the PHP filter module and then created a new page with PHP code input format with drupal_goto(‘node/2’) to redirect the current page to a specified node. Bad idea.

I’ve noticed that, after this change, cron.php operations failed, if you have Search module enabled. On cron new contents are indexed by the Search module: when it got my PHP page, it tries to index it but suddenly is redirected to another. You can also found an error like “Maximum function nesting level of ‘100’ reached” on php error log, symptom of an indexing blocked by pages with drupal_goto inside.

Solution:

  1. Comment all drupal_goto() instruction in your site within pages.
  2. Use an alternative method to redirect from a node to another.
  3. Run cron from Status Report page: you can adjust indexed content per cron on Search configuration page (admin/settings/search on 6.x)

You can add a new block with PHP code inside or (better) create a new module for this simple block (with a simple PHP switch statement as content), displaying it only on specified pages (on the bottom of block configuration). But if you create a PHP block via UI, and you put that block on every page, your site could be loop, so creating a module is the cleanest and secure way (if something go wrong, you can delete your module from the codebase and correct it). You can also find some contrib modules for redirect on drupalmodules.com.

See also:

Fatal error: Maximum function nesting level of ‘100’ reached, aborting!

PEAR install quick howto on Debian Lenny: Image_Graph

Tested on Debian Lenny.

Time elapsed before a working script < 10 minutes 😉

Install PEAR installer:

# apt-get install php-pear

First, type:

pear -h

to get help.

On root shell type:

# pear install --alldeps Image_Graph

Install failed with error:

Failed to download pear/Image_Graph within preferred state “stable”, latest release is version 0.7.2, stability “alpha”, use “channel://pear.php.net/Image_Graph-0.7.2” to install

Retype as:

# pear install --alldeps Image_Graph-0.7.2

Repeat for other alpha dependencies the same procedure, eg:

pear install --alldeps Numbers_Words-0.15.0

At last, you got:

Nothing to install

Now, you can find all installed packages on /usr/share/php.

ImageGraph is installed on /usr/share/php/Image/Graph.php.

You have to include it on your PHPs.

You can follow the Ian’s howto on:

http://www.phpbuilder.com/columns/ian_gilfillan20060503.php3

To start using PEAR Image_Graph.

Here the result using Ian’s code on my Linux box:

Image_Graph test, using Ian code

Image_Graph test, using Ian code

Additional info:

Hosting providers (eg. HostMonster) sometimes furnish simplified PEAR installation via CPanel. So, you can use local PEAR installation for developement, and online installation for production use.

Remember:

  • To check differences between local and production versions
  • To change include path! (say no to white screen!)

See also:

Raise Drupal files upload limit

  1. In your Drupal site, go to admin/settings/uploads
  2. Raise upload limits. If the value you want to insert is greater than PHP allowed value:
  3. On bottom of your .htaccess file (drupal/.htaccess) add these lines:
    # upload settings
    # cfr. http://it.php.net/manual/it/function.ini-set.php
    php_value post_max_size 50M
    php_value upload_max_filesize 50M
    php_value memory_limit 128M

    Change values as you wish.
  4. Return to admin/settings/uploads and watch changes on PHP limit.

See also:

HTTP 301 Page moved in Drupal

Scope: Move an old document to a new URL with HTTP 301 Permanent Redirect.

Module used (required if you don’t want to modify an existing page): URL aliases.

  1. Create a new page (or edit an old one). Select “PHP code” in input type.
  2. Fill the url alias box, with the old document path
  3. Insert the following code: <?php
    // Permanent redirection
    header(“HTTP/1.1 301 Moved Permanently”);
    header(“Location: http://www.yoursite.tld/location&#8221;);
    exit();
    ?>
  4. Save.

Note: Obviously you can’t see the page (you’ll be redirected to specified address), but you can edit or delete the page from admin/content/node. As admin, if you try to unpublish the page you’ll be redirected anyway: you have to delete the page to fix it.

See also: http://www.somacon.com/p145.php