Caching Minified files with Nginx

Save on DeliciousShare on Facebook+1Pin it on PinterestSubmit to redditSubmit to StumbleUponShare on TumblrShare on Twitter

A quick introduction to Minify

Minify is a popular PHP5 library best described by the Minify site itself:

Minify is a PHP5 app that helps you follow several of Yahoo!’s Rules for High Performance Web Sites.
It combines multiple CSS or Javascript files, removes unnecessary whitespace and comments, and serves them with gzip encoding and optimal client-side cache headers.

The uses of Minify are twofold:

  1. Speed up websites by combining and minifying files
  2. Help eliminate the problems of cached css and javascript files in users browsers.

Minify is fairly easy to setup and can dramatically increase the performance of websites, but as mentioned in the FAQs because each request is served by PHP, it can actually slow your site down (for example if your site recieves a lot of traffic or you have are on a feeble shared server)

A simple solution to this problem – suggested in the Minify FAQs  is to serve your Minified files through a reverse proxy.

An even quicker introduction to Nginx

Nginx is popular high performance web server and reverse proxy server. I’m not going to try and summarise what it does here – if you are reading this, the chances are you already know.

We have used Nginx for a while in a standard way, using Nginx in front of Apache. This approach is easy to configure and is also to revert to a pure Apache setup. We served all static content (css, js, images etc.) directly from Nginx and passing requests for dynamic content to a backend (PHP on an Apache server) using the Nginx HttpProxyModule.

Despite the fact that our Minified css and Javascript rarely change with this setup a request for either is still a PHP request. We can avoid this request by using the Nginx HttpProxyModule  to cache responses from the backend making a faster response and reducing the load on the backend server.

Nginx configuration

This is a simple Nginx configuration taken from a development server running Ubuntu. For ease of maintenance, the configuration file is split up into several files using sites-available and sites-enabled directories (Debian/Ubuntu style). In this case Nginx is listening on Port 80 and Apache is listening on Port 8080

nginx.conf

The main file is nginx.conf which then includes all the live virtual hosts that are in the sites-enabled directory (often organised symbolic links to actual files in the sites-available directory)

The white-space in the config files doesn’t matter so it makes sense to organise your .conf files in the way you find most readable.

worker_processes  1;
 
events {
    worker_connections  1024;
}
 
http {
    include mime.types;
    default_type application/octet-stream;
 
    log_format  new_log
    '$remote_addr - $remote_user [$time_local] $request '
    '"$status" $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';
 
    # Proxy cache and temp configuration.
    proxy_cache_path 	/var/www/nginx_cache levels=1:2
			keys_zone=main:10m
			max_size=1g inactive=30m;
    proxy_temp_path 	/var/www/nginx_temp;
 
    sendfile on;
 
    include /etc/nginx/sites-enabled/*; 
}

The key directive here are the lines that setup the Proxy cache and which allow us to cache the results of scripts that are returned by the backend server. (see documentation)

    # Proxy cache and temp configuration.
    proxy_cache_path 	/var/www/nginx_cache levels=1:2
			keys_zone=main:10m
			max_size=1g inactive=30m;
    proxy_temp_path 	/var/www/nginx_temp;

proxy_cache_path – where on the filesystem the cached files will be stored
levels – helps define the structure of files stored in the cache directory
max_size – the maximum size of the cache
inactive – timeout period for requests to the cached files
proxy_temp_path – a buffer for requests from the file system

example.conf

This is a simple example of one of the virtual host .conf files this is included using the include /etc/nginx/sites-enabled/*; directive in the nginx.conf file.

server {
    listen 80;
    access_log /var/log/nginx/www.example.com.access.log;
    error_log /var/log/nginx/www.example.com.error.log;
    root /var/www/www.example.com/;
    index index.php index.html;
    server_name www.example.com;
 
    # send appropriate headers to enable browser caching for static files
    # static files are identified by file extension
    location ~* ^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)${
    access_log off;
    expires 30d;
   }
 
    # Set the proxy cache key
    set $cache_key $scheme$host$uri$is_args$args;
 
    location ~/min/ {
        # Set proxy headers for the passthrough
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_pass http://192.168.1.5:8080;
        proxy_cache main;
        proxy_cache_key $cache_key;
        proxy_cache_valid 30m; # 200, 301 and 302 will be cached.
        # Fallback to stale cache on certain errors.
        # 503 is deliberately missing, if we're down for maintenance
        # we want the page to display.
        proxy_cache_use_stale 	error
               	              	timeout
                       	      	invalid_header
                      		http_500
                       		http_502
                       		http_504
                       		http_404;
	} 
    # proxy any other requests back to the Apache server listening on Port 8080
    location / {
        more_clear_headers 'Content-Length' 'Transfer-Encoding';
	proxy_cache_bypass 1;
	proxy_no_cache 1;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_pass http://192.168.1.5:8080;
    }
 
}

The virtual host .conf file above contains 3 different location blocks each with a different role.

The first location block catches requests for static files and serves them directly from the file system – in this case the Apache server is not accessed at all. Unfortunately request to minify are all .php requests so cache headers are not sent an every time a minified file is requested it will be passed back to Apache.

The role of the third location block is to send any request that hasn’t already been dealt with already back to the Apache server.

The role of the second location block is deal with any requests to minify. We have set a cache timeout of 30 minutes and any requests within that time period will be served from the Nginx cache – if there is no match in the cache, the request will be passed back to Apache. If you are using minify_group, when the underlying Javascript and CSS files change, the timestamp on the minified URL will change and therefore no matching cache file will exist, so the Nginx cache will update.

How do I know if it’s working

The easiest way to see this is working is if you have Apache mod_status enabled. You will know it works because you will no longer see any requests to /min/?

XPath to the rescue. Again?

Save on DeliciousShare on Facebook+1Pin it on PinterestSubmit to redditSubmit to StumbleUponShare on TumblrShare on Twitter

It’s odd. I don’t think about think about XPath from one month to the next. But once in a while, when my usual solutions have all come up blank. Ta-Da XPath to the rescue!

Recently as part of a site I was working on, the design basically required that I inject a block of content into a nested list (part of an elaborate menu actually) – bit of a fiddle because modifying the code that generated the list was not an option.

My first avenues of attack were just simple str_replace() and a regex replace, but I just couldn’t get it work consistently – there were two many variables – additional attributes, white-space etc. The two constants were that the code fragment validated as xhtml (hence xml) and I would always have a class that I could use as a hook.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
 
// $menu - fragment of html consisting of nested UL's 
 
$xml = simplexml_load_string($menu);
 
$nodes = $xml->xpath('//*[@class = "current"]');
 
if(!empty($nodes)){
 
	$nodes[0]->addChild('div', 'text_to_replace');
	$menu = $xml->asXML();
	echo str_replace('text_to_replace', $str, $menu);
 
} else {
	echo $menu;
}

//*[@class = "current"] finds nodes at any level with where the class attribute contains ‘current’.

node[0] is the first instance of a node with this attribute and inject some place-holder text.

$menu = $xml->asXML(); our modified list.

Hey presto!

Quick debug function

Save on DeliciousShare on Facebook+1Pin it on PinterestSubmit to redditSubmit to StumbleUponShare on TumblrShare on Twitter

It’s not pretty and it’s not elegant, but we’ve all got a bag of quick and dirty functions that help make life just a little bit easier.

This is a simple print_r() but I finally got fed up of loosing my quick debug statements and so added the line name and file number from a stack trace (substitute var_export() to taste).

1
2
3
4
5
6
7
8
9
10
11
12
13
 
function pr($str){
 
        echo('<pre>');
        print_r($str);
        echo('</pre >');
 
        $d = debug_backtrace();
 
        echo $d[0]['file'] . '<br />Line: <b>' . $d[0]['line'] . '</b>';
        echo '<hr />';
 
}

Reorder a nested HTML list in PHP

Save on DeliciousShare on Facebook+1Pin it on PinterestSubmit to redditSubmit to StumbleUponShare on TumblrShare on Twitter

Recently I was working on website where I had to re-order a nested list (part of a navigation menu) – unfortunately I only had access to fragment of HTML so I couldn’t just manipulate the arrays from which it was built. The menu was compiled from various arrays and months within years sometimes came out all wrong.

So I thought, I’d just treat it as a bit of XML (which obviously it is) and re-order it using PHPs native XML handling classes. XML is one of those things that I use frequently, but never really do anything with, and finding a solution took me rather longer than I had expected. It is a mixture of simpleXML and XMLDom.

One of the main problems was the lack any real examples.

If anyone can suggest a more elegant solution, I would love to hear it.

Here is my code:

The UL to re-order

As you can see the month names are in the wrong order

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<ul id="menu">
    <li class="first"><a href="/news/q/date/2011/">2011</a></li>
    <li class="current last">
        <a href="/news/q/date/2010/">2010</a>
            <ul>
                <li class=""><a href="/news/q/date/2010/07/">July</a></li>
                <li class=""><a href="/news/q/date/2010/06/">June</a></li>    
                <li class=""><a href="/news/q/date/2010/11/">November</a></li>
                <li class=" last"><a href="/news/q/date/2010/10/">October</a></li>
                <li class=""><a href="/news/q/date/2010/09/">September</a></li>
                <li class=""><a href="/news/q/date/2010/08/">August</a></li>
                <li class="first"><a href="/news/q/date/2010/12/">December</a></li>
                <li class=""><a href="/news/q/date/2010/05/">May</a></li>
           </ul>
    </li>
</ul>

My (woeful) solution

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
$xml = simplexml_load_string($string);
 
// pull a node tree as an Array out using simpleXML xpath
$trees = $xml->xpath('/ul/li/ul');
 
$array = array();
$order = array();
 
$i = 0;
 
// we only need to delve into XML if there are any nested <ul>s
if(isset($trees[0])){
 
	foreach($trees[0] as $var){
 
		// store each node in an indexed array
		$array[$i] = $var; 
		// store the month number in an index array
		// based on the text node value of the <a> tag
		$order[$i] = date('m', strtotime((string) $var->a)); 
 
		$i++;
	}
 
	// sort the month number array descending, but maintaining the keys
	arsort($order); 
 
	// create a new XML Dom object to manipulate stuff
	$dom = new DomDocument();
	// create a holder node <ul>
	$ul = $dom->createElement('ul');
 
	// iterate through the array of simpleXML objects 
	// based on the order in which their keys appear in the re-ordered array
	foreach($order as $key => $value){
		// get the simpleXML objects into a string
		$node = dom_import_simplexml($array[$key]);
		// get the string into an actual DOM node
		$node = $dom->importNode($node, true);
		// append
		$ul->appendChild($node);	
	}
 
	//$dom->appendChild($ul);
 
	// unset the contents of the original <ul> node that we have resorted
	$parent = $trees[0]->xpath( 'parent::*' );
	$parent[0]->ul = NULL;
 
	// turn our simpleXML object into a DOM object
	$ixml = dom_import_simplexml($xml);
	$new = new DOMDocument('1.0');
	$ixml = $new->importNode($ixml, true);
	$ixml = $new->appendChild($ixml);
 
	// fire up a DOM xpath object
	$xpath = new DomXpath($new);
	// pull a node tree out using simpleXML xpath
	$tree = $xpath->query('/ul/li/ul');
 
	// add our newly created DOM node conatining the re-ordered <ul> after the existing node
	$tree->item(0)->parentNode->appendChild($new->importNode($ul, true));
	// delete the original empty node
	$tree->item(0)->parentNode->removeChild($tree->item(0));
 
	echo $new->saveHTML();
 
} else {
 
	echo $string;
 
}

MySQL DATE_FORMAT() AND PHP date() formats

Save on DeliciousShare on Facebook+1Pin it on PinterestSubmit to redditSubmit to StumbleUponShare on TumblrShare on Twitter

I’m always finding myself looking up strings to format PHP and MySQL dates. A simple d/m/Y is pretty straight forward but sometimes things can get a bit fiddly. Where ever possible I try and leave my date formatting in the database, but I still find myself doing it in PHP all the time.

Here are a few simple cut and paste date formats.

PHP Date MySQL Date PHP Format MySQL Format Notes
7/9/2009 7/9/2009 j/n/Y %e/%c/%Y
07/09/2009 07/09/2009 d/m/Y %d/%m/%Y
7/9/2009 8:07 7/9/2009 8:07 j/n/Y G:i %e/%c/%Y %k:%i
7/9/2009 8:07 AM 7/9/2009 8:07 AM j/n/Y G:i A %e/%c/%Y %k:%i %p
07/09/2009 8:07 AM 07/09/2009 8:07 AM d/m/Y G:i A %d/%m/%Y %k:%i %p
07/09/2009 08:07 AM 07/09/2009 08:07 AM d/m/Y H:i A %d/%m/%Y %H:%i %p
07/09/2009 8:07 am d/m/Y G:i a
Mon 7th Sep 2009 Mon 7th Sep 2009 D jS M Y %a %D %b %Y
Monday 7th September 2009 Monday 7th September 2009 l jS F Y %W %D %M %Y
2009-09-07T08:07:50+01:00 Monday 7th September 2009 Y-m-d\TH:i:sP %W %D %M %Y PHP DATE_ATOM constant
Mon, 07 Sep 2009 08:07:50 +0100 Mon, 07 Sep 2009 08:07:50 D, d M Y H:i:s O %a, %d %b %Y %T PHP DATE_RSS constant

The PHP way

 
echo(date('d/m/Y', strtotime('2009-09-07 08:07:50')));
 
//or
 
$dateTime = new DateTime("now");
echo $dateTime->format("Y-m-d H:i:s")

The MySQL way

 
$sqlstring = "SELECT DATE_FORMAT('2009-09-07 08:07:50', '%d/%m/%Y') AS formated_date FROM some_table" ;

If you haven’t looked at the PHP manual closely for a while you should definitely (re)acquaint yourself with the DateTime Class.