Resize images on the fly with Nginx try_files

Nginx is fantastic as a high performance server that can serve static files very, very quickly with minimal, predictable use of resources. Static files are the CSS, Javascript and boilerplate images that make up the design of a website. As the name suggests, static files tend not to change very often. These days the chances are that they have already been optimised, minified and compressed before being deployed to the web server.

Nginx doesn’t handle requests for scripts like PHP or Python directly, but instead acts a reverse proxy and will pass requests to backend server. We tend to use php-fpm but the backend can be anything, even Apache.

Websites typically have a lot of images uploaded by users that are needed in lots of different variants (thumbnail, crop, large etc.). These user generated images obviously can’t be optimised and resized ahead of time like all the boilerplate images.

One solution is to generate all the different image variants when an image is uploaded, but this isn’t very flexible, can create a noticeable delay as all the variants are generated, and things can get complicated if you need to create a new variant for thousands and thousands of images.

There are lots of solutions that will resize images dynamically but these tend to send all requests for images to a script that will serve existing images (hopefully from a cache) or attempt to generate a new image on the fly, serve the new image and save it in a cache for future use. This approach is tried and tested but it is unnecessary overhead calling a script for every image request. It is much more efficient to serve images straight from the disk, something Nginx excels at, and not have to involve a script at all.

A nice solution is to use the Nginx try_files directive to serve an image directly, if it exists and only send the request to the backend if the image (variant) requested doesn’t exist.

From the documentation for try_files:

Checks the existence of files in the specified order and uses the first found file for request processing; the processing is performed in the current context. The path to a file is constructed from the file parameter according to the root and alias directives. It is possible to check directory’s existence by specifying a slash at the end of a name, e.g. “$uri/”. If none of the files were found, an internal redirect to the uri specified in the last parameter is made. For example:

location /images/ {
    try_files $uri /images/default.gif;
}
 
location = /images/default.gif {
    expires 30s;
}

In the above example if the specified image is missing, /images/default.gif is returned instead and has it’s ‘Expires’ header set to 30 seconds.

Simplified Nginx conf file

See how we can use this in practice

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
server {
    listen 80;
 
    access_log /var/log/nginx/example.com.access.log main;
    error_log /var/log/nginx/example.com.error.log;
    root /var/www/vhosts/example.com/htdocs;
    index index.php index.html;
    server_name example.com;
 
    # Check if the requested image exists and if not pass to @backend
    location ~* ^.+.(jpg|jpeg|png|gif)$ {
        try_files $uri @backend;
        add_header Pragma "public";
        add_header Cache-Control "public";
        expires     1y;
        access_log  off;
        log_not_found off;
     }
 
 
 
    # Other static files
    location ~* ^.+.(ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|eot|woff|svg|htc)$ {
        add_header Pragma "public";
        add_header Cache-Control "public";
        expires     1y;
        access_log  off;
        log_not_found off;
    }
 
    location / {
        # Check if a file exists, or route it to index.php.
        # Configure depending on application used
        try_files $uri $uri/ /index.php?$query_string;
    }
 
    location @backend {
        rewrite ^(.*)$ /images.php?url=$uri;
    }
 
    location ~\.php$ {
        try_files $uri =404;
        fastcgi_pass unix:/dev/shm/apache-php.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include /etc/nginx/fastcgi_params;
    }
 
}

In the solution above @backend passes requests to /images.php. Images.php bypasses the rest of the application and is simply contains code to necessary resize images on the fly. Write your own or make use of an existing library. Personally, I like Intervention Image.

Quickly duplicate a row in MySQL

A quick way of duplicating a row in a table without running into duplicate key errors or having to type out all the field names…

Not quite a one-liner, but very straightforward.

/*
Duplicate row 58 from mytable
*/
CREATE TEMPORARY TABLE tmptable  SELECT * FROM mytable WHERE id = 58;
 
/*
Change the unique key
*/
UPDATE tmptable SET id=0; 
 
/* 
Insert the duplicate row into the original table
*/
INSERT INTO mytable SELECT * FROM tmptable;
 
/* 
Drop the temporary table
*/
DROP TABLE tmptable;

Installing Homebrew & Fabric on OSX 10.9 Mavericks

I’ve just upgraded my Mac at work to 10.9.1 Mavericks. No messing about – completely clean install on a formatted disc. Last year I wrote a quick post about installing Fabric on my Mac, this is an update of that post to reflect a few changes.

If you don’t know…

Fabric is a Python (2.5 or higher) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.

I think Fabric is a good solution to manage the deployment of websites and applications – it is an integral part of my workflow these days.

Homebrew is a package manager for OSX and is an alternative to the commonly used MacPorts. You can find out a bit more about the differences between them in this thread on arstechnica.

Install Command Line Developer Tools

These were previously part of Xcode but now need to be installed separately, open a terminal and type:

xcode-select --install

…and follow the prompts.

Install Homebrew

The address of the download appears to have chnaged slightly since last year…

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

…and follow the prompts.

When you see the successful installation message, type:

brew doctor

This will alert you to any problems that you need to resolve.

If everything is good you should see the following message:

Your system is ready to brew.

Install Git, Python and Fabric

brew install git
brew install python

You now need to edit your system paths so that Python can be found:

I edited my .bashrc file: nano ~/.bashrc and added the following line:

export PATH=/usr/local/bin:/usr/local/sbin:~/bin:$PATH

And then /etc/paths sudo nano /etc/paths so that usr/local/bin (the location of packages installed by Homebrew) is first:

/usr/local/bin
 
/usr/local/sbin
 
/usr/bin
 
/bin
 
/usr/sbin
 
/sbin

And finally install Fabric. There isn’t a Homebrew package available by default, so I just installed Fabric using pip – the pip package manager for Python is installed at as part of Python.

pip install fabric

Check if it works…

Let’s check and see if everything has worked…

$ fab
 
Fatal error: Couldn\'t find any fabfiles!
 
Remember that -f can be used to specify fabfile path, and use -h for help.
 
Aborting.

In this case the Fatal error is good. It means that Fabric has installed correctly and you now need to write some fab files.

Linux recursive directory listings

It’s sometimes really useful to recursively generate a list of directories, excluding the files.

I use this in build scripts to create lists of directories that will need to be generated on the remote server – this is useful for things like upload directories for user generated content that wouldn’t necessarily be under version control.

 
find . -type d

To exclude a directory within this path you can use:

 
find . -type d -not -path "./directory_to_ignore/*"