Following on from last week I've been working on more ways to make websites faster and save precious bandwidth.
I've also enabled gzip compression in PHP. Because of shiftlib this was easy to add to multiple sites at the same time.
To enable gzip in apache you need to do the following:
# nano /etc/httpd/conf.d/mod_deflate.conf
Then restart apache:
To enable expiration headers in apache - you need to do the following:
Then restart apache:
Up until now my main PHP thumbnail script has generated images on the fly. PHP is so fast that it doesn't take very long even for a page full of images. However when you have lots of visitors all those clock cycles add up and there are significant savings to be made. My thumbnail script works by pointing the image source to a php script - and passing in the filename as a variable, e.g
There is a big problem with this. Because it's pointing to a php script the cache headers won't work. You could use the php header function to send the correct cache headers but I still suspect that it won't cache as well as a regular image. There is another problem - search engines won't know that it's an image - especially if it's in a link. They would just think it's a regular php page. So they will go ahead and pull down a bunch of images without needing to.
So my new approach is a little different. I have a function that checks if there is a cached thumbnail before creating one if necessary. It then returns the image tag of the resulting thumnail. e.g.
I checked the web-stats of some of the busier websites that I host. There was a significant chunk of bandwidth being used by search engines - around 10%. Search engines will index an entire site several times over a month - but they shouldn't have to pull down every single image. This may be related to the thumbnail issue outlined above but I also have taken another precaution. I've created a robots.txt file and am now preventing access to uploads folders and thumbnail scripts and anything else that a search engine doesn't need. I may need to add an exception for google image search, but for the time being I will monitor it and see how it goes.
18/07/2010 permalink | Posted in web development | 0 Comments »
Server bandwidth has been a tad high recently - so I decided to profile some of the busiest sites using the excellent resource measuring tools in Google Chrome. The most obvious culprit on the sites I looked at was AJAX libraries like Prototype, jQuery and extJS. I needed to speed this up, now I already know about Google AJAX libraries API - so that seemed like a good place to start.
Google AJAX libraries API provides an interface to load remote AJAX libraries using Google bandwidth. Google automatically handles caching and minifying to make the files load as quickly as possible. This is great but there are a few limitations.
You have to declare which version of the library to load. I manage a lot of sites and generally I just want the most efficient, up-to-date version without having to update many different websites.
So I've added a PHP function to ShiftLib called load_js(). You can send this function an array of libraries and it will load the most up-to-date versions using Google AJAX Libraries.
I've also added support for lightbox and extJS - which are not supported by Google AJAX Libraries. My function will also detect if the page is running under SSL and subsequently load all the scripts by HTTPS if it is. It loads all the scripts in the correct order to avoid conflicts and works out dependencies - e.g. lightbox requires prototype to work.
So instead of having a block of code like this:
I now have:
12/07/2010 permalink | Posted in web development | 8 Comments »
Database queries can very often be the bottle-neck that slows down a webpage. If you have a busy server with lots of sites or lots of webpages it's not always apparent which database queries are causing the problems.
Logging slow queries
A good place to start is by logging slow queries. You can do this by adding the following lines to your MySQL config file (usually /etc/my.cnf)
Create the log file and give it write access:
You will then need to restart mysql:
After a while the log file should start to fill up with a list of queries that have taken longer than 1 second.
Adding indexes can have a dramatic affect on speed. You should look at the table joins and WHERE conditions of the query to see where indexes could be used.
Take these examples.
Adding an index to the "name" column will significantly improve this query.
Adding one combined index for "name" and "surname" will help here.
In this case I would use a unique index for the email column. Unique means that the email address can only appear once in the table. This is much faster than a regular index and also enforces the database integrity.
Note that indexes take up space and slow down insert/ updates - but this is generally a small price to pay for much faster select statements.
Correct field types
It's worth looking over the database structure and seeing if a TINYINT could be used instead of an INT. Or an ENUM instead of a VARCHAR. Also check the size of the VARCHAR columns and adjust accordingly. Check if INT fields should be UNSIGNED. An UNSIGNED INT can only be positive (>=0). Unsigned INTs should always be used for ID columns - and again this helps with your database integrity. You may need to run "OPTIMIZE table" before you see any benefit. You probably won't see a drastic improvement - but these little changes all add up.
Joins are an expensive operation and should only be used when absolutely necessary. Check the fields you are selecting and your WHERE conditions to see if you really need each join.
Only select what you need to, so if you only need to fetch an ID instead of doing:
And you can use limit to prevent searching of the entire table.
Try not to include MySQL queries in PHP loops - you might be able to get the same outcome using a join or a sub-select at a fraction of the time.
This should be enough to get you started. For even better performance you could try tweaking MySQL itself or upgrading your hardware.
07/07/2010 permalink | Posted in web development | 13 Comments »
About meAdam Jimenez is a freelance web developer who has been professionally developing websites since 2000.