I attended the excellent Percona Live conference earlier this week, and I have to say it was easily the best conference I’ve ever attended. There were some great, very in-depth technical talks, which I’ll blog about later.
This post is prompted by this question posted on Twitter. At the closing talk given by Domas of Facebook, he mentioned that one should use FORCE INDEX when optimising for performance.
Why is this?
If you look at the manual page explaining how MySQL uses indexes you’ll see that they are broadly selected based on which will find the smallest number of rows. This means that as additional data is added to a table the ‘best’ index use to can change.
Using FORCE INDEX tells MySQL to use a specific index (or indexes) for a query. By using it, you’re avoiding the need to calculate which index to use for each query. You’re also ensuring (or at least making it more likely) that the indexes that are most used stay in the buffer pool, thus removing the need to read them from disk. MySQL also gets the opportunity to replace infrequently used indexes with data in the buffer pool.
I suspect that the benefits of FORCE INDEX don’t apply so much for those with smaller data sets, but when one has as massive a set of data as Facebook, many small optimisations make a big difference.
Update: See also this post by the big man himself 
I just released version 1.3 of our Nagios check_sip plugin, with contributions provided by Ryanny Lin. The changes are support for SIP re-transmission, and a fix for a corner case where the RTT may appear to be negative.
Download it here.
We look after quite a few servers running Django apps, and latterly we’ve been running these on Ubuntu 10.04 LTS, using nginx + fastcgi.
On other systems we’ve used supervisor to launch the Django fastcgi process, but on Ubuntu 10.04, and now on Red Hat/CentOS 6, we have the wonderful upstart system built in which provides everything we need.
Below, you can see the upstart config I’ve written to start fastcgi. It assumes you have python-flup installed.
We have one of these per Django project hosted on the machine – you’ll need to vary the port number specified for each project you have. All you’ll need to alter otherwise is the path to run in, and the path to manage.py. If you’re running under EL6, you’ll probably want to change www-data to apache, but beware that I haven’t actually tested this config in such an environment.
Save the following in “/etc/init/djangoproject.conf”, where djangoproject is the name of your django project.
description "Basic Django fcgi Upstart config"
author "Sam Bashton"
start on (net-device-up
and local-filesystems
and runlevel [2345])
stop on runlevel [016]
respawn
chdir /var/www/djangoproject
exec su -s /bin/sh -c 'exec "$0" "$@"' www-data -- \
/var/www/djangoproject/manage.py runfcgi \
method=prefork host=127.0.0.1 port=32769 minspare=4 maxspare=8 \
maxchildren=128 maxrequests=65535 daemonize=false
Once you’ve added the file, you can start it with the command “start djangoproject”, stop it with “stop djangoproject”, and restart it (for example, if you’ve updated the code) with “restart djangoproject”, replacing djangoproject with the name you gave the config of course.
Should the fastcgi process die for some reason it will be automatically be restarted.
We’re currently looking for another Linux Consultant to join our team. The role primarily involves looking after Linux servers on behalf of our customers, as well as helping to implement new Linux based solutions.
Day to day you’ll be working with a wide range of Open Source software, but we’ve deliberately not included a list of apps – if you’re the right person for us, you’ll be able to pick up new technologies very quickly. The interview process will contain a full technical assessment.
If you’re interested, you can see more details and apply by visiting our job page. Please note that we’re really, really not interested from hearing from recruitment agencies and the like.
I posted here in March speculating when RHEL6 would be released. I’m pleased to say that the final version of Red Hat Enterprise Linux 6 has now been officially released!
Red Hat 5 was getting rather long in the tooth, and I’m sure many users will be happy to see the inclusion of more up to date packages – MySQL 5.1, Python 2.6 and PHP 5.3 amongst the more notable that will have been giving some users of RHEL5 headaches.
Red Hat 6 provides much more than just a version bump to packages though, there are some key changes. KVM is now the default virtualisation technology. Xen is still supported, and RHEL6 will work as a RHEL5 guest out of the box, as long as one uses an ext3 /boot partition. The para-virt drivers for VMWare are also included, making those deployments simpler too.
ext4 is now the default file system, which provides support for file systems up to 1 exabyte in size and files up to 16TB. Red Hat only officially supported file systems of up to 8TB in size on RHEL5, although it was possible to make them larger than this. ext4 also brings much better performance, and removes the limit of 32000 subdirectories in a single directory.
Upstart is now used for the boot process, replacing the old /etc/init.d scripts. Upstart will be familiar to those who have used more recent versions of Ubuntu Linux, where the project was started. It provides support for automatically respawning failed daemons – something that will be music to the ears of anyone who has ever tried to log into a machine that has had the SSH daemon killed by the out of memory killer. It also always for parallel execution of startup scripts, meaning that RHEL6 boots significantly faster than previous versions.
One other piece of good news for sysadmins is that sendmail is now no longer the default MTA – I can almost hear the cheers of readers from here! Postfix is now provided by default, with the distinct advantage of configuration files that are easily distinguishable from line noise.
Overall, Red Hat Enterprise Linux 6 is a massive step forward from the previous version – as you’d expect, given RHEL5 was released in 2007.
