Home arrow Load-Balancing
Load-Balancing


 

Load-Balancing

Hosting issues related to traffic volume are discussed here:

Shared hosting
VPS hosting
Dedicated hosting
Managed dedicated servers
Dedicated server maximum loads
Server cache
Load-balancing
Network monitoring
Alternative server software for dedicated servers


Shared hosting

Most businesses start off with shared hosting. This is the best option for the majority, as it allows you to pay only for the facilities you need when starting up. Only a minority of sites ever need to upgrade to a better service, since shared hosting will always handle 1,000 visits a day and should go to 2,000 or more without issues. Few normal business sites go over this traffic volume.

Shared hosting of good quality will allow you to service up to 5,000 visits a day - and perhaps even more, on very good quality shared hosting, the type with about 50 sites on a server. Of course, this is not available cheaply, it normally costs about £150 / $225 a year or so. Cheaper shared hosting will start to crack with less visits - the page load time will increase to an unacceptable time, and there may be occasional offlines.

VPS hosting

The next step is to go to a VPS (virtual private server) or a dedicated server. VPS hosting means that there are from 2 to 10 sites on a server, and each has its own partition on the disk. This means that your own server app such as Apache can be run. However there are many issues with the precise allocation of server resources, particularly RAM, and in practice many VPS hosting arrangements are not satisfactory. For this reason we do not use VPS, we advise that as soon as funds are available, a dedicated server is used.

It's true that some VPS accounts allow the RAM resource to be expanded, but to achieve a sufficient amount to prevent occasional offlines it is often necessary to spend the same or more as a dedicated server. The minimum needed to avoid issues on busy sites running CMS or forums is about 1GB of exclusive RAM, and this is normally just too expensive. A VPS with 512MB RAM plus 'burstable' RAM might be OK for sites with little traffic but a requirement to host their own software. For busy sites this is not a good arrangement.

Dedicated hosting

A dedicated server is one that you rent exclusively - only your own website or sites are on the server and it is entirely your own, to set up as you wish.

The cost starts at about £100 / $150 per month, which means that you need a reasonable revenue to support it. However any site that needs it can afford it, as with 5,000 visits a day or more there must be enough income for this or something is wrong.

Some users have a dedicated server (aka a 'dedibox') even when they have no traffic need for it, as they want the far superior facilities it offers:
  • Better security
  • Install whatever software you like
  • Restart your server app, or reboot the server even, whenever you want
  • Have whatever software versions or combinations you like
  • Install your own advertising server software to display the ads you want
  • No page load delays when a 3rd-party ad server on another site is bogged down
  • ...and plenty more...
So there are many attractions to a dedibox, especially for ecommerce sites as the security will be better.

Managed dedicated servers

A dedibox can be a 'managed dedicated server' or an 'unmanaged dedicated server'. The cost difference is normally quite small, even though the workload can be higher than the cost indicates. We always advise a managed server, as the cost of the server management work if outsourced will be far higher than the host's charge. Hosts discount their charges here and supply server management as a 'loss leader' - that is to say, they bear the cost as there are many advantages to them. These include:

  • Doing the work themselves for virtually nothing is preferable to later having to sort out big problems on servers that customers have trashed
  • Customer-managed servers are far more likely to host malware, thus ruining the hosts' rep all round
  • Malware at a hosted site might affect all of a range of IPs in one way or another
  • Customers cannot be trusted to get it right - the first thing to suffer is security 
Some hosts don't even supply unmanaged servers, for these reasons - and you can't blame them. The fact is, the people who are most likely to want to cut $20 off a monthly hosting fee in this area are those most likely to have problems on their server later. And those problems might affect the entire hosting operation in some way.

Dedicated server maximum loads

High traffic is always a hosting issue and has to be dealt with by hosting arrangements. The first stage for a high-traffic site is to host with a service that provides top-quality dedicated servers. This level of quality is not necessarily the most expensive: like all hosting issues, cost is no indicator of quality. Some of the most expensive dedicated server hosts have poor-performing servers.

Another factor is that hosts are notoriously unwilling to commit to any discussion of exactly how much traffic their servers will handle. To be fair, it depends entirely on the site, ie the webapps serving the site. However, they could easily quote an average for each type of site if they wanted to.

As they won't, here is our guide to the exact figures:
  • A good-quality basic spec server will handle 60,000 visits a day running a flat HTML website.
  • A good quality basic specification server will handle 33,000 visits a day running a PHP CMS. It will probably be right on the limit.
  • A good quality basic-spec server will handle 20,000 visits a day running a forum.
A good dedibox handles a PHP CMS like Joomla or Drupal with 33k visits per day (1 million visits a month). After all that's only one new visit every 2.5 seconds or so, and in practical terms it's an average of about 600 online at any given time.

And that's with 1GB of RAM. Yes, just 1GB. A good server of average size (nothing special in the way of CPU etc) only needs 1GB of RAM to perform to a reasonable level. However this spec is right on the limit here and there will be times when the site (or part of it) goes down - most likely when the server is running a backup. It really needs more RAM.

A low-quality dedibox starts to crack with 15k visits per day - and it can have cartloads of RAM - as much as you like. 8GB of RAM for example will make no difference. Trying to add more RAM to a poor dedicated server to fix the crashes seems to make no difference.

However a forum puts a heavier load on a server than a CMS, so the visit figures are slightly lower. A poor dedicated server, even with plenty of RAM, can start to crack up with as few as 12k visits a day, running a forum as the main webapp.

The server will most often crash when running backups or if you have big file downloads, as these take more server resources than simply serving pages. If you have popular file downloads and run a daily backup, then the server will soon show its limit.

If you have a good basic server with 1GB of RAM that handles 30k or 33k visits a day on a CMS, then upgrading to 4GB of RAM will provide a lot of extra capacity. When that runs out, it's time to go to a bigger server or load-balancing.

Top-class support
However it is possible in very rare cases, but only if you have one of the best server support techs in the world, to get over 100,000 visits per day from a CMS on one server. We've seen this done but you should note that these are unusual cases with very solid CMS, not overly extended, run by someone who is best described as 'not run of the mill'. This level of achievement is simply not possible for hosts' tech support or indeed most server techs of course. The 'rules' don't seem to apply to the best of the best.

Server cache
There is a 'memory pool' on the server, and also on dynamic webapps if you run one (ie CMS, forums), that holds recently-served pages in memory. To put off the date when you need to upgrade, you should very carefully test and trial cache timings on your main webapp and on the server.

This pool / reserve means that popular pages become closer to flat HTML pages, in that there is no server overhead to build them - they can be served immediately without timelag or load due to having to build them on-the-fly. This server load is appreciable and contributes in a big way to how soon you run out of server capacity. For example Apache server under ideal conditions (which never exist in production though) can serve 400 flat pages a second, but only 4 dynamic pages. There is also the varying load of exactly how many HTTP request are made for a page - forums are the heaviest here.

Therefore the cache timings are critical to server and site performance. Too short and there is no benefit; too long and the RAM gets overloaded, and page changes are not seen.

For practical reasons it may be better to go straight to load-balancing - a 'bigger' server will not normally double the load capacity, but load-balancing (almost) does. And of course, servers can then be added in order to increase capacity, as required.

Load-balancing

This is the method by which you and your webhost deal with increasing traffic to your dedicated server. There are several methods of increasing capacity, and these all depend in the first place on whether you have a database-driven site or not: 
  • Have 2 or more servers loaded with clones of the site - traffic is split between them - this works for HTML sites
  • Have 2 or more web-facing servers fed from one database server - this is how dynamic sites are set up
  • If you have static content (an HTML page website), you can also use cloudservers - this route is cheaper than dedicated servers, but you will need your own server tech

You cannot use cloudservers if you have a busy interactive dynamic site (DB-driven) as it is not possible to sync all the DBs. It can work for DB-driven sites with no content changes, though.

The next question is how you will feed the web traffic to the servers. Here are some options:

  • Split the DNS between 2 or more servers - each handles a proportion of the requests - no load-balancer is needed so it is far cheaper
  • Use RRDNS, round-robin DNS, which rotates the traffic sequentially between the servers - no load-balancer needed
  • Install a software load-balancer on one server, which accepts all requests, and routes requests beteen the boxes.
  • Install a load-balancer 'black box' - this is a safer system because it will only route requests to servers that are online, but costs from $150 a month and up 
Which route to take is entirely decided by your host as they have hardware issues to resolve here. All dedicated server hosts have a preferred solution, and you may find the first stage in load-balancing is described and priced on one of their website pages.

For example, they may well show the monthly cost for a load balancer plus two servers. You'll also need a switch.
 
Switches
Note that whenever you have more than one server box (it's possible to have several server apps on one box, which is different), you need a way to connect them. For normal PCs this is called a LAN, with servers it is called a 'switch' although it is the same thing. Always use a 'gigabyte switch', this is the fastest type - the cheaper / slower versions can create issues due to slow data speed, for example to/from a DB server.
 
All the servers connect to the switch (the hub), which routes traffic to all the boxes.

Network monitoring

The only way to tell your server status is to set up your own network monitoring service. The page load times can go bad, and the site can go offline frequently, and you will never know. However the commercial results will start to reflect the site problems.

A good network monitoring service (see our links page) will give a weekly report on page load times and the number of times the site was found to be offline.

The host may or may not employ status monitoring, for example with the Sonar system. They will react to offlines (eventually) and fix them. However due to workload and so forth, they are unlikely to rush to fix your issue. On the other hand if you employ a short-interval network monitor service, you get an email and/or text as soon as your site goes down. An email to the host then tends to put you at the front of the queue.

It is vitally important to note that hosts do NOT know what is happening to your site from the perspective of your site visitors - or indeed care about fixing issues like slow page loads or intermittent offlines, unless you note these issues and put in a support ticket to fix them. Hosts do not work proactively, you need to tell them what is wrong and ask them to fix it. This is why you need your own server support tech if you run a busy site.

Network monitoring frequency
A free service is available at 15-minute check intervals. This is sufficient for start-ups and non-commercial sites, but is not good enough for business sites. This is because a site can frequently go offline for 30 seconds or 1 minute, and the service will only see 10% or so of those offlines. It may report 5 offlines that week but the total may actually be 50 - which is not good...

A paid network monitoring service will be at 5-minute or 1-minute intervals, and this is what you need for a business site as it is the only way you can keep an eye on the hosting performance. They may quote an uptime of 99% but this is meaningless. Sites with overloaded database servers, or poorly-run Windows servers, often drop out for 30 seconds, dozens of times a week. A 15-minute check interval is not good enough to see many of these.

However your website customers, and search engines, will see far more of these offlines than you. And that will hurt you badly.

Alternative server software for dedicated servers

Apache server is normally thought of as the best software for shared servers. However it has to make some operational adjustments due to the job of handling many sites. Even though most shared servers have around 200 or 300 sites on them, there are some that have as many as 3,000 sites on them. And we have heard of economy hosts that are claimed to have peaked at 30,000 sites on a server.

In order to run this number of sites, the server app has to make numerous sacrifices simply for the ability to handle a large number of them. However, Apache is known to have less efficient handling of RAM and CPU cycles than is possible for a server designed to handle just one site, or a small number with the same owner.

Therefore, busy sites with high traffic, running on dedicated servers under heavy load, have been found to work better by using server software specifically for this task. You can see a list of these alt server apps on another page.

The most popular of these are Lighttpd (called 'lighty'), Nginx (called 'engine-x') and Zeus. Lighttpd and Nginx are open-source with commercial support available if needed, Zeus is commercial. Zeus also has an associated load-balancer as part of the suite. Some monster sites run these apps in preference to Apache, such as Ebay, WordPress.org and Amazon. However support may not be available at your own hosts, who may prefer to run Centos / RedHat Fedora - Apache, or Unix - Apache.

These server apps are members of the LAMP stack and run the same ancillary software in the same way. They use PHP / MySQL / Perl, and there is usually an htaccess file, for example. In most ways they are a direct Apache replacement.

Fedora - Zeus is a commercial combination, and Centos - Lighttpd / Nginx the open-source equivalent, that will become more popular for dedicated servers, as they are proven more efficient under load. Therefore hardware costs (eg for more servers) are reduced.

Litespeed is another LAMP server app replacement. It is a direct Apache replacement which means in theory Apache can be changed out for Litespeed (cPanel etc run on it), and is said to be between 50% and 600% faster than Apache. It is a commercial app with a free version to trial. The free version only accepts 150 simultaneous connections so that it has no use for busy sites. Litespeed can be used for shared servers as well as dediboxes, and it costs $500 to $2,000 to buy outright, depending on CPU type. There is also a load-balancer.

Nginx

Most server admins with experience of many different server apps will tell you that Nginx is the fastest. Remember, though, that there are drawbacks. The alternatives to Apache have pros and cons: 

  • For: much faster than Apache for heavily loaded sites
  • better for single sites as there is less 'baggage'
  • big cost savings as you need less servers, for sites with very high traffic
  • Against: these server apps have no WHM / cPanel - there is no control panel
  • they are entirely, 100%, SSH managed - that means via the console
  • the site manager has zero access to any server configs, stats or anything else - he is entirely dependent on the server tech
  • even if the site manager uses SSH, there is no GUI so the data is less useful
  • there are many htaccess file issues - special code may need to be written - plugins may have issues

Essentially, you are out on the bleeding edge when you install Nginx or similar. There are far fewer resources than for Apache, and community support is not good for many of these apps. Our advice: don't do it unless you have a server tech who can write custom htaccess code from scratch, that does not conform to Apache conventions, without blinking an eye.

Your own server tech

You may have deduced from some of the previous information that we recommend you do two things if you run a busy site: 

  • Set up network monitoring
  • Get your own server tech
It's a simple fact that hosts' tech support cannot work proactively because they are under too much pressure, since they are running hundreds of sites. You need your own guy to tune the server and your main webapp to its maximum potential. For example you can find a vBulletin forum and server ace, or a Drupal CMS and server ace, who would be able to set up your server to perform perhaps as much as 100% better than the hosts' best efforts. In fact we have seen over double the capacity improvement in visits on one server that was set up by such an ace tech, even though the server was with excellent dedicated server hosting - but of course such personnel are not easy to find. Another example: an ace Drupal/server admin can get 100k visits a day from one server. If the box is set up by the hosts, you would most likely get around half that capacity - or less.

 
Web Business Managers