Estimating WordPress traffic on AWS servers.

5 mins read

Estimating WordPress Traffic on AWS medium servers.

When we first shifted to AWS I wondered how much capacity we really needed and was there a mechanism that I could use to right size our servers? Where would you start if you had to estimate traffic on AWS servers. Given a choice between small, medium and large, which server family to opt for? How many visitors would my AWS server be able to support if we go featured on Hacker news, Techcrunch, Reddit, Mashable or CNN. Would we be left standing if they all wrote a piece about us?

The other interesting variation is when your servers get hacked or hijacked. How much bandwidth and CPU cycles would  a site with 1,000 visitors a day consume? CPU cycles and bandwidth theft is here to stay. If you don’t know the answer you could be in for a nasty surprise when your server hosting and bandwidth bill comes in at the end of the month.

A recent walk on the server optimization side lead to some basic insights. These are really crude approximation and before you take a commercial decision based on these, talk to a professional or do your own maths.  But they provide a basic framework for estimating traffic loads under normal as well as peak conditions.

WordPress Server setting and context

We run a portfolio of content rich WordPress sites hosted across one small and one medium AWS m3 server with 3.75 GB of RAM and a 60 GB SATA hard drive. The configuration runs Ubunut, MySQL, Apache. Our primary site receives between 600 – 800 visitors a day and servers 2K – 3K+ page views per day.  Peak load gets as high as 70 – 100 visitors / 400 page views an hour when we get featured or some of our content gets retweeted or shared on social media. When we run marketing campaigns numbers jump to as high as a few hundred visitors per hour.

Estimating WordPress visitors and traffic load

Every time a visitor arrives Apache spawns a child process to handle the request. If you run htop you can get a presentable breakdown of what the load on your server looks like.  Against each Apache process you can see the resource requirements under the RES column. The resource number for this specific instance range between 7MB to 35MB though I have seen processes grow as much as 50MB. The variation   This resource estimate is our first input. If you need help in estimating the average size here is a neat command line script that will do that for you.

HTOP output-childprocess

We use the process size and the number of visitors to estimate the ability of our servers to handle peak load as well as a theoretical maximum number of visitors they can support.  It is a three step process.

We first estimate the server load generated by each visitor.

WebsiteTrafficEstimation-Process

Then adjust the number of visitors for a per minute, per hour and per day load.

TrafficEstimation-visitorsload

Then multiply the visitors per second by the average page size to estimate the bandwidth requirements.

WordPress traffic estimation models

To crudely estimate traffic load we can run two separate models. The first assumes that as traffic rises MySQL load is minimal and we won’t use anything beyond the buffers and caches allocated to MySQL. The second assumes a marginal load assigned to each MySQL thread.  The first model allocates all available memory to Apache. The second would distribute available resources between Apache and MySQL. We show results for both but use the conservative estimate for site traffic.

To begin we need to estimate free memory on our server before it gets hit by a wave of traffic. Command line free -m does that for us and produces the following output.

Available memory

Depending on how you slice the numbers (and free buffers) we have between 2.9 to 3.4 GB of available memory on our server.  How much load can this support?

Here is the simple model.

TrafficLoadEstimation-1

The conservative estimate assumes that there is a MySQL overhead and at max our server will only be able to support 73 concurrent threads or connections at a given point in time where each thread or connection maps to a visitor (not true, but it serves as a crude proxy).

Assuming an Apache child process stays alive for 3 seconds gives us the capacity to support fifteen hundred visitors a minute, ninety thousand visitors an hour. That is more than sufficient for the traffic load that we have been witnessing or expect to clock if we ever get featured by Hackernews.

The three big drivers here are process size, MySQL thread size and process life.  If you change these to 50 MB (bigger pages), 40 MB (dynamic content) and 4 seconds (lots of requests), the revised profile becomes:

TrafficLoadEstimation-2

Changing the three variables brings our traffic capacity down to a third of our original forecast. So theoretically speaking an AWS M3 medium server under conservative conditions should be able to support somewhere between 1.5 million to eight hundred thousand visitors a day. That is a bit excessive and we wonder if we would be better off with something a wee bit smaller.

But this is before any bandwidth utilization considerations. When we plug in an average page size of 2MB we see the bandwidth load at:

Trafficloadestimation-3

A wowza forum thread shares some results on bandwidth load testing on AWS EC2 environment.  So the estimated MBS load on the higher side is just within the AWS range for a medium sized server. The estimated numbers also tie in with results we have been observing under peak traffic conditions on our site. If you push site visitors higher you may run into some bandwidth bottlenecks.

AWS-Bandwidth-Estimates

Extending the same traffic estimation and modeling approach to a small AWS EC2 instance suggests that under ideal conditions a small instance should easily support between 4,000 to 8,000 visitors per day. Which is more like loads we are likely to experience. Which then suggests that if you have a farm of low traffic sites than a small EC2 instance should be more than sufficient to host as many as 10 – 12 such sites on one server.

However there is a catch. If traffic spikes on multiple sites or if you have a plug in heavy implementation you may run into trouble. Even relatively visitors statistics (15 – 30 visitors a minute, a visitor every alternate second) can trigger bandwidth requirements exceeding 25 – 30 MBS.

One load testing metric is that your server should be able to scale between 3 – 6 times your average traffic. So if you average about 1,000 visitors a day, your servers should have sufficient capacity to scale to a peak load of 3,000 to 6,000 visitors per day. As per this rule of thumb a small server should suffice for sites averaging under 1,000 visitors a day under both normal as well as peak loads.

AWSEC2SmallServerTrafficLoad

That is all there is to estimating WordPress traffic on AWS servers. To summarize. An AWS EC2 m3.medium server will support anywhere between 1.5 million to 0.8 million visitors a day depending on how your site is configured and the type of content you are serving.  A small EC2 instance with 1 GB of RAM will support traffic loads between 4,000 to 8,000 visitors per day. If you need more capacity opt for the m1.small server that comes with 1.7 – 2GB of RAM which should scale nicely. While you won’t run into any bandwidth bottlenecks, speed and server response may be an issue at that level.

Your numbers vary depending on how you have configured and optimized your servers and to tweak better performance you need to manage the size of your Apache processes, MySQL threads and KeepAlive settings. If you do a significantly better job on tuning the above, you may run into some bandwidth issues since the m3.medium configuration will support about 140 MBS and the m1.small will do about 100MBS. We have seen throughput touching 30 – 50 MBS on loads as low as 20 – 30 visitors per second.

Finally before you commit to an AWS instance you should also read the datadog report on common AWS performance issues.