top image

Speed Optimization Primer

Benefits, finding bottlenecks and tools for speed optimization

Speed optimization primer

Optimizing websites and web applications for high speed and fast page load times has never been more important than now.

There is a huge and increasing gap between what users need in terms of speed and what the average websites and web applications are giving them. That gap represents big business opportunities and big potential savings in hosting equipment.

In this article you will get an overview of:

  • The potential benefits of speed optimization
  • The state of website speed out there
  • How to find the biggest bottlenecks
  • Tools and methods for testing website speed 

In the article front-end optimization you will get more specifics for harvesting your speed potential.

 

Definition of front-end and back-end

Like it is common practice in the speed optimizing community, we will use following definition:

Back-end is First Byte, the time it takes from the user started navigating to the page, until the first data from the server arrives at the user. Front-end is everything else.

In this definition, back-end includes DNS lookup and initial connection time, even if those have nothing to do with back-end servers.

In other words, back-end time is close to the time it takes for the browser to receive the main HTML document, except the actual content download of the main HTML document is considered to be front-end.

The waterfall diagram below is generated from www.webpagetest.org. It shows a subset of the requests on the home page of LinkedIn.

Around 780 ms is in the back-end and includes DNS lookup, initial connection and SSL negotiation, as LinkedIn runs on HTTPS. The rest of the 3.5 second page load time, only partly visible here, is in the front-end.

The back-end/front-end time distribution for linkedin.com is close to the typical 20/80%, which is the reason it is often in the front-end that most improvements can be harvested:

Waterfall Diagram Linkedin

Finding the areas with the biggest bottlenecks

To figure out where to begin and what areas holds back the speed of your web application, we recommend that you record following 3 metrics on first-view from webpagetest.org using your typical users Internet connection speed and distance from the web servers:

  • Page Load Time
    Record just Document Complete on first view for now. This is when the browser considers the page loaded and rendering is completed. It means that the page is ready to use. Everything above 1,0 seconds is potential for improvement. Above 4,0 seconds is a very nice oppertunity for improvement :-)

  • First Byte
    First Byte is by definition the back-end time. It is the time from the user started navigating to the page until the first data from the server arrives. It contains the DNS lookup, the initial connection and TTFB (Time To First Byte). High First Byte values will indicate connectivity problems or inefficient/busy back-end. The First Byte value should be below 300 ms and fast values are in the 30-100 ms region.

  • TTFB (Time To First Byte) for the main HTML document
    Time To First Byte is the time it takes for the server to respond to a received request. TTFB is after the DNS lookup and after initial connection. A high TTFB value is an indicator of an inefficient or busy back-end. The TTFB value should be below 250 ms and fast values are in the 20-80 ms region.

Now have a look at those 3 values to see what is worth optimizing first:

  • TTFB is high
    The back-end is holding your application back. Either it is too busy or it is ineffecient.

  • First Byte is high, TTFB is ok
    Check for DNS and connectivity problems to the web servers.
     
  • First Byte is ok, TTFB is ok, Page Load Time is high
    Front-end is the main problem, not the back-end and not DNS/connectivity. Go to front-end optimization to figure out what can be done.

  • First Byte is high, TTFB is high, Page Load Time is high
    Both the back-end and the front-end is holding your application speed back.

As an example we will use a large Danish ministry website which will remain anonymous, let's call it example.dk. This website is neither better nor worse than most big websites out there. It just serves as an example of what can be done in terms of speed optimization.

For example.dk, the speed test tool webpagetest.org give us these numbers from the measuring point in Stockholm, using a mainstream 5/1 Mbit Internet connection on the Chrome browser and taking the average of 5 runs:

Page Load Time: 3.27 seconds
First Byte: 0.44 seconds
TTFB (Time To First Byte): 0.39 seconds

From these example numbers it is clear what to focus on:

Page Load Time of above 3 seconds is far too high. There is plenty of room for improvements here, as the goal is 1.0 second page load time.

DNS/connectivity seems ok, as the difference between First Byte and TTFB is low.

Clearly the most gains in this example are to be had in the front-end, but the back-end could also do with some optimization, where we are looking at a 200-250 ms speed improvement potential.

 

Testing tools and methods

There are many tools for speed testing. We recommend following tools for testing page load speed:

  • www.webpagetest.org
    An online tool capable of testing speed from different locations around the world with different Internet connection speeds. It is one of the few tools that will give you measurements of perceived speed with a metric called Speed Index.

  • Firebug
    A plug-in to the Firefox browser. It will give you the classic waterfall diagrams and a host of other things.

  • Yslow
    A plug-in to Firebug in the Firefox browser. It will give you speed grades, a list of speed issues and aggregated reports of objects loaded.

  • PageSpeed
    An online tool also available as plug-in to Firebug and Chrome. Similar to the Yslow plug-in it will give you speed grades and a list of speed issues with recommended fixes.

Be aware how you use the tools and how you test speed. To get speed testing results in usable absolute numbers, comparable to what your users experience, use webpagetest.org. To find and analyze speed issues and to record relative progress, use Firebug, Yslow and PageSpeed.

 

Speed optimization is rarely a topic for companies (but it should be)

Speed optimization is not hard, but it does of course require some basic knowledge. A technical- or development minded person will be able to understand and use for example the basic front-end optimization principles in less than an hour. The many finer details that make up the last 20% potential will take a while longer to master, but it is still far from rocket-science.

The most important obstacle in making faster websites is not that it is hard to understand the optimization principles or hard to implement them, but the fact that speed is not even a topic for most companies, their developers or their managers.

In stark contrast we have the millions of daily visitors for whom the speed of the websites they are visiting – or the low speed rather – is very much a topic, and a reason for frustration and abandonment.

So why is speed optimization not popular among companies? There are probably many reasons. To list a few of them: 

  • High speed is rarely noticed
    Like good usability, fast responding websites rarely leaves a wow-impression. The high speed is more likely to go unnoticed, because the high speed just removes issues and obstacles for users.

  • Most developers do not know about front-end speed
    The average developer is probably good at optimizing back-end code. It is part of their trade, education and training. Front-end optimization techniques on the other hand is typically not part of their education at all. The increasing use of popular bulky coding frameworks and libraries are not helping speed optimization either.

  • Nobody set goals for speed
    Because speed is rarely a topic, high speed is rarely listed as requirement when the systems and platforms for the website or web application are being specified. Or if it is, the requirement is not listed in easily measurable metrics, rather in subjective terms like “the application should be fast” or similar.

  • 'It's fast enough'
    It rarely is, of course. If the bulk of your user base experience sub-second page load times, then you might have an argument for the claim of 'fast enough'. But be aware that there are still good speed benefits to be harvested below the 1 second page load mark.

 

Think of speed as a way of doing things, not a project in itself

Consider approaching speed optimization as a way of doing things, a mindset, not a project in itself. Just like writing semantically correct and validating HTML code, just like writing structured object-oriented CSS code and just like applying good usability practices in UX.

Following good speed optimization practices will for example mean that some solutions and functionality in your web application should be re-written, implemented in a different way or that other solutions need to be found. Just like in many other areas of application development.

Consider also to adopt speed optimization in QA to make sure your speed efforts remain on track.

 

Further reading

Benefits of Speed Optimization

Many different tests have shown that slow responding websites is a reason for users abandoning websites.

A research conducted by Forrester Consulting and published by Akamai in 2009 highligted following for visitors to Ecommerce sites:

  • 47 percent expect page load time below 2 seconds
  • 40 percent will leave the site if the page load time is above 3 seconds
  • 52 percent expressed that fast page load times have an impact on their website loyalty

Now take a look at the page load times of the Top 100 Ecommerce websites in the world plotted in the diagram below. 96% of them are slower than 1.0 second, 60% are slower than 2.0 seconds, and 17% are slower than 3.0 seconds .

Diagram Page Load Speed Ecommerce Top 100

So if you have an Ecommerce site with a page load speed slower than 2.0 seconds, you disappoint 47% of your visitors. If your page load speed is slower than 3.0 seconds, 40% of your visitors will leave your site.

This also means that Ecommerce sites with 3.0+ seconds page load times have an additional sales potential of 40%, just by reducing page load speed. 

Similar patterns exist for blogs, but with even more improvement potential. Below is page load times for the Top 100 Blogs in the world. All are slower than 1.0 second, 97% are slower than 2.0 seconds, and 79% are slower than 3.0 seconds.

Diagram Page Load Speed Blogs Top 100 

It follows that 79% of the blogs out there, maybe (likely even) yours also, have a conversion improvement potential of 40%. Not bad getting 40% more subscribers, followers or whatever your main blog conversion is, just by reducing page load speed, is it?

On top of the business potential from fulfilling user needs, speed optimized websites and web application have the added benefit of:

  • Putting less load on application- and database servers. It is not uncommon that the number of application servers can be cut by 50% or more as a result of careful speed optimization.
  • Reduced traffic consumption, reducing cost for you and your visitors. Up to 70-80% reduction is not uncommon.
  • Better SEO (Search Engine Optimization) as page load speed since 2010 is part of Google's algorithms for ranking pages.

 

Speed potential and about setting goals

For the human interaction with systems like websites and web applications, usability expert Jacob Nielsen highlighted in 1993 a set of time limits when optimizing for speed.

These time limits has since been recognized throughout the usability community to be universally applicable:

  • 0.1 second
    Instant response in the user interaction

  • 1 second
    Delay is noticed, user remain focused

  • 10 seconds
    Delay is making users change focus and many will leave

Others have been suggesting 2.0, 3.0 and 4.0 second limits and shown how many users will leave the application around those limits. The fact remains that above 1.0 seconds, some users' focus will start to drift away, some will reach the limit of their patience, and some will leave.

It is perfectly possible to optimize web pages to load in less than 1.0 second. Some highly optimized websites even have page load times below 0.1 second. Most websites and web applications however have page load times in the 1.0 – 10 second range, yours probably also. So there is plenty of room for improvement.

The question is how much business value you will get from improving speed of your website or web application. And the next, what speed goals would be sensible to aim for.

Following are business results from speed improvements reported by some of the bigger brands out there: 

If you need more solid facts than these, there are methods to find out what a speed improvement of your website or web application will give you in terms of business value:

  • Method 1 (easiest)
    If your web analytic tool is recording page load speed, use it to segment visits in page load time brackets, like 0-1,0 second, 1,0-2,0 second, 2.0-3,0 seconds and up. Make sure the segments contain new visitors and visits from the same device and the same OS. Now compare user satisfaction metrics and conversions between the timing brackets.

    The downside of this method is that your segmentation might not be representative of your users. Still, the method can give you clear indications for the significance of speed.

  • Method 2 (easy)
    Add a JavaScript on your pages that will block page rendering for a set amount of time, for example 1,0 or 2,0 seconds. Now record and compare user satisfaction metrics and conversions.

    The downside of this method is of course that you will record the effect of slower page speed only, not faster. It will however give you a clear indication how much user satisfaction and conversions depends on speed.

  • Method 3 (harder, more accurate)
    Make a handcrafted web page with the same UI and design as one of your existing high traffic web pages (usually your home page), but make it optimized for high page load speed.

    Make your back-end system switch between the 2 variants, or use a tool like Visual Web Optimizer to control the switching. Or let a load balancer do the switching for you.

Now record and compare user satisfaction metrics and conversions for the variants. You also have the option of controlling the speed range of the optimized page by using the slowing JavaScript from method 2).

Armed with facts about the business benefits of speed, you can now hold them against the needed investments for making your web application faster.

Recommended end-goal: Get below 1.0 second

The recommended end-goal is fascinating clear: Get below 1.0 second page load speed for the bulk of your user base.

You might want to set intermediate goals like 2.0 seconds page load time, if you are challenged by restrictive systems and platforms.