Web Performance Watch

A 10-Point Checklist To Ensure Site Uptime When Switching Web Hosts

There comes a time in every website's life when it must grow up and leave behind the infrastructure that got it to where it is. As traffic scales up, site uptime and speed become paramount; Web hosting companies are compared, and finally it is time to move house, i.e. to switch Web hosting companies. Moving sites without a glitch is important, even critical, and with some experimentation, I hit upon a checklist that should help anyone plotting a move. Follow these steps and your site will remain up throughout the move and your site visitors and online business teams will thank the IT department for its forethought.

1. Move When the Traffic is Low. Just like the maintenance crews working on our freeways between 2-5 am, find a 8-12 hour window where it matters less if someone can't get to your site, or if you can't get to your email. I started Friday evening, with a glass of wine in hand, but decided to only start the DNS name server move on Saturday morning, when I would be wide awake and the developers available to fix problems.

2. Switch Your Email Profile at Both Web Hosting Companies. Let's say your domain is mysite.com, and your email address registered at the Web hosting accounts is me@mysite.com. Change that to a Gmail address, because if, during the name server move, if your domain's email goes down, you will at least be able to create support tickets and respond to them using your Gmail address.

3. Create a Mirror Site and Monitor It. First, completely replicate your site from Web hosting company A to B, and setup site performance monitoring on both sites. Run it for a few days to ensure that there are no content, database, or networking problems. 

4. Copy and Import Your Zone File. Your DNS entries are stored in a file called a Zone file. Find out if your current Web host gives you the ability to export that Zone file, and if so, import that Zone file as is at your new host. Sometimes, you can't do this, in which case you will need to manually recreate the DNS settings on the destination Web host. Remember to print out your DNS records so that you know exactly which A, CNAME and other entries you need to make.

5. Setup a Virtual Host at the Destination Server. Since you already setup your mirror site at the destination host, all you need to do is create a "symbolic link" to your domain. As an example, if your original site is mysite.com, and you setup a mirror site mysite-copy.com, what you need to do is to setup your destination servers to accept visits to mysite.com, once the DNS move begins. I was using Linode, and while they didn't specifically give me the recipe to do this, I configured a virtual host to point to the servers that I had already setup at the destination Web host. Linode in particular has great documentation, so it's easy to figure it out.

6. Start Your DNS Move. Once you're ready, go to your source Web host, and change your DNS servers for your domain - this begins the move! Pour yourself another glass of wine, and keep the bottle handy for refills. My poison of choice was a Chianti Ruffino.

7. Monitor the Move Using Website Performance Monitors. If you have setup 24x7 performance monitors that robotically visit your site from locations around the world, expand the geographies from which you're monitoring your site. You want to know if there's a problem in a particular city. I used Keynote and, for extra measure, Pingdom, to keep an eye on both performance and availability of the sites. Something I was able to do with Keynote, but not Pingdom, was to alert me if content was missing on the site after the move. 

8. Setup Availability Alerts to Fire If Content is Missing. Site availability alerts usually fire off in Keynote or Pingdom if your site is completely down, or (in the case of Keynote) if page performance falls below a threshold you previously set. I configured Keynote alerts so that if a page was missing content, that should be considered as if the site were not available. Initially, Keynote alerts were firing off like crazy, as the picture below shows - all the yellow data points show individual tests Keynote made, and where the base page returned successfully, but was missing content. 

Drilling down into any of the yellow data points showed me where the problem was - a variable was undefined, and as a result, incorrect content was being displayed on the page. Keynote told me that that there was a JavaScript error somewhere, and we quickly eliminated it by editing the code. 

I also discovered from the waterfall charts (performance timeline charts) that a JavaScript file was missing, and the developers were able to correct that rapidly as well.

9. Conduct a Traceroute Test from All Over the World. I remembered that Keynote allows you to run network diagnostic tests, such as ping, traceroutes, and TCP traceroutes, from any of their thousands of agents located in cities all over the world. You can only do a test from 16 locations at a time, but that was enough to show me that DNS remapping had not completed, and therefore, the site was not resolving to the servers at the destination host in all locations. In the tests below, you can see some of the windows where the traceroute resolved to my old Web host, and so I knew that those locations were not seeing content from the new Web host.

 

10. Tweet and Enlist Your Customers. Finally, as the DNS change was propagating around the world, I also asked the site developers to throw out a tweet to thousands of their followers, and enlist the help of the crowd to check site uptime from places all over the world. Keynote actually has a crowd-testing tool to allow you to do this, called WebEffective, but I took the free route. Within minutes we got email reports from followers, telling them that the site was up or not, and we were able to isolate problems to a handful of ISPs, where, apparently, the DNS propagation had not reached, 24 hours after the move.

And that's all folks! These 10 steps ensure that your website remains up and running during any change to its infrastructure. Follow them, and mimize or eliminate all downtime. And, in tribute to the Italian wine that keeps you company during the website move, Cin Cin!

Understanding the Impact of Web Attacks - the User Perspective

Recently, Keynote was asked to comment on a widely publicized attack against a company online. The attack, known as a distributed denial of service, was targeted at a network of web servers primarily located in Europe. DDoS attacks against websites have occurred periodically for years. Unlike past incidents, the nature of this particular attack was characterized as severe enough to have caused broad disruption on the Internet. The suggestion made was that users like you and I may have found our email delayed and websites either slow or unavailable.

While many methods exist to understand the impact of web attacks like this, Keynote's web performance monitoring service and global network provide a unique perspective--the end users' perspective. Our business is monitoring websites for our customers so that they have a consistent and accurate source of information about their site's performance. But we also use that same technology to monitor select sites across the Internet and make this data publicly available in the form of an index--actually, 43 indices. And we also use it to provide another free service called The Internet Health Report that shows what's happening across the major U.S. Internet Service Providers in real time.

Our Performance Indices allow us to "see" whether the Web is slow in Europe, for example. This is because we periodically test a wide range of websites from our network of hundreds of monitoring agents connected to different ISPs all over the world. These agents pretend to be a site visitor and measure performance during their visit.

One of our indices monitors the U.S. banking sector. The financial services industry has been hit hard of late by DDoS attacks. And we've been seeing the impact in terms of the inconvenience these attacks have imposed on e-banking customers. One way to look at this impact is by the availability of the banks' home pages. This graph shows how home page availability for the 15 banks we monitor has trended over the past year:

A site's home page might go down due of a number of reasons other than an attack, including things like planned maintenance, or an unprovoked failure in the server/network/application. But this graph shows a strong correlation between when the recent wave of DDoS attacks on the banks were reported to have started (September, 2012) and outages.

Our Performance Indices are a great source of information for benchmarking and comparisons. But in order to use them for real time comparisons, like whether the rest of the Web is experiencing an issue that you might be observing (perhaps with users in a certain region), our weekly web tables are not very helpful. The good news, is that our KB40 index can be accessed in real time programmatically. Read our post on open API access to the KB40 index for step by step instructions.

 

How Do I Compare Thee? Linode vs Bluehost Web Host Performance Shootout

If you are evaluating or switching hosting companies for your Web site, don't just ask for an SLA guarantee, but do a site performance shootout before you cut the check. Recently I wrote a post on Beta Program, a site that I created to highlight how businesses can use Web technologies to run their operations better, for everything from using Web-based accounting software to building a better, faster website. The article, Mirror Mirror On The Wall, Who’s The Fastest Web Host Of Them All, demonstrated the performance gains I experienced when moving a website from Web hosting company Bluehost to Linode. For that article, I had focused on the overall user experience, concluding that the same site performance fluctuated between 1-2 seconds for Linode, compared to between 2-4 seconds for Bluehost.

A commentator asked "Vik– Just curious, when you ran your monitoring experiment comparing Linode to Bluehost, did you notice any trends in the performance details? Like significant differences in DNS lookup time, versus time to first byte, versus content components. Just wondering if the data reveals any specific “soft spots” with Bluehost?" I thought it would be useful to share my findings from this experiment on the Keynote Web performance blog. 

Methodology. I took a website that was built on the LAMP stack - Linux, Apache, MySQL, and PHP - and duplicated it on both Bluehost and Linode Web hosts. In both cases, I used the lowest plan that was available, and with Linode this gave me a Virtual Private Server (VPS). I made very minor modifications to the site to make it work right on the Linode environment, along the lines of changing the value of some variables that referred to the root URL. Then, I used Keynote's IE browser monitoring agent to run measurements every 5 minutes from the US-8, a group of 8 cities in the US. You could use other products as well, including the free WebPageTest service, which I used to create the video of the two sites.These were all high-speed, high-bandwidth connections, and I was able to ensure that I had a clean lab of performance monitoring agents to conduct this test. My assertions here are made after observing over 2000 datapoints per day on each website, from Mar 6 until today, for about 3 weeks. For all datapoints, I used Arithmetic Mean (though I could have used 95th percentile or median or geometric mean if I so chose, and if I remembered high school math better).

Visually Comparing Site Speed. Using WebPageTest, I first ran a site speed comparison. This measurement was taken from a server in Dulles, VA, and the video shows how long it takes for the two sites to load in the same browser. It's a great first step to help you understand the site visitor's experience. If you don't see the widget below, watch it on Youtube.

 

 

Measuring the User Experience Time. UX time is the time elapsed from when the browser started navigating to the page until the browser finished loading the page contents. This includes DNS lookup time (the time it takes for the browser to translate the URL you typed in to a host address), time to process all JavaScript on the page, to load the base HTML page, and all objects on the page such as images and JavaScript files. Here is how the two hosts stacked up over 3 weeks:

The site performance on Bluehost fluctuated between 2-3 seconds, and on LInode between 1.5-2 seconds. To answer the question on whether there were "soft spots" in Bluehost, I calculated the arithmetic mean on each of these performance metrics as well:

As you can tell, the big gains are in the network and server infrastructure. For example, it took Bluehost 448ms, almost half a second, to return the first byte of the Web page, whereas for Linode it was 39ms. If you look at all the content downloaded, there isn't much of a difference between the hosting companies - Linode is only 4% faster (though, at almost 1.5s, there is room for improvement with speed optimization on the website content itself). 

Linode recently wrote about its network upgrades in an initiative called Linode Nextgen. It appears that their work is paying off, and if you host a website, you and your users would be well served by Linode.

Super Disappointing

The performance community has grown quite a bit over the past few years. Web Performance meetup groups now can be found in almost every large metro in the U.S. and around the world. The Velocity conference continues to grow with another venue added this year. And vendors have been providing better technology to help site owners monitor and improve performance. The result: many of the top websites we visit regularly deliver a pretty solid user experience.

It’s heartening to see that web performance seemingly matters to web businesses, and that the value of high performance has become understood.

 

And then the Super Bowl happens.

Each year’s edition of the big game represents a huge opportunity for advertisers to capitalize on their significant investment in air time by engaging consumers beyond the television—on their laptops, tablets and smartphones. Advertisers know that this produces huge surges in traffic, and without proper planning, can create a very poor customer experience.

This year, we closely monitored the advertisers across all 3 screens to see how well they prepared to engage with consumers. Here are the results, and some observations which should serve as reminders of how web performance considerations are just as important today as they were 5 years ago:

1. Mobile performance is tragically misunderstood

Over in our sister blog Mobility, we’ve talked about the differences between traditional desktop Web performance, and mobile performance. What seems clear is that most companies still do not understand how forcing the desktop experience down onto a smartphone impacts the end user experience. With no other Super Bowl advertiser is this more clear than Century 21 Real Estate. Their promotional site was not optimized for the smartphone, sending a staggering 94 objects to the browser. Because our smartphone testing agents connect to wireless carrier networks, the average response time during the game was over 54 seconds, compared to the 2.97 second average measured by our Internet Explorer agents. Such a huge difference illustrates how important it is to treat the mobile experience uniquely—even for tablet users.

On the other end of the spectrum, Axe presented a nicely optimized site for smartphone browsers. Coming in it nearly 7 seconds, Axe maintained a low page weight of 270KB. In contrast, one image on the Century 21 page was over 1,000KB. Axe also decided to provide their tablet visitors with the optimized smartphone site, instead of the fuller featured desktop site—emphasizing their focus on a high performing experience.  

2. 40 million viewers… you should probably run some web load tests

While the 49ers repeated failures on defense and special teams were shocking to many fans, the most surprising crash was that of the Coke website.

You can see how the site availability deteriorates rapidly once the ads aired.

Fans of the iconic brand took to Twitter to vent their frustrations:

Coke reported that its campaign was a success by many measures. However, their public statements also reveal how significantly the brand probably under-estimated capacity for the website component of the campaign. Keynote worked with other Super Bowl advertisers to test their sites in advance of the game by generating real traffic from the Internet—a practice known as Web Load Testing. Those advertisers planned for 1.5 – 3.0 million site visitors. Coke’s estimate was less than 1 million. Doing a bit of ‘Monday morning quarterbacking’ here, this low estimate likely explains much of why Coke’s sites performed so poorly during the Super Bowl.

By contrast, the online retailers (having weathered many a Cyber Monday) such as Best Buy performed flawlessly with 100% availability. Best Buy also delivered some of the fastest response times across all 3 screens.

3. The (good) old rules still apply

It was surprising to find that many the advertising sites did not make use of CDNs and common optimization techniques. It’s clear many site owners can benefit from taking a fresh look at their pages. So revisit those web performance best practices and take an inventory of how well your site is delivering end user experience.

 

Cyber Monday Breaks Records, But Could It Have Been Even Better?

Oh, what a difference a day makes!

By many accounts, Cyber Monday was another record setting day for online retailers. Some estimates have suggested that total sales reached nearly $2 billion! At the handmade & vintage goods marketplace Etsy, more sellers completed deals on Cyber Monday than at any time in their 7-year history. And IBM analytics reported that 18% of all site traffic came from mobile devices.

This is great news for e-commerce companies. But for many, Cyber Monday was not the smooth sales day we observed in our retail indices for Web and mobile performance over the Thanksgiving/Black Friday weekend.

The average Total User Experience Time for home pages in our Top Retailers index was 2.98 seconds, 7% slower than the previous day. As the amount of traffic surged in the afternoon and evening hours, many sites began to struggle. In the graph below, we’ve overlaid the real time sales volume that IBM tracked throughout the day against the hourly average of Total User Time for the home pages in our index.

Although the number of retailers represented in the index is relatively low, there does appear to be a correlation between increased load (sales volume) and performance. We’ve observed a similar correlation in the past.

Overall site availability was high, however, some retailers experienced significant issues. Third party content failures appeared in many measurements, and back-end problems appeared in others. The Saks Fifth Avenue website was unable to successfully complete transactions for nearly 2 hours from 1:39pm ET to 3:26pm ET.

Their Twitter feed confirmed the site’s difficulties:

Keynote tracks the performance of 22 different retail transactions which allows us to monitor the user experience of someone visiting a site, searching for a product and then adding it to the shopping cart. The data from these transactions inclusive of Cyber Monday will be available next week.

Our Mobile Commerce index also observed that mobile website performance slowed across the board for home pages accessed on smartphones. At a couple points during the day, average page load time exceeded 18 seconds, 2X the previous week’s average and well below user expectations.

Mobile sales nearly doubled this year to 13% of all receipts online. But given the functionality issues our recent study of several major retailer mobile sites revealed, and the slow response times of the sites in our index, mobile commerce still has a long way to go. This may be why bounce rate for mobile is 20% higher than the desktop bounce rate (39.42% vs. 32.77%).

Overall, retailers performed well this Cyber Monday—especially for their desktop customers. Record setting traffic and sales volume is the ultimate testament to the preparation and execution of web operations professionals across the retail industry. But of course the holiday shopping season has only just begun!

Retailers Zip Through Black Friday

U.S. online retailers once again set record sales for the long Thanksgiving weekend and we’re already receiving reports that today—Cyber Monday—is setting up to be massive as well. Looking at the performance of major e-commerce sites reveals that most sites were well prepared for the heavy traffic and transactions of the past four days.

The average Total User Experience Time across the Keynote Top Retailers index was 2.77 seconds and Time to Interactive Page coming in at 2.27 seconds. Average Time to First Paint—the point at which a new user first sees content in their browser—clocked in at 789 milliseconds.

While retailers across the board should improve their initial impression with first-time customers (250 milliseconds is recommended best practice), most home pages were complete or perceptibly usable within the recommended tolerance of 3 seconds.

A few retailers, however, did experience some performance issues. Keynote’s Retail Apparel transaction index showed that starting at 8:05 a.m. PT on Black Friday until 8:40 a.m., the Foot Locker site struggled with internal server errors. Transactions during this period could not be successfully completed. You may have been able to get to the home page, but performing an item search or adding it to the shopping cart would not have worked.

But while Foot Locker’s site may not have been completely prepared for the onslaught of Black Friday traffic, most of the other 47 sites in the Top Retailers index experienced no major outages during the weekend. The number of Black Friday shoppers grew 18% over last year and the preparations of online retailers appear to have paid off—to the tune of a more than $1 billion!

Keynote’s monitoring of retail mobile sites also showed that most sites performed consistently through Black Friday. As with the desktop, these sites optimized for smartphones and tablets delivered comparable performance to non-peak periods. However as we had previously reported, the performance of most retail sites on smartphones across major U.S. cellular carriers was generally slow, relative to user expectations. Our Mobile Commerce Index results for the week will be published shortly in Internet Retailer with more detail.

We’ll wrap up our online retailer performance analysis for Cyber Monday tomorrow.

Firefox & User Experience Metrics, Now in KITE

We're happy to announce the latest release of KITE--the Keynote Internet Testing Environment. This upgrade includes several new features, most of which have come from the direct user feedback we receive at our Web Performance Community forum. One especially noteworthy improvement is the inclusion of the Firefox 12 browser instance for recording and playback of Firefox-based user interactions.

Firefox 12 allows KITE to capture User Experience metrics, just as it has for several months now on IE-based scripts and test runs. Browser event timings are visible in the Transaction Performance Details pane:

For a full list of all the enhancements in this release of KITE, visit the Community.

Most importantly, be sure to upgrade your copy of KITE today! If KITE doesn't remind you itself, you can force an update by selecting Options -> Check Now.

Happy testing!

From Network Performance to User Experience

Earlier this year, Keynote announced the addition of User Experience metrics to our web monitoring products.  These metrics allow you to broaden the conversation about web performance.  In addition to seeing the performance of the "bits and bytes" going over the wire, you can now also understand how quickly your end users get to the "clicks and pics" on your site.

We thought it would be fun to take a look at how some of these metrics compare across sites and industries.  Did you know that it takes about 1.5 seconds longer to interact with most of the content on news sites than on social networking sites?  Or that it takes over twice as long to load the home page on Foursquare than Twitter?  These are some of the factoids included in the following infographic:

Is an Outage Sometimes Your Best Strategy?

Today's launch of the iPhone 5 followed a similar pattern to Apple's other previous product launches. And while Apple seems to be training the media to certain launch event expectations, it also seems to be training its Apple Store customers to expect closed doors at its e-commerce website.

Just as with the last major product launch (remember the "new" iPad?), the Apple Store was taken offline immediately preceding, and for some time after the media event. In this instance, for 7 hours. Keynote measurements of the Apple Store, a member of the Top Retailers (US) performance index, show the outage and captured the state of the website before, during and after.

 

Scatterplot graph of performance for the Apple Store home page. The screenshot images--captured by Keynote Transaction Perspective--have been imposed onto the graph for illustrative purposes.

 

A 7 hour outage for any online retailer is huge, and Apple's outages appear to be intentional. The closing of the Apple Store in this instance raises interesting questions about managing customer expectations and online experience. Are launch event outages a defensive practice, or an intentional component of the overall launch experience? Is it worth handling the spike of non-buying visitors (pre-sales won't be available until a couple days later) during the launch event, when most 'lookey-Lous' will likely get their fix through the subsequent media tsunami? At what point does opacity change from creating a sense of mystery to confusion?

The overall performance and availability of the Apple store is quite high, with it consistently ranking above the Index average for online retailers. Indeed, the past 6 weeks reveal improvement in Total User Experience Time and solid availability:

Apple appears to take Web performance seriously. Launch events such as today's are consistenly  ochestrated to careful detail. So while most online retailers strive for 5 nines, is there something to be learned from Apple? Could an intentional full outage (outside of planned maintenance/change) be a viable e-commerce strategy in certain circumstances? If so, when?

Filtering Out Web Performance Monitoring Traffic From Google Analytics

If you're a webmaster, site owner, or digital business manager, picture this scenario. You open your Google Analytics dashboard and you see a big spike in traffic. At first, you're really excited - wow, look at all those site visitors! You run off to tell your boss that she can give you that bonus she promised if you got that online marketing campaign to work. And then... you learn that someone in the Site Ops team added web performance monitors, and so it's all so-called robots - synthetic traffic generated for the purpose of keeping tabs on your site's response time and availability. 



You think, no problem, I could filter that out from Google Analytics, and then you learn you can't. It's always going to be there, like, forever. You go crazy, emailing the GA team, posting on forums, and then slowly resign yourself to forever dealing with that spike. It will disappear from your default view, maybe after a month, but that's a long month to wait. And heaven forbid if your execs ask you for a site traffic report for the past year, try explaining why you can't filter out that spike - what, you weren't thinking ahead? What kind of guy did we hire to run our online business, anyway?

Don't let this happen to your career. Plan ahead and learn how to use Google Analytics to filter out all web performance monitors from your site analytics reports. Here's the recipe:

STEP 1: Find Out the User Agent String for your Web Performance Monitors

Keynote's monitors, like all other web performance monitors, insert a special marker in the browser, called an user agent string. So all you have to know are the user agent strings that Keynote adds to the browser. Keynote has several performance monitors - using real IE and Firefox browsers, or mobile browsers. Each performance monitor comes with its own special marker, so you have to construct a regular expression to filter all of these out.

Here are the browser markers that you have to use to filter out web performance monitors:

 Keynote Systems - Use "KHTE" (for Application Perspective monitors), "KTXN" (for the real browser Transaction Perspective monitors), and "Keynote" for Test Perspective load testing agents.  
 Compuware Gomez - "GomezAgent"  
 Smartbear AlertSite - "AlertSite"  
 Pingdom - "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"  
 Yottaa - "YottaaMonitor"  

If you are using a web performance monitor not listed above, Google "<insert your monitoring vendor> user agent string" and you will definitely find the user agent string you need to know.

STEP 2: Change Google Analytics Tag to Ignore Web Performance Monitors

What you do next is to put a conditional wrapper around your Google Analytics tag to never log visits from testing companies. This requires you to edit the JavaScript code on your site. An example Google analytics tag is shown below – the actual Google code depends on your implementation. Note that the Google Analytics account ID below is an example – you should replace it with your own account ID.

   <!-- Google Analytics tracking code that eliminates Keynote robots -->  
   <script type="text/javascript">  
     // These are the strings that Keynote adds to the User Agent string:  
     var gk_KEYNOTE_TXP_MONITOR = "KTXN";   // Transaction Perspective agents  
     var gk_KEYNOTE_APP_MONITOR = "KHTE";   // Application Perspective agents  
     var gk_KEYNOTE_TSP_MONITOR = "Keynote";  // Test Perspective agents  
   
    // if the User Agent string indicates Keynote monitors, then don't track the visit in Google Analytics  
    if (navigator.userAgent.indexOf(gk_KEYNOTE_TXP_MONITOR, 0) == -1 &&   
      navigator.userAgent.indexOf(gk_KEYNOTE_APP_MONITOR, 0) == -1) &&  
      navigator.userAgent.indexOf(gk_KEYNOTE_TSP_MONITOR, 0) == -1) {  
     // Ok, this is not a Keynote monitoring agent, so go ahead and log the visit with Google:  
    var _gaq = _gaq || [];  
    _gaq.push(['_setAccount', 'UA-12345678-9']);  
    _gaq.push(['_setCustomVar',1, 'User Type','Public',2]);  
    _gaq.push(['_trackPageview']);  
    (function () {  
       var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;  
       ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';  
       var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);  
    })();  
    }  
   </script>  
 <!-- Source code formatted using http://codeformatter.blogspot.com -->  

That's it. At this point, Google Analytics will not log your robot traffic (at least for Keynote web monitoring traffic, using the above example) and you can safely keep your job. The code snippet can be easily extended to filter out Gomez, AlertSite, Yottaa, as well.

Next, we will create an Advanced Segment in Google Analytics to filter out any synthetic traffic that has gotten through the cracks.

STEP 3: Create An Advanced Segment to Filter Out Additional Web Performance Monitors

Some monitoring tools don't modify the User Agent string, and instead use other techniques. Ensuring that you are in the MySite tab, click on Advanced Segments.

In the bottom right area of the Advanced Segments dialog box, click on this button:    and name it "Real People". We will now create a segment that filters out the synthetic traffic from web performance monitors. Here's the Advanced Segments dialog box:

Now, here's the tricky part - writing the regular expression that the Advanced Segment requires. Here is the regexp that filters out both the Keynote and Gomez monitors. Be careful to use the string exactly as shown, with the periods and asterisks: .*(KHTE|KTXN|GomezAgent).*

It's critical that you use a regular expression correctly, and getting it wrong is why I suspect many people believe that Google Analytics can't filter out this traffic. Once an advanced segment is created, then all traffic AFTER today will be filtered in the reports, but this will not apply to traffic that was generated prior to your creating this advanced segment. That's what I believe, from trial and error, though Google Analytics help says that you can filter out historical traffic. In any case, it's important you setup these monitors anyway, because you have no control over someone else setting up web performance monitors - even if your company didn't create performance monitors, your competitors could be monitoring your site's performance and creating all this traffic to your site - it is the world wide web, after all.

STEP 4: Select the Real People Advanced Segment When Viewing Data

Drop down the Advanced Segments dialog box, and select "All Visits" on the left hand side, and the advanced segment that you created, "Real People" on the right hand side.

Click on Apply, and view your data:

Now, it would be nice if I could choose "Real People" as the default segment to apply to all my reports, but I can't do that in Google Analytics yet. Nevertheless, you now have a handy way to view all the traffic and exclude web performance monitors, including Keynote.