Web Performance Watch

Test Your Site on IE 9 and Measure User Experience

Last month, we announced that the Keynote Global Network was being updated with Internet Explorer 9. As a result, our real browser monitoring service, Transaction Perspective™, is now measuring the performance of Web applications and sites using Microsoft’s latest Web browser. This makes Keynote the first on-demand monitoring service built on IE 9, which is pretty cool. But what’s even cooler is the ability that IE 9 gives us to measure a new class of performance metrics we call user experience metrics.

As we’ve discussed here previously, IE is still the big kid on the block when it comes to browser usage. With the demise of IE 6 in the United States, and the rise of Firefox and Chrome, it’s clear that users are quickly leaving “old” browsers for “modern” ones like IE 9. With high performance and broad support for open Web standards, browsers like IE 9 make it easier for companies to create a rich and snappy experience for consumers. In response, 34% of the top Internet sites now use HTML 5, and the use of JavaScript continues to rise. Transaction Perspective built on IE 9 allows customers to get a more precise view of their sites’ performance, especially those leveraging new Web standards.

Our new Live Beta preview of MyKeynote 11 with Transaction Perspective lets you see performance in very important ways:

• Time to First Paint – This new metric tells you when a user begins to see your site render in the browser.
• Time to Interactive Page – Tells you when the Document Object Model (DOM) begins to process user events for the document.
• Total User Experience Time – With User Experience Time, you know how long a page (or series of pages) took to render and become usable for a real user.  It is the ultimate measure of a page’s speed, factoring not only the time it took for data to be downloaded, but also rendered and made interactive.

These key moments are just a handful of the browser events we capture. If you’re a performance expert, you’ll appreciate that we measure all the Browser Navigation Timing events and can graph them individually over time (multiple measurements), as well as display them in a timeline view for an individual measurement.

KITE (Keynote Internet Testing Environment) lets you to test for free your site’s performance from 5 cities on the Keynote Global Network, on demand. But if you’d like to test drive User Experience monitoring, click the “24x7 Monitoring” button in KITE.

Once you’ve activated your trial, click the “Try Beta Version” link in MyKeynote. There, you’ll be able view User Experience metrics for everything monitored during your trial.

Soon, you’ll be able to see these new browser event metrics from your desktop in KITE, as well. Your copy of KITE should automatically update itself to version 5, or you can manually check for the new version. 

Let us know what you think of these new features!

Up on the Roof Top… Click, Click, Click

Hi. My name is Ben, and I plan to visit your website next year a lot! I’m a casual user that likes to browse the Internet, and like everyone else I know, I hate to wait. Although your website is really cool, and I love your products and services, the Internet is full of interesting places and I’m easily distracted. Oh, and did I mention that I love using my Android smartphone and iPad to play, connect, shop, bank and book travel when I’m not at work? All I want for the holidays is a speedy web and mobile site.

Here are my top 10 wishes for you and your site in 2012:

1. Please test your code/site on IE and not just Firefox before you launch it. I am one of the 50% that will continue using IE, even after they start auto-upgrades!
2. Understand the difference between browser execution and network/back-end performance. Most pages have both and you need to know which is which to optimize the page/site. One way is to monitor using a real browser.
3. Understand how your page renders. Focus on reducing upfront (pre-render) delay. It’s the one me and your other users feel the most!
4. Please make sure your third party tags (analytics and others) are below your visual content. (Did I mention rendering delays and blocks are aggravating?)
5. Please combine your external JS and CSS files. I’ve been saying this for years, but very few sites seem to follow this recommendation. Do it, and I’ll see a major improvement in the speed of your site.
6. Understand the quality your Content Delivery Network is providing. Every website is unique, and not all CDN providers are created equal.
7. Don’t worry so much about overall page size but instead focus on individual file/resource sizes. Keep them under 100K and you will limit the impact of slower connections. (Did I mention I love my smartphone?)
8. Don’t just push your desktop website to mobile. You will fail.
9. Test your mobile site… please!
10. Read Keynote’s Page Construction Guidelines. They’re chocked full of goodies to help you optimize the performance your Web pages and keep visitors like me happily clicking through them, instead of away to your competitor’s site.

What’s on your wish list for better web and mobile performance in 2012? Let me know!

Cyber-Awesome: Killer Performance Helps Drive Record Revenue

For years, Keynote has been monitoring the ‘net’s best websites and reporting on notable outages during the peak Black Friday and Cyber Monday online shopping period. Typically, we find that many online retail sites—both pure Internet and those with brick and mortar stores—experience some issues. But the trend has been towards better performance and higher availability. This year marks yet more progress toward the goal of a trouble-free and virtually instant shopping experience. And the numbers suggest the payoff for retailers has been huge: a record setting $1.25 billion in online sales for Cyber Monday, 2011.

The Keynote Top Retailers Index tracks 47 online shopping destinations. This year, the average website response time was 3.05 seconds. According to a study by IBM, online sales peaked at about mid-day, and spiked again later in the evening. Average response times across the Top Retailer Index reflected this pattern as peaking demand slowed sites. But the absolute impact even at its worst was only about 1 second of delay. That’s great performance across the industry as a whole.

Additionally, the availability of these websites remained extremely high throughout the period. Out of the 47 sites tracked, only 10 (21%) experienced any downtime at all and their average availability was around 99.0%.

So how did online retailers get to this pinnacle? We see that customers are maturing the technology, processes and culture of their Web Operations around a passion for performance and deep preparation. One example of this is Keynote customer Karmaloop.com who rang up their biggest day in company history and over 100% more sales than the same period last year. Though careful planning and web load testing, they were able prepare their site for the demands of such a dramatic increase in traffic and order transactions.

The improving trend in website performance throughout the increasingly demanding Black Friday and Cyber Monday period is encouraging. But it also means that if your site experiences failure or less than stellar performance, your competitor is in a much better position to attract and retain your abandoning customers. Congratulations to the online retail industry for hitting a remarkable milestone over the past six days. Keep pushing performance higher through December!

The Target.com "fail doggie" -- it didn't have to happen

 
When Target thought ahead about the headlines they would make with the launch of a limited partnership with designer Missoni, it probably didn’t expect this one:

“Demand at Target for fashion line crashes web site”

While we usually hear about retail Website crashes on or near Black Friday and Cyber Monday, Target.com’s recent site outage is remarkable for its atypical timing, severity and length. Careful investigation into the origins and impact of the incident reveal important lessons for all online retailers. Ultimately, failing to properly prepare for an incident like this is no option for merchants striving for success in today’s connected consumer market.

What Happened?

Keynote Systems' measurements of the target.com website indicated that traffic began overwhelming Target.com around 8:00am EDT. The intense wave of traffic initially resulted in server errors. Quickly, the operations team responded by serving “friendly” error messages around 8:14am. After that, only about 7% of the visitors were able to successfully reach the real home page throughout the remainder of the day. “Fail puppy” error pages were still being measured by Keynote well into the following day.

 

(click image for larger version)

Fallout

Credit the marketing and merchandising teams for generating the hype the drove the demand for Misonni. Were their efforts wasted when Target.com went sideways for two days? A better question might be what destruction did they wreak on sales as a whole during that time?

What drives the financial impact of a site outage or performance issues is abandonment. It’s important to understand that visitors experiencing a Website with issues don’t result in lost revenue, per se. Only when those visitors don’t return, and/or go somewhere else is revenue lost. A shopper’s tolerance for errors is called “tenacity” in Web load testing parlance. Low tenacity shoppers bail from slow searches and hanging shopping carts in a dash. Missoni brandistas, however, were very high tenacity shoppers. They kept trying to buy despite the errors, which also likely contributed to the outage’s long duration.

So while Target’s outage surely didn’t deter some of the Missoni faithful from eventually returning to place orders on another day, the real financial impact was to the other product category sales that couldn’t take place during that time. The iPad shopper went to one of Target’s competitors, instead. According to Nielsen NetRatings, for every 1000 consumer electronics shoppers that experience three errors, the cost to the site is nearly $100,000. The Target.com site gets 1.3 million visitors a day this time of year, and it presented repeated errors to 93% of visitors on Tuesday. You can start to get a sense of how that adds up!

 

(click image for larger version)

Realistic Web Load Testing Prevents Outages

The only way to prevent events like this from happening is to perform realistic Web load tests that prepare for these scenarios. In this case, Target claims to have prepared for the kind of traffic they typically experience on Black Friday—a build-up over the course of the day. Clearly, this was not the realistic scenario to prepare for. The Missoni line went available at a certain time and day, more akin to a ticketing merchant where a concert sells out within minutes of its availability. They did not expect the suddenness of the demand surge. 

If a load test had been run that adquately modeled the impact of the planned launch, the damage would have been done outside business hours and Target management would have been able to decide whether to make changes to their systems and retest, postpone or restructure the launch or just "risk it" and see what happens.  We don't really think they ever had the chance to make those decisions.  Surely they didn't conduct a load test that predicted such an epic fail.  But should they have?

Realistic Web load tests model site usage and shopper behavior. Systems are deployed to simulate high levels of demand from multiple geographically disperse areas. Once the load is generated, the infrastructure and application’s response are watched carefully to identify bottlenecks and breakage points as the entire mesh of the Website’s interconnecting parts are stressed. Only this level of testing can accurately inform e-commerce teams of their preparation adequacy.

 

Script and Monitor RESTful APIs in Public and Private Clouds

By Ian Withrow

In the 4.1 release of KITE we believe that we’ve significantly improved the ease and power with which users can create scripts to monitor APIs. Web APIs, aka Web Services, are an important way for companies to integrate 3^rd party data and service providers both in the browser and in the data center. Web API’s that need to be monitored tend to fall into one of three categories.

• First, all the widgets and mash-ups from companies with Facebook, Google, and Twitter are built upon web APIs.  However, even traditional companies like Best Buy, Hoovers, and the New York Times offer APIs.
• Second, most companies talk to one or more 3^rd parties directly from their data centers.  Examples from this category might include payment vendors like Zuora and PayPal and range to specialized service providers like Quova (IP Geolocation) and Netbiscuits (Mobile site delivery).
• Third, many companies are starting to build their applications using a Web Oriented Architecture, which calls for modularizing different components of the application and exposing them via a Web API. 

Regardless of the scenario, the reason for monitoring these APIs is the same. API performance can have a significant impact on user experience especially when it drives critical transactions like checkout or the delivery of customized content. Unlike other monitoring solutions Keynote actually interacts with the underlying API, enabling you to test not only performance but to ensure responses conform to specific parameters, e.g. number of widgets in inventory is a number between 1 and 100. In this way you can reproduce the logic found in the actual application that consumes the API.

In this two part blog series I’ll walk you through exactly how to script and monitor APIs with Keynote both inside and outside of your cloud. This post will cover RESTful APIs. The second post will cover SOAP APIs. This article is long and in depth so below I’ve listed the major sections and the associated topics:

• Step 1: Create a basic web services script
• Step 2: Techniques for iterative development
• Step 3: Advanced script editing
• Selecting a Keynote monitoring product for your script

 

Step 1: Create a basic web services script

For the purposes of this demonstration we need an API, I’ve chosen the Active.com API because it’s a real, working API that has rich features like developer keys and multiple format responses.

To get started you’ll need KITE 4.1 or newer. If you don’t have KITE a copy can be obtained for free here. Launch KITE, and in the top left corner there should be a button that looks like a big red record button. Click the bottom half of this button which opens a drop down menu and then select ‘Create Web Service Script’ as shown below.

You will be asked to select a service type, go ahead and select REST.

Upon doing this you’ll be presented with a wizard, as shown below, that will gather input from you in order to construct an advanced script. First, you enter the base URL for the RESTful API function you wish to access. Next you specify the response format type you will receive, such as JSON, XML, or unstructured text. Then you can proceed to fill in all the URL query parameters that will be passed with the request. The Wizard supports all standard HTTP request such as POST and DELETE. You can also enter any special headers at the bottom; standard headers will be created for you. In this API, the license key is added as a parameter but other APIs may call for a header.

Don’t worry if you don’t have everything perfect just yet. You can manually edit the script later or simply recreate the Web Services action and the Wizard will remember the last inputs you gave it. When you are ready, click ok and you’ll see a new action added for you in the Script Viewer. 

 

The best practice suggests that at this time you should rename the action something logical by clicking on ‘Call method’ and edit the Script Property shown below. My API call retrieves events in the San Diego area.

 

Now we are ready to view our completed script, you can do this by clicking on ‘REST Script’ on the left, note that this too can be renamed. When you do you’ll get a new window, called Advanced Scripting. Please note if you have a lot of windows open in your KITE instance you may need to close or resize some to get the expansive view that I have below. We’ll explore the different elements of this script later but for now the key portion is boxed in red, this is the RESTful query that you’ve just created. You can now double check and edit it manually using JavaScript.

 

Once done, you click the ‘run’ button as normal to test out the script and see what results you get.  If everything works you’ll get a screen like the below shaded in green.

 

For comparison here is what failure looks like, in this case I deliberately mistyped my license key resulting in a 403 error.

Step 2: Iterative Development

In addition to the Wizard we’ve added three great tools for iterative development, which we’ll examine in turn within this section:

1. Advanced script log
2. HTTP request/response header payload viewer
3. Using saved variables in subsequent actions

Advanced Script Log

After you run a script, the Advanced Script Log will be automatically updated and added to your screen as shown below. The log displays the request made as well as any error response details received. If the response was a successful, then the results are store into variables. For example, 0.42716 is saved in the variable [searchTime], more on how to use this later.

 

HTTP Request/Response Header Payload Viewer

Sometimes I find it helps to see the exact request and response payloads. Now there is an easy way to do this. In the Transaction Performance Details window, which is typically right below the Advanced Script Log, right click the step or object that you are interested in and select View HTTP Payload from the context menu.

Back on top you’ll get the below screen which shows you the exact headers and payloads, up to a character limit.

 

Using Variables in Subsequent Steps

The real power of this tool is to create a sequence of API calls that utilize results received in prior steps, just like a real program would. Let’s say that after our query of local events in San Diego we now want to get the event details for one of the results returned. A quick review of the API’s developer documentation shows that this is possible via the ‘assets’ API method which uses the variable assetId in its base URL. Let’s add that step using a saved variable from our first step. First we should copy and save the variable we want to use, the assetId of the first event in the list. We can find our quarry in the Advanced Script Log as shown below; we want everything inside and including the brackets.

 

Next we need to add a new step. This is done, as normal in KITE, by right clicking the script name in the Script Viewer window and selecting a new REST action as shown below.

Now we are back at the Web Services Wizard. We again start by entering the base URL, but this time we need to append an assetId at the end of the base URL. As you can see below, we can paste the bracketed saved variable directly into the wizard. Next we need to set the Results Format, let’s try XML this time. Finally we need to input our developer key and any other relevant parameters.

Now when we check the results of our new script we see that the code generated is retrieving our saved variable. This could also be done manually, as sharp readers will no doubt guess, but this way saves a lot of time and effort.

We can now run this two-step script and check the HTTP Payload of the second object to confirm we got the results we expected, in this case by double checking the assetId.  As we’ll explore in the next section, it is possible to extend the script to add your own custom validation logic on elements like this and throw errors depending on the results.

The script is now done so we can save and upload it to Keynote for deployment to a monitoring agent.

Step 3: Advanced Script Editing

As I alluded to earlier, it’s entirely possible to manually rewrite the script created by the wizard and even modify the template itself. First though let me explain the organization of the built-in script template. Each script, both REST and SOAP, is organized into three logical sections as shown below. The red section is the parser which takes the results and saves them in variables. By default the script will usually only save one version of a variable, i.e. if you have multiple assetID’s it will save only one, the last. This is fine in most cases, however there may be times when you want to enhance this behavior and that can be done here. The blue section is where the actual query is built, this code block will obviously be pretty different between REST and SOAP but the organization is the same. Finally, the purple section is where error handling is done. The built in error handling can catch HTTP as well as API errors; however you may wish to extend this functionality further. As mentioned previously, all of this is done in JavaScript.

Let’s say you find yourself frequently updating the built in script functionality and want to make a global change. This can be done by modifying the files named wsdl.template and rest.template. These files are found in your application data folder under …/Keynote/Record/Config/. So for example in Windows 7 this would be username/AppData/Roaming/Keynote/Recorder/Config/. I encourage you to back up the originals before experimenting with this though.

Selecting a Keynote Monitoring Agent for Your Script

Now that you have created a brand new Web API monitoring script you’ll need to select some agents to run it. You have two choices, our Application Perspective (ApP) and Cloud Application Perspective (CApP) products. ApP is a collection of globally distributed monitoring agents that are ideal for monitoring your own APIs that are accessed by a broad range of consumers and those 3^rd party APIs integrated at the client or browser. However, for APIs and partners that your own applications require direct access too you can deploy your own CApP agent. CApP is your very own Keynote agent software which you can install in your own private cloud or even within an important business partner’s cloud. The short rule of thumb is you want to monitor the API from where it will actually be used.  If it’s consumed by end users around the world go with ApP. If it’s consumed by applications within a private cloud or network then deploy your own CApP agent there.

 

Script and Monitor SOAP APIs inside Your Private Cloud

By Ian Withrow

SOAP APIs, or Web Services, frequently form the backbone of private enterprise to enterprise communication.  Companies use these APIs to integrate value chains, share information, and integrate applications that depend on partners. While RESTful APIs may be taking the technology world by storm, SOAP still has an established place in private enterprise communications. Even as organizations migrate from dedicated physical resources to private and public clouds, SOAP APIs are likely to continue to play a role for now. As a result, web applications often depend on the information delivered by a SOAP API. As such if the API is slow then so too shall the application be. Thus it makes sense for businesses serious about performance to monitor their partner APIs, whether they are SOAP or otherwise.

In this post I will show you how to easily create an advanced SOAP script using the new enhanced functionality available in KITE 4.1, topics covered here include:

• Step 1: Create a basic web services script
• Step 2: Techniques for iterative development
• Step 3: Advanced script editing
• Putting your script to work with a Keynote monitoring agent

Note that I discuss how to script RESTful APIs in another post.

Step 1: Create a basic web services script

Public SOAP APIs are rare these days, most are private and require authorization.  For the purposes of this demonstration I will utilize the free, public Holiday Web Service API which supplies holiday dates for a small selection of countries.

To get started you’ll need KITE 4.1 or newer. If you don’t have KITE a copy can be obtained for free here. Launch KITE, and in the top left corner there should be a button that looks like a big red record button. Click the bottom half of this button which opens a drop down menu and then select ‘Create Web Service Script’ as shown below.

You will be asked to select a service type, go ahead and select SOAP.

After clicking ok a Wizard will pop-up on your screen. This wizard will automatically generate an advanced script for you that will make a properly formatted SOAP request, parse the results into a log, and capture errors. First, load the WSDL file for your API. Next you select the function you want to call; in this case I want a list of supported countries. A sharp eye might to detect what seems to be a duplicate list of functions. In fact there are multiple versions of SOAP and you will see duplicates when the WSDL specifies the same method for multiple versions. For Keynote monitoring purposes they work the same.

When you are ready, click ok and you’ll see a new action added for you in the Script Viewer. 

The best practice suggests that at this time you should rename the action something logical by clicking on [Action] and editing the Script Property shown below. 

Now we are ready to view our completed script, you can do this by clicking on ‘SOAP Script’ on the left, note that this too can be renamed. When you do you’ll get a new window, called Advanced Scripting. Please note if you have a lot of windows open in your KITE instance you may need to close or resize some to get the expansive view that I have below. We’ll explore the different elements of this script later but for now the key portion is shown below from row 36 to 53, this is the SOAP query that you’ve just created.  You can now double check and edit it manually using JavaScript. Note that the RequestXML variable contains just the SOAP envelope body if you are comparing it to a reference example.  If you needed to add login and authentication information this is the section of the code to do so, either be adding an additional KNWeb.AddHeader command or by including it in the RequestXML string.

Once done, you click the ‘run’ button as normal to test out the script and see what results you get.  If everything works you’ll get a screen like the below shaded in green.

For comparison here is what failure looks like, in this case I deliberately messed up the envelope by deleting a tag.

Step 2: Iterative Development

In addition to the Wizard we’ve added three great tools for iterative development, which we’ll examine in turn within this section:

1. Advanced script log
2. HTTP request/response header payload viewer
3. Using saved variables in subsequent actions

Advanced Script Log

After you run a script, the Advanced Script Log will be automatically updated and added to your screen as shown below. The log displays the request made as well as any error response details received. If the response was a successful, then the results are store into variables. For example, country code GBSCT is saved in the variable:

[GetCountriesAvailableResponse.GetCountriesAvailableResult.diffgr:diffgram.NewDataSet.Countries.Code]

I’ll show you more on how to use this in a second.

HTTP Request/Response Header Payload Viewer

Sometimes I find it helps to see the exact request and response payloads. Now there is an easy way to do this. In the Transaction Performance Details window, which is typically right below the Advanced Script Log, right click the step or object that you are interested in and select View HTTP Payload from the context menu.

Back on top you’ll get the below screen which shows you the exact headers and payloads, up to a character limit.

Using Variables in Subsequent Steps

Retrieving the list of supported countries is nice, but next let’s try to get some real information like a list of holiday’s for one of those countries. In this way our script can behave much like a real program on whose behalf we are monitoring performance. We know, from the parsing of the WSDL, that there is a GetHolidaysAvailable function. Let’s add a step using that method together with a saved variable from our first step. This would require the country code shown in the previous section from the Advanced Script Log: GBSCT.

Next we need to add a new step. This is done, as normal in KITE, by right clicking the script name in the Script Viewer window and selecting a new SOAP action as shown below.

We are at the Web Services Wizard again. This time we click on the ‘GetHolidaysAvailable’ function and a second popup is displayed asking us for input parameters. In this screen we will use the saved parameter as the input variable as shown below. Be sure to include the brackets then proceed as before by clicking ok.

Now when we check the results of our new script we see that the code generated is retrieving our saved variable.  This could also be done manually, as sharp readers will no doubt guess, but this way saves a lot of time and effort.

We can now run this two-step script and check the HTTP Payload of the second object to confirm we got the results we expected, in this case by double checking the countryCode.  As we’ll explore in in the next section, it is possible to extend the script to add your own custom validation logic on elements like this and throw errors depending on the results.

The script is now done so we can save and upload it to Keynote for deployment to a monitoring agent.

Step 3: Advanced Script Editing

As I alluded to earlier, it’s entirely possible to manually rewrite the script created by the wizard and even modify the template itself. First though let me explain the organization of the built-in script template. Each script, both REST and SOAP, is organized into three logical sections as shown below. The red section is the parser which takes the results and saves them in variables. By default the script will usually only save one version of a variable, i.e. if you have multiple countryCode’s it will save only one, the last. This is fine in most cases, however there may be times when you want to enhance this behavior and that can be done here. The blue section is where the actual query is built, this code block will obviously be pretty different between REST and SOAP but the organization is the same. Finally, the purple section is where error handling is done. The built in error handling can catch HTTP as well as API errors; however you may wish to extend this functionality further. As mentioned previously, all of this is done in JavaScript.

Let’s say you find yourself frequently updating the built in script functionality and want to make a global change. This can be done by modifying the files named wsdl.template and rest.template. These files are found in your application data folder under …/Keynote/Record/Config/. So for example in Windows 7 this would be username/AppData/Roaming/Keynote/Recorder/Config/. I encourage you to back up the originals before experimenting with this though.

Putting Your Script to Work with a Keynote Monitoring Agent

Now that you have created a brand new SOAP script you’ll need to setup some agents to run it. Since most SOAP APIs involve private application to application communication, Keynote’s Cloud Application Perspective (CApP) product is the ideal tool for the job. CApP is your very own Keynote agent which you can deploy in your own private or public cloud and even within an important business partner’s network. In this way you can monitor the SOAP API from where it is actually queried in order to get an accurate picture of performance.

 

Deep Link to Keynote Website Performance Data

By Ian Withrow

MyKeynote is designed so you can easily move from high level trends and events down to detailed data points. Users start with an alert or a scatter plot and then iterate down into successive levels of detail, typically culminating in a waterfall chart and details for a specific measurement that we took for them. However, sometimes a customer already knows what specific data point they wish to drill down onto.  For example, if an APM or RUM solution that you have captures a Keynote transaction you would naturally want to compare the user perspective of that transaction side by side with the system perspective provided by your other tools.  In this way you could see both endpoints.  Deep linking to detailed Keynote data points could also be handy in highly automated operations enviroments.  Imagine if your trouble tickets included direct links to poorly performing data points associated with an alarm?

As you may have guessed by now, in this post I’m going to show how you can link directly to a data point or waterfall in Keynote. I expect this will prove handy for operations tools, devops, and performance analyst types who want to compare the Keynote data with APM data or possibly want to iteratively retrieve waterfall visualizations of data that meets specific criteria. In fact Keynote partners with a number of APM vendors like OpTier, ExtraHop and OPNET so you may soon see the ability to link from one of these APM solutions to Keynote visualizations.

Before we proceed I must mention that you will need our Data Feed or DataPulse service.  DataPulse provides a near real stream of Keynote data. Data Feed provides the same data but every 15 minutes.

Keep in mind that I'm appropiating functionality here that wasn't designed for this purpose. Yes it's a creative hack, but don't let that stop you from enjoying the benefits.

 

How To Deep Link Into Keynote

MyKeynote offers two types of visualizations that you can link to directly if you know when the data point was collected, which agent took it, and which transaction or script it is associated with. Happily all of this information can be retrieved from the aforementioned data services. In the below example I use excerpts from Data Feed XML files. DataPulse has all the same information; the XML file is just formatted slightly differently. Either way Keynote has user guides for both of these services if you want more details on them. From this point I’m going to assume some familiarity with these XML files.

 

Step 1 – Gather the Variables Needed for the MyKeynote Request

In XML Data Feed, each record starts with the following tag. The variables we care about are colored green.

<TXN_MEASUREMENT agent="45537" slot="845480" datetime="2011-JUL-05 22:37:51" target="1034270" agent_inst="45538" profile="0">

Additionally if we have a multiple page transaction we’ll need to know what page we care about. Each page record in Data Feed has the following tag which specifies page information.

<TXN_PAGE page_seq="1">

The hardest part is that we need to convert ‘datetime’ to a base UTD number. In English, this is the number of seconds that had passed since 01-01-1997 12AM (GMT) when the data point was captured. This may sound hard to figure out but writing a program to do this conversion is fairly straightforward. Here is how to quickly do the conversion by hand.

1. Go to: http://www.epochconverter.com/ a handy site that lets you quickly convert data and time formats between human readable version and Unix Time
2. Copy ‘datetime’ from your XML file, in our case the value is 2011-JUL-05 22:37:51
3. Enter this value into the “Human date to timestamp” converter and run it as shown below
4. Subtract 852076800 from the result; this converts the number from Unix Epoch time to base UTD.
5. In this case we get the value 457853871, save this result for later since we will need it in the next step.

 

Step 2 – Build the MyKeynote Request

First, a caveat these requests are only fulfilled if the user is already logged into MyKeynote. If not they’ll be asked to login and will need to try again. As mentioned previously we have two options.

1. Display a page waterfall
2. Display a transaction summary which compares multiple pages in a single transaction.

If what these two options represent isn’t immediately clear to you that’s ok. Below I describe how to form a request for each along with a screenshot that shows what you’ll get for your efforts.

 

Waterfalls

A waterfall can be requested using the below URL with the variables you need to fill in denoted in {brackets}.

http://my.keynote.com/newmykeynote/transpagesingledrilldown.aspx?transid={target}&agentid={agent}&butd={UTD}&ksid=5&profid={profile}&pageid={page_seq}&mskin=0

As you may recall in the previous step I described how to find each of these inputs.  Remember if you have only one page in the transaction then the value there is of course 1.  Below is a completed example and the resulting output.

http://my.keynote.com/newmykeynote/transpagesingledrilldown.aspx?transid=1034270&agentid=45537&butd=457853871&ksid=5&profid=0&pageid=1&mskin=0

Transaction Drill Down

For many transactions each data point will be comprised of multiple pages. If you want a detailed view comparing each page, instead of a waterfall for one page, the following request will do the trick.  Again variables are specified in {brackets}.

http://my.keynote.com/newmykeynote/scatterplotdrilldown.do?transid={target}&agentid={agent}&butd={UTD}&profid={profile}&pseq=-1

Note that in this case we don’t care about page number, if you did specify a pseq value it would actually take you to the waterfall for that page.  A completed example below:

http://my.keynote.com/newmykeynote/scatterplotdrilldown.do?transid=1034270&agentid=45537&butd=457853871&profid=0&pseq=-1

Building a Bridge from a Log

Now some of you may be saying wait a minute, what if I only have the log or the header from the Keynote requests to start with?  How am I going to get from that to the Data Feed or DataPulse record?  Fortunately we add a header to each Keynote agent request formatted as below.  In the header is the target and UTD value which should be sufficient for you to find the right XML record.  Example:

X-KTRACE

457853871|45538|1034270|0|1\r\n

Or rewritten using our variable names:

UTD|45538|Target|0|1\r\n

Wouldn’t it be nice if the agent number was in there too?  Sorry I warned you this functionality was developed for another purpose and that we are being opportunistic here.

 

Conclusion

I recognize this won’t be for everyone. For those operational groups that are trying to shave off every bit of time they can from their MTR numbers then being able to jump directly to detailed Keynote data points from say a Nagios or an APM system will save you valuable time. Do let your account team know if you like this an want an expanded solution.

How To: Monitor Web APIs with Cloud Application Perspective

By Ian Withrow

Keynote is known as the end user experience monitoring expert, specifically for humans browsing a website with a web browser. However, there is another set of users that we can monitor: applications. More specifically we can monitor the experience of any application consuming a Web API or Web Service. Of course many of our customers understand that we can monitor any HTTP traffic and have been using us to monitor RESTful API and SOAP calls for a long time, but it has always been a subset of our business.  In the immediate future we see that changing with Cloud Application Perspective. (CApP) I believe two cloud trends will drive this change, which I’ll touch on next. However, the ultimate goal of this post is to show you an example of a RESTful API I scripted for our CApP product.  (Although it would work for our public agents too)

The first trend that will foster an increase in critical API traffic is the adoption of hybrid clouds, specifically applications distributed across them. By hybrid clouds I mean organizations with elements of their applications both on a public cloud and a private cloud computing platforms. This is a popular deployment model for business reasons ranging from the regulatory to the religious to the practical. For example, a company might plan to deploy the core of their application in a public cloud to take maximum advantage of scale, geographic expansion, elasticity, and bursting for unexpected demand etc. However, a key piece of the application will remain in a private cloud or possibly on dedicated hardware. It could be that migrating some legacy code isn’t feasible or perhaps there is data there that the company wants/needs to keep under its direct control. Whatever the reason, the public cloud portion must now send queries back to the enterprise data center. A robust and manageable way of handling this is via a Web API. In this case CApP is then deployed at one or both sides of this link to monitor performance.

The second trend will be driven by companies who aggressively embrace public cloud computing solutions and design their applications for them. A core tenant of this approach is to design your application to scale and contract automatically, as well as terminate and replace instances that become unhealthy or even need to be updated. How will all of this be done? The cloud computing provider’s APIs. Interestingly the performance and availability of the management APIs now become very important and thus merit monitoring. Imagine what happens to your application if you can’t complete requests to this API in a timely fashion or at all? To monitor this interaction, CApP is deployed wherever you need to query these APIs, likely in the cloud itself and near your operations center or devops group.

There are of course many ways to monitor APIs. Many are rather superficial, like a ping or response time test. This of course tells you that the physical system is up and responding in a basic sense, but it doesn’t mean you can actually do something useful. This is the strength of Keynote’s API monitoring, something which I’ll demonstrate here with an example. I’ve picked the Active.com RESTful API because it’s a good modern RESTful API with concepts like developer keys and nested queries with the added benefit that it’s free to use.

To start scripting a RESTful API, enter KITE and create a new ApP script as shown below.

Next you’ll be asked for a URL to start with.  You have two options here. You could input the entire URL for your first API request. KITE will understand that. However, if you aren’t sure yet of the parameters you want to use you can also just enter the base URL for the API query you want and define the variables in KITE later. In our case the base is:

http://api.amp.active.com/search

KITE will then bring up a browser in an attempt to visit this webpage. Just immediately stop the recording window and ignore the likely 403 error you get. At this point you’ll see something like this:

As a procedural note in future screen shots I may show only a subset of this window, in particular just the script viewer, but I started with a full screen shot so you can orient yourself within KITE. Now we want to add query parameters. To do so, right click “Navigate to” and select “Add Queries.”

Doing so will add a Queries section to this step in KITE which is basically a place to specify variables to include in the URL request. To add a name/value pair or just a name right click “Queries” and choose the option you desire.

If you choose to paste the exact request string for the first query when you create a script then these fields will be pre-populated. Now we can generate our full request, note that if you have an API key of some sort it can be specified here. For this query we are looking for activities in Active.com’s database near San Diego. To do this we add a Name/Value pair and edit as shown below in the “Script Properties Editor” box.

A sharp observer might have noticed that Type is static, implying we could pass a non-static piece of data here. In fact you can retrieve/save data from a previous step to use in subsequent actions. Let’s say we want to see all the events going on in San Diego and retrieve the event details for the first one. Active.com offers another query for retrieving event details based on assetid:

http://api.amp.active.com/resource/assetservice/asset/<put assetid here>

So how do we tell the script to pull this value from the first step for use in the second? Well for starters if you are a visual person like me it may help to actually run the first stage to see what we are dealing with.

I’ve highlighted the field we are interested in, also note that the first entry is the World Run Day at Chula Vista. Now we know what we need to in order to create the query. First we add a new action by right clicking the Script name and selecting “Add Navigate.”

Next click on the new step we just added which will have blank URL. Type in the base query URL in the script properties editor. When done click on “Edit” so we can add some logic to find the correct assetid and append it to this request.

You’ll get an “Edit Parameter” box, select “Append” and then “Ok.”

Now we’ll deviate from using static values. Click the drop down menu and select “Text found from page”. This will allow us to search the previous page for some information. Note that we have a number other options ranging from a saved value to Javascript parameters.

From our previous query we know we want text between two XML tags. In fact we can even copy and paste the tags out of that test query in order to avoid typos. There are other optional parameters here, for example if we wanted the second event we could increase the index value.            

One final step not picture here is that I needed to add a name/value pair to this query just like I did in the previous step for my API key. Once done I can run the query and see that I do in fact get the World Run Day record.

Ok that’s enough for our simple starter example. In a production script you may wish to consider adding other details, like error handling, depending on the API. For example what happens if we run a query that has no entries? Is that a failure to you? KITE provides you the flexibility to add this logic to the measurement and report/alarm on it in MyKeynote.

To review, whether you have a hybrid cloud or you are going whole hog into a full public cloud, entering this world means you’ll likely depend on some Web APIs. CApP can easily be scripted to monitor these which will reduce down time and increase visibility into your systems.

 

How To: Setup First Mile Monitoring with Cloud Application Perspective

By Ian Withrow

One of the most straightforward uses for our new Cloud Application Perspective (CApP) product is monitoring the performance of a website or API up to the edge of your private network. To be sure other use cases, which I plan to talk about soon, focus more purely on internal monitoring. However, this is a good one to start with if you are already using or familiar with Keynote services for monitoring user experience from outside your firewall. In fact if you already have Keynote then this is a great complement for your existing monitoring via our Transaction or Application Perspective products. At a high level you will learn the following: how to request a new CApP agent, where to place it, how to create and deploy a measurement, and of course how you can make use of the resulting data.

Ok but first, why should you care? Well there is a direct and an indirect benefit here. By monitoring first mile performance you can quickly identify if performance problems are inside your data center or outside of it. This need not be limited to consumer facing applications, you could easily monitor the first mile performance of B2B applications or even internal/partner applications. In fact, these scenarios might be more compelling as they are often associated with SLAs or clear expectations of performance. Moreover, I talk about monitoring a private data center here but it could just as easily be a private or public cloud. This brings us to the second benefit: proof. In short, gathering first mile performance data with CApP gives you evidence of where a problem is located. Who doesn’t want to avoid finger pointing?

Hopefully by now you are asking how you can get one of these. Well for starters you have to be a Keynote customer, but assuming you are you can easily request one online in the Keynote Service Center. First, select the “Agent” tab. If you don’t have this tab in your portal then you haven’t been setup for CApP, we’ll address that in a second. Second, click on “Request CApP Agent”. Finally, fill in and submit the resulting form. The only input that involves some research is supplying the public IP address range from which the agent will contact our servers. A full Class A is ok though. If you don’t know the public IP address of a given device then a number of free websites can answer that question, like IP Chicken.

So what if you have Keynote service but haven’t signed up for CApP? Well in the “Add Measurement” screen you may have noticed that you now have a CApP section. You can select this option and choose “Getting Started.” This takes you to a form that will nudge your sales team to get in touch with you to get your contract amended. I know not very SaaS’y, but we wanted to get version 1.0 out quickly. In the future, look for this process to be further automated.

 

After going through all of this you’ll get an email from us with a download link and a license key for your brand spanking new CApP agent. So where to put it? Typically you’d deploy CApP for this use case behind your firewall or possibly in the DMZ, this would provide visibility up to the edge of your network. You can then compare this data with measurements from Keynote’s global test and measurement network. Since CApP is designed to operate on end user machines for remote testing, it doesn’t need much in the way of horse power and can easily be run on shared or virtualized systems. Fair warning, it is Windows only right now. The setup itself is a breeze; you basically speed click through a Windows installer. You know, where you quickly click “yes” and “next” while you unwittingly consign your first born over to Keynote? After that there is a simple dialogue window where you enter the license key we provided you. From here on out CApP runs as a Windows service behind the scenes and only requires outbound Port 443 access, so it can communicate with our SaaS portal.

So speaking of instructions, how do you create a measurement for CApP? CApP, if you didn’t already guess, generates synthetic measurements based on predefined instructions we call scripts. You use KITE to create an Application Perspective (ApP) script. In fact if you already have ApP scripts you can skip this step and use them with CApP. For in depth KITE tutorials or to download this free tool, then check out its web page here. What I’ll show you here is how to create a basic script using the point and click functionality of KITE.

Launch KITE and choose the “Record” drop down menu, not the big red button tempting though it is.  Then select “Record Simulated Browser Script”.

KITE will ask you for a URL to start out with, I chose Keynote’s website for this demo.

Next you’ll be presented with the KITE web browser, which loads the URL you provided. Now simply navigate through this browser session as normal and click “stop” once you’ve completed the transaction you wish to record.

Once you are done, KITE will test out the script you’ve defined right away. Afterwards you can add advanced steps, logic, rules, or edit any errors that are present. If you’ve purchased CApP you should know that you also have a support contract with Keynote, this allows you to get script help from us among other things.

 

For now let’s keep things simple and just save and exit. With this we are ready to provision a measurement in the Keynote Service Center. CApP is provisioned like other Keynote measurements, except that it has its own product box, shown below. The only other important difference is that measurements can be as frequent as 1 minute or infrequent as 60 minutes.

With a measurement, like this one, setup you can make use of the data in all the current ways that are possible in MyKeynote, whether it’s the dashboard, graphs, or alarms. Below I copied a graph that highlights the value of CApP for this particular use case: a graph that compares inside (You) and outside (Me) measurements.

Just to wrap up, you’ve seen at a high level how to setup CApP from scratch in order to get improved visibility into how your website or API performs up to the edge of your private network or cloud. This is a great addition for any customer who is already monitoring their properties externally.

The Website Sausage Factory and Impact on Performance – a TechCrunch Case Study

By Ian Withrow

As we’ve discussed in various blog posts, websites are like a sausage.  Ok maybe not so directly, but like a sausage they are made of ingredients than come from many sources even though they are presented in one tidy package to the user.  Today I’m going to break apart the sausage that is the TechCrunch blog, show how this can be easily done for any website using KITE, and as a special bonus show how a sausage maker can monitor all the pieces of their links using a Keynote technology called Virtual Pages. 

TechCrunch Composition

First a fair warning: just like with sausage making, finding out what is inside your favorite website is not always pretty.  If you feel you are of a squeamish disposition then you have been warned.  Second, note that all details in the post are from the time of writing and the balance of ingredients is likely to change overtime.

TechCrunch is a behemoth of a site, weighing in at just over 4 MB of data, 329 page elements, and a whopping 65 domains.  About half of this comes from them directly or really via Wordpress who is evidently the platform they use to power their blog.  The rest comes from over 20 3^rd parties.  You read that correctly, 65 domains in total and half the content originates from someone else.  After direct content the next biggest category is from social sites; Facebook, Twitter, and tools related to these properties total about 1 MB.  Google is about a 500 KB and the ‘Misc.’ category of various 3^rd party tools that TechCrunch uses to improve user experience is about 200 KB.  Ads and ad related content are about 180 KB.  Below is a snapshot from KITE breaking it out for you.  (Note you need to complete the process in the next section to actually get this view)

Now it’s time for a few fun observations.  The amount of content from Facebook and Twitter is huge!  Each alone is bigger than most websites are in total.  Digging into this is unfortunately off topic for this post but it is definitely something on my radar screen for the future.  Another interesting area is the level of user tracking that goes on.  I could identify at least 6 different 3^rd parties that were tracking TechCrunch visitors, not including Google and Facebook.  TechCrunch knows what you are, if not who.

Finally, while these stats make it seem like TechCrunch is hardly advertising, understand that TechCrunch is a very, very, very long page (vertically) and all the ad content is at the top where the user is most likely to see it.  They aren’t dummies giving away their yummy sausage for free.

Scripting Sites for 3^rd Party Monitoring & Analysis

To make sense of this mess I used Keynote’s KITE product.  There are a lot of other great, free products out there that one can use to view all the content and domains of a page.  However, KITE has the ability to permanently parcel out these domains into what we call Virtual Pages for ongoing monitoring and analysis.  Note this section won’t be a detailed how-to; I’m going to focus on highlighting what is possible with the tool.  After which you should be prepared to experiment or watch this training video depending on your learning style.

After downloading TechCrunch in KITE I organized the content by domain as shown below.

This lets me easily see the composition and breakout of a page in a manual fashion.  If I just want to see the domains I can simply collapse the domain groupings.  There are tons of options that I can add to this view like content size and various time breakouts based on my interest.  Here is a complete list:

With just this you can see that I can casually learn a lot about the page.  However, if I’m serious about how TechCrunch and its 3^rd parties perform then I need ongoing data points.  If I’m going to gather a lot of data then I don’t want to do this parsing and analysis manually, it just won’t scale.  The solution is to organize this content into permanent logical pieces.  For example, in a simple scenario I’d carve out a Virtual Page for my advertising so I could monitor and analyze the performance of that content separately from my content with Keynote.  As you can imagine the more complicated your site becomes, the more important this exercise is.  True you can always pick through a waterfall manually to see who did it in the event of the problem but if you want to have ongoing data about 3^rd party performance or be proactive with alerts then you’ll need something like Virtual Pages.  The nice thing is once you’ve designated content into a Virtual Page you can monitor and analyze it like a regular page.

Let’s discuss how I broke-up and organized TechCrunch.  Please note I’m not holding this up as the standard for the best or only way to use Virtual Pages.  One thing we need to keep in mind is cost.  Each page (virtual or otherwise) adds to the cost of the measurement and so in the real world we probably can’t go hog wild with these.  Given an unlimited budget I’d define a Virtual Page for each 3^rd party, possibly even one for each domain if I was especially crazy for detailed data.  My guess is you live in the real world and even if your site isn’t as complicated as this one you’ll need to create some buckets.  Most likely you’d start with prior experience, defining Virtual Pages where you knew or suspected there was a problem.  Here I simply broke the site into the following logical categories:

• TechCrunch direct content plus AOL
• Google (but not Google owned advertising)
• Facebook
• Twitter (and related tools like Postup)
• Wordpress (even though this is the core of the site, I want to evaluate my vendor here)
• Misc. Tools and Widgets for the users
• Analytics and User Tracking
• Ads and Ad related content

Why no CDN category? We certainly encounter CDN’s here but each is tied to a specific 3^rd party.  Facebook has its own CDN, the ad platforms have CDNs and so forth.  So instead I left the CDN’s with their respective masters.

Here is a brief teaser for how this is done in KITE

Step 1) Pick the URL you want to virtualize and run the page once (we did this already)

Step 2) Right click on the page in question and select ‘Insert Virtual Action’

Step 3) We now have a new Virtual Page at the bottom of your script.  Right click ‘Match Page Elements’ and select ‘Add URL Match’.  Here I’ve used the naming convention “vp:TechCrunch” to distinguish Virtual Pages from real pages.  You can name them anything you want in practice though.  There are other options that you can use to construct Virtual Pages, such as content type, that have interesting possibilities but to address 3^rd party content, URL seems ideal to me.  As you see below I’ve created a list of URL matches that should capture all the differently named TechCrunch domains.

Step 4) In the Script Properties Editor you can create the settings for each URL Match.  Note in my script I used a variety of regular expressions so that I could get away with far fewer rules than the 65 domains and still cover all of the page content.

Note that to do this I never had to write any code or do any advanced scripting.  It was all point, click, and form completion.  Hopefully by now you can see how easy it is to create Virtual Pages in KITE and have an idea of its possibilities.

How can I Benefit from Virtual Pages?

The obvious and immediate answer is you can now isolate and monitor the performance of certain 3^rd parties or subsections of you website.  If Facebook slows down you’ll know immediately and explicitly that this is case regardless of the overall impact on your performance.  Moreover, you can easily track and directly report on the performance of these guys overtime without needing to manually crunch the data and objects yourself.  Another interesting possibility is you could monitor your own additions to your site to see how they fair.  Finally, another angle might be to isolate and monitor all the Javascript that your site utilizes.  There are a countless number of ways that Virtual Pages might be used, and my list probably just scratches the surface.  Have fun with it!