Web Performance Watch

Test Your Site on IE 9 and Measure User Experience

IE9Last month, we announced that the Keynote Global Network was being updated with Internet Explorer 9. As a result, our real browser monitoring service, Transaction Perspective™, is now measuring the performance of Web applications and sites using Microsoft’s latest Web browser. This makes Keynote the first on-demand monitoring service built on IE 9, which is pretty cool. But what’s even cooler is the ability that IE 9 gives us to measure a new class of performance metrics we call user experience metrics.

As we’ve discussed here previously, IE is still the big kid on the block when it comes to browser usage. With the demise of IE 6 in the United States, and the rise of Firefox and Chrome, it’s clear that users are quickly leaving “old” browsers for “modern” ones like IE 9. With high performance and broad support for open Web standards, browsers like IE 9 make it easier for companies to create a rich and snappy experience for consumers. In response, 34% of the top Internet sites now use HTML 5, and the use of JavaScript continues to rise. Transaction Perspective built on IE 9 allows customers to get a more precise view of their sites’ performance, especially those leveraging new Web standards.

Our new Live Beta preview of MyKeynote 11 with Transaction Perspective lets you see performance in very important ways:

  • Time to First Paint – This new metric tells you when a user begins to see your site render in the browser.
  • Time to Interactive Page – Tells you when the Document Object Model (DOM) begins to process user events for the document.
  • Total User Experience Time – With User Experience Time, you know how long a page (or series of pages) took to render and become usable for a real user.  It is the ultimate measure of a page’s speed, factoring not only the time it took for data to be downloaded, but also rendered and made interactive.

These key moments are just a handful of the browser events we capture. If you’re a performance expert, you’ll appreciate that we measure all the Browser Navigation Timing events and can graph them individually over time (multiple measurements), as well as display them in a timeline view for an individual measurement.

Timeline

24x7_monitoringKITE (Keynote Internet Testing Environment) lets you to test for free your site’s performance from 5 cities on the Keynote Global Network, on demand. But if you’d like to test drive User Experience monitoring, click the “24x7 Monitoring” button in KITE.

Once you’ve activated your trial, click the “Try Beta Version” link in MyKeynote. There, you’ll be able view User Experience metrics for everything monitored during your trial.

Try_beta

Version_checkSoon, you’ll be able to see these new browser event metrics from your desktop in KITE, as well. Your copy of KITE should automatically update itself to version 5, or you can manually check for the new version. 

Let us know what you think of these new features!

Posted by Aaron Rudger on February 21, 2012 at 07:07 AM in Application Performance Testing, Test Website, Web Page Monitoring, Web Performance, Web Performance Testing, Website Performance Monitoring | Permalink | Comments (0) | TrackBack (0)

| |

What Happens When IE Has Less Than Half?

For the last decade, developers have been told that because Microsoft’s Internet Explorer dominates the Internet population’s Web browser usage, applications and sites should be built for IE first. To a certain extent, 50% has been a magic number. It’s guided app design, test designs and QA processes. Now, it’s clear that IE will soon lose its greater-than-half leadership in the global marketplace. The latest statistics from Net Applications show IE at 52.6% share worldwide and on pace to go below 50% in two months.  

Net_applications-Oct_trend
What happens then? Will the Internet suddenly seize up like our computers did on 1-January 2000?

Clearly, the passing of the 50% threshold is purely symbolic. IE’s share in various regional markets has already dipped below half, and has long since broken through the mark according to StatCounter (which uses a different methodology than Net Applications). Developing exclusively for IE is the exception, not the rule today. But this milestone eliminates yet another reason for not building and testing websites across multiple browsers. Even as modern browsers adopt greater standardization, the differences still matter—especially when it comes to performance.

For Microsoft, the challenge is no longer to maintain IE’s super-majority status. Rather, it is to effectively compete against Chrome, and Firefox on the Windows 7 (and beyond) platform.

Net_applications-Oct_win7

This strategy appears to be working. Since the introduction of IE9, IE’s share in North America has remained stable, despite Chrome’s gains.

Statcounter-Oct_NA_trend

But as the mobile browser continues to grow in popularity and usage with smartphone and tablet consumers, the trend is undeniable. It’s a multi-browser world that requires multiple screen solutions. What do you think is the significance of IE’s market share falling below 50%?

 



Posted by Aaron Rudger on November 01, 2011 at 04:07 PM in Current Affairs, Test Website, Web Performance | Permalink | Comments (0) | TrackBack (0)

| |

Deconstructing the Target.com “Fail Doggie”: A Keynote Perspective

Have you met the Target.com “fail doggie?”

He’s cute, but if you are a Target customer, you don’t really want to encounter him in a browser.  I met the “fail doggie” while using the MyKeynote portal to research the nature and extent of the major outage that occurred on Tuesday, September 13th 2011.  That was the day that Target began allowing online purchases of “Missoni by Target” items.

Fail-doggie-intro

A lot at stake

According to one source, the Target site re-build from scratch, launched just weeks before this incident, drew on the talents of over 20 vendors, including many of the biggest names in the e-commerce technology space.

Imagine spending millions of dollars and two years of time to “create a more user-friendly, reliable experience” and then have this happen.  Not fun.  When a major event like this outage happens, it can be difficult to get complete details from any one participant.

Passions are high when a crisis like this occurs.  Where can you go to get an objective vantage point from which to make accurate assessments and analysis?  At Keynote Systems, we’ve long been looked to as a neutral third-party with accurate and actionable Internet and mobile performance data, and that day proved to be no exception.

In this blog post, I’ll explain how I used the tools that every Keynote customer has in MyKeynote along with measurements being run for two of our public web performance indices to determine what users were seeing that day and the following morning and to determine just how extensive the outage was.

Target.com appears in a number of our public index measurements, so I had several to pick from – note that we never publish insights based on a customers’ private data, but only based on publicly available data that we collected without payment by any company. Two were particularly useful for figuring out what was going on and capturing screenshots as the day went on. The first measurement visited their home page only, while the second arrived at the home page and then performed a multiple step transaction, just as a customer would when shopping, placing items in a cart and checking out. 

First a brief bit of background: both scripts were written in the Keynote Internet Testing Environment, known as KITE and both measurements were being run with our real browser product, Keynote Transaction Perspective. I point that out so you’ll know that none of what you will see here was retrieved by a “bot” or other emulation system; we were getting the experience of a user launching an actual Internet Explorer browser to go visit Target.com.

How things went down that day

As word of the outage quickly spread, we started taking calls from various news media companies asking if we had data.  We took a look at the home page monitoring scatter plot below and saw what looked like a brief spike in response times followed by a restoration of reasonable response times shortly thereafter. 

02-firstScatterPlot

(click image to view full size)

At first glance, it was tempting to say that the outage had been brief and thus was relatively uneventful.  But a deeper look at the chart revealed telltale signs that something was not right.  Notice the distribution of the dots before and after the spike?  See how they are randomly distributed somewhere between two and six seconds before the spike (time on the network is on the left scale) but then they are all tightly packed down below the one-second mark afterword?

Each of those dots represents a full visit with a real browser, so I could click down on any of them to see a listing of what was on each of those pages and how long it took to get to the browser.  I did that eventually, but much like an operations team technician would be at such a moment, I was in triage mode at this point.  The first thing I wanted to do was to look for clues as to why those dots were organized as they were.

First, I hovered my mouse over a datapoint from before the spike in performance.   This would be my “normal” baseline to compare to.  Notice that the page contained 147 elements (separate downloaded objects) and a total size of about 2 MB.  The time on the network to download the page and all elements was 5.156 seconds. 

03-baselineScatterPlot

Next, I took a look at the two red triangles, which represent pages where we know there was an error of some sort or another:

04-SpikeScatterPlot

In this case, the element count and page size were lower than the baseline but the time to download was skyrocketing to a number eight times higher. The lower object count and page size were due to timeouts being hit… we couldn’t get all of the objects into the browser before time was up, so the agent running the browser quit trying and reported the error.

What was really interesting was the object count from the very next green dot after the red triangles.

05-1ElementScatterPlot

That data point showed only one page element and a very small download size of 640 bytes. 

One element?  I knew that if we had downloaded only one element, that it must be the base page of html, but a page with only 640 bytes couldn’t possibly have much to say.  That was my first clue that visitors were probably getting raw error messages.

I quickly scanned over the remaining “spiky” datapoints after the incident began and found more of the same: just one page element and a size of 639 or 640 bytes.   Here’s the last point caught during that initial spike:

06-639byteScatterPlot

So that was all fairly consistent with a site failure; something went very wrong around 8:00am EDT and the server took a long time to send out a very small page that was probably just an error message. 

What about all of those green dots hugging the bottom line after the spike had subsided?   I hovered over a few and got the same thing each time: 5 page elements and a page size of 31550 bytes. 

07-5ElementScatter

This wasn’t some random subset of the real page and no error was being recorded, so clearly these speedy responses were something altogether different. 

Now it was finally time to start drilling in for some details.  I clicked one of those data points and made my way to the page detail waterfall graph:

08-PageDetailWaterfal
 

(click image to view full size)

Welcome to our waiting room

I hovered over each of the bars and noticed that each object came from a folder called “spawaitingroom.” The image files consisted of a red stripe, a Target logo and a photo of the Target dog posing next to a tool box.  I had met the “fail doggie” for the first time:

  09-WaterfallPopupDoggie

(click image to view full size)

So let’s recap what I had learned so far:

  1. Initial data points have no page objects… just the base page, and that base page was TINY (640 bytes) in comparison to the pages that preceded it, but the amount of time the server took to return that page was HUGE.
  2. The datapoints following the big spike downloaded much faster and had larger base pages but only had five page elements instead of the 140+ found in the normal pages.
  3. Drilling in on the five-element page datapoints, I found that all five of the objects came from a folder called “spawaitingroom” – the “fail doggie” was part of a “waiting room” feature that was being served in lieu of the real home page. 

Getting the whole picture: what were people actually seeing?

I was making good progress, but I still had a lot of questions to answer.  I could guess that a Target dog next to a toolbox was probably some variation of an “under construction” page, but I didn’t have the text of the page yet.  I really wanted to know what those pages looked like but all I had were some pieces to the puzzle.

The thing that was complicating my sleuthing was that neither the original 640-byte server error pages nor the fail doggie pages were being sent as an “error” (http status of 4xx or 5xx) – they were being sent as successful pages (http status 200).  That prevented me from getting as much diagnostic information as I otherwise might have.  I could poke around at each scatterplot and piece together what I was seeing, but without an error, I wasn’t going to have the html from the page or any screenshots, both of which MyKeynote stores on a fatal error. Fortunately, all was not lost.  There's a benefit in more than 400,000,000 objects per day stored away inside MyKeynote.

Web Content Trending to the rescue

Here’s where the other measurement that was scripted to go five steps deep into the target.com site came in very handy.  The second monitoring script was set up to do the following:

  1. Go to the target.com home page.
  2. Perform a search for “lil wayne”
  3. Filter the search results to the “music” category.
  4. Click on the first album in the resulting list to view the details.
  5. Click the “add to cart.” button on the details page and then confirm that “1 item added to cart.” appears on the screen.

Without the real site being served up, there was no way the script could complete the search for an album.  When the Keynote agent piloting the Internet Explorer browser went to find the search box and type “lil wayne” into it, there was no search box. The resulting error provided a steady stream of screenshots of the home page throughout the day.

Let me explain a little more about why I got the screenshots.  With Keynote’s Web Content Trending option turned on, every screen is proactively captured and if an error is detected in subsequent steps, all captured screenshots are saved to the MyKeynote portal to support troubleshooting by our customers.  If there is no error, the proactive screenshots are discarded before ever being sent to the database.  Every time the second step failed, (which was on EVERY visit at this point), the screenshot of the home page taken on arrival was stored.

To get those screenshots, I simply had to drill into the scatterplot chart and I saw this:

10-EisForError

(click image to view full size)

I clicked on that “E” which is the error recorded for the second step and saw the page details screen below. I have added callouts so you can see where the links to the screenshot and html are:

11-SnapshotAndHTML

(click image to view full size)

Things were about to get a lot clearer in a hurry. 

The wrong kind of "direct communication"

I clicked on the thumbnail of the screen snapshot and sighed as I saw what visitors had seen in the moments when the site became unusable around 7:58-8:00am:

12-ServerError

We captured the above screenshot at 8:01am.  It shows what the 640 and 639 byte html pages with no images looked like in the browser.  This is a raw server error that was passed through all the way to users’ browsers (something developers and operations teams work very hard to avoid).

In the minutes that followed, the target.com team took rapid action to replace the cryptic server error with something more friendly.

By 8:14 am, we had captured the first of those friendlier images; the “fail doggie” page made its debut. About 17 minutes later we captured a new version of the page.  We saw additional changes again at 11:30 and 12:38.  It should be noted that these are times when we visited with the transaction-based measurement and that particular measurement was only set to visit ten times per hour.  The point is that these times are when we observed the pages and made captures, not necessarily the exact times that they changed.

Here, then, is the full gallery of “fail doggie” pages we recorded:

8:14 am EDT – “Oh no”

This page requested the user to “please try again” and provided a single link to “Target help”

13-OhNo

(click image to view full size)

8:31am EDT – “Hello”

This page let folks know the team was “hard at work making the site better” and dropped the “please try again” in favor of “Sorry for the inconvenience – we’ll be back up and running shortly.  It also featured links to three services that were still online: redcard, weekly ad, and find a store.

14-Hello

(click image to view full size)

11:30 – “Woof” #1

Around 11:30 the message changed to “We are suddenly extremely popular.”  Visitors were also asked to “Please stay here and we’ll try to get you in as soon as we can!”  Finally, this version also explained what the three links that began appearing in the previous version were, saying “We are up and running here” just above the links.

15-Woof-01

(click image to view full size)

12:38pm EDT – “Woof” #2

This version added a plea to not keep hitting refresh, something that can make bringing a site back up very difficult when a large group is all doing it at the same time: “Please know that there is no need to refresh your browser.  Your request will automatically retry in 30 seconds.”  This was the most well organized page, broken into four separate paragraphs, and adding back in an apology with the sentence, “Thank you and our apologies for the inconvenience.”

16-Woof02

(click image to view full size)

Sizing up the impact: just how bad was it, and for how long?

I was starting to put it all together.  Now I knew what the raw server error looked like and I had a play-by-play set of screen captures showing how the “waiting room” page evolved over time that morning. 

What I still wanted to know was just how “unavailable” the site had been.  Were any users getting through to the real home page at any point?  The fail doggie page said it would auto-refresh in 30 seconds and implied that perhaps one might eventually get through.  Was the real home page ever turning up, and if so how often? 

I waited until the next morning to size up the duration and intensity of the outage.  My goal was to confidently establish an “end point” for the incident and then do the numbers.

How can you measure “availability” when the site is displaying “OK” pages, but not the right ones?

The home page only measurement had the highest frequency (about 40 per hour) so I wanted to use that to calculate availability.  There are many different reports and graphs in the MyKeynote portal, and most of them will tell you at a glance what your “availability” is – that is, what percentage of the measurements succeeded versus failed in some way.  The catch here was that server error and “fail doggie” pages were sent to the browser as “OK” (http status 200) pages, not errors. 

Scripts can be easily enhanced with validations that look either for required text that should be there or error text that should not be there.  Either validation option would have marked the server message or fail doggie pages as errors and impacted the built-in availability calculation, but the index measurement I was using didn’t have that validation in place.  In practice, validation is usually used at the end of a multi-page script to be sure the right final page had been reached.  Fortunately the information we needed to answer the questions was readily available anyway (more on that below). We just had to step back and consider the available tools to size it up.

The tool I chose to use was MyKeynote’s Object Trending report.  This report is available for any measurement that has been configured with our Web Content Trending (WCT) option.  WCT stores performance information for every object in the page, not just a roll-up for the page as a whole. Once again, storing all those details day and night was about to become very handy.

The Object Trending report has several options, all of which provide ways to view the performance of page elements over time.  To display it, I chose the Target measurement, selected Object Trending and set the date range to the 24 hour period after the incident had begun:

17-ObjectTrendingSelected

(click image to view full size)

Here’s what that report looks like by default, which is a separate line for each domain that objects originated from:

18-ObjectTrendingGraphDisplayed

(click image to view full size)

The above default view is great for determining if one or more domains are particularly slow or unavailable, but I needed even more detail than that, so I dropped down the menu at the bottom of the graph and changed it to “Object data by object without parameters:”

19-OTWithoutParams

This option would give me each separate element in the page, ignoring any query-string data that might be appended to the object name.  Think of the home page as a stage with a cast of characters.  The object trending report was about to show me who had appeared in what number of performances. 

I clicked “Generate Graph Now” and scrolled down to the table below the graph. Now I had a listing of the frequency of every object that had appeared on the home page from 8:00am EDT onward:

20-ObjTrendTableRoughCutEdges

(click image to view full size)

What I was particularly interested in was the object name on the left and the “Included datapoints” on the right.   Every visit always had retrieved at least the base page (www.target.com). By comparing the count of base pages observed to the count of all the other objects, I could make some meaningful conclusions about how often the “fail doggie” had been turning up. 

I needed to do a little math, so I pulled the results into an Excel spreadsheet with a simple copy and paste, sorted by the datapoints column and added a new column to compare the count of each element to the count of the base page.  I did this a number of times, varying the time period a bit to narrow in on useful takeaways.

For starters, I re-ran the report to look just at the objects observed in the first hour between 8:00am EDT and 9:00am EDT.  The cast of characters is quite small; we either got just the base page of html, or the base page plus the elements of the fail doggie page.  Remember, these aren’t observations of all user traffic, they are the results observed by the visits of our real browser agents. While we were just a drop in the bucket of actual visitors, we did make it through to the site 31 times in that first hour.  Here’s what we saw: The “fail doggie” appeared just 55% of the time and we got nothing but the base page in the remaining 45% of visits.  The hundreds of thousands (or millions) of users attempting to visit during that same period likely saw a similar mix.

21-ExcelFirstHour

(click image to view full size)

So I could see that things had been pretty “messy” during the initial confusion of the incident.  An hour of outage is worth a lot of money to a site like target.com, and cryptic errors couldn’t have been good for inspiring confidence, but an hour is just an hour and perhaps many people hadn’t even tried to visit the site yet. 

The next question I wanted to answer was, “What percentage of Target.com’s traffic was able to make it to the home page throughout the rest of the day?”

Here’s a view of my spreadsheet based on running the Object Trending report for the period starting one hour after the incident (9:00am EDT) and ending at midnight that evening:

22-Excel-9-Midnight
(click image to view full size)

So taking the big view of that entire day up to midnight EDT (which is 9pm Pacific time), I determined that 93% got the fail doggie and no more than 7% got through to the real home page.

Not pretty no matter how you slice it

I re-ran the report with various time windows, and the results varied only slightly.  Most surprisingly, even pushing the end time all the way to 9am the following day, the stats still showed that 85% of all visits got the fail doggie page elements.  Focusing on Midnight 9/14/11 to 9:00am 9/14/11 the number was still high at 75%.  Was this all cleared up by the start of business on 9/14/11?  Looking at just the hour between 8:00am and 9:00am that second day 9/14/11, the number was still an amazing 50%.

By this time my boss was wondering why I was spending so much time with all those spreadsheets and charts, so I stopped my investigation there and moved on to share updated results with all the folks that had been asking for data.  I annotated a screenshot of a scatterplot graph and wrote a narrative to go with it, sharing the details with several media outlets and even conducting a radio interview with Wall Street Journal Radio.  Here are a couple of links to places the results showed up, along with a copy of that annotated scatter-plot graph.

Investor’s Business Daily: Target Website Crash Offers Lessons

Retail Online Integration: What You Can Learn From Target's Site Crash

23-target-scatterplot

(click image to view full size)

Epilog:

The one question I left unanswered was just how many truly useful pages there had been in the 7% that were not “fail doggie” pages.  I’m curious how many would have actually allowed a customer to purchase.  Given the size of Target.com’s typical traffic this time of year (reported as 29.5 million for the month of the prior October by Investor’s Business Daily), I was pretty content to stop with the work I had done over those two days, observing with a long sigh that the folks at Target, who are no amateurs at online retail, had missed out on a LOT of potential transactions by turning away something north of 93% of all visitor attempts that first day.

Posted by Dave Karow on September 30, 2011 at 05:10 PM in Application Performance Testing, Current Affairs, Load Testing, Test Website, Testing Web Applications, Transaction Monitoring, Web Load Test, Web Page Monitoring, Web Performance, Web/Tech, Website Availability Monitoring | Permalink | Comments (1) | TrackBack (0)

| |

The Target.com "fail doggie" -- it didn't have to happen

 
When Target thought ahead about the headlines they would make with the launch of a limited partnership with designer Missoni, it probably didn’t expect this one:

“Demand at Target for fashion line crashes web site”

While we usually hear about retail Website crashes on or near Black Friday and Cyber Monday, Target.com’s recent site outage is remarkable for its atypical timing, severity and length. Careful investigation into the origins and impact of the incident reveal important lessons for all online retailers. Ultimately, failing to properly prepare for an incident like this is no option for merchants striving for success in today’s connected consumer market.

What Happened?

Keynote Systems' measurements of the target.com website indicated that traffic began overwhelming Target.com around 8:00am EDT. The intense wave of traffic initially resulted in server errors. Quickly, the operations team responded by serving “friendly” error messages around 8:14am. After that, only about 7% of the visitors were able to successfully reach the real home page throughout the remainder of the day. “Fail puppy” error pages were still being measured by Keynote well into the following day.

 Target-scatterplot

(click image for larger version)

Fallout

Credit the marketing and merchandising teams for generating the hype the drove the demand for Misonni. Were their efforts wasted when Target.com went sideways for two days? A better question might be what destruction did they wreak on sales as a whole during that time?

What drives the financial impact of a site outage or performance issues is abandonment. It’s important to understand that visitors experiencing a Website with issues don’t result in lost revenue, per se. Only when those visitors don’t return, and/or go somewhere else is revenue lost. A shopper’s tolerance for errors is called “tenacity” in Web load testing parlance. Low tenacity shoppers bail from slow searches and hanging shopping carts in a dash. Missoni brandistas, however, were very high tenacity shoppers. They kept trying to buy despite the errors, which also likely contributed to the outage’s long duration.

So while Target’s outage surely didn’t deter some of the Missoni faithful from eventually returning to place orders on another day, the real financial impact was to the other product category sales that couldn’t take place during that time. The iPad shopper went to one of Target’s competitors, instead. According to Nielsen NetRatings, for every 1000 consumer electronics shoppers that experience three errors, the cost to the site is nearly $100,000. The Target.com site gets 1.3 million visitors a day this time of year, and it presented repeated errors to 93% of visitors on Tuesday. You can start to get a sense of how that adds up!

 Target-faildoggie-woof

(click image for larger version)

Realistic Web Load Testing Prevents Outages

The only way to prevent events like this from happening is to perform realistic Web load tests that prepare for these scenarios. In this case, Target claims to have prepared for the kind of traffic they typically experience on Black Friday—a build-up over the course of the day. Clearly, this was not the realistic scenario to prepare for. The Missoni line went available at a certain time and day, more akin to a ticketing merchant where a concert sells out within minutes of its availability. They did not expect the suddenness of the demand surge. 

If a load test had been run that adquately modeled the impact of the planned launch, the damage would have been done outside business hours and Target management would have been able to decide whether to make changes to their systems and retest, postpone or restructure the launch or just "risk it" and see what happens.  We don't really think they ever had the chance to make those decisions.  Surely they didn't conduct a load test that predicted such an epic fail.  But should they have?

Realistic Web load tests model site usage and shopper behavior. Systems are deployed to simulate high levels of demand from multiple geographically disperse areas. Once the load is generated, the infrastructure and application’s response are watched carefully to identify bottlenecks and breakage points as the entire mesh of the Website’s interconnecting parts are stressed. Only this level of testing can accurately inform e-commerce teams of their preparation adequacy.

 

Posted by Dave Karow on September 23, 2011 at 12:33 PM in Application Performance Testing, Current Affairs, Load Testing, Test Website, Web Load Test, Web Performance Testing | Permalink | Comments (0) | TrackBack (0)

| |

Useful Paranoia

 

[108/365] Ill-advisedThe recent Sony and RSA hacks likely threw the IT leaders at those companies into a state of panic. Unanticipated events like breaches and outages stress the organization fabric of IT teams, and can result in pain. It’s almost like tearing a muscle. But pain, like many things, is an experience to be viewed in context and can be rationalized beyond the difficulty it imposes.

For example, if you’re an athlete, pain is expected. Athletes train hard, repetitively breaking down muscle in order to build it. The best athletes are training and workout fanatics. They throw themselves intentionally into stressful situations. No pain, no gain!

Some IT organizations do this too. They contract with outside firms to attempt breaches of their systems. They throw their teams into chaos drills where resources and assets become unavailable. The adage “people, process and technology” applies supremely here, with heavy emphasis on “people.” For example, we’ve seen that the best load testing projects on which we’ve worked are those that involve extended teams and intentionally simulate situations for which they were unprepared.

If you’re feeling comfortable today, tomorrow’s the day to throw a wrench in the system. A little paranoia is healthy. How are you stressing your people, process and technology?

Posted by Aaron Rudger on June 02, 2011 at 02:19 PM in Current Affairs, Load Testing, Test Website, Testing Web Applications, Web Load Test | Permalink | Comments (0) | TrackBack (0)

| |

Is it possible to be “point and click easy” but also precise and accurate? I set to find out by using KITE to make a random departure into space

KITE is the tool we use at Keynote to build performance monitoring scripts.  One question I get from time to time as the KITE product manager is, “Is it just point and click to record a script?” and another, related question is “Can it really do exactly what I need it to do if it’s that simple?” 

Yes, KITE can capture multi-step transactions by just watching you point and click in its browser window. But there is more to it than that, and in this blog post and some others I’m working on down the road, I’ll demonstrate why that’s a good thing.

KITE is built more for power and precision than to eliminate the need for any effort.  We care a lot about accuracy here at Keynote and our demanding clientele insist on a lot of control over what gets monitored and how, so KITE has evolved over many years to reflect those values.  In other words, after you capture a recording, you may need to roll up your sleeves a bit if you want to take full control.

Here is an example, based on a fictional web site the folks at dynaTrace put together that lets you book space travel.  I want to select my origin on Earth and then search for available departures into space.  Here’s how that looks while recording the session with KITE:

Recording-gospace-01 
KITE is pretty smart.  It knows which element of the page you dropped down and records both the text label for it (what you see) and the index value (the unambiguous reference to that choice).  Here’s how KITE would show the choice of ‘France; Paris’ after performing a recording:

Select-france-paris 

The thing is, you would probably want your monitoring script to exercise all of the values, not just one.

So “point and click to record” then?  Check.  But now how do I go beyond the simple case to get what I really want, the random choice of origins each time the script runs?  Do I need to write code?  Nope, not this time.  Over time, we’ve put most common solutions right into KITE’s UI so code isn’t needed.  Here are the three steps I took, without writing any code, to accomplish my goal:

Step one:  Tell KITE to ignore the text value (France; Paris) and use the index instead

Select-use-index 

 

Step two:  Tell KITE that instead of a static (i.e. “hard coded”) value, that I want to use a random number for the index

Static-value 

Random-value

Step Three: Type in the minimum and maximum values I want KITE to select randomly from (there are 26 cities in the list):

Range1-26
 

That’s it.  When I’m done, the section of the script we originally looked at looks like this:

Select-index-random

So with a few clicks and a total of three characters of typing, I’m all set to head off to space from a random origin.   I rolled up my sleeves but barely broke a sweat. 

In my next post, I’ll go a little deeper, helping KITE know when some dynamic page content is just the way I want it before moving on to the next step.  I hope you’ll join me for that voyage J

Posted by Dave Karow on September 16, 2010 at 09:50 AM in Application Performance Testing, Test Website, Testing Web Applications, Transaction Monitoring, Web Page Monitoring, Web Performance, Web Performance Testing, Website Monitoring Software | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: Keynote, KITE, Monitoring, Recorder, Recording, Script, Scripting

| |

Search

Connect With Keynote

  • Subscribe to RSS
  • Follow us on Twitter
  • View us on YouTube
  • Signup for our Newsletter
  • Other Keynote Blogs:
    • Keynote Web Privacy

    • Keynote Mobility

    • Performance Watch - Le
     blog

    • Cloud Testing und Performance Monitoring

Keynote Web Performance Watch Blog

A forum for discussion and commentary on technology, trends and touchpoints of interest to the Web Performance community.

Recent Posts

  • A 10-Point Checklist To Ensure Site Uptime When Switching Web Hosts
  • Understanding the Impact of Web Attacks - the User Perspective
  • How Do I Compare Thee? Linode vs Bluehost Web Host Performance Shootout
  • Super Disappointing
  • Cyber Monday Breaks Records, But Could It Have Been Even Better?
  • Retailers Zip Through Black Friday
  • Firefox & User Experience Metrics, Now in KITE
  • From Network Performance to User Experience
  • Is an Outage Sometimes Your Best Strategy?
  • Filtering Out Web Performance Monitoring Traffic From Google Analytics

About This Blog

  • • About
  • • View Archives
Copyright © 1995-2012 Keynote Systems, Inc. All rights reserved.


  • • Terms of Service
  • • Privacy Policy
  • • Site Map
  • • Support