Deep Link to Keynote Website Performance Data

By Ian Withrow

MyKeynote is designed so you can easily move from high level trends and events down to detailed data points. Users start with an alert or a scatter plot and then iterate down into successive levels of detail, typically culminating in a waterfall chart and details for a specific measurement that we took for them. However, sometimes a customer already knows what specific data point they wish to drill down onto.  For example, if an APM or RUM solution that you have captures a Keynote transaction you would naturally want to compare the user perspective of that transaction side by side with the system perspective provided by your other tools.  In this way you could see both endpoints.  Deep linking to detailed Keynote data points could also be handy in highly automated operations enviroments.  Imagine if your trouble tickets included direct links to poorly performing data points associated with an alarm?

As you may have guessed by now, in this post I’m going to show how you can link directly to a data point or waterfall in Keynote. I expect this will prove handy for operations tools, devops, and performance analyst types who want to compare the Keynote data with APM data or possibly want to iteratively retrieve waterfall visualizations of data that meets specific criteria. In fact Keynote partners with a number of APM vendors like OpTier, ExtraHop and OPNET so you may soon see the ability to link from one of these APM solutions to Keynote visualizations.

Before we proceed I must mention that you will need our Data Feed or DataPulse service.  DataPulse provides a near real stream of Keynote data. Data Feed provides the same data but every 15 minutes.

Keep in mind that I'm appropiating functionality here that wasn't designed for this purpose. Yes it's a creative hack, but don't let that stop you from enjoying the benefits.

 

How To Deep Link Into Keynote

MyKeynote offers two types of visualizations that you can link to directly if you know when the data point was collected, which agent took it, and which transaction or script it is associated with. Happily all of this information can be retrieved from the aforementioned data services. In the below example I use excerpts from Data Feed XML files. DataPulse has all the same information; the XML file is just formatted slightly differently. Either way Keynote has user guides for both of these services if you want more details on them. From this point I’m going to assume some familiarity with these XML files.

 

Step 1 – Gather the Variables Needed for the MyKeynote Request

In XML Data Feed, each record starts with the following tag. The variables we care about are colored green.

<TXN_MEASUREMENT agent="45537" slot="845480" datetime="2011-JUL-05 22:37:51" target="1034270" agent_inst="45538" profile="0">

Additionally if we have a multiple page transaction we’ll need to know what page we care about. Each page record in Data Feed has the following tag which specifies page information.

<TXN_PAGE page_seq="1">

The hardest part is that we need to convert ‘datetime’ to a base UTD number. In English, this is the number of seconds that had passed since 01-01-1997 12AM (GMT) when the data point was captured. This may sound hard to figure out but writing a program to do this conversion is fairly straightforward. Here is how to quickly do the conversion by hand.

1. Go to: http://www.epochconverter.com/ a handy site that lets you quickly convert data and time formats between human readable version and Unix Time
2. Copy ‘datetime’ from your XML file, in our case the value is 2011-JUL-05 22:37:51
3. Enter this value into the “Human date to timestamp” converter and run it as shown below
4. Subtract 852076800 from the result; this converts the number from Unix Epoch time to base UTD.
5. In this case we get the value 457853871, save this result for later since we will need it in the next step.

 

Step 2 – Build the MyKeynote Request

First, a caveat these requests are only fulfilled if the user is already logged into MyKeynote. If not they’ll be asked to login and will need to try again. As mentioned previously we have two options.

1. Display a page waterfall
2. Display a transaction summary which compares multiple pages in a single transaction.

If what these two options represent isn’t immediately clear to you that’s ok. Below I describe how to form a request for each along with a screenshot that shows what you’ll get for your efforts.

 

Waterfalls

A waterfall can be requested using the below URL with the variables you need to fill in denoted in {brackets}.

http://my.keynote.com/newmykeynote/transpagesingledrilldown.aspx?transid={target}&agentid={agent}&butd={UTD}&ksid=5&profid={profile}&pageid={page_seq}&mskin=0

As you may recall in the previous step I described how to find each of these inputs.  Remember if you have only one page in the transaction then the value there is of course 1.  Below is a completed example and the resulting output.

http://my.keynote.com/newmykeynote/transpagesingledrilldown.aspx?transid=1034270&agentid=45537&butd=457853871&ksid=5&profid=0&pageid=1&mskin=0

Transaction Drill Down

For many transactions each data point will be comprised of multiple pages. If you want a detailed view comparing each page, instead of a waterfall for one page, the following request will do the trick.  Again variables are specified in {brackets}.

http://my.keynote.com/newmykeynote/scatterplotdrilldown.do?transid={target}&agentid={agent}&butd={UTD}&profid={profile}&pseq=-1

Note that in this case we don’t care about page number, if you did specify a pseq value it would actually take you to the waterfall for that page.  A completed example below:

http://my.keynote.com/newmykeynote/scatterplotdrilldown.do?transid=1034270&agentid=45537&butd=457853871&profid=0&pseq=-1

Building a Bridge from a Log

Now some of you may be saying wait a minute, what if I only have the log or the header from the Keynote requests to start with?  How am I going to get from that to the Data Feed or DataPulse record?  Fortunately we add a header to each Keynote agent request formatted as below.  In the header is the target and UTD value which should be sufficient for you to find the right XML record.  Example:

X-KTRACE

457853871|45538|1034270|0|1\r\n

Or rewritten using our variable names:

UTD|45538|Target|0|1\r\n

Wouldn’t it be nice if the agent number was in there too?  Sorry I warned you this functionality was developed for another purpose and that we are being opportunistic here.

 

Conclusion

I recognize this won’t be for everyone. For those operational groups that are trying to shave off every bit of time they can from their MTR numbers then being able to jump directly to detailed Keynote data points from say a Nagios or an APM system will save you valuable time. Do let your account team know if you like this an want an expanded solution.