By Ian Withrow
In the 4.1 release of KITE we believe that we’ve significantly improved the ease and power with which users can create scripts to monitor APIs. Web APIs, aka Web Services, are an important way for companies to integrate 3rd party data and service providers both in the browser and in the data center. Web API’s that need to be monitored tend to fall into one of three categories.
- First, all the widgets and mash-ups from companies with Facebook, Google, and Twitter are built upon web APIs. However, even traditional companies like Best Buy, Hoovers, and the New York Times offer APIs.
- Second, most companies talk to one or more 3rd parties directly from their data centers. Examples from this category might include payment vendors like Zuora and PayPal and range to specialized service providers like Quova (IP Geolocation) and Netbiscuits (Mobile site delivery).
- Third, many companies are starting to build their applications using a Web Oriented Architecture, which calls for modularizing different components of the application and exposing them via a Web API.
Regardless of the scenario, the reason for monitoring these APIs is the same. API performance can have a significant impact on user experience especially when it drives critical transactions like checkout or the delivery of customized content. Unlike other monitoring solutions Keynote actually interacts with the underlying API, enabling you to test not only performance but to ensure responses conform to specific parameters, e.g. number of widgets in inventory is a number between 1 and 100. In this way you can reproduce the logic found in the actual application that consumes the API.
In this two part blog series I’ll walk you through exactly how to script and monitor APIs with Keynote both inside and outside of your cloud. This post will cover RESTful APIs. The second post will cover SOAP APIs. This article is long and in depth so below I’ve listed the major sections and the associated topics:
- Step 1: Create a basic web services script
- Step 2: Techniques for iterative development
- Step 3: Advanced script editing
- Selecting a Keynote monitoring product for your script
Step 1: Create a basic web services script
For the purposes of this demonstration we need an API, I’ve chosen the Active.com API because it’s a real, working API that has rich features like developer keys and multiple format responses.
To get started you’ll need KITE 4.1 or newer. If you don’t have KITE a copy can be obtained for free here. Launch KITE, and in the top left corner there should be a button that looks like a big red record button. Click the bottom half of this button which opens a drop down menu and then select ‘Create Web Service Script’ as shown below.
You will be asked to select a service type, go ahead and select REST.
Upon doing this you’ll be presented with a wizard, as shown below, that will gather input from you in order to construct an advanced script. First, you enter the base URL for the RESTful API function you wish to access. Next you specify the response format type you will receive, such as JSON, XML, or unstructured text. Then you can proceed to fill in all the URL query parameters that will be passed with the request. The Wizard supports all standard HTTP request such as POST and DELETE. You can also enter any special headers at the bottom; standard headers will be created for you. In this API, the license key is added as a parameter but other APIs may call for a header.
Don’t worry if you don’t have everything perfect just yet. You can manually edit the script later or simply recreate the Web Services action and the Wizard will remember the last inputs you gave it. When you are ready, click ok and you’ll see a new action added for you in the Script Viewer.
The best practice suggests that at this time you should rename the action something logical by clicking on ‘Call method’ and edit the Script Property shown below. My API call retrieves events in the San Diego area.
Now we are ready to view our completed script, you can do this by clicking on ‘REST Script’ on the left, note that this too can be renamed. When you do you’ll get a new window, called Advanced Scripting. Please note if you have a lot of windows open in your KITE instance you may need to close or resize some to get the expansive view that I have below. We’ll explore the different elements of this script later but for now the key portion is boxed in red, this is the RESTful query that you’ve just created. You can now double check and edit it manually using JavaScript.
Once done, you click the ‘run’ button as normal to test out the script and see what results you get. If everything works you’ll get a screen like the below shaded in green.
For comparison here is what failure looks like, in this case I deliberately mistyped my license key resulting in a 403 error.
Step 2: Iterative Development
In addition to the Wizard we’ve added three great tools for iterative development, which we’ll examine in turn within this section:
- Advanced script log
- HTTP request/response header payload viewer
- Using saved variables in subsequent actions
Advanced Script Log
After you run a script, the Advanced Script Log will be automatically updated and added to your screen as shown below. The log displays the request made as well as any error response details received. If the response was a successful, then the results are store into variables. For example, 0.42716 is saved in the variable [searchTime], more on how to use this later.
HTTP Request/Response Header Payload Viewer
Sometimes I find it helps to see the exact request and response payloads. Now there is an easy way to do this. In the Transaction Performance Details window, which is typically right below the Advanced Script Log, right click the step or object that you are interested in and select View HTTP Payload from the context menu.
Back on top you’ll get the below screen which shows you the exact headers and payloads, up to a character limit.
Using Variables in Subsequent Steps
The real power of this tool is to create a sequence of API calls that utilize results received in prior steps, just like a real program would. Let’s say that after our query of local events in San Diego we now want to get the event details for one of the results returned. A quick review of the API’s developer documentation shows that this is possible via the ‘assets’ API method which uses the variable assetId in its base URL. Let’s add that step using a saved variable from our first step. First we should copy and save the variable we want to use, the assetId of the first event in the list. We can find our quarry in the Advanced Script Log as shown below; we want everything inside and including the brackets.
Next we need to add a new step. This is done, as normal in KITE, by right clicking the script name in the Script Viewer window and selecting a new REST action as shown below.
Now we are back at the Web Services Wizard. We again start by entering the base URL, but this time we need to append an assetId at the end of the base URL. As you can see below, we can paste the bracketed saved variable directly into the wizard. Next we need to set the Results Format, let’s try XML this time. Finally we need to input our developer key and any other relevant parameters.
Now when we check the results of our new script we see that the code generated is retrieving our saved variable. This could also be done manually, as sharp readers will no doubt guess, but this way saves a lot of time and effort.
We can now run this two-step script and check the HTTP Payload of the second object to confirm we got the results we expected, in this case by double checking the assetId. As we’ll explore in the next section, it is possible to extend the script to add your own custom validation logic on elements like this and throw errors depending on the results.
The script is now done so we can save and upload it to Keynote for deployment to a monitoring agent.
Step 3: Advanced Script Editing
As I alluded to earlier, it’s entirely possible to manually rewrite the script created by the wizard and even modify the template itself. First though let me explain the organization of the built-in script template. Each script, both REST and SOAP, is organized into three logical sections as shown below. The red section is the parser which takes the results and saves them in variables. By default the script will usually only save one version of a variable, i.e. if you have multiple assetID’s it will save only one, the last. This is fine in most cases, however there may be times when you want to enhance this behavior and that can be done here. The blue section is where the actual query is built, this code block will obviously be pretty different between REST and SOAP but the organization is the same. Finally, the purple section is where error handling is done. The built in error handling can catch HTTP as well as API errors; however you may wish to extend this functionality further. As mentioned previously, all of this is done in JavaScript.
Let’s say you find yourself frequently updating the built in script functionality and want to make a global change. This can be done by modifying the files named wsdl.template and rest.template. These files are found in your application data folder under …/Keynote/Record/Config/. So for example in Windows 7 this would be username/AppData/Roaming/Keynote/Recorder/Config/. I encourage you to back up the originals before experimenting with this though.
Selecting a Keynote Monitoring Agent for Your Script
Now that you have created a brand new Web API monitoring script you’ll need to select some agents to run it. You have two choices, our Application Perspective (ApP) and Cloud Application Perspective (CApP) products. ApP is a collection of globally distributed monitoring agents that are ideal for monitoring your own APIs that are accessed by a broad range of consumers and those 3rd party APIs integrated at the client or browser. However, for APIs and partners that your own applications require direct access too you can deploy your own CApP agent. CApP is your very own Keynote agent software which you can install in your own private cloud or even within an important business partner’s cloud. The short rule of thumb is you want to monitor the API from where it will actually be used. If it’s consumed by end users around the world go with ApP. If it’s consumed by applications within a private cloud or network then deploy your own CApP agent there.
Comments