This article is also available on the Microsoft TechNet Wiki.
Microsoft Application Insights is a new service in Microsoft Azure. It is currently available in the Azure portal.
With Application Insights, you can test the availability of any endpoint that is available over the internet. By setting up web tests, you can simulate web requests into your website or web application from around the world.
This wiki article will show how to set up availability testing, how to access error reports, and how to read the metrics returned.
Application Insights offers two types of tests: a URL Ping test and a Multi-Step web test. A URL Ping test is just as it sounds in that it will send out web requests to a site to see if it is responsive. This test can be created within the Azure portal.
A Multi-Step web test is created using Visual Studio Ultimate or Enterprise. It is designed to test a series of steps in succession that are to be performed exactly the same way each time. The test is created by using a recorder to chart all movements so that they can be uploaded to Application Insights and then replayed through the test.
Creating A URL Ping Test
A URL Ping test is created within an Application Insights resource in the Azure portal. If you have already used Application Insights before in a previous project then you have already have configured an Application Insights resource. If you need to create a new one, see Creating a Microsoft Application Insights resource for detailed instructions.
Once your Application Insights resource is available, browse to it within the Azure Portal so that you are on the Overview blade. Scroll down the blade past the Application health section until you see a group of tiles. Within this group you will see one marked Availability.
Click the Availability tile to start creating your first web test. The Web tests blade will appear.
Click the Add web test button. The Create test blade will then open. Notice that the system tried to guess which URL you wished to test and so it created an Azure Websites address using the Application Insights resource name.
To start configuring the web test, enter a descriptive name as the Test name. The Test type defaults to URL ping test but as you can see in the figure below the Multi-Step test is also available. Next, enter the URL to test with this web test. The URL must accessible on the public internet. It can also include query strings and it will follow up to 10 redirects. For illustration purposes I have entered Microsoft.com but for your test you would enter the URL of the website or web application you wish to test.
Next, click the Enable retries for web test failures check box if you want the test to retry the URL 20 seconds after a failure. After it fails three times in a row it will register a test failure. The regular interval will then resume and retry will be suspended until there is a successful test. This setting can be useful when your system has a momentary hiccup as you can be assured the recording of failures will only happen when something is truly wrong.
The main point of performing Availability tests is to be able to simulate calls into your web applications from around the world. Clicking the Test Locations button will cause the Test Locations blade to appear. The location of San Antonio, Texas will be selected by default. There are 16 locations available in total. They can be selected individually (for example, you may wish to create a test for all points within the U.S.) or you can choose them all using the Select/Deselect All check box at the top. Microsoft advises clicking more than one location so you distinguish between website and network issues. Once you have made your selection, click OK to close the blade.
For this example, I will be selecting five random points around the world.
Clicking the Success Criteria button will cause the Success criteria blade to appear. This section sets the conditions on whether the test is successful or not. The blade will be pre-populated with the HTTP status code check box selected and with a status code of 200 entered. A 200 status code represents a successful HTTP request.
This setting will verify the URL under test will return a 200 OK value. If this occurs the test is successful. Any other return code will result in a test failure. On the flip side, this setting can also be used to test the security of your site. You could create a test that points to a page that needs authorization to view it. Creating a test that verifies a 401 Unauthorized is returned shows that the page is still secure and that a setting has not changed.
A Content match setting also exists on this blade. This can be used in conjunction with the HTTP status code or on its own. Content match can be used to make sure the HTTP request returns a string value that matches the text entered into the Content must contain box. For example, if you are expecting a 200 OK and you want to make sure the page contains the standard welcome message of your site you could use this setting. As well, when a 401 is returned you could ensure the correct error message is displayed.
For this example we will leave the blade at the default setting. Click OK to close it.
The final setting is in the Alerts section. This Alerts blade determines whether alerts will be raised on failures and if emails will be generated when they are. They are enabled by default.
Alerts can be turned on or off through the Status setting. When alerts are raised is determined through the Sensitivity setting. The default Low setting means an alert is generated when all sites fail within 15 minutes of the test. Medium means an alert is raised if half of the locations fail within 10 minutes. The High setting raises an alert on any failure at any time.
The Send alert emails to subscription admins check box means alerts will be sent to administrators of the web test. Additional emails will also be sent to email addresses entered into the Send alert emails to these email addresses box that are separated by a semicolon. Click OK to save your settings.
For this example we leave the default settings intact.
Once your test has been configured, click the Create button at the bottom of the test.
After the test has been created, there will be a success message under Notifications.
The newly created test will also appear on the Web tests blade in the All web tests section.
Accessing Availability Test Results
Once the test has been created it will be scheduled to run immediately. It may initially take several minutes for some metrics to be returned.
To view the results, click the name of the test in the Web Test column. The Microsoft web test will appear in a new blade. The top of the blade contains tools that can be used to adjust the test as needed. For more information on these, see Modifying The Test below.
The blade contains a chart under the Web test execution time section. All of the tests results show as dots on a graph and they are grouped by time. The Average Response Time as well as the number of Successful and Failed Tests are displayed.
All components of a web page (text, CSS, images, etc.) are considered part of the web test. If any portion of the page fails to load then the test is considered a failure. The page’s response time is recorded once the entire page has loaded.
Hovering over a data point on the chart will display the information associated with it.
Clicking on the data point will display the values in a new blade titled Web test result.
By clicking the request message, you can drill down one final time to see the details of the HTTP request details. In the Web test result details blade you can inspect the Response Headers, Response Body, Rules, and Exceptions.
Close the Web test result and Web test result details blades by clicking the X in the upper right to return to the Microsoft web test blade.
Microsoft web tests are also grouped by location under the Success ratio by location section. This graph can quickly show you where tests are passing or failing by locale.
Clicking a location will open a blade that contains those test results only. You can then hover over and drill down through the results the same as above.
Reading All Web Test Metrics
Returning to the main Web tests blade, the accumulated information of all web tests should now be displayed under the All web tests response time (ms) section. (If it is not, then click the Refresh button as the blade does not auto refresh.) This information is useful when running multiple tests for various sites. The principles of hovering over data points and drilling down applies here too.
As well, the Web test execution time section will show the range of how long the accumulated tests are taking on average.
On the blade you can also filter the results using the Time range button on the tool bar. Clicking Time range will open the Time Range blade. Within this this blade you can set a time range of 30 minutes to 30 days The default setting is Last 24 hours. You can also set a Custom Start and End period. When you pick a new time range the Update button becomes enabled. Click it to save the new value.
Modifying The Test
Clicking on a web test will show the toolbar that can be used to manage it:
- Edit test: Used to change the test settings. An Edit test blade will appear that has the same fields as the Create test blade. You can make changes and click OK to save them.
- Time range: Allows you to scope down the web test results. For more information, see above.
- Refresh: Access the current test results by reloading them within the blade.
- Enable/Disable: Turn off and on the web test without deleting it. Means you can enable a web test for a short period of time without having to delete it. You may also want to disable a test if your doing maintenance on your site.
- Delete: Permanently remove a web test.
Checking For Failure
When setting up your tests you may wish to also check your process on test failure to ensure the correct alerts and emails are being generated. A simple way to do this is to use a blatantly false URL. Notice in the image below how I have modified the Microsoft URL to include number 9s. This will ensure the URL fails.
The Web test result blade also includes the Open in Visual Studio button. Clicking this button will open the error message information within Visual Studio for more detailed debugging.
Clicking through to the Web test result details we can see that the URL name could not be resolved in the Exception
section. Seeing this message tells us immediately that there is an error in the URL.
Since our test has generated failures it has also crossed our sensitivity threshold as well. The test has generated an alert and sent an email notification. It has also raised an alert within the Notifications area.
The cost of Application Insights falls under three pricing plans: Free, Standard, and Premium. The difference between them is the number of allowed data points captured each month (5 million, 15 million, and 50 million), the number of days raw data will be help in your account (7, 15,and 30) , and whether you can extract data for offline processing (continuous data export is only available under the Standard and Premium options). Availability testing is available on all three plans. For more information on pricing, see Visual Studio Application Insights Pricing.