Feed: Application Performance Management Blog - Shunra Software - AggScore: 47.7
Summary: Application Performance Engineering Blog - Shunra Software
Supporting application performance management for IT professionals
Manage user expectations in your data center relocation
This is the final post in our “Five Tips for Data Center Relocation Success.”
Our first tip highlighted the need to proactively identify which applications are most susceptible to performance issues prior to relocating or consolidating data centers. The second tip emphasized the importance of performing deep transaction analysis on under-performing applications in order to identify the root cause of performance bottlenecks. In tip three, we discussed risk mitigation, and the need to remediate performance problems and validate solutions in a proof of concept lab. In tip four, we discussed assessing the latency impact on interdependent back-end servers and establishing an appropriate move plan.
Tip Five: Manage user expectations and get business buy-in commitments through hands-on acceptance and performance reports.
Does your DCR team and your organization understand the magnitude and nature of the challenge to move the servers and applications to a new location? Change/impact information should be shared on a per application and per user group basis. You cannot do this alone. Establishing an executive sponsorship team and service level objectives will help you manage user expectations.
Sometimes, it doesn’t make sense to set a post-relocation Service Level Objective (SLO) that is identical to the SLO before the move. If it originally took a local user three seconds to execute a task, it is very unlikely that the task will take the same amount of time after servers are moved. Therefore, an SLO of 5 seconds, for example, may be more reasonable to expect.
It’s critical to address users’ service level expectations up front. If you wait until after the move and tell users they have to live with what you can deliver, you’re setting yourself up for a battle. But if you can get buy-in beforehand as part of the planning process, you can avoid such hassles and ensure that no one has unrealistic expectations.
To achieve this pre-relocation acceptance, IT must be able to predict and accurately emulate post-relocation performance. These predictive and emulation capabilities enable IT to set up “acceptance environments” where users can experience post-relocation performance before the move is actually executed – and where they can accept post-move SLOs before a single server is relocated.
One way to do this with a Performance Engineering Dashboard. You can analyze performance results with remediation suggestions, in separate reports per business process. Learn which of the tested applications in each department will have no impact, minimal impact, significant impact, or fail completely. Users who hear that a transaction response time will go from two seconds to five may object out of sheer reflex. When users are allowed to directly experience post-move application performance in advance, on the other hand, they can give their informed consent to the relocation plan.
Performance Dashboard Example
Establish SLO baselines by measuring application performance before your move. These pre-move transaction response times provide essential data to begin managing user expectations. End-users should be given the opportunity to directly experience post-move application performance in advance, so they can offer informed consent to the relocation plan. With users conducting their day-to-day operations with real-world network conditions in a lab, you empower them to experience first-hand what the move will be like for them.
Review this sample report for a DCR of Fortune 500 company.
By following these five tips, IT organizations can substantially reduce risk, eliminate unnecessary infrastructure spending, accelerate time-to-benefit, and overcome a wide range of potential political pitfalls. The exclusion of any of these ways, however, greatly increases the likelihood that unforeseen problems will sabotage the project.
Unexpected impacts of latency in data center relocations
This post is the fourth of five in our “Five Tips for Data Center Relocation Success.” Our first tip highlighted the need to proactively identify which applications are most susceptible to performance issues prior to relocating or consolidating data centers. The second tip emphasized the importance of performing deep transaction analysis on under-performing applications in order to identify the root cause of performance bottlenecks. In tip three, we discussed risk mitigation, and the need to remediate performance problems and validate solutions in a proof of concept lab. Stay connected to the tips, and sign up for the entire five-part series.
Tip Four: Assess the latency impact on interdependent back-end servers and establish an appropriate move plan.
In addition to addressing network latencies between clients and servers, your DCR team should fully understand the impact of latencies that may be created between servers during transitional stages of the move.
Understanding the dependencies among the systems will determine how all the servers of a particular application should be moved to the new data center. Will they move all at the same time? When is best to move them? There will be times when elements are co-located, and times when only part of the new infrastructure is active and you still leveraging the old infrastructure. Assessing the latency impact will help you build an effective move plan.
It is a dangerous misconception to equate network latency to application latency. The relationship between network delay and application response time is far from one-to-one, and depends on factors such as:
- The number of messages exchanged between the client and the server for each transaction (often referred to as application “turns” or “chattiness”).
- The synchronous nature of the application and how many turns can be processed in parallel. Does the application wait for a certain object (e.g. css, js, jar) to download before it can continue processing other data?
- The location and distribution of application code (js, jar, asp, php, swf), meta data (ini, XML, etc.) and the actual data (tables, images, objects, XML, etc.) and how these resources are fetched/downloaded in each transaction.
- The configuration of the protocol on the server and client (TCP buffer sizes, authentication methods, time out settings, maximal number of sessions and sockets, etc.).
One way to accomplish this is by creating “move groups.” A “move group” is a collection of servers and/or applications that will relocate as a single managed event (even if this event takes days or even weeks). By learning the dependencies among your systems, and how latency will be impacted, you should be able to move all the components of a system on the same “move group.” If you need to split a system into several “move groups,” testing the impact of the individual moves is critical to engineering for performance.
For example, based on a log-in transaction for a CRM application, Chart 1 (below) provides a relatively simplistic example of the relationship between network latency and application latency. For a local user with less than a millisecond of network delay between the client and the server, the transaction was completed in three seconds. When just 50 milliseconds of network delay were introduced (representing a typical cross-country WAN connection), the performance of this transaction did not slow down by only 50 milliseconds. Instead, this same login transaction took a full 30 seconds to complete. This example dramatically illustrates how small changes in network latency result in major problems with application performance.
Transaction response time – local user vs. remote user. Note: The “login page download” arrows represent over 500 network packets broken into 270 application turns.
An appliance-based, network emulation solution is invaluable for accurately measuring the impact of these server-to-server latencies. Assessing the latency impact on interdependent back-end servers and establishing an appropriate move plan will help your DCR team understand the impact of latencies that may be created between servers during transitional stages of the move.
Read this case study of Utah State University’s challenges testing their new distance learning system in their lab.
Keep reading Five Tips for DCR Success: In tip five, we discuss managing user expectations through performance reports.
How to mitigate risk with your data center relocation
This post is the third of five in our “Five Tips for Data Center Relocation Success.” Our first tip highlighted the need to proactively identify which applications are most susceptible to performance issues prior to relocating or consolidating data centers. The second tip emphasized the importance of performing deep transaction analysis on under-performing applications in order to identify the root cause of performance bottlenecks. Sign up for the entire five-part series.
Tip Three: Remediate performance problems and validate solutions in a proof of concept lab.
Risk mitigation is essential to the success of a DCR project. Proactively testing performance remediation solutions in a test lab provides insight into expected performance gains achieved from efforts such as code fixes, WAN acceleration, and virtualization. The remediation lab should include a complete, virtual model of the pre- and post-relocation environment, as well as all planned transitional phases.
As we’ve discussed, this test (or proof of concept) lab must mirror real-world network conditions in order to deliver reliable insight into application behavior, and in the case of this tip, into potential performance gains from remediation efforts. Tips 1 and 2 highlighted how applications can be evaluated and at-risk applications can be analyzed and prioritized for remediation. Tip 3 takes us to performance remediation and acceleration options that, like your initial tests, must be implemented, evaluated and validated under the precise same real-world network conditions, all before your DCR move.
Also, recognize that you cannot always control the changing user environments, but you can always plan to test and re-test your application and remediation efforts as network conditions change. Look ahead. Be ready when change hits your environment or software portfolio by capturing and virtualizing real-world network conditions.
Your best chance of success comes if you adopt a proactive practice of testing applications, remediating performance issues, and validating solutions in a controlled lab environment that includes virtualized real-world network conditions, including bandwidth, latency, jitter and packet loss. Waiting until you move or consolidate data centers to test for application performance can cost your business dearly – having your end users become your performance testers puts revenue, productivity, customer satisfaction and brand at risk. Testing in a lab with virtualized network conditions is an industry best practice and provides the only reliable means for validating performance gains due to remediation efforts.
Review this case study of a membership warehouse club testing a document scanning application prior to their DCR.
From tip two, read this case study from a large, retail sportswear company applying this tip prior to relocating their data center relocation from Portland, OR to Denver, CO.
From tip one, read this case study of a U.S. Civilian Agency experiencing data and productivity loss after a move, and how Shunra helped solve the issues.
How do you view the cost of not validating your options, given the cost of WAN acceleration? I’m interested in your thoughts – please feel free to comment below.
Keep reading Five Tips for DCR Success: In tip four, we discussed assessing the latency impact on interdependent back-end servers and establishing an appropriate move plan.
Which applications are most susceptible to latency and other network impairments in a DCR?
This post is the second of five in our “Five Tips for Data Center Relocation Success.” Sign up for the entire five-part series.
Tip Two: Perform deep transaction analysis on under-performing applications in order to identify the root cause of performance bottlenecks.
After identifying which applications are most susceptible in your DCR (Tip One), the recommended next step is to analyze and pinpoint performance bottlenecks. Taking an empirical approach to understanding which applications to test and how to structure the move plan is critical in accounting for application dependencies and effectively formulating a move strategy that minimizes risk. Which applications are most susceptible to latency and other network impairments?
For example, let’s take a deeper look at an under-performing, “out of the box” application.
Figure 1: Performance Reporting – Single User
Figure 1 is an example of what a typical single user performance test for the new, emulated location looks like in terms of results. This type of analysis is powerful in that it enables a comparison of performance across many locations relative to the baseline for a single user – which in many circumstances can determine if the root cause of the performance issue is application or network centric. This is also the first step clients can take to start addressing problematic apps at specific locations before moving on to the more complex load tests.
Once problematic apps have been identified, we need to find out if it is the application or the network that could be causing the slow response times. First, identify if the problem is a network issue as this is usually the easiest to rule out. We can look at the Bandwidth Utilization chart from our testing to check if we ever hit 100% utilization. If we haven’t, as shown in Figure 2, then we know that no matter how much bandwidth we add it will not add any performance increase to these slow response times.
Figure 2: Bandwidth Utilization Chart
In this specific example, at this point, we have ruled out bandwidth as the issue, and we can now analyze performance within the application so we can improve response times.
When we look at this same application from a transactional analysis point of view we can see that this transaction was extremely chatty with almost 5k app turns carrying only 1k of data each time… not a very efficient transaction within the app. The result of this is that it severely increased client time processing and was actually the cause of this particular delay. In Figure 3, transaction analysis of this “out of the box” application tells us that we should not always accept that the base application is correctly configured.
Figure 3: Transaction Analysis from Shunra Reporter, available in PerformanceSuite and vCat
Most companies concern themselves more with testing the final revision of a new application prior to release to ensure that they will be able to meet their SLAs. When looking for performance improvements you don’t want to assume that the “out of the box” app provided is completely optimized for your requirements.
Read this case study from a large, retail sportswear company applying this tip prior to relocating their data center relocation from Portland, OR to Denver, CO.
From tip one, read this case study of a U.S. Civilian Agency experiencing data and productivity loss after a move, and how Shunra helped solve the issues.
Keep reading Five Tips for DCR Success: In tip three, we discussed risk mitigation, and the need to remediate performance problems and validate solutions in a proof of concept lab.
Which applications will fail in your data center relocation?
Recent survey results show that one of the top concerns of Data Center Relocation (DCR) Project Managers is that application performance will degrade as a result of their DCR. In fact, research validates that concern – 39% of applications will experience performance problems after a data center move without application engineering. Most organizations are aware of the risks, but not sure how to plan for them.
This blog post is the first installment of five in the “Five Tips for Data Center Relocation Success” series that will help you implement a preventative and proactive performance methodology. Sign up for the five-part series.
Tip One: Prior to your move, identify applications that will experience performance degradation under the post data center move conditions.
With more than one-third of applications failing to perform as expected or required, knowing which applications will perform poorly before a move will drive out much of the risk.
Why is this so difficult?
- Even the smallest data center relocation impacts performance
- Lack of time does not always allow for upfront planning
- Hard to account for which applications will fail before the move
- Inability to capture and emulate real-world network conditions
Why is this happening?
DCRs introduce performance risk. Even a small amount of network latency (10-30 milliseconds) can dramatically degrade application performance and dramatically increase transaction response times. Lack of time doesn’t allow for proper upfront planning, which causes a great deal pain for DCR teams. The result is application failure or degradation. “Is it worse than it was?” is not an acceptable business metric or service-level objective. Latency affects each application turn, so even minimal latency adds up quickly.
By profiling your network environment and quantifying application performance before the move, you will know in advance whether the move is viable. Which applications will be impacted? What will end users experience? Should we create move groups? Knowing these answers can drive much of the risk associated with failed relocation efforts out of the process. You can isolate problematic applications early on and apply remediation strategies well before the move.
Plan ahead
- Simulate post-move conditions
- Identify which applications will degrade in performance
- Identify performance impact for individual move group scenarios
- Analyze and pinpoint performance bottlenecks
- Provide remediation and optimization suggestions
- Validate performance fixes before the migration
You can overcome the challenge of identifying applications that will fail prior to your move by precisely recreating network conditions for performance testing.
How you can solve this problem:
- Capture real-world network conditions, enabling you to understand and emulate network behavior for existing and target conditions.
- Access our network library of 15 million recordings (from broadband and mobile users) from more than 52,000 cities, providing you with typical/average best case and worst case conditions. With the ability to record actual network conditions, you have the true “last mile.”
- Replay real-world network scenarios in the lab with 100% accuracy to optimize applications and ensure the conditions remain the same for all your tests. Capture a recording of the WAN environment including network impairments such as latency, packet loss, jitter, and available bandwidth.
- Impair those conditions, hit “play” and see the actual condition of what it would be like to work from that location on that specific client. For example, you can work in Philadelphia, and recreate complex network topologies in Australia, Hawaii, United Kingdom, etc. This testing freedom enables you to pinpoint bottlenecks, identify remediation possibilities, and test optimization suggestions.
Taking an empirical approach to understanding which applications to test and how to structure a data center move plan is critical in accounting for application dependencies and effectively formulating a move strategy that minimizes risk. Which applications are most susceptible to latency and other network impairments?
Most companies concern themselves more with testing the final revision of a new application prior to release to ensure that they will be able to meet their SLAs. When looking for performance improvements you don’t want to assume that the “out of the box” app provided is completely optimized for your requirements.
Identifying applications that will experience performance degradation under the post data center move conditions PRIOR to your move is critical. Read this case study of a U.S. Civilian Agency experiencing data and productivity loss after a move, and how Shunra helped solve the issues.
Keep reading Five Tips for DCR Success: Our second tip emphasizes the importance of performing deep transaction analysis on under-performing applications in order to identify the root cause of performance bottlenecks.
Why Mobile Apps Fail After Deployment
Research shows that less than 50% of organizations are prepared to establish, validate, and predict mobile application performance. Without accurately predicting how applications and network services will perform for your mobile users before launching, some portion of your deployment will fail, resulting in unhappy users, lost revenue, and decreased productivity. To make matters worse, end users do not necessarily consider the inherent constraints presented by mobile networks, yet latency, bandwidth limitations, packet loss and jitter all affect their experience.
So, why are mobile apps crashing after being deployed? On April 16, 2012, Shunra experts John Curtis and Liam McCamley answered this question on a Shunra webinar, and profiled three customers who overcame common challenges of mobile application deployment.
Potential Challenges of Mobile App Development Teams
If we ask the team developing the native mobile app to define its biggest challenges, we might hear some of the following:
- R&D might look to define and constrain the specific operating systems, the hardware configurations, the number of different screen sizes and their maximum dimensions, and so forth. Unlike the desktop client where we had a few browsers and plug-ins to support on a relatively stable environment, the mobile landscape is ever-changing and very dynamic ALL THE TIME.
- QA might make unfortunate choices about ever-smaller subsets of regression tests. Since it is impossible to predict the millions of mobile configurations and network conditions, sometimes choosing worst-case configurations and hoping that it provides sufficient coverage is the “answer.”
- Customer Support, likewise, will be totally overwhelmed trying to address the sheer volume of platforms and radio access technologies and complete lack of onboard diagnostics to help reproduce or identify a problem. Moreover, given the urgency to deploy quickly, QA is shortchanged, products don’t enjoy sufficient pre-deployment test coverages, and Customer Support becomes a branch of QA, albeit in conjunction with unwitting customers who downloaded the latest, greatest, untested code.
With all of these formidable challenges, it’s easy to forget that there is an even greater problem lurking in the target environment: Unless the apps were tested in the conditions of the target environment, the risks are enormous.
Identifying these three, common challenges, John and Liam then exampled cases where customers believed that they were testing “worst case,” only to discover that the application would fail as soon as they deployed it.
Solve Performance Problems for Mobile Applications
On the webinar, we recommended the following steps for successful application performance:
- Discover what the network looks like and how users will actually use the application. Directly involving the user or business community in testing so you can identify expectations.
- Recreate accurately the true dynamic mobile network conditions and account for variation.
- Test current conditions and what-if scenarios.
- Analyze problems and solutions to determine impact and ROI.
- Correct issues and optimize for performance.
In January 2012, Shunra announced the development of a mobile app performance platform that delivers the capabilities, services and partner integration points organizations need to reliably test for and predict mobile performance. This open platform includes integration with hardware, software, services and delivery mechanisms from a growing network of Shunra mobile testing partners.
Savings, Checking, Pay Bills – Wait
Savings, Checking, Pay Bills – Wait
My money. When I need access to my account, I want to make sure it’s secure and also that I can reach it whenever I want, from wherever I am. Recently, when a large European bank introduced their mobile app, they imagined a sizeable portion of their retail banking customers using the app from various locations across Europe. Since their website had launched successfully, they figured they could easily package up their website application, tweak it a bit and go mobile. However, almost as soon as the mobile app was launched, initially just to banking customers in France, complaints began to roll in. Response time was so poor – up to 40 seconds just to start the app – that users were abandoning the mobile app at record rates.
Industry research shows just how much a single second of delay can impact business. According to one case study published by Strangeloop’s Joshua Bixby, a single second of delay can result in a loss of 3.5% of customer conversions. The result of this lack of conversion: revenue loss from unfinished transactions and abandonment of other consumer services.
The Bank realized the risk to their bottom line when their mobile app did not perform up to par. Knowing they needed to improve app performance and deliver a positive end user experience, The Bank asked Shunra to determine the source of the problem and provide recommendations as to how to improve response time.
Shunra’s Professional Services team conducted a performance analysis of the mobile app across multiple simulated locations. To determine actual network conditions being experienced by the Bank’s customers, Shunra’s team leveraged NetworkCatcher’s extensive global library of mobile and broadband conditions. The library provided sample network conditions experienced by mobile users from many locations across the country. These network profiles were imported into Shunra vCat, enabling the Shunra team to virtualize real-world network conditions – including bandwidth, latency and packet loss – in the performance test environment.
SETTING UP THE PERFORMANCE TEST
To virtualize the network and performance test the application, the Shunra vCat client was installed on a Windows 7 workstation, and an Android mobile handset was connected to the workstation using WiFi. The workstation was connected to the internet using a high-bandwidth low-latency cable network. Network traffic flowed from the handset via the WiFi connection, to the internet via the workstation, and back, as depicted below:
Because Shunra vCat supports the definition of test processes (steps) to be executed on the mobile app, a specific user scenario was scripted and tested:
1. Startup
2. Payment abroad
3. Find a branch
4. Search
A baseline measurement was first taken on the LAN, where no network effects were present, to establish a comparative performance level. The test process was then repeated from different virtualized locations, so the effect of network conditions on user experience could be measured from close and afar. Detailed performance reports of these scenarios were generated and analyzed, including traffic analysis of data flowing between the mobile device and servers and analysis of the communications. Finally, specific performance optimization recommendations were generated based on Shunra’s proprietary rule set.
RESULTS
The following chart shows the comparative performance of specific actions within the Bank’s Android app across various virtualized network conditions representing:
- LAN connection (no network effect)
- 3G connection in the capital
- 3G connection in the countryside
These results immediately demonstrated to the performance test team that the application slows considerably once the app is used at a remote location, even within the same city, with some transactions taking more than 30 seconds.
To understand the application performance in more detail, additional tests were then executed under controlled network conditions to identify the impact of various bandwidth constraints.
The following test results showed the delay incurred on each transaction under various bandwidth conditions. While almost no effect was noted on ‘Search’, for the other three transactions, the effect of the WAN was clear. For users with slow connections, the effect was even more pronounced:
Next, the test was repeated with higher latency and packet loss conditions. Once again, the correlation between poor network conditions and increasingly higher response times was evident:
These results demonstrated that the mobile app was sensitive to both reduced bandwidth and increased latency/packet loss conditions. The testing team concluded that users would experience degraded performance under most mobile network conditions, and therefore, performance would be an issue for the vast majority of users.
DETAILED PERFORMANCE ANALYSIS
In order to diagnose the root cause of the performance issue, the team used Shunra vCat’s Analytics Module which supports the detailed analysis of network communications across transactions and networks.
The following graph shows the HTTP traffic between the mobile app and the server during the application startup process under LAN conditions:
The analysis showed that 256 KB of data are downloaded for the application startup, and that these data were delivered across 23 HTTP elements. All elements were downloaded sequentially, with no concurrent data transfer. The serial request/response sequence, combined with the amount of data downloaded during this step, explained the sensitivity to reduced bandwidth and increased latency and packet loss conditions.
The following graph shows the download sequence for the “Payment abroad” step:
This analysis showed that the “Payment abroad” transaction resulted in the download of 288KB of data in 40 elements. Several components were downloaded in parallel, but the number of elements and the volume of data explained the sensitivity of this step to mobile network conditions.
OPTIMIZATION RECOMMENDATIONS
With an understanding of the mobile app’s sensitivity to mobile network conditions, the team was ready to make recommendations to improve performance. Although the analysis only included a small number of application functionalities, several key recommendations were made that improved the application’s performance across all mobile network conditions. Of their recommendations, many apply to mobile development efforts of all types:
- The earlier developers consider the significant slowdown that occurs on the mobile network, the more effectively they will be able to meet performance testing objectives.
- For hybrid apps (a native app with embedded HTML), it is crucial that the main content of the app be presented by a browser embedded within the app, as opposed to widgets provided by the platform.
- A hybrid app doesn’t leverage parallel downloads as well as a browser, nor does it cache as well. Therefore, developers must engineer this behavior to ensure that it is optimized for mobile conditions.
Another suggestion provided by the Shunra team was to integrate Application Performance Engineering (APE) earlier in the development process. Instead of waiting until the app is in production – or worse, abandoned by customers – Shunra showed the Bank how to mitigate risk to both time and effort by shifting performance resources and efforts earlier in the software development lifecycle. This also represented tremendous potential savings to the Bank, as remediation efforts and business impact due to application performance issues is estimated at around $45,000 per hour of performance degradation. Therefore, ensuring that performance is optimized before the app is handed off to QA or Production can have a positive effect on a company’s bottom line.
Additional recommendations included:
- Reducing the amount of data transferred during startup
- Enabling parallel data transfer of elements during startup
- Compressing web elements downloaded in other steps
- Hosting content on multiple domains to speed network transfer
- Consolidating duplicate elements from multiple paths to be common between application steps
- Considering a mobile CDN to improve network performance for downloaded elements
- Considering replacing mobile web pages delivered in a window with a mobile app framework which only requests dynamic data
SUMMARY
To improve the performance of a leading Bank’s mobile app, Shunra’s NetworkCatcher and vCat were employed to recreate their users’ actual network conditions in a test environment; application performance and response times were then analyzed under various mobile network conditions. Shunra’s recommendations included:
- Consideration of the slowdown that occurs in hybrid apps
- Integration of Developer Performance Engineering (DPE) into the development process
- Customized recommendations based on the Shunra Analytics Optimization reports.
Once implemented, these recommendations provided the methodology and process improvements by which the Bank’s mobile app performance was optimized. Initial estimates from the Bank showed monthly savings of over US$100,000, and an ROI that would be measured in months, not years.
Twelve Best Practices of Data Center Relocations
On average, 39% of your applications will fail or experience significant impact after a data center relocation (DCR). DCRs introduce performance risks like latency, jitter, and packet loss. Proactively engineering for performance prior to moving is critical to reducing this risk of failure.
On a recent webinar, Shunra’s experts shared a holistic approach to performance engineering that includes insight into how to capture real-world network conditions and virtualize those conditions in the test environment. Leveraging a virtualized network to create reliably predictable performance tests helps you understand the impact of bandwidth and latency on user groups, resulting in steps that can be taken before a datacenter move to maximize success. On this webinar, our attendees learned how to:
- Discover the most commonly overlooked factors of a server move or consolidation
- Avoid relocation pitfalls
- Understand the risks that make testing mandatory
- Know which applications are going to fail post-move
- Know precisely how end users, in different locations, will be affected
The twelve best practices shared on this webinar were:
- Know what and when to move
- Understand dependencies among systems
- Know who uses what and where
- Identify network changes by location, time, and user
- Know the impact of your move before the move
- Create move groups
- Ensure your organization knows the impact of the move
- Test and assess
- Manage user expectations
- Be ready to deal with issues
- Consider remediation options
- Validate your move
Watch this webinar to learn recommendations for each of these best practices.
How do you calculate the cost mobile application failure?
There is no doubt, mobile is everywhere. Shared by mobiThinking in January 2012, there are over 5.9 billion mobile subscribers worldwide (roughly the equivalent of 87% of the world population). Within three years, it is expected that the Web will be browsed from mobile devices more often than PCs. What does this mean for you? Proactively designing, developing and testing for mobile application performance is no longer an option. It is a necessity.
But what happens if you don’t engineer for performance in your mobile application deployment? Without accurately predicting how applications and network services will perform for your end users before launching, some portion of your deployment will fail. Guaranteed. Despite this potential failure, less than 50% of companies are prepared to establish, validate, and predict mobile application performance (per a March 2011 Shunra survey on mobile application performance).
If you don’t engineer for performance, what is the potential cost of failure to your business?
Many factors influence the cost to your business when an application doesn’t perform as expected. You need to consider two types of costs – remediation costs and business costs. Remediation costs are simple to calculate, you just need to understand how many resources are required to troubleshoot and resolve problems, how long it takes those resources to resolve a problem, and what is the hourly cost of those resources. With this information, you begin to understand how much a failure costs.
You also have to consider how the application affects your business and end users. How does company brand relate to the app? What is the impact of bad reviews? Good reviews? The Android market and the Apple AppStore offer immediate feedback – positive and negative. These reviews can – and do – make or break an application’s success. You also have to ask yourself: how do performance degradations affect revenue? How do you quantify productivity losses? Consider these two calculations of productivity loss:
“Productivity loss due to application performance issues is estimated at a loss of $42,000 per hour for each user group impacted.” (IT Management News)
“The loss associated with IT issues has been quantified as a 63% reduction in productivity, “resulting in an average of 552 people hours lost per year per company.” (CA Technologies, 2011 survey of 1,800 companies)
How does your company’s cost of failure compare? How many post-deployment performance issues can be avoided? What does it cost your business to lose a customer? What does it cost to retain them? What is the lifetime customer value? What does it cost to acquire a new customer? Answering these critical questions could be the key to unlocking the true cost of failure for your mobile application.
True cost avoidance and cost of failure reduction or elimination comes from proactively designing, developing and testing for mobile application performance.
Regardless of how you calculate the cost of failure for your mobile application, you cannot afford to fail. Plain and simple.
Watch this webinar for more information: Take the Guesswork out of Mobile Performance Testing with Shunra and HP
Configuring HTTPS for a standalone server driven by the Windows HTTP Server API.
Introduction
In the following post I am using the term server to denote any server application, which ultimately relies on the HTTP protocol to deliver client requests and the server responses. This is a very broad category, including the general purpose web servers, like apache and IIS, but these are not at the focus of this article. This article is about a particular subset of servers, namely those utilizing the Windows HTTP Server API.
Let me introduce the affected servers more formally, so that you can safely skip this article, if it is irrelevant to you. I am talking about:
- Self-hosted WCF services utilizing the HTTP bindings.
- Any .NET server implemented on top of the HttpListener class.
- Any native Win32 application using the HTTP Server API.
There is a vast class of servers, which are either not driven by the HTTP Server API or expect to be configured through its own dedicated API, for instance:
- The IIS web server. It is based on the HTTP Server API, but every aspect of it should be configured through the dedicated API.
- Java written servers, typically these do not use the HTTP Server API, since the latter does not exist, except on Windows.
- The Apache web server, since it uses the raw sockets.
- Any server using the raw sockets (Windows Sockets 2) for handling the traffic.
So, if your server is based on the HTTP Server API – read on.
What is the goal of this article? The goal is simple – share the knowledge we have to let you master a simple HTTPS configuration. Plus there is a bonus – the full C# source code of the tool we have written to do precisely this – configure HTTPS.
I am going to start with the practical stuff first – using the tool. For that we need a simple demo server (utilizing the HTTP Server API, of course) and a client. Let us refer to them as simply the server and the client. For the sake of the demonstration I am going to use a simple WCF based server. Please, note that in no way do I endorse using the WCF technology. The reason I am using it for my demo is because I find it very easy to create a trivial one – no 3rd party dependencies are needed, just the .NET framework. Plus the Visual Studio lets us consume SOAP WCF services easily.
For the impatient, the complete source code of the tool is available at https://bitbucket.org/markkharitonov/setuphttps/src and is distributed on the terms of the Simplified BSD License. You will need the .NET framework 4 Client Profile in order to build it.
Setting the stage
Let us get started. The source code for the demo server is found beside the tool source code in the DemoServer directory. It contains three C# projects:
- Server – implements the server logic.
-
ConsoleHost – this is our stand-alone server –
courtesy of the WCF framework. It exposes two HTTP endpoints:
- a SOAP HTTP endpoint – http://localhost/Wcf/SimpleDemo/ws
- a plain HTTP endpoint – http://localhost/Wcf/SimpleDemo/web
- ConsoleClient – a simple .NET client to consume the SOAP endpoint of the server. The Visual Studio lets us create it with just a few clicks.
Note the absence of the plain HTTP client. We do not need it – our web browser is the client.
Anyway, here is how the things are arranged on the virtual machine I have used to run the demo (this is exactly the contents of the deploy directory):
(I am using the powershell as my command line shell)
You can utilize the provided binaries or just build from the source code (does not require the Visual Studio).
Before we start with HTTPS let us make sure that the server is working correctly – start ConsoleHost.exe. Ideally, this is what we should see:
However, it is much more likely that we shall get the following “welcome” message instead:
This is what happens if the server is started without elevation and the reason is that the HTTP Server API requires that the HTTP namespace(s) of the particular server be first reserved. The catch is that HTTP namespace reservation demands elevation, although it can be performed just once – upon the installation of the product. Anyway, there are tools to deal with the HTTP namespace reservations, including:
- Windows 7/2008 – netsh
- Windows XP SP2/2003 – httpcfg
- All – SetupHTTPS (our tool!)
You can find a few examples of using the standard tools in the Port binding and Reserving HTTP namespace sections of this article.
This is how our tool can be used to address the HTTP namespace reservation task. Let us start with the following command line:
Where:
- sethttp is one of the tool commands, meaning that the tool is to configure plain HTTP, rather than HTTPS.
- –vp is the shortcut of the –virtualPath option, specifying the third component of an HTTP namespace, the first two being the scheme and the port, which are in our case http and 80 respectively. The resulting HTTP namespace is http://+:80/Wcf/
- -n is the shortcut of the –dry-run option – nothing is actually done, but the tool tries its best to show what it would do if ran for real.
Notice the first line of the aforementioned error message:
The HTTP Server API lets us choose between three different HTTP namespace reservations, all suitable for our server:
- http://+:80/
- http://+:80/Wcf/
- http://+:80/Wcf/SimpleDemo/
The middle one seems to be the right choice, because on one hand we are not claiming the ownership of all the servers listening to the default port (as is the case with the first option). On the other hand we do not want to perform the HTTP namespace reservation for each and every sample WCF server that we might use – something implied by the third option. So we have a go for http://+:80/Wcf/ (note the trailing slash, it is crucial if you are using one of the standard tools to perform the reservation).
Back to the command line. As you can see the tool says that it would reserve http://+:80/Wcf/ for the currently logged on account, which is good enough for us. A different account can be given with the -a (the shortcut of –account) option. In fact, let us run it now without any arguments to discover all the commands supported by the tool and then with sethttp -h to display only the options relevant for the sethttp command:
PS C:work> .SetupHTTPS.exe
Usage: SetupHTTPS.exe [options]
Where can be anything of the following:
SetHttps - Configure HTTPS using the given arguments.
SetHttp - Configure HTTP using the given arguments.
ClearHttps - Clear the given HTTPS configuration.
ClearHttp - Clear the given HTTP configuration.
Execute - Execute the actions logged by a previous invocation of the tool.
ExtHelp - Display the selected extension help.
Run SetupHTTPS.exe -h to get the option help for the particular command
The tool is merged with the following extensions:
- ConfigureWCF.Main
PS C:work> .SetupHTTPS.exe sethttp -h
SetHttp - Configure HTTP using the given arguments.
The tool is merged with the following extensions:
- ConfigureWCF.Main
SetHttp Server options:
-h, -?, --help Show this help screen.
-q, --quiet Be quiet.
-n, --dryRun Dry run - perform all the steps, but modify
nothing. Ignores the --quiet flag, if the latter
is given.
--dontElevate Do not try to elevate.
-e, --extension=VALUE The name of an optional extension type. Must be
an assembly qualified name if the extension is
implemented in a different assembly, otherwise
the full type name is enough. If given, the
extension type must be discoverable by the
standard .NET type loading logic, it must have
the default constructor and implement the
SetupHTTPS.IExtension interface, otherwise the
program is aborted. So, if the executable is
merged with the ConfigureWCF extension assembly,
then passing --extension=ConfigureWCF.Main
activates that extension. Otherwise, it should
be --extension="ConfigureWCF.Main, ConfigureWCF,
Version=1.0.0.0, Culture=neutral,
PublicKeyToken=null" and the ConfigureWCF.dll
assembly must be visible by the .NET type loader.
--ulog, --uninstallLog=VALUE
The path to a file where the uninstall actions
will be logged. An existing file would be
overwritten. The created file can later be
passed to the execute command.
-p, --port=VALUE The http(s) port. The default is 443 for https
and 80 for http.
-a, --account=VALUE The name of the account under which the server
is expected to be run. The default is WIN-
OR05DA2H54Dmarkk
--vp, --virtualPath=VALUE
The virtual application path, will be appended
to http(s)://:/. Must be properly
escaped.
PS C:work>
Of course, your output will show a different default account name
(unless, your machine is also named WIN-OR05DA2H54D
and you are logged on under the markk local
account). But we have strayed aside. Let us run the tool for
real:
And once more in the dry mode:
Everything is in order now. The demo server should start just fine. You should now be able to do any of the following:
- Navigate to http://localhost/Wcf/SimpleDemo (no trailing slash!) – presents the WcfDemoService Service welcome page (courtesy of the WCF framework).
- Navigate to http://localhost/Wcf/SimpleDemo/web/Add?x=12&y=13 – you shoud see something like this:
- Run the ConsoleClient – it should print “OK”.
We are ready to configure our demo server to work over the HTTPS.
Using the tool
The main scenario
Core configuration
First, make sure the firewall allows the traffic on the TCP port 443 (the tool does not deal with the firewall, maybe in a future version). Next, run the tool in the dry mode to see the tool intentions first:
Oops… If this is what you see, then your machine does not have the makecert.exe tool (part of the Windows SDK). The remediation is simple – just do any of the following:
- Copy makecert.exe from the Dependencies folder found in the tool source code tree into the same directory where the tool executable (SetupHTTPS.exe) resides.
- Pass the path to makecert.exe in the –makeCert argument, for
instance
--makeCert=Dependenciesmakecert.exe - Install the Windows SDK (overkill, if you ask me).
- Specify the path to makecert.exe in the MAKECERT environment variable.
Anyway, once this issue is resolved we should get an output like this:
According to the output, the tool would:
- Create a self-signed CA certificate with the subject name of
WcfDemo by invoking the following command line:
C:workmakecert.exe -n "CN=WcfDemo" -sr localmachine -ss root -a sha1 -cy authority -r. The new certificate would be placed in the Trusted Root CA folder of the system certificate storage associated with the local machine (LocalMachineRoot). - Create a Server certificate signed by the aforementioned CA
certificate for the host
WIN-OR05DA2H54Dby invokingC:workmakecert.exe -n "CN=WIN-OR05DA2H54D" -sr localmachine -ss my -cy end -pe -sky exchange -a sha1 -is Root -ir LocalMachine -in WcfDemo. The new certificate would be placed in the Personal folder of the system certificate storage associated with the local machine (LocalMachineMy). - Allow the account
WIN-OR05DA2H54Dmarkkto read the private key of the aforementioned Server certificate. - Associate the aforementioned Server certificate with the TCP port 443.
- Reserve the HTTP namespace of our demo server
(
https://+:443/Wcf/) for the accountWIN-OR05DA2H54Dmarkk
Notes:
- We can make the action list shorter by combining the two
certificates into one self signed Server certificate. Having two
distinct certificates, however, is much more flexible, it allows
for a variety of scenarios, impossible with a single combined
certificate, for example:
- Using an existing CA certificate to sign the new Server certificate. One might already have a certificate signed by VeriSign or alike and wish to use it instead of creating a self signed certificate in step 1.
- Revoking the Server certificate, without touching the CA certificate used to sign it.
- The Server certificate’s subject name is the same as the machine’s host name.
- The HTTP Server API does not allow to associate two certificates with the same TCP port, but one certificate can be used with more than one port. In other words, two servers wishing to utilize two distinct server certificates must be bound to different TCP ports.
The following table summarizes the parameters used by the process:
| Parameter | Default value | Command line option | Used in steps |
|---|---|---|---|
| CA certificate subject name | N/A | –caSubject | 1, 2 |
| Machine name |
%COMPUTERNAME% (.NET: Environment.MachineName)
|
–host | 2,3,4 |
| Server account | %USERDOMAIN%%USERNAME% (.NET: WindowsIdentity.GetCurrent().User) | -a or –account | 3,5 |
| TCP port | 443 | -p or –port | 4,5 |
| Server’s root url path component | N/A | –vp or –virtualPath | 5 |
| Certificate maker | makecert.exe | –makeCert | 1,2 |
| Certificate maker’s options to generate the CA certificate |
-n "CN={0}" -sr localmachine -ss root -a sha1 -cy
authority -r
|
–createCACertArgs | 1 |
| Certificate maker’s options to generate the Server certificate |
-n "CN={0}" -sr localmachine -ss my -cy end -pe -sky
exchange -a sha1 -is Root -ir LocalMachine -in {1}
|
–createServerCertArgs | 2 |
Running the tool with sethttps -h prints the complete help on the sethttps command options:
PS C:work> .SetupHTTPS.exe sethttps -h
SetHttps - Configure HTTPS using the given arguments.
The tool is merged with the following extensions:
- ConfigureWCF.Main
SetHttps Server options:
-h, -?, --help Show this help screen.
-q, --quiet Be quiet.
-n, --dryRun Dry run - perform all the steps, but modify
nothing. Ignores the --quiet flag, if the latter
is given.
--dontElevate Do not try to elevate.
-e, --extension=VALUE The name of an optional extension type. Must be
an assembly qualified name if the extension is
implemented in a different assembly, otherwise
the full type name is enough. If given, the
extension type must be discoverable by the
standard .NET type loading logic, it must have
the default constructor and implement the
SetupHTTPS.IExtension interface, otherwise the
program is aborted. So, if the executable is
merged with the ConfigureWCF extension assembly,
then passing --extension=ConfigureWCF.Main
activates that extension. Otherwise, it should
be --extension="ConfigureWCF.Main, ConfigureWCF,
Version=1.0.0.0, Culture=neutral,
PublicKeyToken=null" and the ConfigureWCF.dll
assembly must be visible by the .NET type loader.
--ulog, --uninstallLog=VALUE
The path to a file where the uninstall actions
will be logged. An existing file would be
overwritten. The created file can later be
passed to the execute command.
-p, --port=VALUE The http(s) port. The default is 443 for https
and 80 for http.
--host=VALUE The host name. The server certificate will be
bound to that name. The default is WIN-
OR05DA2H54D
-a, --account=VALUE The name of the account under which the server
is expected to be run. The default is WIN-
OR05DA2H54Dmarkk
--vp, --virtualPath=VALUE
The virtual application path, will be appended
to http(s)://:/. Must be properly
escaped.
--caSubject=VALUE The CA subject name.
--makeCert=VALUE The path to a certificate creation utility. The
default value is C:workmakecert.exe
--createCACertArgs=VALUE
The certificate creation utility command line
arguments to create the CA certificate. {0}
would be replaced with the CA subject name. The
default value is
--createServerCertArgs=VALUE
The certificate creation utility command line
arguments to create the Server certificate. {0}
would be replaced with the given host name,
which becomes the certificate's subject name.
{1} would be replaced with the CA subject name.
The default value is
PS C:work>
The thing to be noticed in the help output is the default values
of the --account, --host and
--makeCert options. The last one in particular – if
you do not see the default value of the --makeCert
option, then the tool has failed to locate the makecert.exe
utility on your machine and you will have to help it by using one
of the methods mentioned before.
Having said all that let us run the tool for real:
And in the dry mode again to compare with the previous dry run:
Now the server:
Seems OK, does it? Not quite. The truth is that our server is still working over HTTP, rather than HTTPS. Two reasons why it starts OK at this point (instead of complaining about something being broken):
- There is only one possible intersection between the HTTPS and HTTP configurations – the TCP port. As long as the HTTP and HTTPS ports are different the two configurations can happily live side by side. This is exactly our case, because we utilize different TCP ports for HTTP and HTTPS (80 and 443 respectively).
- Our demo server is a WCF server and it is configured (in the application configuration file ConsoleHost.exe.config) to expect HTTP traffic on port 80.
What we did until now is the mandatory HTTPS configuration for any server belonging to the category described at the beginning of this article. But a server may require additional server specific configurations. In the case of ours we have to tweak its config file as well as that of the console client, because this is what the WCF technology used by them requires. This tweaking can be done manually, but we can structure these config files in a way that makes it possible for the tool to modify them without our intervention.
WCF configuration
The tool can be extended by means of the --extension
command line option and in fact the prebuilt binary is already
merged with the ConfigureWcf extension dll, which enables to
tweak WCF configuration files structured in a certain well
known way. Please, observe:
PS C:work> .SetupHTTPS.exe exthelp --extension=ConfigureWCF.Main
The ConfigureWCF.Main extension allows to modify XML attribute values in an XML configuration file using simple rewrite rules.
The given extension supports the following options:
--config=VALUE The path to the server configuration file.
Specify --mode=client to indicate a client
configuration file instead.
--rules=VALUE The path to a xaml file containing the rewrite
rules for the configuration file specified by
the --config option.
--showSampleRules Displays a sample configuration rewrite rules
file
PS C:work> .SetupHTTPS.exe exthelp --extension=ConfigureWCF.Main --showSampleRules
PS C:work>
Before I show the extension in action, allow me to sketch in a few words how it works:
- Given a WCF server/client application, the extension modifies its config file by applying the given rewrite rules.
- There are only two “interesting” rewrite rule types:
-
RewriteValue– replaces the value of an attribute at the given XPATH. Besides the XPATH itself, the rule specifies two values – theSecureValueand theNonSecureValue. The former is taken when configuring HTTPS and the latter when HTTP is being configured. -
RewriteUri– modifies the URI at the given XPATH to match the scheme being configured – HTTPS or HTTP. Note, that it also understands HTTP namespaces like http://+:80/Wcf/ (which is not a URI).
-
- The rewrite rules can be nested within a
RewriteRuleGroupelement, which specifies the XPATH prefix shared by all the rules within. Naturally, theRewriteRuleGroupelements themselves can be nested too. - In addition to the rewrite rules, there are predefined
tokens, which are replaced by the appropriate values at runtime.
Currently, only one token is supported –
CertificateThumbprint. Specify%{CertificateThumbprint}as theSecureValuefor some attribute and it will be replaced with the thumbprint of the relevant Server certificate.
As one can see, the extension does nothing WCF specific, it just rewrites XML attributes at the given XPATHs. So what is the deal with this “structured in a certain way” requirement? The ConfigureWCF extension is very simple, it only knows to modify the value of an existing attribute. It knows to neither add new nor delete existing elements or attributes. It so happens, that this limitation is not a problem for the config file of a WCF server/client – it can be arranged in such a way that the two setups (HTTPS vs HTTP) differ only in the values of existing attributes.
Back to our demo server. Let us, finally, run the tool with the
extension activated to modify the relevant config files. We shall
use the rules displayed by the --showSampleRules
option, because they conveniently address all the configuration
needs of the demo server. As always, we start in the dry mode
first:
According to the output:
- Nothing is to be done in the certificate, port binding and HTTP namespace reservation department – previous runs of the tool have covered that area.
- The tool wants to modify certain attributes in the server
config file, all of them within the
/configuration/system.serviceModelelement.
Running the tool for real (without the -n option)
produces the same output (only without the === DRY RUN ===
preamble). And if we run it yet again:
Make note of the last line – the tool finds nothing to change in the server config file.
At this point the server should start listening on port 443 over HTTPS and we should be able to navigate to https://localhost/Wcf/SimpleDemo/web/Add?x=12&y=13:
Just click on “Continue to this website (not recommended).” IE
sees that the Server certificate is signed by an unknown
certificate, hence the “not recommended” notice.
Everything seems to work fine, except for the “Certificate error” indication in the url input text box of the browser.
There is one more client to test – the ConsoleClient application. It should be obvious by now, that it would not work – its WCF configuration still targets the old url of the server. Please, observe:
PS C:work> .ConsoleClient.exe System.ServiceModel.EndpointNotFoundException: Could not connect to http://win-or05da2h54d/Wcf/SimpleDemo/ws. TCP error code 10061: No connection could be made because the target machine actively refused it 192.168.220.141:80. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 192.168.220.141:80 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception) --- End of inner exception stack trace --- at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context) at System.Net.HttpWebRequest.GetRequestStream() at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream() --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream() at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout) at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at ConsoleClient.ServiceReference.IWcfDemoService.Add(Int32 x, Int32 y) at ConsoleClient.ServiceReference.WcfDemoServiceClient.Add(Int32 x, Int32 y) in c:workSimpleDemoConsoleClientService ReferencesServiceReferenceReference.cs:line 161 at ConsoleClient.Program.Main(String[] args) in c:workSimpleDemoConsoleClientProgram.cs:line 64 PS C:work>
Luckily, the ConfigureWCF extension can be used to configure the client configuration file as well:
Of course! Neither the --vp nor the
--caSubjectName option make sense in the client mode
(requested by the --mode=client option).
Running the tool with sethttps -e ConfigureWCF.Main -h
--mode=client prints all the options relevant to the
client mode. Actually, the tool by itself does not support this
mode at all, because the core HTTPS configuration does not
involve the client side. It is the ConfigureWCF extension, which
makes this mode available. Let us proceed with a dry run first:
Looks very similar to the changes previously applied to the server configuration file ConsoleHost.exe.config. Running for real produces exactly the same output, only without the === DRY RUN === banner. At this time, the console client should print OK (the very first run might take a few long seconds).
Two machines
We are not through, however. Our HTTPS configuration procedure cannot be taken seriously until it is tested on two separate machines – one for the server and another for the client (make sure the firewall on the server does not block the incoming traffic). As was mentioned before, the demo server exposes two endpoints:
- plain HTTP at
https://WIN-OR05DA2H54D/Wcf/SimpleDemo/web - SOAP HTTP at
https://WIN-OR05DA2H54D/Wcf/SimpleDemo/ws
So, given another machine, we should be able to access the plain HTTP endpoint from the web browser in exactly the same manner as we did before on the server machine:
The Chrome browser on my dev machine warns against accessing the plain HTTP endpoint of our demo server, because it does not recognize the certificate used to sign the incoming Server certificate. Recall, that the Internet Explorer browser on the server machine behaves exactly the same. Anyway, ignoring the warning takes us to the desired result.
Now the console client. First, we need to configure it:
Notice the presence of the --host option, which is
necessary, because by default the host name of the local machine
would be used and, of course, the server is not there.
Unfortunately and despite these efforts, running the console
client does not seem to yield a satisfactory outcome yet:
PS Z:devpocInternalToolsSetupHTTPSbinMerged> workvswcfSimpleDemoConsoleClientbinDebugConsoleClient.exe System.ServiceModel.Security.SecurityNegotiationException: Could not establish trust relationship for the SSL/TLS secure channel with authority 'win-or05da2h54d'. ---> System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure. at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result) at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.ConnectStream.WriteHeaders(Boolean async) --- End of inner exception stack trace --- at System.Net.HttpWebRequest.GetResponse() at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason) at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at ConsoleClient.ServiceReference.IWcfDemoService.Add(Int32 x, Int32 y) at ConsoleClient.ServiceReference.WcfDemoServiceClient.Add(Int32 x, Int32 y) in C:WorkVSWcfSimpleDemoConsoleClientService ReferencesServiceReferenceReference.cs:line 161 at ConsoleClient.Program.Main(String[] args) in C:WorkVSWcfSimpleDemoConsoleClientProgram.cs:line 64
PS Z:devpocInternalToolsSetupHTTPSbinMerged>
Examining the error message reveals the following interesting
phrase – The remote certificate is invalid
according to the validation procedure.
What happens is that the WCF console client follows the same
route as the browser – it tries to validate the Server
certificate first. Unlike the browser, which defaults to asking
the user whether the received Server certificate is to be
trusted, the WCF framework defaults to aborting with the
aforementioned exception. It is possible, however, to override
this behaviour by intercepting a dedicated event and performing
the certificate validation ourselves. In fact, our console client
knows to do it already, it is just not its default behavior:
Our demo client is a WCF client, hence it validates the server
certificate using the procedure prescribed by the WCF framework.
Or is it the .NET framework? I am not sure myself. Anyway, a java
(python, ruby, etc…) client may require a different Server
certificate validation procedure. The important detail here is
that the client machine has neither the Server nor the signing CA
certificate, although the --trustThis option does
require the client to have some knowledge of the incoming Server
certificate, namely its thumbprint.
Obviously, the demo client could have acted exactly like the browser – asking the user to approve or reject the received Server certificate.
Some of those who have survived this post until this point may have a question – why the console client does not complain when ran on the same machine as the server, whereas the browser behaves consistently and always warns the user about the Server certificate? It all boils down to the CA certificate used to sign the Server certificate. The answer contains two parts:
- When the client runs on the server machine, the CA certificate is visible to it and so it can verify that the CA certificate is indeed trusted, i.e. it is found in the Trusted Root Certificates Authorities folder of the system certificate storage. This CA certificate, however, is unavailable to the other machine, hence the default validation fails there.
- Apparently, the browser and a WCF client disagree as to what a “good” CA certificate is. A WCF client seems to be satisfied, if the CA certificate is trusted, never mind the fact that that certificate may be self signed. The browser, on the contrary, is not fond of the fact that the CA certificate is self signed – it wants it to be signed by a well known certificate authority, like VeriSign and such.
It follows, that if the CA certificate is exported from the server machine (without the private key, of course) and then imported on the other machine as trusted, then the default server certificate validation procedure employed by our WCF client would not fail it.
This concludes our exploration of the main scenario.
Scenario II – bring it back
The tool is installer friendly:
- it attempts to return a distinct exit code for each recognized error condition (there are around 25)
- error messages are output to the standard error only
- there is a quiet mode, where only important messages are displayed
- it is possible to create an uninstall log, which can be used at a later time to revert to the old configuration
This scenario shows off the last feature – reverting back to the
old configuration. It is enabled by the
--uninstallLog command line option, which is
recognized by all the commands of the tool, but the
exthelp command. For the sake of the demo, let
us setup a slightly different HTTPS configuration from the one we
have done before, namely we shall install our demo server as
a windows service running with the Network Service credentials.
So, as the first step please run the powershell
script InstallAsWinService.ps1 under elevated permissions.
This registers the demo server executable as a windows service to
be run with the Network Service credentials.
(Make sure the powershell’s execution policy is set to RemoteSigned. You can do it by running the Set-ExecutionPolicy RemoteSigned -Force command as elevated, see the SetPSExecutionPolicy.cmd script).
Now the server is installed, but it must be configured to serve requests over HTTPS. This time we would like to have the uninstall log, which may be used to revert to the old configuration:
Things to note:
-
-a(--account) command line option specifying the server account (Network Service) -
--ulog(--uninstallLog) command line option specifying where the uninstall log is to be created. - The tool reuses the existing certificates, which stay unaffected by this change in the configuration.
- Notice the conflict between the old HTTP namespace
reservation record and the desired one. This happens, because the
same HTTP namespace (
https://+:443/Wcf/) cannot be reserved both forWIN-OR05DA2H54Dmarkkand forNT AUTHORITYNETWORK SERVICE– we have to delete the reservation for the former in order to be able to add it for the latter.
(I strongly advise running the tool in the dry mode before making any changes. This is what I do. However, this post is pretty long already and the dry mode output is almost identical to the output of the real run, so I am going to skip it for brevity from now on)
At this point the server should be expecting HTTPS (you can start
it by net start "Simple Demo Server") and both the
browser and the console client should be able to connect to it
exactly as before.
Now suppose that we have second thoughts about this configuration
and wish to revert back to the old one, the one where the server
was running as WIN-OR05DA2H54Dmarkk. One way is
simply invoke the tool with the appropriate parameters, like so:
The second way is just use the previously saved uninstall log:
The result is the same, but the two commands have different side effects. Notice, how using the uninstall log undoes granting the Network Service the read access to the Server certificate’s private key. It is important to understand that an uninstall log is the true delta between the old and the new configurations. The commands it contains make sure that the current configuration matches the new configuration at the time the log was created. They refuse to change anything if there is a mismatch. This can be easily demonstrated by applying the same uninstall log the second time in a row:
As you can see, no changes are made here.
Let us conclude by having a look at the uninstall log itself:
The execute command accepts the –ulog parameter as well, making it easy to redo the undo.
Scenario III – wiping the slate clean
Sometimes, we would like to clear the existing HTTPS configuration, i.e. remove the certificates, delete the bindings, etc.. The tool command for this task is clearhttps:
PS C:work> .SetupHTTPS.exe clearhttps -h
ClearHttps - Clear the given HTTPS configuration.
The tool is merged with the following extensions:
- ConfigureWCF.Main
ClearHttps Server options:
-h, -?, --help Show this help screen.
-q, --quiet Be quiet.
-n, --dryRun Dry run - perform all the steps, but modify
nothing. Ignores the --quiet flag, if the latter
is given.
--dontElevate Do not try to elevate.
-e, --extension=VALUE The name of an optional extension type. Must be
an assembly qualified name if the extension is
implemented in a different assembly, otherwise
the full type name is enough. If given, the
extension type must be discoverable by the
standard .NET type loading logic, it must have
the default constructor and implement the
SetupHTTPS.IExtension interface, otherwise the
program is aborted. So, if the executable is
merged with the ConfigureWCF extension assembly,
then passing --extension=ConfigureWCF.Main
activates that extension. Otherwise, it should
be --extension="ConfigureWCF.Main, ConfigureWCF,
Version=1.0.0.0, Culture=neutral,
PublicKeyToken=null" and the ConfigureWCF.dll
assembly must be visible by the .NET type loader.
--ulog, --uninstallLog=VALUE
The path to a file where the uninstall actions
will be logged. An existing file would be
overwritten. The created file can later be
passed to the execute command.
-p, --port=VALUE The http(s) port. The default is 443 for https
and 80 for http.
--host=VALUE The host name. The server certificate will be
bound to that name. The default is WIN-
OR05DA2H54D
-a, --account=VALUE The name of the account under which the server
is expected to be run. The default is WIN-
OR05DA2H54Dmarkk
--vp, --virtualPath=VALUE
The virtual application path, will be appended
to http(s)://:/. Must be properly
escaped.
--caSubject=VALUE The CA subject name.
--keepCACert Indicates to keep the CA certificate intact
while clearing the current HTTPS configuration.
Should be used if the CA certificate was not
created by this tool. The default is false, i.e.
to delete the CA certificate.
PS C:work>
Let us use it to clear all the HTTPS setups we have configured until now, namely:
- The server setup.
- The WCF client setup on the server and the second machine.
Clearing the server setup
Note, that we cannot just clear the HTTPS configuration from the server/client config file. The proper thing to do is reconfigure it as HTTP and this is exactly how the ConfigureWCF extension implements the clearhttps command.
Clearing the client setup on the server machine
Clearing the client setup on the second machine
Actually, this is the same as on the server machine, except that
the --host option must be given:
Of course, we could have used the --ulog option
to create the uninstall log, which could have been later used to
restore the cleared HTTPS configuration.
Under the Hood
Why does the whole thing look so complex? After all, it is so simple in java (ruby/python/you-name-it) ? I claim, that configuring HTTPS with the HTTP Server API requires essentially the same steps as configuring HTTPS with java (ruby/python/you-name-it). Unfortunately, I am only familiar with the HTTPS setup of a standalone Restlet server using the Simple HTTP server framework (Restlet comes already integrated with it). So, I will take my chances, that other systems are no more complex.
It may seem unrelated to this post, but bear with me – I believe it is educational. So, what would one do to configure a standalone Restlet server to use HTTPS? Easy:
- Create the Server certificate using the keytool ( http://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html). This would create a keystore file. The convention is to give it the jks extension (not the best sounding extension in the world, if you ask me). Let this file be c:devmy-cool-server.jks and let abracadabra be the password used to protect the access to both the keystore file and the Server certificate within.
- Next, we have to inform the HTTP server framework to use the
Server certificate from this file during the HTTPS sessions. A
code snippet like the one below could be used to serve this
purpose:
Server server = component.getServers().add(Protocol.HTTPS, 8182); Series parameters = server.getContext().getParameters(); parameters.add("sslContextFactory", PkixSslContextFactory.class.getName()); parameters.add("keystorePath", "c:\dev\my-cool-server.jks"); parameters.add("keystorePassword", "abracadabra"); // I do not know if there is a way to avoid having the passwords in the code in clear text, parameters.add("keyPassword", "abracadabra"); // but I will not be surprised if there is parameters.add("keystoreType", "JKS");
That’s it. Simple and less steps. Or is it? Let us compare. Configuring HTTPS using HTTP Server API requires 4 steps:
- Creating the Server certificate, which may be self signed or signed by another certificate, which in turn may be self signed (as is the case in the examples here) or signed by yet another certificate and so on.
- Granting the account under which the Server is running the read access to the private key associated with the Server certificate.
- Binding the HTTP port (actually this is a TCP port) used by the Server to the respective Server certificate.
- Reserving the HTTP namespace used by the server to the account under which the server is running.
Let us examine these steps more closely and see how (and if) they are matched in the Restlet example above, referred to as simply Restlet below.
The Server certificate
HTTPS communication with the server implies the presence of a Server certificate. Period. No matter the server technology. Both HTTP Server API and Restlet demand it. One can argue, that using keytool is easier than makecert. Inconclusive.
The certificate is used by the Server to identify itself before
its clients, which may or may not have apriori knowledge of the
certificate details. The example clients used in this post fall
in the latter category (let’s call them naive
clients), except when using the --trustThis
command line option of the console client, which demands from the
client knowing the thumbprint of the expected Server certificate.
Of course, naive clients are more vulnerable to malicious actions.
The exact command line used to create the certificates can be found in the tool output.
Permissions
The Server certificate is associated with a pair of keys – the private key and the public key. The public key is known to the world and gets transmitted to the client along with the Server certificate. The client encrypts the HTTP traffic using this key. Only the owner of the respective private key can decrypt the traffic, which is our server and here we come to the point where the server must be able to access the private key associated with the Server certificate. This is an explicit configuration step when configuring HTTPS with the HTTP Server API, but where is it when doing it with Restlet?
Notice, that we set the value of the keystorePath parameter to be the path to the keystore file. The file system permissions associated with this file must be sufficient for the server account to read it, otherwise the server will have access denied. And if the server is to be run under account, which currently does not have the read access to the keystore file – an explicit configuration is in order. Either grant that account more rights or change the permissions on the file. This way or that way an explicit configuration step is required, just like it is with the HTTP Server API. Is it simpler with Restlet? I tend to think it is.
(The scheme presented here is oversimplified. It is entirely possible that the private key is needed at an earlier stage. It is also possible that not all the traffic is encrypted using public/private keys, but only the initial conversation, during which a symmetric key is created and exchanged between the parties and the rest of the session is encrypted using some kind of a symmetric encryption).
It is also possible to set the necessary permissions from the command prompt. The process is to locate the file containing the private key of the certificate and change its ACL to allow the relevant account the read-only access to it. The following command shell session demonstrates the whole thing on a Windows 7 workstation:
PS C:work> .SetupHTTPS.exe sethttps --vp=Wcf --caSubject=WcfDemo Using the existing CA certificate 'WcfDemo' (Issuer = 'WcfDemo', Hash = 63FFA5AF16EA9466AF9F5BCE5FE62EFB3F2E004C) Using the existing Server certificate 'WIN-OR05DA2H54D' (Issuer = 'WcfDemo', Hash = E611348193F737CEBBE0B46C6A129AFBD2314A3E) WIN-OR05DA2H54Dmarkk appears to have the read access to the private key of the certificate 'WIN-OR05DA2H54D' (Issuer ='WcfDemo', Hash = E611348193F737CEBBE0B46C6A129AFBD2314A3E) already. [MATCH] The port 443 is bound to the certificate 'WIN-OR05DA2H54D' (Issuer = 'WcfDemo', Hash = E611348193F737CEBBE0B46C6A129AFBD2314A3E). [MATCH] The HTTP namespace https://+:443/Wcf/ is reserved for 'WIN-OR05DA2H54Dmarkk' (SID = S-1-5-21-202045982-2235527663-522765838-1000). PS C:work> certutil -store My E611348193F737CEBBE0B46C6A129AFBD2314A3E My ================ Certificate 0 ================ Serial Number: afaa6635e7365ca64d55e3b834987846 Issuer: CN=WcfDemo NotBefore: 3/28/2012 9:32 PM NotAfter: 1/1/2040 2:59 AM Subject: CN=WIN-OR05DA2H54D Non-root Certificate Template: Cert Hash(sha1): e6 11 34 81 93 f7 37 ce bb e0 b4 6c 6a 12 9a fb d2 31 4a 3e Key Container = 5a98dd76-1a30-4dc1-848f-352514632440 Unique container name: e722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 Provider = Microsoft Strong Cryptographic Provider Encryption test passed CertUtil: -store command completed successfully. PS C:work> icacls C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 /save aclfile processed file: C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 Successfully processed 1 files; Failed processing 0 files PS C:work> cat aclfile -Encoding Unicode e722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 D:AI(A;;FA;;;SY)(A;;FA;;;BA)(A;;FR;;;S-1-5-5-0-97349)(A;;FR;;;S-1-5-21-202045982-2235527663-522765838-1000) PS C:work> icacls C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 /remove:g WIN-OR05DA2H54Dmarkk processed file: C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 Successfully processed 1 files; Failed processing 0 files PS C:work> icacls C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 /save aclfile processed file: C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 Successfully processed 1 files; Failed processing 0 files PS C:work> cat .aclfile -Encoding Unicode e722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 D:AI(A;;FA;;;SY)(A;;FA;;;BA)(A;;FR;;;S-1-5-5-0-97349) PS C:work> .SetupHTTPS.exe sethttps --vp=Wcf --caSubject=WcfDemo -n | Select-String "(===|access)" === DRY RUN === Granting WIN-OR05DA2H54Dmarkk the read access to the private key of the certificate 'WIN-OR05DA2H54D' (Issuer = 'WcfDemo', Hash = E611348193F737CEBBE0B46C6A129AFBD2314A3E). PS C:work> icacls C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 /grant WIN-OR05DA2H54Dmarkk:R processed file: C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 Successfully processed 1 files; Failed processing 0 files PS C:work> icacls C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 /save aclfile processed file: C:ProgramDataMicrosoftCryptoRSAMachineKeyse722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 Successfully processed 1 files; Failed processing 0 files PS C:work> cat .aclfile -Encoding Unicode e722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 D:AI(A;;FR;;;S-1-5-21-202045982-2235527663-522765838-1000) (A;;FA;;;SY)(A;;FA;;;BA)(A;;FR;;;S-1-5-5-0-97349) PS C:work> .SetupHTTPS.exe sethttps --vp=Wcf --caSubject=WcfDemo -n | Select-String "(===|access)" === DRY RUN === WIN-OR05DA2H54Dmarkk appears to have the read access to the private key of the certificate 'WIN-OR05DA2H54D' (Issuer = 'WcfDemo', Hash = E611348193F737CEBBE0B46C6A129AFBD2314A3E) already. PS C:work>
Note, how changing the ACL of the Server certificate’s private
key container file affects the dry run output of the tool.
Port binding
In order to understand the meaning of this step, one has to
understand how the HTTP Server API works.
The essence of the HTTP Server API is that all the HTTP traffic
is handled by a dedicated kernel mode driver http.sys (http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/a2a45c42-38bc-464c-a097-d7a202092a54.mspx?mfr=true).
A server built on top of the HTTP Server API registers itself
with that driver in order to receive notifications when there are
incoming HTTP requests for it. The idea is that http.sys handles
all the routine chores related to parsing the TCP traffic and
converting it to HTTP requests, so that high level servers could
concentrate on their logic. In particular, it hides the HTTPS
related details. But, that means that when a client establishes
an HTTPS connection with such a server, it is the http.sys driver
which “greets” it. Sending the Server certificate is part of this
greeting protocol. But how does the driver know which certificate
to send? And here is where the port binding kicks in – someone
must associate (bind !) the Server certificate with the port used
by that server, this way the driver would know which certificate
to send the next time a client tries to establish an HTTPS
connection on that port.
Do we have this step in the Restlet example setup above? Of course, we do! It is not called port binding, though…
One can use the netsh tool on Windows 7/2008 or httpcfg on Windows XP/2003 to bind a certificate to a port.
Windows 7/2008:
PS C:work> .SetupHTTPS.exe sethttps --vp=Wcf --caSubject=WcfDemo Creating new CA certificate with the subject name WcfDemo C:workmakecert.exe -n "CN=WcfDemo" -sr localmachine -ss root -a sha1 -cy authority -r Creating new Server certificate with the subject name WIN-OR05DA2H54D issued by WcfDemo C:workmakecert.exe -n "CN=WIN-OR05DA2H54D" -sr localmachine -ss my -cy end -pe -sky exchange -a sha1 -is Root -ir LocalMachine -in WcfDemo Granting WIN-OR05DA2H54Dmarkk the read access to the private key of the certificate 'WIN-OR05DA2H54D' (Issuer = 'WcfDemo', Hash = E611348193F737CEBBE0B46C6A129AFBD2314A3E). [NOT FOUND] The port 443 is not bound to any certificate. Binding the port 443 to the certificate 'WIN-OR05DA2H54D' (Issuer = 'WcfDemo', Hash = E611348193F737CEBBE0B46C6A129AFBD2314A3E). [NOT FOUND] The HTTP namespace https://+:443/Wcf/ is not reserved for any account. Reserving the HTTP namespace https://+:443/Wcf/ for 'WIN-OR05DA2H54Dmarkk' (SID = S-1-5-21-202045982-2235527663-522765838-1000). PS C:work> netsh http show sslcert ipport=0.0.0.0:443 SSL Certificate bindings: ------------------------- IP:port : 0.0.0.0:443 Certificate Hash : e611348193f737cebbe0b46c6a129afbd2314a3e Application ID : {00000000-0000-0000-0000-000000000000} Certificate Store Name : My Verify Client Certificate Revocation : Enabled Verify Revocation Using Cached Client Certificate Only : Disabled Usage Check : Enabled Revocation Freshness Time : 0 URL Retrieval Timeout : 0 Ctl Identifier : (null) Ctl Store Name : (null) DS Mapper Usage : Disabled Negotiate Client Certificate : Enabled PS C:work> certutil -store My e611348193f737cebbe0b46c6a129afbd2314a3e My ================ Certificate 0 ================ Serial Number: afaa6635e7365ca64d55e3b834987846 Issuer: CN=WcfDemo NotBefore: 3/28/2012 9:32 PM NotAfter: 1/1/2040 2:59 AM Subject: CN=WIN-OR05DA2H54D Non-root Certificate Template: Cert Hash(sha1): e6 11 34 81 93 f7 37 ce bb e0 b4 6c 6a 12 9a fb d2 31 4a 3e Key Container = 5a98dd76-1a30-4dc1-848f-352514632440 Unique container name: e722d2765ccc6930858e5e871c12d73c_114e56eb-b518-4612-9dba-7c6625f6c531 Provider = Microsoft Strong Cryptographic Provider Encryption test passed CertUtil: -store command completed successfully. PS C:work> netsh http delete sslcert ipport=0.0.0.0:443 SSL Certificate successfully deleted PS C:work> netsh http show sslcert ipport=0.0.0.0:443 SSL Certificate bindings: ------------------------- The system cannot find the file specified. PS C:work> netsh http add sslcert ipport=0.0.0.0:443 certhash=e611348193f737cebbe0b46c6a129afbd2314a3e appid='{00000000-0000-0000-0000-000000000000}' clientcertnegotiation=enable SSL Certificate successfully added PS C:work> netsh http show sslcert ipport=0.0.0.0:443 SSL Certificate bindings: ------------------------- IP:port : 0.0.0.0:443 Certificate Hash : e611348193f737cebbe0b46c6a129afbd2314a3e Application ID : {00000000-0000-0000-0000-000000000000} Certificate Store Name : (null) Verify Client Certificate Revocation : Enabled Verify Revocation Using Cached Client Certificate Only : Disabled Usage Check : Enabled Revocation Freshness Time : 0 URL Retrieval Timeout : 0 Ctl Identifier : (null) Ctl Store Name : (null) DS Mapper Usage : Disabled Negotiate Client Certificate : Enabled PS C:work>
Windows XP (SP3):
PS C:work> .SetupHTTPS.exe sethttps --vp=Wcf --caSubject=WcfDemo Creating new CA certificate with the subject name WcfDemo C:workmakecert.exe -n "CN=WcfDemo" -sr localmachine -ss root -a sha1 -cy authority -r Creating new Server certificate with the subject name MARKK-0F449E6D7 issued by WcfDemo C:workmakecert.exe -n "CN=MARKK-0F449E6D7" -sr localmachine -ss my -cy end -pe -sky exchange -a sha1 -is Root -ir LocalMachine -in WcfDemo MARKK-0F449E6D7markk appears to have the read access to the private key of the certificate 'MARKK-0F449E6D7' (Issuer = 'WcfDemo', Hash = 0090902A4C418D2A8162993BBBCE2946C650D7CC) already. [NOT FOUND] The port 443 is not bound to any certificate. Binding the port 443 to the certificate 'MARKK-0F449E6D7' (Issuer = 'WcfDemo', Hash = 0090902A4C418D2A8162993BBBCE2946C650D7CC). [NOT FOUND] The HTTP namespace https://+:443/Wcf/ is not reserved for any account. Reserving the HTTP namespace https://+:443/Wcf/ for 'MARKK-0F449E6D7markk' (SID = S-1-5-21-725345543-1292428093-1177238915-1004). PS C:work> httpcfg.exe query ssl -i 0.0.0.0:443 IP : 0.0.0.0:443 Hash : 090902a4c418d2a8162993bbbce2946c650d7cc Guid : {00000000-0000-0000-0000-000000000000} CertStoreName : My CertCheckMode : 0 RevocationFreshnessTime : 0 UrlRetrievalTimeout : 0 SslCtlIdentifier : (null) SslCtlStoreName : (null) Flags : 2 ------------------------------------------------------------------------------ PS C:work> certutil -store My 0090902a4c418d2a8162993bbbce2946c650d7cc 402.203.0: 0x80070057 (WIN32: 87): ..CertCli Version ================ Certificate 0 ================ Serial Number: 57aececbc0a495b74524e376e388dee0 Issuer: CN=WcfDemo Subject: CN=MARKK-0F449E6D7 Non-root Certificate Cert Hash(sha1): 00 90 90 2a 4c 41 8d 2a 81 62 99 3b bb ce 29 46 c6 50 d7 cc Key Container = 7ac32ae9-56f1-4652-811f-bef80f313b53 Provider = Microsoft Strong Cryptographic Provider Encryption test passed CertUtil: -store command completed successfully. PS C:work> httpcfg delete ssl -i 0.0.0.0:443 HttpDeleteServiceConfiguration completed with 0. PS C:work> httpcfg.exe query ssl -i 0.0.0.0:443 HttpQueryServiceConfiguration completed with 2. PS C:work> httpcfg set ssl -i 0.0.0.0:443 -h 0090902a4c418d2a8162993bbbce2946c650d7cc -f 2 -c My HttpSetServiceConfiguration completed with 0. PS C:work> httpcfg.exe query ssl -i 0.0.0.0:443 IP : 0.0.0.0:443 Hash : 090902a4c418d2a8162993bbbce2946c650d7cc Guid : {00000000-0000-0000-0000-000000000000} CertStoreName : My CertCheckMode : 0 RevocationFreshnessTime : 0 UrlRetrievalTimeout : 0 SslCtlIdentifier : (null) SslCtlStoreName : (null) Flags : 2 ------------------------------------------------------------------------------ PS C:work>
Notes:
- The certutil may also accept the space
delimited certificate hash, i.e. something like “
00 90 90 2a 4c 41 8d 2a 81 62 99 3b bb ce 29 46 c6 50 d7 cc“. - The default value of the certificate store name parameter of
the netsh and httpcfg
commands is
My, so specifying it is optional. - Windows XP initially comes with neither
certutil nor httpcfg. These
commands must be installed explicitly:
- Install Windows Server 2003 Administration Tools Pack for certutil. You have to install it, simply extracting certutil.exe from the installation would nor work.
- Install Windows XP Service Pack 2 Support Tools for httpcfg. Unlike certutil, this one can be simply extracted from the installer.
- Beware of a peculiar bug in the httpcfg utility . In the
query output it replaces some (not all!) zeros of the certificate
hash with spaces, so
“
0090902A4C418D2A8162993BBBCE2946C650D7CC” becomes" 090902a4c418d2a8162993bbbce2946c650d7cc"!
Reserving HTTP namespace
This step is not HTTPS specific. Plain HTTP configuration requires it as well.
So, what is it? As I have already mentioned, all the HTTP traffic is handled by the kernel mode driver http.sys, which dispatches the incoming HTTP requests to the registered servers. But what is the registration key? Enter the HTTP namespace.
Please, notice that:
- Two different HTTP namespaces can have identical ports (like
http://+:8888/YabaDabaDooandhttp://+:8888/YogiBear) and even identical prefixes (likehttps://+:443/YabaDabaDooandhttps://+:443/YabaDabaDoo/XYZ) - The http.sys driver is the sole owner of the TCP sockets involved in the HTTP communication.
It is therefore possible to have two different servers share the same HTTP port, all they need is register themselves with the http.sys driver under different HTTP namespaces.
This is all cool, but there is a tiny problem – without any special arrangements one server could “steal” the traffic from another server. Indeed, what prevents the YabaDabaDoo server from registering two namespaces – one for itself and the other on behalf of the YogiBear server? Then once the YogiBear server tries to register itself it would be refused on the basis that the registration already exists. In effect, the YabaDabaDoo server “steals” the traffic intended for the YogiBear server.
The HTTP Server API solves this problem by requiring that an HTTP namespace be associated with an account (it is said to be reserved for the account) and only that account (or the administrator) is able to register for the HTTP traffic coming through the namespace in question. Now, if the YabaDabaDoo server wishes to do its dirty trick on the YogiBear server, it would have to impersonate the account used by the latter. The downside is that reserving an HTTP namespace for an account requires elevation, but it needs to be done just once.
Note, that in the world of pure socket based servers nothing prevents one server from opening two sockets, thus preventing another server from running. This can be partially addressed by having firewall rules, which should refuse forwarding the traffic intended for the YogiBear server to the YabaDabaDoo server. This way the YabaDabaDoo server would not be able to “steal” the traffic, but it can still prevent the other server from getting it by keeping the socket open. (In reality, though it is not a problem, because netstat should immediately disclose an offending server, which would never be allowed to run again)
Clearly, no such step exists in the aforementioned Restlet configuration sample. On the other hand, two standalone Restlet servers cannot share the same HTTP port. However, HTTP port sharing does exist outside the HTTP Server API and is widely used. Any application server or general purpose web server (like Apache or IIS) provide it. Indeed, a Tomcat application server may host many different servers (or should I say servlets), all of which would share the same HTTP port – the port used by the Tomcat instance itself. In effect, the http.sys driver can be viewed as a mini application server provided by the OS.
Here as well, the netsh tool on Windows 7/2008 or httpcfg on Windows XP/2003 can be used to reserve the HTTP namespaces.
Windows 7/2008:
PS C:work> .SetupHTTPS.exe sethttps --vp=Wcf --caSubject=WcfDemo | Select-String namespace [MATCH] The HTTP namespace https://+:443/Wcf/ is reserved for 'WIN-OR05DA2H54Dmarkk' (SID = S-1-5-21-202045982-2235527663-522765838-1000). PS C:work> netsh http show urlacl url=https://+:443/Wcf/ URL Reservations: ----------------- Reserved URL : https://+:443/Wcf/ User: WIN-OR05DA2H54Dmarkk Listen: Yes Delegate: No SDDL: D:(A;;GX;;;S-1-5-21-202045982-2235527663-522765838-1000) PS C:work> netsh http delete urlacl url=https://+:443/Wcf/ URL reservation successfully deleted PS C:work> netsh http show urlacl url=https://+:443/Wcf/ URL Reservations: ----------------- PS C:work> netsh http add urlacl url=https://+:443/Wcf/ user=WIN-OR05DA2H54Dmarkk URL reservation successfully added PS C:work> netsh http show urlacl url=https://+:443/Wcf/ URL Reservations: ----------------- Reserved URL : https://+:443/Wcf/ User: WIN-OR05DA2H54Dmarkk Listen: Yes Delegate: No SDDL: D:(A;;GX;;;S-1-5-21-202045982-2235527663-522765838-1000) PS C:work>
Windows XP (SP3):
PS C:work> .SetupHTTPS.exe sethttps --vp=Wcf --caSubject=WcfDemo | Select-String namespace
[MATCH] The HTTP namespace https://+:443/Wcf/ is reserved for 'MARKK-0F449E6D7markk' (SID = S-1-5-21-725345543-1292428093-1177238915-1004).
PS C:work> httpcfg query urlacl -u https://+:443/Wcf/
URL : https://+:443/Wcf/
ACL : D:(A;;GX;;;S-1-5-21-725345543-1292428093-1177238915-1004)
------------------------------------------------------------------------------
PS C:work> httpcfg delete urlacl -u https://+:443/Wcf/
HttpDeleteServiceConfiguration completed with 0.
PS C:work> httpcfg query urlacl -u https://+:443/Wcf/
HttpQueryServiceConfiguration completed with 2.
PS C:work> httpcfg set urlacl -u https://+:443/Wcf/ -a 'D:(A;;GX;;;S-1-5-21-725345543-1292428093-1177238915-1004)'
HttpSetServiceConfiguration completed with 0.
PS C:work> httpcfg query urlacl -u https://+:443/Wcf/
URL : https://+:443/Wcf/
ACL : D:(A;;GX;;;S-1-5-21-725345543-1292428093-1177238915-1004)
------------------------------------------------------------------------------
PS C:work>
Conclusion
- The http.sys driver maintains server registrations mapped by the respective HTTP namespace. However, it allows one certificate per port, rather than per HTTP namespace. There may be good reasons for it, I just do not know them.
- Sharing the same HTTP port implies using the same Server certificate.
- Two servers running under the same account may register for the traffic of each other. If it matters, do not use standard accounts like Network Service, rather create a dedicate account.
- The same HTTP namespace cannot be used for both HTTPS and
HTTP. Hence
http://+:1234/ABCandhttps://+:1234/ABCcannot be both reserved, even if they do denote different namespaces.
Testing matrix
The presented tool is based on another tool that we use in production to configure HTTPS on the server side. Our production server is only installed and tested on the Windows Server OS versions. Consequently, there may be problems on the client versions of the Windows OS (XP, Vista and 7). I have tested the tool in these configurations:
- The demo server on Windows XP SP3/2003/7
- The demo console client/IE9/Firefox 11/Chrome 17.0.963.83/wget 1.8.2 (windows build) on a second Windows 7 box, where the console client talks to the SOAP endpoint of the server while all the rest – to the plain HTTP.
The following table summarizes the results:
| Server | Client | Result |
|---|---|---|
| Windows XP SP3 | console | OK |
| Internet Explorer | Failure (5) | |
| Firefox | Failure (ssl_error_rx_record_too_long) | |
| Chrome | Failure (Error 107 (net::ERR_SSL_PROTOCOL_ERROR) : SSL protocol error) | |
| wget | Failure (OpenSSL: error:1408F0C6:SSL routines:SSL3_GET_RECORD:packet length too long) | |
| Windows 2003 | console | OK |
| Internet Explorer | OK (1), (2),(4) | |
| Firefox | OK (1) | |
| Chrome | OK (1), (2),(3) | |
| wget | OK | |
| Windows 7 | console | OK |
| Internet Explorer | OK (1), (2),(4) | |
| Firefox | OK (1) | |
| Chrome | OK (1), (2),(3) | |
| wget | OK |
Notes:
- The console client must be run with either the
--trustAnyor--trustThiscommand line option. - (1) A security exception must be confirmed in order to proceed.
- (2) Neither Internet Explorer nor Chrome would proceed if the
client machine has the WcfDemo root certificate, but it
is not the same as the one used to sign the incoming server
certificate. One can easily get this situation when playing
around with the tool on both machines and reusing the same value
for the
--caSubjectparameter. The WcfDemo root certificate must either be the same on both machines (through export-copy-import) or not exist at all on the client machine. Neither Firefox nor wget have this issue. - (3) Chrome may attempt to authenticate the client before the server by asking the user to select a client certificate from a list. The server does not require this – it is safe to cancel this attempt by clicking the Cancel button on the client certificate selection dialog.
- (4) It may take Internet Explorer ages to present the security exception page and then another epoch to display the requested page. Sometimes, it is just stuck before even showing the security warning, in which case restarting the browser helps.
- (5) Importing the WcfDemo root certificate from the server machine (XP) to the client makes Internet Explorer work without a glitch! The rest of the browsers, including wget, fail with the same result.
Unfortunately, I failed to make work the plain HTTP endpoint when the server is running on Windows XP SP3. All of the HTTP clients I have tried have had problems with it. This issue is not critical though, since no server application should be installed on Windows XP anyway. Still I am curious as to the reasons for the problem. The console client, by the way, works as expected, which implies that the problem is with the WCF configuration of the webHttpBinding on Windows XP, rather than with the core HTTPS configuration performed by the tool. Internet Explorer, however, does succeed, if the WcfDemo root certificate is visible to it – see (5).
What’s next?
- The SetupHTTPS tool is based on a tool we use in production to configure HTTPS for our server, which consists of two parts – an ASP.NET application and a standalone business server. The SetupHTTPS tool covers only a half of the HTTPS configuration we do there, the other half is configuring HTTPS for an ASP.NET application hosted by IIS6 and above. It might be useful to add this functionality to the SetupHTTPS tool.
- The HTTPS configuration examined in this article does not utilize mutual authentication, because the latter demands that a client be equipped with its own certificate. Moreover, the server and the client certificates must be signed by the same certificate. Satisfying this demand makes the HTTPS communication less vulnerable to all sorts of mischievous deeds. However, it also presents a deployment challenge. The certificate signing process may only occur on a dedicated machine (which may or may not coincide with the server machine) called the CA ( ertificate [A]uthority) server. So, when the server or a new client is installed, it must obtain its certificate from the CA server. Implementing this process automatically is not that trivial and it may provide enough meat for yet another tool and an article. I have some working code that does about that, but it requires polish and more testing.
Visitor Rating: 5 (1) (Rate)
 |
|
Story Clicks: 0
Feed Views: 57
Lenses (Add|?)
Comments (Log in to add)
Feed Details
Date Approved: 01/28/2011
By: Anonymous
Search FeedAgg.com
3600 sp6976 serv 4.5624 seconds to generate.