What’s Real User Monitoring, anyway?
We use the term Real User Monitoring to explain what Coradiant’s technology does. The term sounds a bit nebulous, but it does the job. Of course, there are lots of people who think they do real user monitoring; so I’m going to try and explain the differences between us and some of the distinctions.
Synthetic tests
First off are the synthetic testing companies. Their tools—usually sold as recurring monthly services—run scripts at regular intervals from all over the world. These scripts simulate what an ideal user would do: Transactions like checking in, putting something in a cart, or getting an account balance.
Lots of people like synthetic tests because they’re repeatable and predictable. They’re great for baselining; in fact, Chris Looseley of Keynote Systems did a great job explaining this for us at the Webops sessions of Interop Las Vegas.
But they’re not monitoring real users. They’re simulating idealized users from controlled environments. Real users might be miserable while the synthetic tests work just fine.
These tools are essential to web operators; but they won’t tell you anything about the volume of traffic to a site, or whether end users are actually getting the performance that the tests report.
Web log analysis
A second way of collecting information on web health is via weblogs. Each time a server gets a hit, it writes down information on that hit in a logfile (usually following a format called ELF, or Extended Log Format.) The logfile tells you a lot about the request: Where it came from, what it requested, and when it occurred. It might even tell you about the timing of the request.
Web logs are monitoring real user activity. On their own, they’re not that useful. But feed them into a web log analysis tool (like Analog, Webtrends, Sane, or Sawmill) and you’ll find out lots of details: What people searched for, where they went on the site, what browser they used, and so on. More commonly, companies use a web analytics firm like CoreMetrics, WebTrends Live, Omniture, Clicktracks, or WebSideStory that collect activity based on Javascript. Often, activity is displayed in funnels of user activity by step, cross-referenced with search terms.
Web log analysis doesn’t offer much performance data. It won’t split requests down into the elements of latency, or show network forensics. But it’s also aimed at the public-facing, B2C sites. Analytics products are seldom used to explain activity on an intranet or a back-end B2B application.
Sniffers
Technically, sniffing traffic is real user monitoring—after all, real users made all those packets. But even viewing the traffic in a sniffer screen doesn’t tell you much about users. WildPackets, Network General, Niksun and ClearSight are good examples of sniffers I’ve seen, but most people I know use Ethereal, which is free and amazing.
Flow monitoring products
Higher up the stack than sniffers are what I call “flow monitors.” These work in a variety of ways, generally by asking other devices about traffic they saw (using RMON or Netflow). A more open version of Netflow, called IPFIX, is making this more and more attractive to people.
Response time monitoring
Another way to measure application response time is to sniff traffic from span ports and measure the round-trip time of sessions (rather than collecting flow data from network devices.) For example, NetQOS’ SuperAgent measures the end-to-end time between networks and hosts by listening to span ports or taps.
We announced a partnership with NetQOS back in April. Their reporter/analyzer product collects NetFlow and IPFIX data; and their SuperAgent product is a response-time monitoring product that watches the TCP/IP sessions between networks and hosts. It assembles and aggregates these so you can see how much traffic flowed from what network to what port on a server. And it measures performance data—how long the packets took to travel, how long the server thought about them, and so on.
What does what
A flow monitoring product summarizes things at the time of collection (i.e. on the router) so it can’t peer within the flow. Response time monitors can look within the traffic, but are generally protocol-agnostic: They don’t “understand” a web, e-mail, or IM session across individual traffic flows. This means that if a protocol-agnostic monitoring tools sees that there were 50 Kbytes of data between a network and a web host, the operator still doesn’t know whether that was one 50Kbyte object, or 50 1-Kbyte objects.
As a result, I don’t know if this session was one user, or 10 users behind a NATting firewall. I don’t know how long individual pages took, or how many pages a user requested in a visit, and so on. And I can’t tell things like browser type or search string, or what they entered in a form.
On the other hand, flow monitors and response time monitors are great for comparing the amount of traffic across all kinds of applications. A sudden increase in Voice-Over-IP (VOIP) traffic might mean that web traffic takes longer to get through; someone running a backup late at night might inadvertently make late-night shoppers miserable. And this kind of activity is completely invisible to a product that’s watching HTTP. So if you’re trying to troubleshoot and measure networks, you need a flow monitor (preferably from our friends at NetQOS; they also have great SNMP monitoring tools to collect device health, and a centralized performance console.
Real User Monitoring products
TrueSight falls into this class. Basically, it’s able to discern individual users and page load times.
So a complete web operations team has a variety of monitoring tools at their disposal:
- Synthetic testing to detect problems when there’s no activity and set baselines for controlled, known processes.
- Web analytics to show conversion rates, funnels, search terms, and the like to marketing.
- Sniffers to capture traces for network engineers.
- Flow-based monitors to understand the breakdown of traffic across all protocols and how one application impacts the others.
- Real user monitoring to measure the performance and availability experienced by actual users, diagnose individual incidents, and track the impact of a change.So when Coradiant partners with NetQOS, it’s a way of giving customers the best of both worlds: Deep web analysis, alongside broad multiprotocol monitoring.
