I’ve been thinking a lot about a company that we met with recently. Their web site is one of many media that they use to promote their products. Each medium lends its particular talents to the promotion of the products. They drive their traffic across these multiple channels as a means of increasing sales.

Given the immediacy of the web and their other channels, their promotion campaigns are incredibly brief – typically on the order of a few hours. Moreover, they often run more than one campaign per day. They have engaged their customer base extremely well and are using the immediacy of their channels to drive revenue sky-high.

However, each campaign requires new content for their web site. Moreover, that content must be removed when the campaign is over. Think about that for a minute. They are altering the content of their site several times a day, every day. It’s almost an hourly release cycle. As an IT department, what would that rate of change do to you?
Oh look, here comes the tide …

 

And this trend is not just limited to web content. The release cycle is shrinking on all fronts – infrastructure changes, application updates, etc. As businesses try to tap into the immediacy of the internet, they are going to expect their IT department to be equally nimble, moving swiftly to add servers for increased capacity, deploy new content to support campaigns or apply software upgrades to resolve issues. Where this type of activity used be allotted several months to ensure quality before deployment, we are now seeing that reduced down to weeks or even – in the case of this one company – down to a matter of hours.

In the IT world, change = instability and instability leads to support problems and customer calls. As I’ve mentioned before (see Do you know who your users are?), IT departments will typically only catch 3% of the problems before their customers are affected by them. That’s not a great track record. If the rate of change continues to increase, so will the number of problems until the IT department is in danger of drowning completely. Many IT departments we have met with are already in serious trouble. They can’t afford to have things get any worse. What are they expected to do?
 

Perpetual beta

 

Welcome to the world of the “perpetual beta” where you can leave your obligations at the door.

The need to accommodate an increased rate of change was the main reason that the term “perpetual beta” was coined. It effectively announced to the world that “you should expect this product to have problems, so don’t get too upset or hassle us about it”. Techies love this term because it absolves us of our obligation to provide good service and reliable products, allowing us to focus on being “innovative” instead – as though the two concepts were incompatible. In my opinion, the need for such a term is actually a deplorable testament to our inability to find and fix problems when it comes to web applications.

The term is already being applied to public web sites (with Google leading the charge). However, we are increasingly at risk of applying this term to corporate web sites. Imagine if the site that handled your on-line banking were a “perpetual beta” site. “Whoops, we just lost $3000 of your hard-earned cash. We’re so sorry, but this is a beta. We’ll get right on that, assuming it does not stop us from rolling out our next great feature.”

Not exactly awe-inspiring, is it?
 

And it’s slow, too …

 

The other main issue that is not discussed very often is performance. Even if you are diligent about finding and resolving crashes and obvious flaws, constant change can also affect performance. And performance is the hardest aspect to monitor effectively. It is usually the last thing that is tested by a QA department. In an accelerated release cycle, it is typically the first activity to be reduced or entirely cut from a schedule.

We have an existing customer whose IT department does not have authority over the deployment of new content, although they are expected to support it once it is rolled out (sadly, this is a fairly typical arrangement). On one particular day, the IT department started getting an enormous number of complaints about the performance of the site. After consulting TrueSight, they noticed that the content had changed. So they called the marketing department. It turns out that the marketing department had rolled out a “new look” for the site that included a lot of high-resolution graphics. Suddenly, delivering the content of the site to the user was like shoving an elephant through a garden hose. The IT department, of course, had never been informed of this since “nothing important was changed, so it shouldn’t make a difference”. Well, that’s comforting. At least we can warm ourselves in that glow while the phones are ringing off the hook.

If you have signed Service Level Agreements with your customers, I’m looking at you. Rather pointedly. You can also include yourself if poor performance tends to cause an increase in support calls (although the SLA victims are worse off, believe me).

All of this to say that any change – any at all – can cause significant problems and cost you real money.
 

Slaying the beta beast

For obvious reasons, corporations cannot – and should not – condone “perpetual beta” status on their web presence. If you cannot declare “perpetual beta” status, what can you do in the face of such a rapid rate of change? How do you take your life back?

You cannot control the QA cycle, so you must work with the hand you are dealt. Given the trend in release cycles, it is increasingly likely – in spite of the best intentions – that you will be expected to support releases that have less and less quality. The onus will be increasingly on you to sniff out and address poor quality before anyone is affected.

To accomplish this, you need the following abilities:

• - To visualize what is happening on your site as it happens, in real time
• - To monitor changes in error rates, traffic levels and performance
• - To provide proof of culpability to other teams and departments

 

With these basic tools, you can take a less-than-tasty rollout, find its primary flaws and schedule fixes before anyone picks up the phone.

Driving quality

Several of our customers have started to use TrueSight’s abilities in these areas to drive greater quality into the development process. One customer used TrueSight’s alerting capabilities to send an email to the entire development team whenever a customer clicked on a broken link. You can imagine that backlash that ensued at first, as developers demanded that the spamming cease and desist. The IT department stuck to its guns with the simple point that the developers could stop the email themselves by fixing the broken links. And guess what? Miracle of miracles, the links were fixed! More importantly, after the initial storm blew over, the developers were intrigued with the possibility of having more direct feedback from production. They now work more closely with that IT department towards producing better quality.

Simple exposure of flaws – with hard evidence to support those assertions – can be a powerful tool in changing the dynamics of departmental relationships and molding corporate policy when it comes to quality.

And we desperately need that type of change in this industry.