Digital Design Blog

Our Brave New Beta Future

What’s the best way to send a room full of designers running toward the exits? Simply mention that “data” or “analytics” should be integral to the creative process and watch what happens. Until recently that is.

As of late, the notion of a data-driven approach to design was considered
heresy in many parts of the design world. Style was the lingua franca of these groups (and still is, in many ways). But that is slowly starting to change in the digital realm as more and more designers are starting to embrace the power of data and are delivering increasingly impressive and effective experiences as a result.

The Art & Science of Design
Much has been said and written about how designers are faced with a new set of interaction paradigms in a Web 2.0 world (emerging navigation, mashups, tag clouds, multi-channel systems, AJAX UIs, etc.). But what about data? How are we as designers and marketers preparing to mine the data-rich platforms that underpin the experiences we create?

How does data transform the art of design into a science?

Brave New Betas
We can start with the ubiquity of the beta release for starters. Web 2.0 has ushered in many different types of innovation, but none, as significant as the beta release. This one little word signifies a whole new approach to design; one that advocates pushing a product into the hands of users as quickly as possible to learn as much as possible. It acknowledges that digital applications are living things that evolve over time and, for them to flourish, they need constant user interaction and feedback.

Google and Yahoo! have led the way here. Many of today’s top digital products and services started out in a beta release. Flickr (http://www.flickr.com), Netvibes (http://www.netvibes.com), Google’s Gmail (http://gmail.google.com) and Maps (http://maps.google.com) are among the top ten, according to the Museum of Modern Betas (http://momb.socio-kybernetics.net) a site that quasi-scientifically tracks the ongoing popularity of Web 2.0 betas.

As we have learned at Avenue A | Razorfish, the best consumer feedback is not gathered in a lab, it’s gathered from users’ real-time interaction with the application or Web site. Want to listen to your consumer? Simply sift through the data for clues and then react.

Eye-Popping Results
Beta ultimately represents a data-driven approach to design and forces us, as designers, to become both artist and analyst. It also changes the role of consumers in the design process. We can thank the likes of Google, Yahoo! and a host of Web 2.0 players for this. No longer do consumers expect to sit back passively receiving the latest digital product with baited breath. Now we are trained to believe that digital products are ongoing “works-in-progress” whose success depends on our interactions with a site or system and the feedback we explicitly supply (comments) and tacitly (clickstreams) provide.

Most digitally focused companies now actively maintain sites inviting users to test beta and alpha releases. Google did this famously with their “labs” (http://labs.google.com) and the other giants have quickly followed: Adobe previews its creations at labs.adobe.com, Microsoft gives users a taste of new Web-based services through Windows Live Betas (http://get.live.com/betas/home) and AOL maintains AOL Beta Central (http://beta.aol.com).

The results for consumers are superior user experiences and, for corporations, digital properties that provide eye-popping performance around key indicators like conversion rates, average order size, time spent on site and over all user engagement. It is not uncommon for these companies to see conversion rates soar by 20% or more over a given time frame. Ditto for revenue where multi-million-dollar increases are increasingly frequent.

The New Science of Digital Design
To successfully design and develop digital products in this new landscape requires a new approach. Here are three key ways to use data to alter your products and organizations.

1. Mine Existing Data For Behavioral Insight: Think of your existing digital property as a living lab where hundreds of thousands or millions of users have left their fingerprints. The first step in any good process is to explore existing data repositories to discover what works and what doesn’t.

• Site-Side Analytics – Mine clickstreams to understand where users go and where they don’t. Look for clusters of activity and orphans. Determine if site issues are design or product-driven.

• Search Logs – A virtual treasure trove of user intent, analyze these reports to find out what users are looking for and what they can’t find.

• Support/Call Center – Call center or support logs yield clues to user frustrations and talking to support reps give insights into true user needs.

• Financial Reports – Ensure that financial reports are integrated with Web performance. There should be strong correlations.

• 3rd Party Reports – ComScore, Nielsen, Alexa and Compete data often surface surprising insights about your properties performance – and your competitors.

• Google Trend Data – Provides an interesting, graphical way to explore the zeitgeist and where interest lies.

• Ad-tracking Technologies – Atlas, DoubleClick and others provide customers innovative ways to track consumer behavior above and beyond your property.

2. Utilize Advanced Analytics: Digital properties are all about “flows” or the paths that consumers take through a site or application on their way to completing a task. Site-side analytic tools like SiteCatalyst, Hitbox and WebSideStory are great for serving up general reports on success ratios but yield few clues as to why users exit a flow. That’s because users exit on a page, not going from page-to-page. To ensure that you are designing for maximum conversion, Avenue A | Razorfish employs a proprietary tool called Advanced Optimization that allows us to track user behavior at the page level. Now we can see exactly where users click on a page, the amount of time spent filling out form fields, how far users scroll, how much time they spend watching a video, interacting with a flash module and more. All of which allows us to understand why a user opts out of an experienceand how to design a better solution.

3. Design for Ongoing Optimization: After launching a digital property, it’s key to understand what performs well and what doesn’t. Usability testing only gets you so far—it’s much better to monitor performance in real-time. And, better yet, test page options with real users. As such, we are huge advocates of multi-variant testing. Think of this as A/B testing on steroids, where every element of the page—from text to graphics to layout—can be tested in various combinations. All of this is done dynamically and yields typically powerful and often surprising results (at least to the design teams and marketers).In the end, a data-driven approach to UI design will force the industry to redefine the notion of “participatory design,” leaping out of the lab and into the world. It’s an idea whose time has come—and it can only be done digitally.

by Garrick Schmitt and Marisa Gallagher