Artience: Art and Science in the World of Design

April 20, 2015
Foundation Capital

A SXSW conversation with Foundation Capital’s Steve Vassallo, Soleio Cuervo – Designer at Facebook & Dropbox, Scott Klemmer – Associate Professor at University of California, San Diego, and Ann-Caryn Cleveland – Co-Founder / Head of UX & Design at BrightSky Labs.

At SXSW Interactive last month, some of the brightest minds in product shared what they’ve learned on the path to harmony in data and design. Moderated by Foundation Capital General Partner, and former product and engineering lead at IDEO, Steve Vassallo — the panel treated a standing-room only audience to a wide-ranging discussion on topics from the breakthrough of the “Like”button at Facebook, to principles of innovation exemplified in Picasso’s work, and advice for creatives on how to drive data-driven design decisions at their organizations. What follows is an excerpt from the day’s conversation.

Steve Vassallo: A couple of years ago, the head of Product at Netflix told me that of the 50 million subs that they had at the time, 33 million of them had a unique user experience—with a truly different interface for every single customer, depending upon their usage modes. I think that statistic is a nice way to kick off the discussion here because the notion is that the data they’re getting from their users is actually telling them how to design their next-generation products.

And so I’d love for the whole group to just give us your perspective on what the biggest differences are between how people years ago would figure out what they were going to build versus what the world of creative development looks like today.

Scott Klemmer: Well, the British Design Council has this wonderful idea they call a “double diamond” —that when you’re designing, you start out by exploring what your users might need, settle on one item you think is most impactful, and then you go back and iterate on alternative solutions before finally narrowing in on one solution. And your Netflix example demonstrates that now you can roll out and iterate multiple different solutions. And I think that that’s a big game-changer.

Double Diamond Design Thinking. Illustration by Scott Klemmer.

Ann-Caryn Cleveland: People have the misconception that design is this magic ah-ha moment. That double diamond model is extraordinary because if you think about that first diamond being the “need finance” stage where you gather data to figure out what’s going on in the world, you only move to the next stage once you start to pull in information. So thinking about design as a structure rather than a lightning bolt is really key for us as creatives, because the lightning bolt is a fallacy and it’s really difficult to incorporate into a cohesive approach.

Steve Vassallo: And in software today, especially social software, we have that feedback mechanism where you can build around hypotheses, do tests across the newest permutations, and use that feedback in the creative process to help curate experiences that are going to be informed by how people actually use products.

The most innovative tech firms today are developing product roadmaps that are guided by learned behaviors, and they’re guided by data. You could say it’s building on that double diamond framework, where you’re constantly iterating and using new releases as a means of guiding what you’re going to work on next. How did that testing work at Facebook?

Soleio Cuervo: I haven’t worked for the Facebook in a couple years but, when I left in 2011, we had this internal tool called “Gatekeeper” that developers could use to roll out changes or tests. But I think over time, what we’re going to start to see is that more and more firms are going to adopt a methodology of using short-term release cycles to gather data around very specific changes and use that as a means of guiding what they work on next and finding out where the opportunities for impact lie.

Steve Vassallo: But what about the times when you’re divergent, when you’re inventing things that don’t exist?

Soleio Cuervo: Well, my favorite example of how data showed how systemically wrong our product intuition was—was with the Like button. There was this prevailing intuition internally that it was going to cannibalize comments, and for whatever reason, that idea was blasphemous. We didn’t want people’s interactions on Facebook to be reduced to just pushing buttons at one another. When we rolled it out to a few countries, over the course of a month, what we found was not only did the Like button not cannibalize comments; it actually increased the likelihood of a person leaving a comment. It acted as this social lubricant where, if people saw that there was an existing uptake on a piece of content in our news feed or on a particular photo, they almost had a built-in audience.

And then on further iterations we started to show people’s names, so instead of two people liking this, a user would see “my brother and my sister like this.” That’s a very different, very personalized picture you just painted in a person’s head. If it wasn’t for that approach to using the data, using experimentation to guide the product’s development, the Like button could easily have gone out the door. So it is a good example, in my mind, of how you use data to show that a company’s product intuition can be just dead wrong.

Scott Klemmer: I love that story, because even if you’re optimizing based on data, sometimes creatives will say something like, “Yeah, but the really big breakthroughs come out of your head, man.” And that’s just not true. One of my favorite stories is about Picasso. In the summer of 1907, he was working on this new painting called Les Demoiselles d’Avignon, and it turned out to be a seminal painting in the cubist revolution. He made 400-some-odd sketches, and the interesting thing is at that time, the ethnographic Museum of Paris had these wonderful Fang sculptures that had just arrived from Africa. In those sculptures were all kinds of ideas about how to represent the face and how to convey expression—those ideas were well known in Africa but were completely unknown in French painting. They were a huge influence on him, and his insight was being able to take ideas from one context and leap—move it to a completely different context. For me, that’s a reminder that creativity is not about trying to completely get away from everything, but about seeing connections that are further afield than others can see.

Picasso’s Les Demoiselles d’Avignon, 1907

The story that Soleio was telling is, I think, a perfect example of the new world that we’re in —where you have the opportunity to iterate online, see what works, what doesn’t, and draw connections based on what you learn.

Steve Vassallo: Do either of you teach people to keep track of when they were right versus when they were wrong? Is it something that some organizations can do successfully, where they build intuition around this information?

Soleio Cuervo: We never do that.

Ann-Caryn Cleveland: I’d argue that it’s really dangerous to do that. We have inherent biases about how we view data, even. “I have a vested interest in this, so I see it in this way. And you see it in that way.”

Scott Klemmer: I actually do keep track of what I think will happen ahead of time and then what happens afterwards because it’s like an hypothesis and result, and hopefully I can build up a set of intuitions that are better. And when we align what happened with what we thought would happen, we learn a lot more.

I think that the critical thing with the insight that Ann-Caryn had is that you don’t use it as a scorekeeping in the organization, to be like, “Well, Scott — not so good whereas Soleio, he’s the guy to pay attention to.” But I do think it’s valuable to close the loop.

Steve Vassallo: What I take away from all of your comments is the notion that the prototype is what really rules as opposed to your intuition. But what about the product experience as a whole? When you think about your favorite products today, it’s my belief that the teams behind them are focused on the entire experience around the product. We used to applaud Apple for a great out-of-box experience, but that is table stakes today. So talk through that a little bit: what you’re seeing, companies that are doing it well, examples you’d point us to where it required another way of viewing the challenge, where designers can bring something unique to the table.

Notes Image by: @erintilley

Soleio Cuervo: We built a team at Dropbox called “Black Ops” which is essentially an internal design / communications team. In a business where you rely so heavily on, say, a sales organization to sell a product, the sales representatives need to understand the value of design, communication, and teaching as a means of accomplishing their tasks.

It was something we kind of stumbled into, but we started to recognize really early on that it was an active investment that we needed to make separate from the product, separate from marketing. The focus around teaching design to everyone helped people start to appreciate the value of being a really strong communicator, a really strong educator, and an advocate for Dropbox and the user experience that we’re trying to develop. Each business will have its own design organization, but I think at a high level, you can enhance the whole product by being a really good educator to every single person in the company that relies on your creative to help them do their job.

Scott Klemmer: I agree completely. I think a lot of the design that we most admire comes about not just when you have a couple of good, talented designers, but when it becomes pervasive in the organization. And so what Soleio and others are doing in teaching design for the other 99 percent of the company, I think that’s an important thing to focus on.

Ann-Caryn Cleveland: As designers, we often feel like we’re the lone person trying to work in this space and trying to convince people of what we’re doing. Aza Raskin has this great quote, where he says: “Create the resort on the island so the team wants to make the boat to get there.” My CEO/CTO Co-Founder also says, “Designers are the compass.” It’s really interesting to think about how design teams inform the whole organization. Thinking about ways that you can help the company by better informing people about how design works is a fascinating idea.

But the information needs to flow both ways. We’re really new to data-informed design, and in my experience, I’ve sometimes seen one person at the company with the keys to the kingdom of data. And they hoard it, because it’s sort of a power thing. But if designers don’t have access to the data, then they cannot make a good product, and then they can’t make that organization awesome. In today’s environment, having those keys, that data, is a super-critical need for designers. And make sure that the data is current—because if you have data that is a month old, it may be dragging you in a different direction than where you need to go.

Continue the conversation with @FoundationCap, @vassallo, @soleio,@DesignAtLarge, and @anncaryn. We want to hear from you. How is data informing your companies design decisions? Tell us more about the challenges and successes you’ve had with your users and products.