UI13 and "Don't Build a Department" for Design

UIE's UI13 conference was fun, even for a volunteer up at 5am every day. A few of the talks really got the adrenaline flowing, including Peter Merholz's "16 (Mostly) Difficult Steps to Becoming a Customer-Experience-Based Organization", and Scott Berkun's talk on "Why Designers Fail".

Merholz's talk outlined a number of points describing successful user experience orgs, including some wise ones I've heard in other similar talks:

• Execute a quick win [to show your value to the company].
• Have an executive sponsor.
• Move up the product planning food chain [i.e., be involved earlier, not just down stream].
• Have an experience strategy [for the company/products/team].
• Think systems, not artifacts [a point also made in a recent talk by Don Norman on operations and services, over here].

His final bullet was the admittedly controversial: "Do not become a department." I thought I heard similar sentiments from Jared Spool too, and I have heard this in different flavors from people who cite Amazon's success in building a business based on A/B volume testing of page designs by marketers without usability or interface designers on staff ("let the customers just tell us which one works"), ebay's early success without an empowered design org, etc. The gist of this argument seems to be: Executive mandate for good user experience trumps individuals in the trenches, and good execution requires everyone to play, not just designers. So, have a design-oriented company, not a bunch of designers trying to change a company.

While I agree we want holistic design-oriented companies for better customer experience, I think designers play an important role, if good design matters. Anything that requires skill, training, and practice to do well should be a job in itself, and therefore be a hired position, not a sideline role for someone who is paid to do something else.

Additionally, if you're talking about companies that succeeded despite not having a staff of interaction designers, you might be talking about companies that (1) might have done it faster or more cheaply WITH a staff of designers, or (2) had talented people who were doing design without that job title - have they checked into how they succeeded?, or (3) companies that won as first movers, but could lose in a crowded space with better design and real usability from their competitors. Yes, design isn't the whole story in business success, but it's often important, depending on the competition. And to me it's a moral requirement for a customer-oriented business.

Scott Berkun's pre-conference survey on reasons for designer failure found that the top 2 reasons were agreed to be "People in non-design roles [are] making design decisions," and the related and subsumed "Managers [are] making design decisions without design training." I believe that if you haven't got a strong design department with a recognized skillset and/or haven't empowered your designers in the org, you'll get a committee effect, and design outputs will be worse as a result. (See also Scott's excellent article "The List of Reasons Why Ease of Use Doesn't Happen on Engineering Projects.") Berkun's audience of designers, managers, project managers, and developers also seems to believe this, contrary to Merholz's last point. (Caveat: It's possible that Merholz's position was "hire designers but don't have them grouped in a department, have them spread throughout the company." But a department makes it possible to argue for headcount, achieve hiring and management consistency, enable organizational empowerment, and accountability; NOT having a department makes these things harder and really kind of a crapshoot long-term.)

Another broad theme of many of the UI13 talks was the importance of the strategic role of design in defining the right projects and requirements before design processes start in earnest. While it was valuable to see how many consultants and design agencies do this--often playing a "facilitation" role with their clients' ideas--the reality of most software companies is that product managers (or their equivalent) are making these decisions, and designers live further downstream. The ideal is otherwise, but that's how it often is.

Pixar on Successful Creative Teams

I've seen this on a number of sites now, but it's rich enough to keep passing on. Requiring payment from HBR, the article is How Pixar Fosters Collective Creativity, by Ed Catmull. Some of his points are business management truisms or even cliches, but as with most management-related things, it's not the concept that's tough, it's execution that's tough. Especially in a creative environment.

On People. Most of his points here are about handling diversity and collecting lots of input from lots of sources. Less dictatorial hierarchy, more feedback and empowerment of teams to decide how to handle the feedback. Some good quotes:

• "If you want to be original, you have to accept the uncertainty, even when it’s uncomfortable, and have the capability to recover when your organization takes a big risk and fails. What’s the key to being able to recover? Talented people!"
• Creativity isn't about finding one big good idea. "However, in film making and many other kinds of complex product development, creativity involves a large number of people from different disciplines working effectively together to solve a great many problems."
• And yet, talent isn't evenly distributed, he acknowledges. But this does not mean anyone tells anyone else what to do - a creative team gets input and makes its own decisions about what to do with it. A "brain trust" of the truly excellent people with track records can be called on for input when teams need help, but they don't dictate anything. Ironically, this frees everyone up to talk and listen more effectively.
• Having talent on staff isn't enough. "What’s equally tough, of course, is getting talented people to work effectively with one another.That takes trust and respect, which we as managers can’t mandate; they must be earned over time." If people trust each other, they can be less polite in meetings, apparently. Ideas are under discussion, not personal status and power.
• "An important lesson about the primacy of people over ideas: If you give a good idea to a mediocre team, they will screw it up; if you give a mediocre idea to a great team, they will either fix it or throw it away and come up with something that works." I note, they can only do the latter if they are given the freedom and authority to do something radical.
• Pixar's "small incubation teams" that consist of a director, a writer, an artist, and storyboard folks. Whereas in my experience most software incubation project teams are weak on the creative staffing and very heavy on the implementation side, not a good balance of skills for the stages of creation.
• It's critical for an incubation team to function well internally: "During this incubation stage, you can’t judge teams by the material they’re producing because it’s so rough—there are many problems and open questions. But you can assess whether the teams’ social dynamics are healthy and whether the teams are solving problems and making progress. Both the senior management and the development department are responsible for seeing to it that the teams function well." I note: presumably there are non-subjective, non-gossipy ways to evaluate social dynamics. I've seen this rhetoric applied to very bad ends at one company.
• Catmull says, "Treating one another as peers is just as important as getting people within disciplines to do so. But it’s much harder. Barriers include the natural class structures that arise in organizations: There always seems to be one function that considers itself and is perceived by others to be the one the organization values the most." Overcoming that is a huge management and process challenge... Catmull seems to be saying that time together helps, but I think the deliberate creation of well-rounded incubation teams is a big aid in changing these biases. None of this "we'll add a user interface designer later" stuff, like you hear from the software company incubation teams!
• Newcomers to an organization can be threatening, because of the "not invented here" syndrome that they may cause with their new ideas. But constant change, not taking success for granted, and acknowledgment of mistakes made can make newcomers less threatening to current employees, he says.

On Processes. So they've got a good staff who encompass both technical and creative backgrounds, now how do they keep it all working and on track?

• Dailies are watched, by lots of people (the animation industry version of footage of the day). Sharing unfinished work and inviting comment helps creatives get over the fear of showing the incomplete, and that in turn means work isn't wasted if it's on the wrong track. I note, a healthy culture of regular software design critique does not exist in most software companies (barriers to this are a political subject for another time). Agile development processes seem to be better off in this regard than waterfall-like models of development: producing and showing in-progress work is critical in that methodology.
• Input on work-in-progress is collected widely, because the work needs to be great before release to the real world. TiVo executed on this principle when I worked there, too; employees all used the beta software at home and we had to like it, too.
• Post-mortems are done regularly. Rather than just "what went well and what didn't go well," his suggestions include having groups list the top 5 things they'd do again, and the top 5 they wouldn't do again. Now, in a creative environment, people often assume that you can't evaluate the creative process. But Pixar uses data to ground the post-mortems (making me wonder how they track it, who does the analysis, etc). "Most of our processes involve activities and deliverables that can be quantified. We keep track of the rates at which things happen, how often something has to be reworked, whether a piece of work was completely finished or not when it was sent to another department, and so on. Data can show things in a neutral way, which can stimulate discussion and challenge assumptions arising from personal impressions." The fact of being "neutral" prior to interpretation is important, from my perspective. Using data in a post-mortem shouldn't lead to finger-pointing so much as conversation about root causes for data peaks and valleys.
• Management challenge for their corporate processes: "Clear values, constant communication [across and around hierarchy], routine postmortems, and the regular injection of outsiders who will challenge the status quo aren’t enough. Strong leadership is also essential—to make sure people don’t pay lip service to the values, tune out the communications, game the processes, and automatically discount newcomers’ observations and suggestions." And I say: Easier to say than to execute. Leadership is so rarely evaluated well, at any company.
• Catmull says they keep up with academic research. Being cutting edge means staying on the bleeding edge, and being able to attract people who want to work on that edge, too. Why do so many companies sneer at research and research conferences?

It's a good article, and I think worth the $6 cost. It does leave a few questions I had unanswered, like how they handled the massive overtime and repetitive stress injuries he describes during one "failure recovery" period.

As a final point, something I've said here before: Post mortems may be unpleasant, but understanding how a team was successful is just as important, or more so, than understanding how it made mistakes. I don't think most companies use the positive particularly well in setting up their downstream teams. I think Pixar probably does, to have such a string of successes.

My Twitter on the VP Debate

I couldn't watch it all, and made a mistake in not having my twitter in front of me while I did. But here's, in a glance, a big reason I love twitter. There's a slim chance that 2 of these people know each other, but I don't think the others do, although I bet they'd like each other if they met. (Assuming they won't mind: tingilinde whom I know from Bell Labs/AT&T, Nancy Baym from grad school mentorship days, Jared Spool from being, well, Jared, and Greg Raiz, a colleague from Boston.)