Designers are Necessary

I seriously had a post in drafts about this very topic when Clint Hocking posts a link to this article by Don Norman of the Design of Everyday Things fame. Always upstaged by better communicators than myself, I point you to go read it. It is everything I wanted to say in a better package. The analogy of local maxima is particularly accurate.

The reason I had a post brewing is that I had a pair of recent interviews that both included a discussion of “Quantitative Design” versus “Intuitive Design”. These are the words that both interviewers used. I do not endorse the terms. The reason I do not endorse “Intuitive Design” is that it seems almost pejorative. I’ve seen truly intuitive designers who made decisions by the seats of their pants where whatever came to mind was right. How can you question them? Their reasons for making the decision is simply that the decision “feels right”. The label of “intuitive design” conjures images of diva-like egos dictating on whim. Scientific minds reject gut feeling and glom quantitative design where you create an A/B test and count up the results. Numbers are concrete. They must provide truth, right?

Yet show me a breakthrough that has come about via A/B testing. A/B testing works in a controlled environment where there is no possibility of a C, D or E and where both A and B provide a similar level of familiarity. So what if the test itself is poorly designed? How do we determine that? By testing the test? Then testing the testing test? Turtles all the way down. At some point you need human creativity to step in and make judgements. That’s obvious, of course, but it is worth noting to those who think quantitative testing reveals the word of God.

Testing in many forms is absolutely crucial. It is the results of playtests that need to inform the decisions of forward-thinking designers. My objection from the start is in the “versus”. It suggests that these two camps in extremis are the only pure methods, that only Farmville and Crazy Indie Game can exist. There has never been an original game designed solely by quantitative design, nor will there ever be. All it can do is take two or more items that already have been designed and judge the merits in isolation. While there certainly have been games designed wholly by intuition, I’ve never had experience where one could not be improved by a little scientific playtesting.

Many designers feel threatened by the recent “social” game trend towards phasing out the opinions of trained designers replacing them with “designers” who simply run A/B tests and interpret results. I am not scared. Studios that value giving customers something new will, by necessity, need trained creative designers.

2 thoughts on “Designers are Necessary”

  1. Quantitative tests should be used to test two (or more–you absolutely can have A/B/C/D/n tests. they are just more complicated/expensive) alternative design ideas in a controlled environment. For example, you might want to test if users of a website will click navigation links more if the links are placed on the side of the page vs the top. You could test more options if you have them, like navigation on top vs side vs bottom. You have a measure (rate of clicking navigation), variables (location of the navigation), and a clear path forward for each outcome (place the navigation on top or side). It is imperative that the other portions of the screen remain constant so you have the best chance of reducing error and measuring an effect.

    You can’t just put two designs side by side and ask people which they like better. Well, you could, but you would get garbage and not know anything of actual use (What did they like? OK, they like it, but does that mean they can use it better? etc.)A good test will layout what is going to be tested, how you will measure it, and what your path forward would be for each outcome.

    I think that by Quantitative design,they are talking about the designers ability to use quantitative results in their design process: know what is something that should be tested, know how to make sure that the test will have the best chance at measuring what you want to measure, know how to incorporate the test results back into your design, etc. If they are really asking you to design solely by quantitative methods then they have no idea what that means.

    In life there are few absolutes and in design, there are fewer still. Just like trying to design solely by quantitative means will often result in bad design, so will designing completely intuitively. There has to be a mix between the two. You need to come up with something to do your initial test on (intuitive design) and then design good tests of particular features with a plan to incorporate the results back into your designs (quantitative design).

    In a perfect world, you will be building on previous designs so that you can come into the new project with an idea of what works (and does not need to be tested) and what is new for this design (and is a candidate for testing). Testing costs lots of time and money and should not be overused.

    Don’t fear testing. Read some intro to experimental statistics books/websites and know that testing is just a tool in your kit for making a good design. I think it is easier for a designer to learn statistics than it is for a statistician to learn design (speaking from experience). Companies these days are just more interested in making sure their designers are aware and open to the ability of testing to better their designs. Any company that is serious about quantitative design is going to have a good usability researcher on hand to help with the hard stuff.

  2. I think we are saying the same thing, Chris. When I was talking about no possibility of C, D or E, I was saying no possibility of n+1, n+2 etc if you are testing n options. It was an overly obtuse way of saying don’t ignore possibly confounding variables or elements outside the scope of understanding.

    I do disagree with the last part though. Companies today are enamored with the Zynga method without really understanding its limitations or sphere of applicability. In a previous employment life I was asked to “put metrics in” because an exec went to a conference where Zynga presented and was sold on the idea of metrics. When I pressed further to find out what questions they wanted answered with those metrics, I couldn’t get an answer.

    Otherwise, you are preaching to the choir. I think quantitative design is a necessary facet of design, but not a panacea for overly egoistic designers. If you were using quantitative design to determine a child’s dietary needs, something tells me you would get overwhelming support for candy and ice cream.

Leave a Reply

Your email address will not be published. Required fields are marked *

Human? * Time limit is exhausted. Please reload the CAPTCHA.