I seriously had a post in drafts about this very topic when Clint Hocking posts a link to this article by Don Norman of the Design of Everyday Things fame. Always upstaged by better communicators than myself, I point you to go read it. It is everything I wanted to say in a better package. The analogy of local maxima is particularly accurate.
The reason I had a post brewing is that I had a pair of recent interviews that both included a discussion of “Quantitative Design” versus “Intuitive Design”. These are the words that both interviewers used. I do not endorse the terms. The reason I do not endorse “Intuitive Design” is that it seems almost pejorative. I’ve seen truly intuitive designers who made decisions by the seats of their pants where whatever came to mind was right. How can you question them? Their reasons for making the decision is simply that the decision “feels right”. The label of “intuitive design” conjures images of diva-like egos dictating on whim. Scientific minds reject gut feeling and glom quantitative design where you create an A/B test and count up the results. Numbers are concrete. They must provide truth, right?
Yet show me a breakthrough that has come about via A/B testing. A/B testing works in a controlled environment where there is no possibility of a C, D or E and where both A and B provide a similar level of familiarity. So what if the test itself is poorly designed? How do we determine that? By testing the test? Then testing the testing test? Turtles all the way down. At some point you need human creativity to step in and make judgements. That’s obvious, of course, but it is worth noting to those who think quantitative testing reveals the word of God.
Testing in many forms is absolutely crucial. It is the results of playtests that need to inform the decisions of forward-thinking designers. My objection from the start is in the “versus”. It suggests that these two camps in extremis are the only pure methods, that only Farmville and Crazy Indie Game can exist. There has never been an original game designed solely by quantitative design, nor will there ever be. All it can do is take two or more items that already have been designed and judge the merits in isolation. While there certainly have been games designed wholly by intuition, I’ve never had experience where one could not be improved by a little scientific playtesting.
Many designers feel threatened by the recent “social” game trend towards phasing out the opinions of trained designers replacing them with “designers” who simply run A/B tests and interpret results. I am not scared. Studios that value giving customers something new will, by necessity, need trained creative designers.