How Zynga Uses Ghetto Testing and Minimum Viable Products

Zynga is a social gaming giant.  Farmville,their largest game, has more than 69 million monthly active users, making it larger than Twitter. Unlike many companies, Zynga also monetizes these users effectively with virtual goods and advertising. Zynga makes a huge amount of real money – while the company is private, 2009 revenues are projected to be around $100 million.

Today I listened to a podcast of a talk from Mark Pincus, CEO/Founder of Zynga and Bing Gordon of KPCB.  In this podcast, Mark shared some of Zynga’s methods of creating phenomenally successful social games.  As with almost all successful start-ups, Mark uses an appropriate version of customer development, growth hacking, and rapid iteration.

Mark talked about how they assess demand for new products and features without taking up engineering hours.  All of his methods are pure customer development/agile start-up.

How Zynga Assesses Market Demand

- Create a 5-word pitch for a new product or feature

- Put it up on a high traffic webpage

- If it gets clicks, collect the emails of interested customers

- Build a ‘ghetto’ version of the feature

- Test everything

- Iterate constantly

Mark Pincus on Ghetto Testing and Minimum Viable Products

(Words are by Mark Pincus, transcribed and slightly paraphrased for clarity and grammar.  Emphases are my own.)

Mark Pincus:  We do something at Zynga that I call “ghetto testing.” I like to take someone who has a gigantic idea, usually a game designer, and they have some gigantic idea that this would just be great…  Maybe they really want a hospital simulation game…

We want to ghetto test it.  Again, we have so many bullets(engineering hours) we can fire, and we’ve got to just treasure and honor our engineers.  If we do our job right, they don’t get burned out.  They have a great life and we have successful products, so that’s what we want.

So I say to the marketing person or the product manager, “Describe it in five words.  It’s built.  If six months from now we built every dream you have, how are you going to market it?  Give me the five words.”

We’ll put that up.  We’ll put up a link for five minutes saying, ” Hey!  Do you ever fantasize about running your own hospital?” (laughter)  And, well, maybe you have!  In this economy, it’s the only growth area.

We’ll put that up for five minutes, and the link will maybe take you to a survey, where you give us your email and we say when this comes out we’ll contact you. If you’re really doing ghetto, it says ’404 not found’.  That’s bad.

So first you try to get the heat around it, you see how much do people like it, then…

(Brief discussion of usage metrics… they’re huge!)

Once we get to the point of actually building a game, or building a new feature, which we love Bing [Gordon's] idea of golden mechanics.  You should take away and steal it from us, the idea of not a game, but a feature that you can deconstruct and see that this interactive feature – a way to do a gift will drive virality or retention or revenues. So we put it in a feature we can build in a week – it’s a ghetto build we AB test it, we flow test it, we put it out to one percent.

We built a data warehouse with a testing platform so we’re running several hundred tests at any given time for every one of our games.  And no single user has more than one test.

So, we love tests.  When we see that it moves our metrics in a considerable way, that’s when we take it to be a full feature roll-out, and then we do the whole 2.0.

So, one example, we just turned on flowers in Farmville.   So now you can plant and grow beautiful flowers.  There’s so many place you could take flowers.  The holiday season is coming up – what happens if we let you level up your flowers and create your own custom bouquets?

But we don’t want to go down those paths until we test them with our users.

What’s amazing, and this is a feature that you will all have available as you enter this third internet era, is that you’re going to run a service, and you’re going to test things every week with your users, something that I never had available to me at previous companies.

It won’t be ‘build it for three months and hope and pray.’




Do you use ‘ghetto testing’ at your start-up?  How does it work? Leave a comment…