How Zynga Uses Ghetto Testing and Minimum Viable Products

Zynga is a social gaming giant.  Farmville,their largest game, has more than 69 million monthly active users, making it larger than Twitter. Unlike many companies, Zynga also monetizes these users effectively with virtual goods and advertising. Zynga makes a huge amount of real money – while the company is private, 2009 revenues are projected to be around $100 million.

Today I listened to a podcast of a talk from Mark Pincus, CEO/Founder of Zynga and Bing Gordon of KPCB.  In this podcast, Mark shared some of Zynga’s methods of creating phenomenally successful social games.  As with almost all successful start-ups, Mark uses an appropriate version of customer development, growth hacking, and rapid iteration.

Mark talked about how they assess demand for new products and features without taking up engineering hours.  All of his methods are pure customer development/agile start-up.

How Zynga Assesses Market Demand

- Create a 5-word pitch for a new product or feature

- Put it up on a high traffic webpage

- If it gets clicks, collect the emails of interested customers

- Build a ‘ghetto’ version of the feature

- Test everything

- Iterate constantly

Mark Pincus on Ghetto Testing and Minimum Viable Products

(Words are by Mark Pincus, transcribed and slightly paraphrased for clarity and grammar.  Emphases are my own.)

Mark Pincus:  We do something at Zynga that I call “ghetto testing.” I like to take someone who has a gigantic idea, usually a game designer, and they have some gigantic idea that this would just be great…  Maybe they really want a hospital simulation game…

We want to ghetto test it.  Again, we have so many bullets(engineering hours) we can fire, and we’ve got to just treasure and honor our engineers.  If we do our job right, they don’t get burned out.  They have a great life and we have successful products, so that’s what we want.

So I say to the marketing person or the product manager, “Describe it in five words.  It’s built.  If six months from now we built every dream you have, how are you going to market it?  Give me the five words.”

We’ll put that up.  We’ll put up a link for five minutes saying, ” Hey!  Do you ever fantasize about running your own hospital?” (laughter)  And, well, maybe you have!  In this economy, it’s the only growth area.

We’ll put that up for five minutes, and the link will maybe take you to a survey, where you give us your email and we say when this comes out we’ll contact you. If you’re really doing ghetto, it says ’404 not found’.  That’s bad.

So first you try to get the heat around it, you see how much do people like it, then…

(Brief discussion of usage metrics… they’re huge!)

Once we get to the point of actually building a game, or building a new feature, which we love Bing [Gordon's] idea of golden mechanics.  You should take away and steal it from us, the idea of not a game, but a feature that you can deconstruct and see that this interactive feature – a way to do a gift will drive virality or retention or revenues. So we put it in a feature we can build in a week – it’s a ghetto build we AB test it, we flow test it, we put it out to one percent.

We built a data warehouse with a testing platform so we’re running several hundred tests at any given time for every one of our games.  And no single user has more than one test.

So, we love tests.  When we see that it moves our metrics in a considerable way, that’s when we take it to be a full feature roll-out, and then we do the whole 2.0.

So, one example, we just turned on flowers in Farmville.   So now you can plant and grow beautiful flowers.  There’s so many place you could take flowers.  The holiday season is coming up – what happens if we let you level up your flowers and create your own custom bouquets?

But we don’t want to go down those paths until we test them with our users.

What’s amazing, and this is a feature that you will all have available as you enter this third internet era, is that you’re going to run a service, and you’re going to test things every week with your users, something that I never had available to me at previous companies.

It won’t be ‘build it for three months and hope and pray.’




Do you use ‘ghetto testing’ at your start-up?  How does it work? Leave a comment…

Comments

  1. Schwabe says:

    A wise business man once said that, “the only risk in business is you ever have to take is a small, inexpensive test. ” Zynga is obviously embracing this principal to the max.

  2. Gabe says:

    Matt,

    Thanks for sharing!
    Very helpful.

  3. BTW, I think Josh Koppelman gets credit for using the adword campaign to determine if there is demand. I remember him explaining this to Mark years before Zynga.

    respect to Mark still for making it happen.

  4. dennisyu says:

    This is a great interview– “ghetto testing” is another way of saying “fail fast”.

Trackbacks

  1. [...] how Zynga uses minimum viable products « Grattisfaction (tags: statups entrepreneurship) [...]

  2. [...] opinion, so let’s play devil’s advocate.  First, let’s review the Zynga Minimum Viable Product testing method: Mark Pincus:  We do something at Zynga that I call “ghetto testing.” I like to take [...]

  3. [...] January 16 tags: agile start-up, Customer Development, lean, social, web, zynga by mattgratt This Post Has Moved.  It is now at http://grattisfaction.com/2010/01/how-zynga-does-customer-develo… Possibly related posts: (automatically generated)UX* DesignGmail Labs – straight from concept [...]

  4. [...] día perfecto. Él lo considera deshonesto, y lo relaciona con otra técnica de marketing llamada Ghetto testing, porque el resultado que llega al usuario final está “pensado y probado” para que [...]

  5. Quora says:

    Why has FarmVille been so explosively successful, compared to other similar games?…

    http://grattisfaction.com/2010/01/how-zynga-does-customer-development-minimum-viable-product/

  6. [...] Wiseman, Software consultant I've only heard this in the Zynga context: http://grattisfaction.com/2010/0…In essence, Zynga tries to validate the idea with real customers quickly and often by investing as [...]

  7. [...] via How Zynga Uses Ghetto Testing, A/B Testing, Minimum Viable Products, and Lean Startup Methods | Grat…. [...]

  8. [...] Read the article and watch the video at Grattisfaction. [...]

  9. [...] group when it comes to internet/consumer tech startups. In the games business, you need hits. Zygna uses “ghetto testing”to make sure that they invest their resources in projects that are going to pay off, based on user [...]

  10. [...] testing has been the secret of successful companies like Google and Zynga, which ruthlessly test search pages and game items to see what appeals the most to their users. But [...]

Speak Your Mind

*