(Words are by Mark Pincus, transcribed and slightly paraphrased for clarity and grammar.  Emphases are my own.)

Mark Pincus:  We do something at Zynga that I call “ghetto testing.” I like to take someone who has a gigantic idea, usually a game designer, and they have some gigantic idea that this would just be great…  Maybe they really want a hospital simulation game…

We want to ghetto test it.  Again, we have so many bullets(engineering hours) we can fire, and we’ve got to just treasure and honor our engineers.  If we do our job right, they don’t get burned out.  They have a great life and we have successful products, so that’s what we want.

So I say to the marketing person or the product manager, “Describe it in five words.  It’s built.  If six months from now we built every dream you have, how are you going to market it?  Give me the five words.”

We’ll put that up.  We’ll put up a link for five minutes saying, ” Hey!  Do you ever fantasize about running your own hospital?” (laughter)  And, well, maybe you have!  In this economy, it’s the only growth area.

We’ll put that up for five minutes, and the link will maybe take you to a survey, where you give us your email and we say when this comes out we’ll contact you. If you’re really doing ghetto, it says ’404 not found’.  That’s bad.

So first you try to get the heat around it, you see how much do people like it, then…

(Brief discussion of usage metrics… they’re huge!)

Once we get to the point of actually building a game, or building a new feature, which we love Bing [Gordon's] idea of golden mechanics.  You should take away and steal it from us, the idea of not a game, but a feature that you can deconstruct and see that this interactive feature – a way to do a gift will drive virality or retention or revenues. So we put it in a feature we can build in a week – it’s a ghetto build we AB test it, we flow test it, we put it out to one percent.

We built a data warehouse with a testing platform so we’re running several hundred tests at any given time for every one of our games.  And no single user has more than one test.

So, we love tests.  When we see that it moves our metrics in a considerable way, that’s when we take it to be a full feature roll-out, and then we do the whole 2.0.

So, one example, we just turned on flowers in Farmville.   So now you can plant and grow beautiful flowers.  There’s so many place you could take flowers.  The holiday season is coming up – what happens if we let you level up your flowers and create your own custom bouquets?

But we don’t want to go down those paths until we test them with our users.

What’s amazing, and this is a feature that you will all have available as you enter this third internet era, is that you’re going to run a service, and you’re going to test things every week with your users, something that I never had available to me at previous companies.

It won’t be ‘build it for three months and hope and pray.’