Companies use focus groups, usability studies, surveys and other methodologies to find out what potential customers want, then try to give it to them.
These methods don't work very well, because the truth is that people don't really know what they want -- at least until they've actually used it in real life.
For example, if Apple had held a focus group and asked users if they wanted an on-screen keyboard or a physical keyboard on their phone, few would have selected the on-screen keyboard.
If Zuckerberg had asked members of the general public if they wanted a new Web site where they could read postings about the minutiae of their family and friends' daily lives, nobody would have raised their hand.
If Twitter founders Biz Stone, Evan Williams and Jack Dorsey asked people in a phone survey if they wanted a new service that's like blogging, but where you're not allowed to exceed 140 characters or post pictures directly, everyone would have said no, they wouldn't want that.
People don't know what they want until they've tried it.
A better way to understand what people really want is a methodology called ethnography. Big companies hire ethnographers to essentially "become" their customers or target audience.
They live with them, work with them and pretend to be them. They learn the language and habits to the point where they have the same thoughts as customers, then they can speak authoritatively about what customers really want.
But good ethnography takes a long time -- months or even years. Silicon Valley just doesn't have that kind of time.
In the case of my examples above, all these visionaries had the same desires as a great many people. Jobs loved music in just the same way as millions or billions of people do. Zuckerberg craved the same kinds of social interactions as millions of other people. Page and Brin wanted better search results, and so did everyone else.
However, it's also very common for designers to design things that only designers can appreciate and for engineers to engineer things only engineers can love.
That's why most consumer electronics fail in the market -- they're either too ugly and complex (because engineers tend to de-emphasize beauty and simplicity) or too limited in function (because designers often prize aesthetics over power). Or they don't work well because the suits think their customers are stupid and won't know the difference.
Another reason products fail is that once companies succeed with the product of their desire, they expand into businesses they're not really passionate about. I somehow doubt Steve Jobs loves TV as much as he does music. Microsoft culture is really built around the PC, not mobile consumer electronics. Googlers are notorious brainy super geeks who have somehow failed thus far in teaching the world a better way to socialize.
The ideal product creator must be simultaneously ordinary and extraordinary. They must be ordinary in their desires, but extraordinary in their ability to deconstruct and understand those desires and maintain product focus while all the designers and engineers try to erode the vision.
But this approach to consumer technology innovation can only be taken so far. At some point, companies need some way to innovate for people who are entirely unlike the people creating the products.
Ultimately, for mature companies, the winning formula evolves into one in which users create their own products -- or at least a formula where you throw enormous numbers of products at users and let the public pick.
This appears to be the Android model, where Google competes with Apple's one iPhone with dozens of partner companies producing hundreds of devices, and with the Apple app store model where development is controlled but pretty easy to do, and a quarter of a million apps vie for supremacy among users.
Still, for startups and product designers, the best approach is to ignore your customers and build the product that you, personally, want to use.
Don't believe me? Just ask Apple customers.