In researching this question for more than eight months I turned my attention to more than five hundred papers, reports and articles on the topic to distil and formulate a comprehensive answer. Not only was this research a consequence of my own curiosity but a concerted attempt on my part to help the C-level executives with whom I am partnering currently gain clarity on the topic.

More specifically, this work also underpins the ongoing development of our Innovation Lab Programme.

Central to my work, which will continue for some time, stands the identification of the qualities, attributes and practices of truly successful product and service innovation, as well as the intention to list organizations that demonstrate exceptional Innovation Management skills. The materials I collected, read and studied covered more than three hundred organizations so far, representing a good cross section of nine industries and both public and private business sectors.

Almost invariably, Innovation Management – stated as such or in comparable fashions – appeared among the top three subjects on their CEO agendas across the vast majority of these organizations, proving that each considers innovation key to achieving higher profitability and growth.

What’s more, my analysis of their financials indicates that organizations with mature – relative to their industry peers - innovation management capabilities seem to achieve levels of profitability that outstrips that of their less innovation-sophisticated peers by as much as 200%, and were able to grow faster than their less innovative competitors at rates that appear to average around 7 percent.

The data and information I sourced for this analysis covers a seven year period – between 2003 and 2010.

So far I have been able to identify 23 characteristics that seem to separate leaders from laggards, of which the top ten are:

1. Having innovation strategies that are "top down" driven – proactively, ‘tenaciously’, or ‘obsessively’ fostered, developed and managed by C-level suites as an integral part of their corporate strategies.
2. Having innovation strategies that are documented, agreed upon – tested and validated by employees and often representatives of wider business networks – which are translated in clear, unambiguous terms into operational targets and performance metrics.
3. Developing corporate cultures that are allowed to be ‘open for ideas’, improvement and innovation, and to take ‘calculated risks’ – also knowing when and how to discontinue unpromising programmes and projects without discouraging participants and owners.
4. The drive and ability to proactively seek and enable collaboration both internally and across external networks, and to coordinate cross-organizational and cross-network innovation programmes in line with innovation strategies, taking the capabilities and constraints of all players into account throughout entire value chains.
5. The drive and ability to manage ideation pipelines proactively, according often unique value assessments that are ‘logically’ based on predicted risks, as well as expected benefits.
6. Standardized, implemented and routinely deployed process and product technologies to increase potential number of solution launches whilst reducing idea-to-cash related costs [‘Time-To-Profit’]
7. Innovation objectives and goals are poured in concrete, religiously tracked and monitored, whilst recording the factors that lead to success or failure to inform subsequent participants, programmes and projects on how future innovation initiatives can be improved and enhanced.
8. Capability management, performance management and reward systems integrate with innovation strategies to enforce the notion that innovation is Business As Usual.
9. Acquisition, procurement and development of appropriate, often web-based technologies to speed up flows of knowledge and information, and to enable employees to better balance core-business priorities and innovation activities.
10. Maintaining laser-like, customer-centric focus when developing new processes, solutions, markets and industries; taking ‘customer and industry foresight’ as foundations for all innovation initiatives.

It is clear that the importance of these success factors differs, which I listed here in random order, depending by industry. I sense that geography seems to play a role as well, but have not been able to determine what that role is and how that influences these factors or their relative importance.

What is equally clear is that the most successful innovators remain focused on not just meeting customer needs but to develop solutions that meet or exceed the ‘unarticulated’ needs of tomorrow’s markets.

As for the question: ‘Why Do We Continuously Achieve Sub-optimal Results From Our Innovation Efforts?’ – may I suggest you re-read this preliminary list to distil your own conclusions?

If you come to the uneasy conclusion that your organization doesn’t quite meet these criteria you may have to consider redeveloping your innovation organization – perhaps even from the ground up.