Best business intelligence and data articles, 2 - 9 February

In an age of computers, GPS devices and fleet GPS transceivers, RFID tag readers, smart meters, embedded microprocessors and sensors, we are now confronted with a tsunami of data.

More than a decade ago, we entered an era of data deluge. Data continues to explode - it has been estimated that for each day of 2012, more than 2.5 exabytes (or 2.5 million terabytes) of data were created. Today, the same amount of data is produced every few minutes!

The history of innovation is scattered with products and processes which only came into being as a result of an accident or pure serendipity. Microwave ovens, X-ray machines, penicillin, Play-Doh - to name a few - all came about not as a result of being designed from a blank sheet of paper, but by innovators who had the vision and motivation to seek wider applications for what was there in front of them.

Just over a quarter of organizations say their big data initiatives are a success, according to a recent Capgemini study. So why are three of four unsuccessful? (Click image for larger view and slideshow.) Nearly eight out of ten organizations have big data projects underway, but only 27% describe their efforts as "successful," and a scant 8% as "very successful."

What if the scientific method were no longer necessary? A bold statement, but one that Wired's Chris Anderson posits in his 2008 article, extolling the virtues of "Big Data." He writes, "Petabytes allow us to say: 'Correlation is enough.' We can stop looking for models.

Business intelligence platforms are set to be in high demand across a variety of industries, according to new research. More firms are working to control the huge amounts of data they are generating. The latest solutions in the market are of great interest to midsize firms aiming to do the same.

The battle of trying to apply traditional waterfall software development life-cycle (SDLC) methodology and project management to Business Intelligence (BI) has already been fought - and largely lost. These approaches and best practices, which apply to most other enterprise applications, work well in some cases, as with very well-defined and stable BI capabilities like tax or regulatory reporting.

In a meeting last week one of our partners, Musgrave Analytics, mentioned something that caught my attention: people are afraid to deploy dashboards to business users until the data is perfect. Although understandable, the problem with this approach is that it can paralyze dashboard and other business intelligence projects and result in a frustrated business community that lacks access to critical information.