TaKaDu, an Israeli tech firm has used data that already exists to better predict leaks in water mains, as well as diversions of the water supply. No new sensors are needed--just better analysis of the data at hand. As water struggles become more intense over the coming years, more efficient use of water is key.The legal world is being changed through artificial intelligence and data mining as well. Blackstone specializes in e-discovery and provides a cheap alternative to expensive law firms and their armies of 1st and 2nd year lawyers. Why pay people to look at thousands of documents when computers can scan millions?
One of the reasons Tesco has dominated the British groceries market is its ability to understand its customers. It does this through collecting information about purchases and then providing price adjustments and customized discounts. Individual stores are tailored according to the demographics of their customers.
Another interesting example involves policing. As funding is cut, police need to be more intelligent about where they patrol. Smart Policing helps police analyze trends to predict where crime will occur in the future, so less time is needed on the beat.
The data revolution will impact health-care, government, marketing, education--you name it. And, of course, all businesses are working internally to deal with the data deluge. Coworkers need need to be able to share documents in a way that is productive. Products like SharePoint and MarkLogic have become increasingly popular to deal with such unstructured data.
The exponential increase in data and the need to wrangle it will be so great that we will struggle to cope for years. MGI predicts that by 2018 there will be a shortage of 140,000 data professionals in the US alone. It's not just DBAs and architects, but philosophers, analysts, miners, mavens, and entrepreneurs that are needed. One reason that Google and Facebook have become so popular is that they help filter out the noise. I don't pretend to know where this is all going, but I can say I'm excited to be a part of the revolution.
Excellent post. I particularly liked the part where you talked about how philosophers will be needed. Ha.
ReplyDeleteWho else will problematize our metadata? ;)
ReplyDeleteZach,
ReplyDeleteFirst, blame Jeff for pointing me here.
This topic is interesting on many levels, and not just for the technical people. With all the talk of privacy both on and off the 'net, I think that the aggregation of data will have to lead to broader discussions about what "privacy" entails.
Along with that, while I generally don't believe in the "technology for technology's sake" sort of solutions, as an IT consultant I find many companies actually have no real idea of what it is that they are sitting on, much less what to do with it. This observation extends from smaller companies without a lot of technical savvy, to financial firms whose bread and butter is analyzing data and drawing conclusions. You are exactly right, most companies are going to have huge struggles trying to figure out what they have and how to use it. And I absolutely agree that asking those types of questions will have to come from a multitude of sources and experts. I mean, who better to ask about the "truth" of data than someone who spends his days debating such a topic?
Great piece.
Thanks for reading, Scout.
ReplyDeleteI think some of the most valuable people in any business are those who understand business processes as well as technology. I would take a novice programmer who understands what they're programming than a wunderkind who does not any day.
Becoming adapt at two knowledge domains is not easy, however. I often recommend my friends and advisees who are not IT people to become business analysts, but the path to BA-dom is not clear. If the numbers of the MGI report are correct, however, we may need to create new disciplines and training for such careers.