Predictive Analytics with Big Data
My job is often to turn industry concepts or “catch phrases” into results. Some of these concepts come and go and some stick around. My organization, like so many others is on the Big Data bandwagon. “Big Data” means different things to different people, just like “Cloud” just a few short years ago.
Turning a concept into a project and subsequently a reportable outcome as any CIO can tell attest is not easy. First we had to understand Big Data and define what it means for our organization. We are a healthcare organization with a large research institute. We finally settled on our definition of Big Data. It is all of our data plus all of the data we are not currently capturing or using. We are capturing terabytes of data each month from our Electronic Medical Records system. This all goes into our Enterprise Data Warehouse (EDW). Our Big Data equation equals our current EDW + medical claims data + vital monitor data + pharmaceutical data + home device data + patient entered data + local public school data + data from other organizations + data from our community physicians + data from our health department + patient genomics + geographical and market data + cost data and + many more sources. You get the idea.
As difficult as the definition process has been, the use case discussion is ten times more complex. There are lots of ideas but a lack of data analysts and data scientists to turn those ideas into reality. So far we’ve identified areas of opportunity in genomics, population health, patient safety, care coordination, research and best outcomes for our patients.
With all the data we will be collecting, often without specific purpose, we need to do analytics and even better, predictive analytics. We debate whether or not we do analytics currently. Everyone has a different concept of “analytics”. I bet we’ve all pulled out a dictionary definition at one point to kick start a discussion. We are starting to define some use cases which will help us drive the organization towards more robust analytics capability.
“The time we are taking to develop our strategy is crucial to success”
The next big discussion will be the value proposition. There is belief that there is long term value in the data. Many of us have been trained to choose and implement projects based on a cost value equation or Return on Investment (ROI). With Big Data, sometimes the ROI is not immediately evident. We start with hypothesis and hope that the data will show us something of value. Not every idea will pan out. It is a bit of a culture shift for CIOs and IT staff to manage uncertainty.
As we hone in on what we want to do, it is time to figure out what resources we need. There are still many questions to answer. Do we build or buy? Can we purchase a prefab appliance type solution? Do we store the data in-house or in the cloud? How do we find the people with the right skill sets or do we have to develop them in house? How much time and money can we afford to spend? How do we measure success?
While much of the time I feel like we are not doing enough or moving fast enough, when I look back at where we were a year ago, we really have made a lot of progress. The time we are taking to develop our strategy is crucial to success. Our ability to tie our Big Data strategy to our overall strategic plan is also critical. I believe we are on the right path and while we may veer off now and then we are going to finish the marathon.