Industry influencers, academicians, and other prominent stakeholders certainly agree that big data has become a big game changer in most, if not all, types of modern industries over the last few years. As big data continues to permeate our day-to-day lives, there has been a significant shift of focus from the hype surrounding it to finding real value in its use.
While understanding the value of big data continues to remain a challenge, other practical challenges including funding and return on investment and skills continue to remain at the forefront for a number of different industries that are adopting big data. With that said, a Gartner Survey for 2015 shows that more than 75% of companies are investing or are planning to invest in big data in the next two years. These findings represent a significant increase from a similar survey done in 2012 which indicated that 58% of companies invested or were planning to invest in big data within the next 2 years.
Generally, most organizations have several goals for adopting big data projects. While the primary goal for most organizations is to enhance customer experience, other goals include cost reduction, better targeted marketing and making existing processes more efficient. In recent times, data breaches have also made enhanced security an important goal that big data projects seek to incorporate.
More importantly however, where do you stand when it comes to big data? You will very likely find that you are either:
- Trying to decide whether there is true value in big data or not
- Evaluating the size of the market opportunity
- Developing new services and products that will utilize big data
- Already utilizing big data solutionsRepositioning existing services and products to utilize big data, or
- Already utilizing big data solutions
With this in mind, having a bird’s eye view of big data and its application in different industries will help you better appreciate what your role is or what it is likely to be in the future, in your industry or across different industries.
In this article, I shall examine 10 industry verticals that are using big data, industry-specific challenges that these industries face, and how big data solves these challenges. I shall additionally mention some examples of big data providers that are offering solutions in the specific industries.
1. Banking and Securities
Industry-Specific big data challenges
A study of 16 projects in 10 top investment and retail banks shows that the challenges in this industry include: securities fraud early warning, tick analytics, card fraud detection, archival of audit trails, enterprise credit risk reporting, trade visibility, customer data transformation, social analytics for trading, IT operations analytics, and IT policy compliance analytics, among others.
Applications of big data in the banking and securities industry
The Securities Exchange Commission (SEC) is using big data to monitor financial market activity. They are currently using network analytics and natural language processors to catch illegal trading activity in the financial markets.
Retail traders, Big banks, hedge funds and other so-called ‘big boys’ in the financial markets use big data for trade analytics used in high frequency trading, pre-trade decision-support analytics, sentiment measurement, Predictive Analytics etc.
This industry also heavily relies on big data for risk analytics including; anti-money laundering, demand enterprise risk management, "Know Your Customer", and fraud mitigation.
Big Data providers specific to this industry include: 1010data, Panopticon Software, Streambase Systems, Nice Actimize and Quartet FS
2. Communications, Media and Entertainment
Industry-Specific big data challenges
Since consumers expect rich media on-demand in different formats and in a variety of devices, some big data challenges in the communications, media and entertainment industry include:
- Collecting, analyzing, and utilizing consumer insights
- Leveraging mobile and social media content
- Understanding patterns of real-time, media content usage
Applications of big data in the Communications, media and entertainment industry
Organizations in this industry simultaneously analyze customer data along with behavioral data to create detailed customer profiles that can be used to:
- Create content for different target audiences
- Recommend content on demand
- Measure content performance
A case in point is the Wimbledon Championships (YouTube Video) that leverages big data to deliver detailed sentiment analysis on the tennis matches to TV, mobile, and web users in real-time.
Spotify, an on-demand music service, uses Hadoop big data analytics, to collect data from its millions of users worldwide and then uses the analyzed data to give informed music recommendations to individual users.
Amazon Prime, which is driven to provide a great customer experience by offering, video, music and Kindle books in a one-stop shop also heavily utilizes big data.
Big Data Providers in this industry include:Infochimps, Splunk, Pervasive Software, and Visible Measures
3. Healthcare Providers
The healthcare sector has access to huge amounts of data but has been plagued by failures in utilizing the data to curb the cost of rising healthcare and by inefficient systems that stifle faster and better healthcare benefits across the board.
This is mainly due to the fact that electronic data is unavailable, inadequate, or unusable. Additionally, the healthcare databases that hold health-related information have made it difficult to link data that can show patterns useful in the medical field.
Other challenges related to big data include: the exclusion of patients from the decision making process, and the use of data from different readily available sensors.
Applications of big data in the healthcare sector
Some hospitals, like Beth Israel, are using data collected from a cell phone app, from millions of patients, to allow doctors to use evidence-based medicine as opposed to administering several medical/lab tests to all patients who go to the hospital. A battery of tests can be efficient but they can also be expensive and usually ineffective.
Free public health data and Google Maps have been used by the University of Florida to create visual data that allows for faster identification and efficient analysis of healthcare information, used in tracking the spread of chronic disease.
Obamacare has also utilized big data in a variety of ways.
Big Data Providers in this industry include: Recombinant Data, Humedica, Explorys and Cerner
Industry-Specific big data challenges
From a technical point of view, a major challenge in the education industry is to incorporate big data from different sources and vendors and to utilize it on platforms that were not designed for the varying data.
From a practical point of view, staff and institutions have to learn the new data management and analysis tools.
On the technical side, there are challenges to integrate data from different sources, on different platforms and from different vendors that were not designed to work with one another.
Politically, issues of privacy and personal data protection associated with big data used for educational purposes is a challenge.
Applications of big data in Education
Big data is used quite significantly in higher education. For example, The University of Tasmania. An Australian university with over 26000 students, has deployed a Learning and Management System that tracks among other things, when a student logs onto the system, how much time is spent on different pages in the system, as well as the overall progress of a student over time.
In a different use case of the use of big data in education, it is also used to measure teacher’s effectiveness to ensure a good experience for both students and teachers. Teacher’s performance can be fine-tuned and measured against student numbers, subject matter, student demographics, student aspirations, behavioral classification and several other variables.
On a governmental level, the Office of Educational Technology in the U. S. Department of Education, is using big data to develop analytics to help course correct students who are going astray while using online big data courses. Click patterns are also being used to detect boredom.
Big Data Providers in this industry include: Knewton and Carnegie Learning and MyFit/ Naviance