"How the Cloud is Allowing Carnival Corporation to Better Manage Big Data"
In the recent past, Carnival Corporation, comprised of 10 cruise line brands with over 100 ships started collecting nautical data from many ships in its fleet. The goal was to have a better understanding of the behaviors and identify potential opportunities for operational improvement and cost reductions using trend and predictive analysis. From a technical viewpoint, the goal was to have a simple architecture to allow the capturing of data and to store it for analytical purposes. Over time, the type and volumes of data would be extended, so an agile and flexible solution was crucial for success.
From a technology perspective, this initiative presented some significant challenges: how to collect, store and process what is to become a very large repository of historical data to perform analytics. For example, the expectation was to have a continuous flow of data coming from vessels that could generate 2 million records per ship per day. Sizing the infrastructure correctly and having elasticity and agility to increase processing capacity on demand would be paramount to the continued success and maintenance of the solution. Storing data in the cloud became the natural alternative, as it allows Carnival to “pay by the drink” and very quickly summon processing and storage resources. Microsoft Azure, with its comprehensive set of services, was the partner of choice.
The application solution, developed according to Carnival’s specifications, consists of a data collector installed in the ship, which in turn connects to different data source entities and is continually capturing and storing data locally. The data available in each ship may vary, depending on the equipment and sensors installed in the vessel. From time-to-time, based on configurable frequency rules and when connectivity is available, data is transmitted from the ship and stored in a data repository in the cloud at Microsoft Azure. It is possible to use tools from the Cortana Intelligence Suite to both monitor and manage the flux of data transmitted and to perform historical data analysis (trending).
In these days, agility and speed to market are critical to maintain any company competitive. And the cloud is one tool that can effectively enable it
Once the data is in the cloud, subject matter experts can perform analytics to “learn what is not known” and recognize correlations using predictive algorithms. For example, the company can potentially identify differences of navigation over similar itineraries and better understand the drivers for those differences. This analysis can help determine the most efficient way to operate equipment, optimize maintenance intervals, or enable failure analysis, which can uncover opportunities to reduce costs.
Solution High-Level Architecture:
Some additional benefits of the solution are:
• One can easily “plug” a business intelligence tool to analyze and visualize data from many different sources and discover correlations between different conditions,
• Identify typical ranges for certain data and highlight off-liners for further investigation (evaluate instances by exception),
• Understand influence of external factors that may affect ship’s operation (e.g. weather, traffic), and
• Eventually, with the use of predictive analytics, identify potential problems before they occur (e.g. condition based maintenance).
The decision to store big data on the cloud proves to be very efficient from both a technical and financial perspective. It allows tremendous speed to make changes to infrastructure resources when needing more processing or storage capacity. It makes quick work to test prototype solutions for presentation and analysis of data. Adding new data types and sources is as quick as “turning the lights on”. All of these allow the company to focus its time, investment, and resources in finding ways to extract value from the data instead of immobilizing financial resources and creating “distractions” by setting up, configuring and testing changes to the system environment.
In these days, agility and speed to market are critical to maintain any company competitive. And the cloud is one tool that can effectively enable it.