We all know that the usage of big data is constantly growing. Starting on terabytes of information, to huge databases of petabytes, zettabytes, and even beyond that. To maximize the value of the expanding data, we have this short guide to learn new ways to make the most of your big data:
Think long term:
Everything is evolving so fast that it’s impossible to know which tools and methodologies to use. For example, relational and online transactions systems (OLTP) will become more efficient and smarter, while techniques will develop to make easier relations between Hadoop and data warehouses. This means, we must stay open to the possibilities of new products that can maintain a business intelligence platform directly connected to a wide variety of formats.
See Beyond the Obvious
Not only Hadoop and data warehouses can work well alongside each other. The data warehouse is best to crunch all the important, structured data to then store it where BI tools and dashboards can find it easily. The data set can be massive, with too little time to prepare it for the data warehouse. When the work is done, it is very probable the data set will be disposed. That’s when Hadoop make presence to store the data, refine it, and send a sample to the data warehouse. As Third Nature CEO Mark Madsen wrote in his article “What Big Data is Really About”: “It’s part of the new IT environment. Don’t fall for the Hadoop-or-data warehouse trick. You can and should use both.”
Big Data Visualization
A 2013 report published by Aberdeen Group found that “at organizations that use visual discovery tools, 48 percent of BI users are able to find the information they need without the help of IT staff.” The report also found that visualization encourages interaction with the data.
Visual analysis allows you to do two things: change the data you’re looking at (since different questions, require different data) or change the way you look at it (because different views answer different questions). “A light bulb goes off; you just don’t get that with a spreadsheet” says Wells Fargo Vice President of Strategic Planning Dana Zuber. With these steps, you can go to the Cycle of Visual Analysis: getting the data, visualize it, make questions, give answers, and repeat. You may also either bring new data or create view after view as your visualization speeds. When you’re ready, you share.
Empower Users for Big Insights
With big data, there’s no time for nothing but self-service data analysis. As explained by Aberdeen Group in one of its reports, organizations with big data are over 70% more likely to have BI projects that are driven primarily by the business community, and not by the IT group, than other organizations less focused on self-service data analysis.
Instead of waiting too long for data to be cleaned and ready to use, it’s possible to apply big data technologies to discover the value in it. If it is valuable, the data can be sent through rigorous processes for a data warehouse. Mark Madsen also said: “Don’t force users to elucubrate. Empower them.”
Union Makes It Stronger
When analyzing the data, we realized that it is made of smaller datasets. Alone they might provide value, but together, they will make a big difference. In the consumer goods industry, executives get full understanding of customer behavior when they blend sentiment data with purchase data. “You get a rich variety of data through loyalty cards,” said Rishi Kumar, Unilever’s director of analytics. “It’s important to blend all this data together to understand why people enter the shop and fill their baskets.” This lets Unilever anticipate for popular products and new trends.
The biggest value goes to companies that combine relational, semi-structured, and raw tone. Whether the data is in a spreadsheet, a database, a data warehouse, open-source file systems, or in all of those together, the flexibility is needed to connect to data and consolidate it. This allows ask and answer questions as they come to mind.