Data Management and Analytics in a World of Big Data

Data management is the process of collecting, validating, integrating, storing, and processing data in a secure, efficient, scalable way for easy access to insights and analytics.

If you look at any successful tech companies in the world now, you will notice that they all continuously collect and analyze massive amounts of data to understand their customers, provide a quality customer experience, and continuously improve operations and efficiency.

If you look at any successful tech companies in the world now, you will notice that they all continuously collect and analyze data to increase their value proposition, understand their customers, and improve their operations. There are an infinite number of big data use cases and increasingly, data provides the competitive advantages and value for these companies.

When we think about data, in its simplest form, we assume customer data is simply a customer name, address, phone number. Data comes in many forms, customer data includes personal information, date/time of logins, preferences, location, financial, industrial, usage activity, device data.

In our seemingly simple example of customer data, an obvious question that comes up: How is Big Data structured? The structure of big data varies depending on the source of the data. It could come in a variety of formats including:

• Structured data - Data that is stored in a database with consistent fields and values such that the format and how to understand the value is known ahead of time. It is the simplest type of data to store, process and analyzed because of its structure. Example: Any database where you can execute an SQL query to look up the data.
• Unstructured data - Data that does not have a fixed format known ahead of time, typically the size is massive, varied and the value derived isn’t predictable. Example: Email messages can contain a mix of content like text, images, video. Difficult to predetermine which is useful
• Semi-structured data - A mix of both structured and unstructured data. When the type of data might have properties and known ahead of time but the content itself is unstructured. Example: the content contained in a JSON structure