5 solutions for customer data management

Customer data resides in many databases and systems, and can be used to formulate insights to help improve the customer experience. But, the precise benefits driven by that data will depend partly upon your unique business needs, and partly on the chosen solution used to manage it, so it’s advisable to think about the desired outcomes before settling on the most appropriate data management tool for you.

Customer data management is a three-pronged process for businesses, whereby customer information is first of all collected and stored in one place. This may be part of a data warehouse or data lake project managed by the IT department, part of a master data management solution, a tailored CRM project, or indeed marketers and businesses may choose to store their data in a Customer Data Platform (CDP).

The benefit of the latter is that, during the process of ingesting data from around the business into a CDP, data transformation also occurs to match, merge, de-duplicate, cleanse and enhance the data to create unified customer profiles. Sometimes referred to as ‘golden records’ these profiles provide a complete view of each customer, including every transaction and touch-point.

The crucial next step for effective customer data management is to ensure you have the right tools and skills in place to analyze and make sense of the data, to determine what lessons and recommendations can be gained from it.

The third step is then to empower business users to take action from those learnings and use the insights gathered to inform better business decisions and achieve improved marketing campaign performance.

Customer data management solutions are key to unlocking better marketing and sales processes, and should be designed to help you derive value and revenue from your first party data. These solutions need to extract data from host systems, such as eCommerce platforms, websites, CRM systems, or apps (to name a few), then convert it into usable formats for many different business units or departments, including, but not exclusively, the marketing team.

Different users, though, will have markedly different needs. For example, a marketer who wishes to create a basic Customer Lifetime Value (CLTV) dashboard will need to rely on a derived sum of all customer transactions to get answers quickly, without being obliged to prepare and analyze raw data, whereas a data analyst may wish to run many variants of more complex CLTV calculations, or run models to calculate a predicted CLTV for new prospects, and will therefore need that granular raw data to work with (and as much of it as possible).

The various data management technologies store and prepare data in different ways, so before investing in a customer data management solution, you’ll first need to consider the departments and users who will rely on the information held and their technical competencies for accessing or using the data. Expecting a marketing team to be able to run SQL queries to get at data they’ll need may be too much to ask, whilst relying on some custom-built applications to fill the gaps of an existing CRM system may lead to lots of expensive covering-over of cracks. As well, a data warehouse or data lake solution may not provide the data standardization, de-duplication or cleansing that is required for better direct marketing.

It is essential to first prepare a detailed map of people, skills, processes, data and technologies to ensure that you invest in the right data solution. Here is an overview of some of the different customer data management options available, along with their typical uses, strengths and weakness:

1. Customer Data Platform

A Customer Data Platform, or CDP, is a customer data management solution designed to be used by marketers, to inform the direction of marketing campaigns. Gartner also defines it as follows:

A Customer Data Platform (CDP) is a marketing system that unifies a company's customer data, from marketing and other channels, to enable customer modeling and optimize the timing and targeting of messages and offers.

A CDP is designed to extract, standardize and clean customer-related data from all platforms across the business, but is not designed to act as a repository for reams of data that might be languishing in other areas of the company, such as procurement, HR or fulfillment. Its primary use cases are relevant to Marketing, but also to any customer-facing department. It works to create a unified Single Customer View, that is, a record of all known online and offline characteristics, behaviors, purchases and interactions associated with an individual, matched using ‘identifiers’ such as an email address, device ID or cookie.

The profiles held can be analyzed, segmented and activated to channel execution tools, such as an Email Service Provider, Mobile Marketing Platform or Marketing Cloud, to enable better personalization and improve cross-channel capabilities. Since the data is standardized, cleansed and deduplicated, it can be queried and sorted into accurate segments without any assistance from the IT department or third party agencies. There’s no need to write code, as marketers can access or process the data with the help of a handy User Interface, and the unified data can be used to power more targeted campaign audiences. In many cases, CDPs will also provide in-depth marketing automation and execution capabilities, making the delivery of cross-channel campaigns part of one seamless data management process.

CDP Pros

• Can create a Single Customer View that becomes a reliable record of the customer, making it easier to understand customers’ needs and power campaigns.
• Standardized, structured data means analysis can be performed by anyone, regardless of skill set, resulting in less delays in getting answers to queries or building campaign audiences.
• Integrates with multiple platforms, customer facing or otherwise, regardless of brand or type, making extracting and sharing data quick and easy.
• It is designed to be marketer-owned, so that all of the information contained within will be relevant for marketing purposes.

CDP Cons

• Structured and standardized data is not a requirement for everyone. CDPs are not primarily built to handle unstructured data, such as social data for sentiment analysis, and do not replace the need for IT-driven data management solutions.
• Since it only contains marketing information, a CDP cannot be used to provide the relevant business insights for other departments within the company that a data warehouse or data lake may provide.
• Although a CDP makes marketers 2.5x more likely to exceed their goals, the initial outlay can be relatively large, since it is pricier to store structured, processed data in an easily accessible form than it is to store raw data.

2. Data Warehouse

A data warehouse is an enterprise-wide repository of data, taken from one or more sources, that stores both current and historic data categorized by subject area. The data held is rigidly structured, it is carefully selected for a purpose and isn’t loaded until its intended uses have been decided upon. If there’s no special use for a piece of data, it won’t be included. Types of analysis that can be performed are also pre-decided.

The loaded data will likely be transactional and definitely quantitative, and it will typically be used to create reports of business changes over time, or to provide a source of business intelligence or data discovery. Different teams can leverage it to mine pre-specified reporting metrics, and other technologies can be integrated for the purposes of speedy analysis (for example, it could act as a source of purchase price data or a resource for predictive modeling if integrated with a CDP).

Without connecting technology, it’s very hard to get swift answers for anything but the pre-programmed questions. To get additional answers, a lot of time and resources would be required, as a complete restructure could be needed. However, the pre-prepared queries can be performed quickly and easily, making data warehouses a great foundation for the execution of complicated analysis, such as Machine Learning and AI.

Data Warehouse Pros

• Can provide the answers to a complicated set questions rapidly, making it a great source of transactional intelligence, and holds ‘hot data’ which might be needed in a hurry.
• Has a large capacity for storage, as data is only activated when it is needed for analysis, so ample space is paired with high performance at a relatively low cost.
• Can be leveraged by business users from all departments for analysis, thanks to pre-set coding making searching for specified answers easy.

Data Warehouse Cons

• This is not a system to be used for ad-hoc queries or to assist with real-time digital marketing requirements. Asking new questions takes time and the structure needs to be set up first.
• It has limited functions, mainly for repetitive KPI or financial reporting and analysis, due to its structure and preference for quantitative data.
• Data warehouses are more expensive than data lakes, as low latency is a functional requirement.

A Marketer’s Guide to Customer Data Platforms eBook

Download our eBook for more CDP advice, information on how a CDP differs from other data management technologies, or to find out more about the different types of CDP and what to look out for when researching technology vendors.

3. Data Lake

A data lake is a sizable repository that ingests every single bit of data from source systems, without exception, and stores it indefinitely in its raw state. This makes it an ideal resource for in-depth analysis, as even data with no identified use will be kept in case it is needed later on, so infinite queries can be applied and retrospective analysis is possible.

This large, varied set of data supports all users, though due to its lack of structure, it’s not easy to extract the answers required, especially quickly. Code will need to be written in, so queries will have to be posed by an engineer. However, with the right expertise, getting quick answers is possible, as no special restructuring or cleaning is necessary and the data is readily available.

The data lake can handle structured or unstructured data, hot or cold, but is most suited to cold data, as the sheer amount of information contained means it takes some wading through to find the right pieces. The fact that data isn’t processed until needed makes this a very low-cost storage option, since low-cost, commodity hardware will support it.

Data Lake Pros

• The data held can be processed over and over again, in all sorts of different ways, so every possible drop of value can be drawn from it. Its lack of structure leads to greater agility as business requirements change.
• Huge volumes of structured and unstructured data can be stored at a low cost, including text and pictures, allowing every department in the business to use it as a place to archive data until needed.

Data Lake Cons

• The data held is difficult to process, due to the high volume and lack of structure, so everyday business users (e.g. Marketing) will need expert assistance to pose queries.
• The majority of the data stored will be of limited use, and more useful pieces are not prioritized to be most easily available, so it’s hard to get quick insights and the ROI is debatable.
• The lack of structure and rules can leave companies open to data privacy regulation breaches, if they are not careful.

4. Master Data Management (MDM)

Once again, we will make use of Gartner’s definition:

Master data management (MDM) is a technology-enabled discipline in which business and IT work together to ensure the uniformity, accuracy, stewardship, semantic consistency and accountability of the enterprise’s official shared master data assets.

It’s clear, then, that master data management (MDM) isn’t intended solely for the use of marketers, but for the whole company. The idea is to create a set of rules and guidance to govern the handling and quality of shared data company-wide. As with a CDP, there is an aim to create a ‘single source of truth’, but this time that unified record will include information on all business functions, including procurement, marketing, fulfillment, finance and so on, so it doesn't specifically look to connect to marketing execution channels or fulfill Marketing's  needs.

Ideally, an MDM project should identify quick wins, such as avoiding the duplication of data on different channels or inconsistent standardization between departments, and aim to start small (for example, with one department), before rolling out more widely. It is important to assign owners of particular pools of data, to ensure practices stay consistent and there’s no confusion when it comes to strategy.

The aims of the project should align to wider company goals, for example customer retention, and associated technology should be chosen to help achieve them. However, despite being technology-fueled, MDM is a strategy/discipline, not a specific tool. It is therefore unlikely that there will be an 'out-of-the-box' way to use it to power marketing campaigns.

MDM Pros

• A single view and format of company data can be used to power business-wide strategy and ensure decisions are based on reliable evidence.
• MDM is a great way of guaranteeing data compliance, as it ensures data is governed, collected and handled consistently around the business, allowing it to be easily audited.
• Consistency and depth means the data can be used to derive insights for different departments, allowing everyone to contribute to achieving the company aims.

MDM Cons

• The large data pool can make it difficult to find the precise data needed for more specific, department-related processing, so it may be that IT will need to provide answers, leading to campaign delays.
• Being a business-wide system, there’s little opportunity for agility or flexibility. The information you need will either be there, or it won’t – there’s no way to tailor it according to departmental needs once the rules are finalized.
• Due to the coordinated nature of master data management, the necessary tools are likely to be part of a suite from a large vendor, so you won’t be able to covet ‘best-in-breed’ and the costs involved could be substantial.

5. Extract, Transform, Load (ETL)

It’s actually a misnomer to call extract, transform, load (ETL) a customer data management solution, as it is really a process for pushing data from one database solution to another, and is therefore encompassed by different solutions, most often a data warehouse, although there are specific 'ETL tools' available on the market.

The process involves first extracting data from an existing data store (the CRM system for example), then transforming or converting it into a different format, and finally loading it into another database (or perhaps back into the CRM, in its new format). ETL provides a means of automating the transfer of data between systems, perhaps after a merger, while at the same time performing any changes needed to make it readable by the other system.

An example use case for ETL software is to extract data from one system (such as a purchase date), transform the date field format (potentially from a YYYY-MM-DD format to a DD/MM/YYYY format), and then load it to another system that will only accept the new format. This means it can be used to move data from one system to another, without the need to rely on a person to do it on a continual basis.

ETL can take place in a bulk deposit, for example when a bank resolves the day’s transactions and all changes to the source systems are frozen until the exchange takes place, but also to automate the movement of data between existing systems at set intervals. The process tends to be scheduled for business down-times, to ensure there is minimal disruption for customer and ETL is not being run when live data is being accessed or altered. 

ETL Pros

• ETL works to unify and standardize different types of data, from different locations, so it has many potential uses.
• It can cope with large deposits all at once, in batches, so it’s very efficient, and processes can be automated, limiting the amount of original coding required and allowing non-technical users to access frequently leveraged data.

ETL Cons

• ETL tools can be dependent on the systems that they need to connect to, and readily available integrations may or may not be available.
• The ETL process is used to push data between systems, rather than to provide one solution that provides a 'single source of truth'.
• The process can take time, so it must be properly scheduled.

Hopefully these explanations will help you to weigh up the pros and cons of varying customer data management solutions in relation to your business requirements. The solution that is most appropriate for you  will depend on whether you’re looking for a business-wide system or a bespoke marketing tool, whether you need quick access to the data or cheap storage, and what you intend to use the data for. A Customer Data Platform is definitely the best solution for marketing use cases, as it’s designed to handle customer data only, and that data is structured, deduplicated, cleansed and enhanced specifically for easy use within marketing campaigns.

However, data warehouses can be a great source of data when connected to a CDP, and the ETL process powers the data preparation for such repositories. Master data management systems won’t have as much to do with marketing processes, but they can ensure that the data fed into a CDP is as reliable as possible and free from regulatory issues. So, a marketing team can benefit from leveraging a number of data management solutions simultaneously, although they will only fully be in control of data held in a Customer Data Platform.

If you want to find out more about what sets a CDP apart from other data management solutions, our informative webinar will teach you more about its capabilities. 

Customer Data Platform vs. Data Management Platform vs. Single Customer View webinar

On Demand

CDPs. DMPs. SCVs. Marketers swim in an ocean of acronyms and the confusion surrounding them is muddying the waters. When building your marketing technology stack, do you need a Customer Data Platform, a Single Customer View, a Data Management Platform – or all three?

In this webinar, Jim Kelly, Principal Consultant at BlueVenn, will guide you through these troubled seas so you can set a course for greater marketing success.