3 Best Practices: Designing CDP Architectures on Google Cloud

Most companies are in various stages of digital transformation. One aspect of this larger enterprise-wide transformation is often a focus on the customer, particularly on improving the customer experience through personalizing their experience with the company, regardless of channel and the type of interaction.

Leveraging Google Cloud for Effective Customer Data Platform Architectures

A Customer Data Platform (CDP) can be a foundational technology to enable and accelerate this more extensive transformation. Done right, the CDP can be the heart of a company’s entire marketing technology and enterprise-wide stack, helping orchestrate customer data and optimising the customer (and business user) experience. 

Must-have marketing capabilities in the digital economy include, but are not limited to:

• Personalisation and segmentation at scale
• Appropriate content and product recommendations
• Relevant, real-time digital experiences
• Unified omnichannel messages
• Cross-domain and cross-channel performance measurement
• A secure, privacy-safe, and well-governed unified architecture

Companies have adopted, or are in the process of adopting Customer Data Platforms, done right (if you ask me) as an integral part of a Unified Enterprise Data Platform to help them collect, transform, analyze and interact with their audiences. A CDP brings together customer data from various sources and creates a unified view of the customer in order to deliver the must-have marketing and compliance capabilities mentioned above.

It is important to view the CDP and its corresponding capabilities as an extension of a strong data management strategy and unified enterprise architecture.

Don’t regard the CDP as an alternative to a data management strategy and a unified enterprise architecture! By adopting this view, the ROI of your CDP investment will be boosted through interoperability in the enterprise.

For example, a bedding company wants to offer their customers to rebuy a mattress after 6 years of purchase because in general, their mattresses have an expected lifetime of 6 years. The customers receive an email with a voucher including the name of the current product in use and the product image that is living in their data lakehouse and not in the CDP. This can be enabled as the data is linked to and harmonised in your CDP and will be readily accessible in your lake house and through the managed services provided by Google Cloud.

Similarly, with connective tissue between your data lake house and your CDP, your data teams will be empowered to infuse their domain knowledge, insights, and models into the customer interactions powered by your CDP.

For companies that have decided to invest in a CDP, the main decision is which components to build versus which components to buy or license, which is an important part of an article I posted recently. Even off-the-shelf CDP platform components require customers to have a strategy for where essential data resides and who has access to the core data versus access to the insights. Generally, I recommend keeping the important data in company-managed data lakes built on Google Cloud Storage and Big Query storage.

The control, access, and management of 1st party customer data, including Personally Identifiable Information (PII) is not only a significant competitive advantage for brands, it is also a regulatory necessity. Data protection regulations, such as GDPR, force enterprises to know where all of their data is housed and how they are keeping it safe.

Even when companies have CDPs, I notice frequently that data sets and insights are still being used individually by teams or company divisions but not being aggregated sufficiently to have a comprehensive view across divisions, platforms, content types, etc. Ultimately, customers are concerned about where their data resides and who can leverage it effectively.

Finally, third-party cookies, which have been important to media marketers and media advertising spends, are disappearing. Businesses that use third-party cookies to track customer journeys increasingly need to think of alternatives to reaching their target audiences, as I have recently written about.

How to build CDP Architecture on GCP

In general, our customers take one of 3 architecture approaches:

1. Fully integrated suite CDP’s 

This approach is best for customers who want one tool, and want it quickly. I see a split between two types of companies. 

Difference between enterprise and small & medium-sized companies

If enterprises want one or two suites combined and have little or no concern for cost, the larger suites may be an option, as shown in the image below. Customers need the appropriate budget and dedicated data engineering resources for integration. A cost component that shouldn't be overlooked here is the need for internal alignment, as out-of-the-box solutions require some end users to accommodate the platform’s tools and workflows. Direct costs include paying for the whole platform and for the data rows that persist in the platform.

Also to consider are lock-in to a single vendor for features, pace of innovation, and flexibility risks, as discussed in this article, 'Should I bring my data and analytics to Salesforce?'

Smaller companies that aren’t too demanding about bespoke business rules, code, and models may also use a suite. As their maturity grows, they might shift to a Modular best-of-breed architecture.

Example high-level architecture of fully integrated suites on Google Cloud Platform:

2. Modular CDP’s

This hybrid approach allows customers to pick which aspects of the CDP components they build in the public cloud, and which they buy/license. This provides the potential for the “best of breed” CDP stack tools for each use case and allows customers to avoid spending money on features they do not need. It also allows customers to add functionality as needed with their expanding needs. Customers will still need to dedicate data engineering resources, and in most cases, this will be a larger time commitment than the full-stack option. Customers can more directly control and reduce CDP costs. You can read in detail about building a MarTech stack on a public cloud platform and packaged apps, as part of a modular architecture, in this blog.

My general advice for mid-to-large-sized companies with large customer databases is to build, compose, and buy what is possible in the public cloud and persist all data here and not in the modules. If you have the normalized 360 customer view in GCP, switching from or testing with CDP modules significantly reduces migrating time. 

Integrate with tools that these clouds don't offer natively. For example, a component as a user interface for creating customer process flows like Salesforce Einstein and Selligent offer. This way you can get the best out of both worlds: integration, orchestration, smart modelling, and adding these tools from other vendors you want to use.

In this high-level example architecture, we have added three logos per component, not that there are 1000s of components to choose from. The technical and operational management complexity is not as hard as it may seem if you implement it well. Crystalloids can guide you in the selection process and architecture that fits your organisation.

3. Fully headless GCP end-to-end

The third best practice is to build the CDP on top of their existing lake house foundation using the full capabilities of Google’s IAAS, PAAS, and SAAS services. This is for companies that have the budget and the internal and/or external partner resources, in most cases, enterprise digital natives. They have complete flexibility, control, ownership, workflow, and data lifecycle management including CI/CD, containerization, etc. The components are built at the pace they require and leverage testing to make data-driven development decisions.

A big difference with the modular approach is that a Google-first approach means functionality is developed in GCP and only buy/license apps if GCP doesn’t offer it, or a third party component has a better fit for a specific reason. Another plus is the native integrations with the Google advertising suite.

Packaged apps such as e-mail engines and visualization SAAS are used in DIY CDP. However, the functionality is reduced to the essentials. The 360-degree customer and analytical views always live in the GCP CDP, not third-party CDP components. 

Our clients Body & Fit, Coolblue, and Rituals Cosmetics are examples of digital natives that DIY on GCP.

Getting started with CDP Architecture on GCP

There is no one “right approach” when it comes to building a CDP – it depends on where your company is in its cloud journey, the complexity and size of your company, the scope of your business needs, and the resources that you have available yourself or with a partner like Crystalloids.

If you do not have the proper internal resources, specifically software development, data engineering, and data science talent, but have the budget, then a full-stack CDP approach is an option.

However, as internal data capabilities mature within an enterprise, it makes sense to outsource development to a specialized partner like Crystalloids, which has expertise in building CDPs on GCP and buying best-in-breed specific functions. 

We have customers who have taken all of these approaches, or started with one and then moved into another as they hired more data team resources. 

Ensure buy-in from various constituents and inform and provide the road map for successful implementation and user acceptance beyond the purchase decision. Inclusive, cross-functional involvement is a key marker of success.

Executive sponsorship and commitment from the enterprise's top leadership also help ensure success. This is easier if the CDP is helping drive the overall strategic objectives and direction of the enterprise.

Why Google Cloud?

What’s important is that with GCP at the core, you can build a modern, flexible CDP that can evolve with your business and scale to your users and use cases in a secure and privacy-compliant manner.

If you want to learn why Google Cloud Platform is the right fit for your requirements, I invite you to read why our clients (past and present) chose Google Cloud to develop their CDP and remain on Google Cloud to this day. 

Conclusion

Our general advice for mid-to-large companies with large customer databases is to build, compose, and buy what is possible in the public cloud. Integrate with tools that these clouds don't offer natively.

For example, a component can be used as a user interface for creating customer process flows like Salesforce Einstein, Tinyclues, and Selligent offers. This way, you can get the best out of both worlds: integration, orchestration, and the addition of tools from other vendors you want to use.