Matt Aslett's Analyst Perspectives

Master data management may not attract the same level of excitement as fashionable topics such as DataOps or Data Platforms, but it remains one of the most significant aspects of an organization’s strategic approach to data management. Having trust in data is critical to the ability of an organization to make data-driven business decisions. Along with data quality, MDM enables organizations to ensure data is accurate, complete and consistent to fulfill operational business objectives.

As engagement with customers, suppliers and partners is increasingly conducted through digital channels, ensuring that infrastructure and applications are performing as expected is not just important but mission critical. My colleague, David Menninger, recently explained the increasing importance of observability to enable organizations to ensure that their systems and applications are operating efficiently. Observability has previously been the domain of the IT department but is increasingly important for business decision-makers as organizations combine machine-generated telemetry data with business event data to understand the impact of a system outage or application performance degradation on their ability to conduct digital business. Companies such as Mezmo are responding with observability platforms designed to facilitate the integration of machine and business data and encourage collaboration between business and IT professionals.

To execute more data-driven business strategies, organizations need linked and comprehensive data that is available in real time. By consistently managing data across siloed systems and ensuring that data definitions are agreed and current, organizations can overcome the challenges presented by data being distributed across an increasingly disparate range of applications and data-processing locations. Maintaining data quality is a perennial data management challenge, often preventing organizations from operating at the speed of business. Our Analytics and Data Benchmark Research shows that almost two-thirds of participants (64%) cited reviewing data for quality issues as being the most time-consuming aspect of analytics initiatives, second only to preparing data for analysis. This is where master data management (MDM) becomes critical, to ensure that organizations have the clean, consistent data needed to operate efficiently and effectively. When organizations control master data, they gain visibility into their overall operations and can provide proper governance, while also having access to reliable, accurate and timely data about customers, products, assets and employees. Reltio offers MDM products designed to help customers improve trust in data by unifying and cleansing complex data from multiple sources in real time.

Data Operations (DataOps) has been part of the lexicon of the data market for almost a decade, with the term used to describe products, practices and processes designed to support agile and continuous delivery of data analytics. DataOps takes inspiration from DevOps, which describes a set of tools, practices and philosophy used to support the continuous delivery of software applications in the face of constant changes. DataOps describes a set of tools, practices and philosophy used to ensure the quality, flexibility and reliability of data and analytics initiatives, with an emphasis on continuous measurable improvement, as well as agility, collaboration and automation. Interest in products and services that support DataOps is growing. I assert that by 2025, one-half of organizations will have adopted a DataOps approach to their data engineering processes, enabling them to be more flexible and agile.

As data continues to grow and evolve, organizations seek better tools and technologies to employ data faster and more efficiently. Finding and managing data remains a perennial challenge for most organizations, and is exacerbated by increasing volumes of data and an expanding array of data formats. At the same time, organizations must comply with a growing list of national and regional rules and regulations, such as General Data Protection Regulation and the California Consumer Privacy Act. While these regulations protect consumers, they increase complexity for governing and providing access to data.

I have previously written about the importance of data democratization as a key element of a data-driven agenda. Removing barriers that prevent or delay users from gaining access to data enables it to be treated as a product that is generated and consumed, either internally by employees or externally by partners and customers. This is particularly important for organizations adopting the data mesh approach to data ownership, access and governance. Data mesh is an organizational and cultural approach to data, rather than a technology platform. Nevertheless, multiple vendors are increasingly focused on providing products that facilitate adoption of data mesh and promote data democratization. Amazon Web Services is one such vendor, thanks to the recent launch of Amazon DataZone, one of the figurehead analytics and data announcements made during the company’s recent re:Invent customer event.

Organizations require faster analytics to continuously improve business operations and stay competitive in today’s market. However, many struggle with slow analytics due to a variety of factors such as slow databases, insufficient data storage capacity, poor data quality, lack of proper data cleansing and inadequate IT infrastructure. Challenges such as data silos can also decrease operational efficiency. And as the data grows, performing complex data modelling becomes challenging for users as they spend more time managing data rather than identifying insights.

The market for data and analytics products is constantly evolving, with the emergence of new approaches to data persistence, data processing and analytics. This enables organizations to constantly adapt data analytics architecture in response to emerging functional capabilities and business requirements. It can, however, also be a challenge. Investments in data platforms cannot be constantly written-off as organizations adopt new products for new approaches. Too little change can lead to stagnation, but too much change can be chaotic, leading to silos of data and data integration complexity. This is one reason why there is growing interest in the concept of data fabric for managing and governing data across distributed environments. In addition to supporting hybrid and multi-cloud strategies, data fabric enables organizations to manage and generate insight from data spread across a combination of long-standing and new data platforms. Promethium focuses on automating data management and data governance across a distributed architecture with a combination of data fabric and self-service augmented analytics capabilities.

Data observability was a hot topic in 2022 and looks likely to be a continued area of focus for innovation in 2023 and beyond. As I have previously described, data observability software is designed to automate the monitoring of data platforms and data pipelines, as well as the detection and remediation of data quality and data reliability issues. There has been a Cambrian explosion of data observability software vendors in recent years, and while they have fundamental capabilities in common, there is also room for differentiation. One such vendor is Soda Data, which offers an open-source platform for self-service data observability that is focused on facilitating collaboration between business decision-makers and data teams responsible for generating and managing data to improve trust in data.

Organizations across various industries collect multiple types of data from disparate systems to answer key business questions and deliver personalized experiences for customers. The expanding volume of data increases complexity, and data management becomes a challenge if the process is manual and rules-based. There can be numerous siloed, incomplete and outdated data sources that result in inaccurate results. Organizations must also deal with concurrent errors – from customers to products to suppliers – to create a complete view of the data. Many vendors, including Tamr, have turned to artificial intelligence and machine learning to overcome the challenges associated with maintaining data quality amid the growing volume and variety of data. I assert that by 2026, more than three-quarters of organizations’ data management processes will be enhanced with artificial intelligence and machine learning to increase automation, accuracy, agility and speed.