Matt Aslett's Analyst Perspectives

Master data management may not attract the same level of excitement as fashionable topics such as DataOps or Data Platforms, but it remains one of the most significant aspects of an organization’s strategic approach to data management. Having trust in data is critical to the ability of an organization to make data-driven business decisions. Along with data quality, MDM enables organizations to ensure data is accurate, complete and consistent to fulfill operational business objectives.

The data platforms market has traditionally been divided between products specifically designed to support operational or analytic workloads, with other market segments having emerged in recent years for data platforms targeted specifically at data science and machine learning (ML), as well as real-time analytics. More recently, we have seen vendor strategies evolving to provide a more consolidated approach, with data platforms designed to address a combination of analytics and data science, as well as hybrid operational and analytic processing. Snowflake, which has been hugely successful in recent years with its cloud-based analytic data platform, is a prime example. The company has expanded its purview to address data engineering and data science, as well as transactional data. Additionally, it now provides users with the ability to access and process data in on-premises environments as part of its strategy to address an increasing range of use cases.

The recent publication of our Value Index research highlights the impact of intelligent applications on the operational data platforms sector. While we continue to believe that, for most use cases, there is a clear, functional requirement for either analytic or operational data platforms, recent growth in the development of intelligent applications infused with the results of analytic processes, such as personalization and artificial intelligence (AI)-driven recommendations, has increasing influence over requirements for operational data platforms to support real-time analytic functionality. Operational data platform vendors, including MongoDB, are responding to these evolving requirements with new functionality to support the development and deployment of intelligent applications. 

As engagement with customers, suppliers and partners is increasingly conducted through digital channels, ensuring that infrastructure and applications are performing as expected is not just important but mission critical. My colleague, David Menninger, recently explained the increasing importance of observability to enable organizations to ensure that their systems and applications are operating efficiently. Observability has previously been the domain of the IT department but is increasingly important for business decision-makers as organizations combine machine-generated telemetry data with business event data to understand the impact of a system outage or application performance degradation on their ability to conduct digital business. Companies such as Mezmo are responding with observability platforms designed to facilitate the integration of machine and business data and encourage collaboration between business and IT professionals.

I have written recently about the increasing importance of managing data in motion and at rest as the use of streaming data by enterprise organizations becomes more mainstream. While batch-based processing of application data has been a core component of enterprise IT architecture for decades, streaming data and event processing have often been niche disciplines typically reserved for organizations with the highest-level performance requirements. That has changed in recent years, driven by an increased reliance on streaming data and events to identify trends and anomalies and enable real-time responses through uninterrupted processing and analysis of data generated by applications, systems and devices on a continuous basis. Companies like Solace have been integral to the development of event-driven applications and increased adoption of event-driven architecture (EDA).

To execute more data-driven business strategies, organizations need linked and comprehensive data that is available in real time. By consistently managing data across siloed systems and ensuring that data definitions are agreed and current, organizations can overcome the challenges presented by data being distributed across an increasingly disparate range of applications and data-processing locations. Maintaining data quality is a perennial data management challenge, often preventing organizations from operating at the speed of business. Our Analytics and Data Benchmark Research shows that almost two-thirds of participants (64%) cited reviewing data for quality issues as being the most time-consuming aspect of analytics initiatives, second only to preparing data for analysis. This is where master data management (MDM) becomes critical, to ensure that organizations have the clean, consistent data needed to operate efficiently and effectively. When organizations control master data, they gain visibility into their overall operations and can provide proper governance, while also having access to reliable, accurate and timely data about customers, products, assets and employees. Reltio offers MDM products designed to help customers improve trust in data by unifying and cleansing complex data from multiple sources in real time.

Data Operations (DataOps) has been part of the lexicon of the data market for almost a decade, with the term used to describe products, practices and processes designed to support agile and continuous delivery of data analytics. DataOps takes inspiration from DevOps, which describes a set of tools, practices and philosophy used to support the continuous delivery of software applications in the face of constant changes. DataOps describes a set of tools, practices and philosophy used to ensure the quality, flexibility and reliability of data and analytics initiatives, with an emphasis on continuous measurable improvement, as well as agility, collaboration and automation. Interest in products and services that support DataOps is growing. I assert that by 2025, one-half of organizations will have adopted a DataOps approach to their data engineering processes, enabling them to be more flexible and agile.

As data continues to grow and evolve, organizations seek better tools and technologies to employ data faster and more efficiently. Finding and managing data remains a perennial challenge for most organizations, and is exacerbated by increasing volumes of data and an expanding array of data formats. At the same time, organizations must comply with a growing list of national and regional rules and regulations, such as General Data Protection Regulation and the California Consumer Privacy Act. While these regulations protect consumers, they increase complexity for governing and providing access to data.

Success with streaming data and events requires a more holistic approach to managing and governing data in motion and data at rest. The use of streaming data and event processing has been part of the data landscape for many decades. For much of that time, data streaming was a niche activity, however, with standalone data streaming and event-processing projects run in parallel with existing batch-processing initiatives, utilizing operational and analytic data platforms. I noted that there has been an increased focus on unified approaches that enable the holistic management and governance of data in motion alongside data at rest. One example is the recent emergence of streaming databases designed to combine the incremental processing capabilities of stream-processing engines with the SQL-based analysis and persistence capabilities of traditional databases.

I have previously written about the importance of data democratization as a key element of a data-driven agenda. Removing barriers that prevent or delay users from gaining access to data enables it to be treated as a product that is generated and consumed, either internally by employees or externally by partners and customers. This is particularly important for organizations adopting the data mesh approach to data ownership, access and governance. Data mesh is an organizational and cultural approach to data, rather than a technology platform. Nevertheless, multiple vendors are increasingly focused on providing products that facilitate adoption of data mesh and promote data democratization. Amazon Web Services is one such vendor, thanks to the recent launch of Amazon DataZone, one of the figurehead analytics and data announcements made during the company’s recent re:Invent customer event.