Services for Organizations

Using our research, best practices and expertise, we help you understand how to optimize your business processes using applications, information and technology. We provide advisory, education, and assessment services to rapidly identify and prioritize areas for improvement and perform vendor selection

Consulting & Strategy Sessions

Ventana On Demand

    Services for Investment Firms

    We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

    Consulting & Strategy Sessions

    Ventana On Demand

      Services for Technology Vendors

      We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

      Analyst Relations

      Demand Generation

      Product Marketing

      Market Coverage

      Request a Briefing



        Matt Aslett's Analyst Perspectives

        << Back to Blog Index

        Streaming Data Success Relies on Managing Data in Motion and At Rest

        I recently noted that as demand for real-time interactive applications becomes more pervasive, the use of streaming data is becoming more mainstream. Streaming data and event processing has been part of the data landscape for many decades, but for much of that time, data streaming was a niche activity. Although adopted in industry segments with high-performance, real-time data processing and analytics requirements such as financial services and telecommunications, data streaming was far less common elsewhere. That has changed significantly in recent years, fueled by the proliferation of open-source and cloud-based streaming data and event technologies that have lowered the cost and technical barriers to developing new applications able to take advantage of data in-motion. This is a trend we expect to continue, to the extent that streaming data and event processing becomes an integral part of mainstream data-processing architectures.

        Data from Ventana Research’s Analytics and Data Benchmark Research highlights both the growing use of streaming data and the reasons for that growth. Currently, less than one-quarter (21%) of organizations are in production with event-streaming technologies, with a further 16% planning to, and 18% still evaluating. Organizations utilizing event-streaming technologies have greater confidence in their ability to cope with the growing volume, variety and velocity of data. Almost three-quarters (73%) of organizations in production with event-streaming technologies are confident in their ability to analyze large volumes of data compared to 49% of those that are not using these technologies. Nearly nine-tenths (86%) of organizations in production with event-streaming technologies are confident in their ability to analyze the variety of data needed to make informed business decisions, compared to 65% of those that have not adopted event-streaming technologies. And almost two-thirds (64%) of organizations are confident in their ability to analyze high-velocity data while less than one-half (43%) of organizations who do not utilize event-streaming technologies are not. Ventana Research’s Streaming Data Dynamic Insights enables organizations to assess their relative maturity in achieving value from streaming data.

        One reason the adoption of streaming data may be highly impactful is that it can be applied to both operational and analytic data workloads and business processes. To date, a slightly higher proportion of organizations have adopted streaming data for operational workloads. Adoption for analytic workloads is growing, which should soon equate to greater association with the generation of business intelligence and delivery of business value. Almost one-half (48%) of participants in Ventana Research’s Analytics and Data Benchmark Research are at organizations that use streaming data in operational processes, while 44% use streaming data in analytics processes. The use of streaming data in analytics processes is expected to accelerate, however, with 16% of organizations planning to use streaming data in analytics processes in the next 12 months, compared with 13% of organizations planning to use streaming data in operational processes in the same time frame. Streaming analytics — the use of technology to analyze information as it is generated in order to respond to opportunities or threats with timely actions — is becoming more mainstream, driven by the desire of organizations to increase the frequency of analysis to deliver more real-time decision-making, and the eagerness of business intelligence and visualization vendors to incorporate support for streaming analytics. My colleague, Dave Menninger, asserts that by 2025, 9 in ten business intelligence platforms will incorporate streaming analytics to mix historical analyses with real-time or near-real-time analyses.

        Support for streaming analytics and real-time data processing is still a work in progress. Fewer than six in 10 (56%) organizations consider their analytics technologies for real-time analysis to be adequate, while less than one-half (49%) of organizations consider their technologies for event streaming to be adequate. Ongoing questions over the adequacy of event-streaming technology can be attributed the relative immaturity of products and services, as well as the historic tendency of organizations to deploy them for standalone, niche workloads. While vendors have focused on developing and refining core stream data processing and analytics functionality, there has been less focus on the management and governance of data in-motion alongside data at rest. This is changing, however. Established data management vendors added streaming and events technologies to their portfolios, and streaming and event specialists have added centralized management and governance capabilities — including data cataloging — to facilitate the discovery and governance of distributed data. Significantly, that means not just managing and governing streaming data in isolation, but embracing real-time data flows as the continuous generalization of batch data processes so that data in-motion and data at rest can be managed and governed holistically.

        For this to become a reality, the capabilities that support the holistic management and governance of data in-motion and data at rest must be made available by the streaming data and event vendors, and data governance specialists. Almost two-fifths (39%) of participants in Ventana Research’s Data Governance Benchmark Research stated that a shortcoming of existing technologies for data governance is a failure to keep up with new data techniques and technologies (such as search, semantics, lineage and streaming data). I assert that by 2024, more than one-half of all organizations’ standard information architectures will include streaming data and event processing, allowing organizations to be more responsive and provide better customer experiences. I recommend that all organizations that have not already done so investigate the potential opportunities for streaming data and analytics in their organization, while those that have already adopted streaming data for niche workloads examine the potential for a more holistic approach to managing and governing data in-motion and data at rest.

        Regards,

        Matt Aslett

        Authors:

        Matt Aslett
        Director of Research, Analytics and Data

        Matt Aslett leads the software research and advisory for Analytics and Data at Ventana Research, now part of ISG, covering software that improves the utilization and value of information. His focus areas of expertise and market coverage include analytics, data intelligence, data operations, data platforms, and streaming and events.

        JOIN OUR COMMUNITY

        Our Analyst Perspective Policy

        • Ventana Research’s Analyst Perspectives are fact-based analysis and guidance on business, industry and technology vendor trends. Each Analyst Perspective presents the view of the analyst who is an established subject matter expert on new developments, business and technology trends, findings from our research, or best practice insights.

          Each is prepared and reviewed in accordance with Ventana Research’s strict standards for accuracy and objectivity and reviewed to ensure it delivers reliable and actionable insights. It is reviewed and edited by research management and is approved by the Chief Research Officer; no individual or organization outside of Ventana Research reviews any Analyst Perspective before it is published. If you have any issue with an Analyst Perspective, please email them to ChiefResearchOfficer@ventanaresearch.com

        View Policy

        Our Analysts

        Subscribe to Email Updates



        Analyst Perspectives Archive

        See All