In recent years, many enterprises have migrated data platform workloads from on-premises infrastructure to cloud environments, attracted by the promised benefits of greater agility and lower costs. The scale of cloud data platform adoption is illustrated by Ventana Research’s Data Lakes Dynamic Insights research: For two-thirds (66%) of participants, the primary data platform used for analytics is cloud based. As the quantity and importance of the data platform workloads deployed in the cloud increased, so did the number of data analysts and business leaders who rely on accessing and querying data in cloud environments.
As adoption has grown, many enterprises have found that the theoretical advantages of data processing in the cloud can be more challenging to deliver in practice, with constant
Keebo was founded in 2019 by CEO Barzan Mozafari to build a business based on academic research conducted as an associate professor at the University of Michigan. The core emphasis of the company is to combine expertise in machine learning and database systems to automate the optimization of data warehouse environments and improve query performance. The company’s proposition is that while the adoption of cloud data warehouses has helped reduce upfront capital expenditure on data warehouse hardware and software, it has increased ongoing operational expenditure on data engineering staff required to monitor and optimize cloud data warehouse deployments to ensure that they are running efficiently.
Keebo provides robotic process automation software designed to automate the configuration of cloud data warehouse environments to optimize data warehouse performance and resource utilization, adapting in real time to workload changes and requirements. The company also offers a complementary service to monitor and improve the performance of business intelligence queries running against cloud data warehouses. Keebo officially launched the Warehouse Optimization product in October 2022 when it also announced that it had raised $15 million in funding from investors, including True Ventures, Neotribe, Pear, 406 Ventures and Uncorrelated Ventures.
Cloud data warehouse services remove the requirement for upfront hardware and software licensing costs and, in theory, make it far simpler for organizations to scale data warehouses
However, failure to make the most efficient use of cloud infrastructure can result in unexpected costs. Data engineers require a thorough understanding of usage requirements, the ability to accurately predict future needs, the time and expertise to monitor data warehouse environments to ensure efficient operation and the skills to take corrective action to address any inefficiencies. Even if data engineers have all these attributes and are initially successful, operational complexity is likely to grow as more workloads and users are added to the cloud data warehouse environment, resulting in operational overheads that may not have been initially identified and budgeted for.
Keebo’s products are designed to alleviate these challenges. Warehouse Optimization learns from an enterprise’s data warehouse telemetry metadata to detect usage patterns and train learning models to optimize warehouse size, clustering and memory. Users can define automation rules, including default warehouse sizes and expansion limitations, in addition to the desired balance between cost savings and query performance for each warehouse. The product’s user interface enables users to view query performance as well as projected credit savings resulting from the optimization.
Keebo generates revenue by charging customers a percentage of the calculated savings. The company also offers Query Acceleration, which monitors the performance of queries against the data warehouse and uses learning models to automatically detect and rewrite low-performance queries on the fly. The service does not require changes to the warehouse or business intelligence tools and provides a dashboard through which users can set parameters for budget and data freshness requirements.
Keebo is initially focused solely on Snowflake environments but might find it advantageous to expand its addressable market and differentiation by offering similar capabilities for other cloud or on-premises data warehouse or data lakehouse providers. The relationship with Snowflake is currently cordial: Keebo became a member of the Snowflake Partner Network in June 2023, and Snowflake is keen to promote options that enable customers to make efficient use of its cloud data warehouse. As such, Keebo could be an acquisition target for Snowflake. Or, Snowflake could potentially develop or acquire directly competitive capabilities. Keebo is not alone in seeing opportunities for cost containment. Nevertheless, I recommend that enterprises using Snowflake and concerned about cost containment evaluate Keebo and its potential to automate data warehouse optimization and query acceleration.
Regards,
Matt Aslett