From Snowflake to Databend: Leading Game Platform replaced Snowflake with Databend Cloud for real-time Data Cloud

Overview

A prominent global gaming platform provides millions of players worldwide with captivating real-time skill-based gaming experiences and seamless interactive gameplay. For this platform, maintaining instantaneous data updates and real-time analytics is critical to enhancing player engagement and retention. However, when leveraging Snowflake for real-time data ingestion and analysis, the platform faced substantial hurdles and escalating costs. To achieve superior efficiency in real-time analytics, they migrated from Snowflake to Databend Cloud, reducing operational costs by 50%, enabling sub-second task execution, and delivering actionable real-time insights.

A leading global gaming enterprise specializes in delivering engaging, skill-based gaming experiences to millions of players worldwide. The company also leads innovation in advertising, payments, and player engagement, developing cutting-edge solutions to facilitate seamless real-time interaction across their popular game titles.

The global game platform previously utilized Amazon Aurora to store real-time customer activities, advertising events, and financial data. Data was exported as JSON files to an S3 bucket and then ingested into Snowflake as a raw table. The data, including player actions, in-game purchases, and matchmaking details, was transformed into multiple refined tables using Snowflake's stream and task features. Accurate, up-to-date data in Snowflake was crucial for enhancing player engagement and retention, ensuring a seamless and responsive gaming experience.

Snowflake's Struggle with Real-Time Data: Cloud Costs Soar

The gaming platform faced critical challenges using Snowflake for real-time data ingestion and analysis. The platform needed to process user data from S3-staged JSON files, generated from Aurora binlogs, in real-time. However, Snowflake struggled with sub-second data ingestion and transformation, crucial for updating raw tables and maintaining a real-time customer activity stream.

Snowflake's pricing model proved impractical for such tasks, as it charged excessive credits for cloud services like tasks. The lack of direct SQL support for sub-second tasks forced the platform to use minute-level tasks with SQL scripts, which drastically increased costs. This inability to handle real-time data led to gaps in their data analysis, hindering their ability to measure customer engagement and advertising performance effectively.

Additionally, despite spending millions on Snowflake, the platform struggled to meet its $$99.9$$ SLA requirement. Even as a high-volume enterprise customer, they received inadequate support. In 2023, a service outage took four hours for a response, which merely stated the issue wasn't Snowflake's fault, without resolution. This inadequate support led to critical delays and affected their ability to maintain high availability.

The platform first discovered Databend Cloud on GitHub and was intrigued by its mission to create an open-source alternative to Snowflake. They were impressed by Databend Cloud's familiar SQL syntax and its powerful streaming and task capabilities. The platform was particularly drawn to Databend's promise of 10x faster performance in data ingestion, transformation, and analytics, along with sub-second task support. Databend Cloud's clear and cost-effective pricing model resonated perfectly with their requirements and goals.

Familiar SQL Syntax from Snowflake

Databend Cloud offers serverless data cloud on AWS with robust SQL support, including full TPC-H and TPC-DS benchmarks. It features transactions, stored procedures, and UDFs in Python/JS, along with support for diverse DML operations like insert, update, and merge into.

On the gaming platform, migrating thousands of SQL queries related to data ingestion, CDC, transformation, and dashboards to Databend Cloud took only three weeks. The seamless transition was possible due to Databend Cloud's compatibility with Snowflake's SQL syntax.

Reliable and Predictable Pricing for Your Data Needs

Databend Cloud separates compute from storage and offers minute-level automatic suspend and resume features, which helps significantly reduce costs. Unlike Snowflake, where task scheduling services could make up to $$50$$ of the total costs and lacked API cost transparency, Databend Cloud provides a more straightforward and predictable pricing model. The gaming platform, which was on Snowflake's enterprise plan with customer-specific discounts, faced high and unpredictable costs, particularly for task scheduling services. Despite the discounts, the lack of transparency in API costs made it challenging to forecast monthly expenses accurately.

With Databend Cloud, they experienced a $$70$$ reduction in costs, thanks to clear and consistent pricing. This transparency allowed the gaming platform to efficiently manage and forecast their monthly data management expenses. They successfully migrated their extensive historical data to Databend Cloud, achieving the ability to handle up to 100 billion rows of data per month with complete cost clarity. This shift has empowered them to provide their customers with comprehensive and real-time business insights without the fear of unexpected financial surprises.

According to the platform’s Head of Technology:"We’re incredibly grateful to Databend for their quick and reliable service. Your product’s performance in streaming data has been outstanding, clearly leading the industry. After a smooth three-week integration, Our platform is excited to have Databend in our production system. Thanks to everyone involved for your hard work. We look forward to seeing Databend grow and achieve even greater things! "

Streamline Your Real-Time Data Pipelines with Ease

Databend Cloud provides powerful streaming and task execution capabilities that deliver superior performance and ease of use for customers. The gaming platform benefits from Databend Cloud by efficiently ingesting large volumes of data into a single raw table. They leverage multiple streams to seamlessly distribute and transform this data using external UDFs for various operational needs.

With Databend Cloud, the platform achieves a steady ingestion rate of 10,000 rows per second and schedules workflows with sub-second task execution, enabling real-time insights. This allows them to process and analyze 100 billion rows per month, significantly enhancing their data operations and driving better business outcomes. The platform now enjoys reliable, efficient real-time data processing, allowing them to focus on delivering exceptional gaming experiences and making data-driven decisions with confidence.

Key Benefits for the Customer

• Effortless migration from Snowflake to Databend Cloud ,thanks to familiar SQL syntax and extensive API support for tasks, streams, and multi-table inserts, dramatically reducing learning curves.
• Achieved true real-time ELT processes with sub-second task execution capabilities, fueling growth in core gaming operations.
• Cost-effective Databend Cloud solutions notably decreased operational expenses, effectively improving cost-efficiency.
• Dedicated, concierge-style technical support services enable the customer to confidently pursue business development and innovation.