Caylent Services
Data Modernization & Analytics
From implementing data lakes and migrating off commercial databases to optimizing data flows between systems, turn your data into insights with AWS cloud native data services.
Innovative Ticket Resale Platform Enhances Data Efficiency and Analytics with a Data Lakehouse on AWS
Migrations
Data Modernization & Analytics
Solution Implemented
→ Established an AWS Data Lakehouse architecture for migrating current datastores and workflows from GCP BigQuery to a highly scalable & cost-effective implementation leveraging Redshift.
→ Modernized ETL and MLOps pipelines by improving and standardizing Kubernetes-based Airflow ingestion jobs and incorporating AWS Glue for datastore processing and metadata.
Outcomes Expected
→ Cost Savings: Automatiq is expecting to see a 30%+ reduction in operational costs due to more predictable and optimized resource usage on AWS Redshift vs. GCP BigQuery.
→ Performance Improvements: Automatiq is forecasting data ingestion speeds to increase around processing using AWS Glue and Airflow batch processing vs. legacy ETL pipelines.
→ Scalability Gains: Automatiq’s next-gen Data Lakehouse platform can seamlessly support future growth, product launches, scalability, and increased volume transactionality without performance degradation or latency issues.
Pioneering Innovation in Secondary Ticket Market Automation
Automatiq is a mission-critical technology leader that enables resellers to leverage best-of-breed technologies across the live event ticketing ecosystem. As the only end-to-end solution in the secondary ticket market, Automatiq integrates dynamic pricing, processing, consignment, distribution, and analytics, offering unmatched value to both professional resellers and individual users.
Operating within the $18+ billion secondary ticket market, which is growing at a 10%+ annual rate, Automatiq integrates with all major marketplaces, including StubHub, Vivid Seats, Ticketmaster, Seatgeek, TicketNetwork, Gametime, TickPick, Ticket Evolution, AXS Resale, and more. Automatiq has over 100k+ clients who do over $5BN of ticket sales annually.
How Caylent Delivered Impact
Pursuing opportunities to improve costs and efficiencies with its existing GCP-based data infrastructure, Automatiq partnered with Caylent to future-proof its enterprise data platform on AWS. The collaboration addressed critical challenges, including the need for better scalability, cost predictability, and streamlined data workflows for its products.
Through a tailored AWS Data Lakehouse solution, Caylent provided Automatiq with a unified, governed data architecture capable of handling structured, semi-structured, and unstructured data at scale. This modernization not only resolved legacy inefficiencies but also laid the foundation for Automatiq’s future ambitions.
Problem
Automatiq identified key opportunities to enhance its existing data infrastructure, which was hosted on GCP BigQuery. While BigQuery provided robust capabilities, the platform’s unpredictable costs made it difficult to optimize budget planning and enable greater resource flexibility for future growth. Automatiq also wanted to accelerate data analytics, ML/AI workflows, and create a richer experience for customers of its flagship products.
They recognized that introducing a more structured data model could simplify workflows and improve the stability of business intelligence and analytical reporting.
As Automatiq continued to grow, its Kubernetes-based data ingestion system, which relied on Airflow and Python routines to pull data from various sources like REST APIs and SQL/NoSQL databases, also struggled to scale efficiently. This presented a need to modernize workflows and ensure scalability to meet the demands of their rapidly growing market and customer base.
By migrating to AWS, Automatiq aimed to unlock these opportunities, transitioning to a cost-efficient, scalable data architecture that could support even more advanced analytics, real-time decision-making, and even further investment into AI/ML initiatives.
“Caylent’s AWS Data Lakehouse solution has been transformative for future-proofing our data evolution strategy at Automatiq. The data migration was clean, and the new governance architecture has streamlined our MLOps workflows, improved datastore query performance and efficiency, while pushing the envelope of innovation even faster. Their expertise and collaborative approach made this critical project an immediate success.”
George Thomas
CTO
Solution
To address Automatiq’s goals, Caylent designed and implemented a tailored AWS Data Lakehouse architecture, leveraging insights gathered during discovery sessions about the company’s use cases and existing workflows. This solution introduced a scalable, governed data platform that streamlines operations, improves efficiency, and lays a strong foundation for future growth.
Key Components of the Solution
1. AWS Data Lakehouse
A centralized data repository was established to store raw, structured, semi-structured, and unstructured data. This allows Automatiq to retain data in its original form and process it on demand for specific use cases, such as analytics, reporting, AI/ML models, and real-time decision-making.
2. AWS Redshift
Data models were optimized for analytical queries and integrated with Automatiq’s business intelligence platform. This significantly reduced query complexity and improved stability, ensuring seamless data performance for Automatiq’s product portfolio.
3. ETL Workflow Modernization
Automatiq’s existing Kubernetes-based ingestion system running Airflow was optimized and standardized, incorporating AWS Glue for data processing to streamline workflows and enhance efficiency.
Implementation
Caylent executed a seamless data migration, prioritizing 44 tables critical to Automatiq’s flagship products. The migration was carried out ensuring business continuity throughout the transition.
As part of the implementation, Caylent developed tailored data models to streamline workflows and improve query stability. These models optimized data organization, data format and reduced complexity, resolving previous challenges with data loads. Automatiq’s new infrastructure was designed to handle growing data demands while maintaining predictable performance.
Ensuring Long Term Success
Caylent provided thorough knowledge transfer, delivering comprehensive documentation and training for Automatiq’s ever-growing team. This enabled Automatiq to independently manage and scale its new AWS data infrastructure into the future.
