SQL Polyglot, our new groundbreaking AI-powered solution, significantly accelerates and simplifies complex database migrations, helping you minimize technical debt. Discover how it can reduce your migration time and costs by automating the translation of stored procedures.
Leveraging our accelerators and technical experience
Browse GenAI Offerings
Learn how to choose between AWS Systems Manager Parameter Store and AWS Secrets Manager for managing sensitive data, by exploring their features, costs, and best use cases based on real-world insights.
Explore how we helped a education-focused technology company modernize their cloud infrastructure and data processes.
Learn how we helped an education technology company with a seamless transition to AWS, delivering high availability, disaster recovery, cost savings, compliance, and improved visibility for the customer's network infrastructure.
Many companies are eager to reduce technical debt and cut licensing costs by migrating away from their current, expensive database systems. However, the technical challenges along with the financial and opportunity costs of such migrations often make them prohibitively expensive. After completing hundreds of migrations, our team decided there had to be a better way. The result of our efforts to simplify the migration process is SQL Polyglot. This groundbreaking generative AI solution makes previously fiscally impossible database migrations possible and cost-effective.
We'll explore the challenges we’ve tackled head-on as part of our comprehensive data and analytics services and how SQL Polyglot was purpose-built to overcome these obstacles.
It's no secret that database licensing costs are a significant burden for many companies. According to The Business Research Company, organizations spend nearly $200 billion annually on database licenses. With such astronomical figures, it's no wonder that a primary motivator for database migration is to reduce these licensing costs. For example, many companies are looking to move away from expensive proprietary systems like SQL Server or Oracle to open-source alternatives like PostgreSQL, which is free.
However, cost reduction isn't the only driving factor. Another compelling reason to migrate is the opportunity to shed technical debt associated with existing implementations. A report by Quod Orbis reveals that technical debt costs organizations over $2 trillion annually. This accumulation of outdated or suboptimal database structures and practices can significantly hinder an organization's efficiency and agility.
Migrating to a new, faster, and less expensive database can be a crucial step in application modernization, allowing companies to improve performance, reduce costs, and enhance security simultaneously.
While the benefits of database migration are clear, the process is far from simple. According to a Bloor Research study, 40% of migrations are delayed due to undiscovered complexity. The planning, design, coding, and testing required can extend the process to a year or more, making it a daunting undertaking for many organizations. This lengthy timeline and the technical intricacies often deter companies from embarking on much-needed migrations without expert guidance. That's why many organizations turn to experienced cloud migration services to navigate the following challenges effectively:
One of the most significant hurdles in database migration is the complexity of the existing logic. Database logic is often spread across stored procedures, functions, and views. Converting all of this logic to work with the new database's capabilities is generally a heavy lift, requiring extensive expertise and time.
Database systems use different syntax and languages for stored procedures and slightly different SQL dialects. While some tools, such as the AWS Schema Conversion Tool (SCT), can automate part of the job of porting, a significant portion still requires intervention by skilled experts.
Due to the technical complexity of database migrations, rigorous and thorough testing is essential. This is especially true if you're taking the opportunity to fix issues along the way (such as type inconsistencies, data formatting problems, or schema issues) to reduce technical debt. Ensuring all data is correctly transferred and all functions work as expected in the new environment is a time-consuming but critical process. This task is extraordinarily time consuming for people to execute and track, but is highly amenable to automation.
Successful conversions require data engineering teams to have in-depth knowledge of both source and destination systems - a rare combination of skills. This dual expertise goes beyond understanding different SQL dialects. It includes grasping the source system's intricacies, the destination system's capabilities, and how to bridge the gap between them.
The knowledge gap challenge manifests differently for in-house teams and external consultants. In-house Subject Matter Experts (SMEs) often need extensive upskilling in the target system and migration tools, extending project timelines. Consultants, while bringing broad expertise, may lack specific organizational knowledge, potentially increasing costs. Organizations must balance these factors, possibly combining targeted in-house training with strategic use of consultants to manage both time and budget constraints effectively.
Database objects like stored procedures don't exist in isolation. They interact with other database objects, such as views and functions, which might also need modification during the migration process. These interdependencies can significantly complicate the migration process and increase the risk of overlooking critical components.
At Caylent, we've accumulated extensive experience in Generative AI and database migrations, having completed hundreds of projects across multiple industries, including Financial Services, Healthcare, Government, and Media/Entertainment. Through these experiences, we've come to several important conclusions:
More specifically, we've found that GenAI excels at converting stored procedures, functions, and views between database engines, and these results have only improved over time as models like Claude 3/3.5 have iterated. This trend suggests we expect even further improvements in the future, making GenAI an increasingly powerful tool for database migrations.
Building on our experiences and insights, we developed SQL Polyglot, a revolutionary tool that creates the framework to translate stored procedures between databases and test the results. We've incorporated our extensive experience with database migrations into the prompts and orchestration, leveraging cutting-edge AWS services to power its AI-driven migration capabilities.
At the heart of SQL Polyglot are LLMs, hosted on Amazon Bedrock, which enable us to continuously improve and deploy our AI models. Bedrock provides the robust machine learning infrastructure we need to deploy Open Source models at scale. It allows us to efficiently manage the entire machine learning lifecycle, from data preparation to model deployment and monitoring. This means we can rapidly iterate on our AI models, incorporating new learnings and improvements as we gather more data from successful migrations.
Amazon Bedrock offers a selection of high-performing foundation models, upon which we have developed a configurable chain of prompts for our use case of database migration. By using Bedrock, we ensure that SQL Polyglot is always powered by state-of-the-art language models, capable of understanding and translating complex database schemas and stored procedures across different SQL dialects.
This powerful combination allows us to continuously improve our AI models based on real-world migration data, rapidly deploy updated models without service interruption, scale our AI capabilities to handle migrations of any size, and ensure high accuracy and reliability in our automated translations. By harnessing these AWS services, SQL Polyglot stays at the forefront of AI-powered database migration technology, offering our clients the most advanced and efficient migration solution available.
Essential to the success of SQL Polyglot translation is the validation of the stored procedures on the target platform. By first creating the database on the target platform, then creating the procedures on the new database as they are validated, we enable the parallel execution of the procedure on both the source and target. To aid in this process, we have developed an automated data validation process that covers scenarios for the use of the procedures.
For more examples of Caylent customers who have found success, check out our case studies page!
We've focused on tuning the system to perform SQL Server to PostgreSQL migrations, and the results have been remarkable. Let's compare the traditional migration process for a database with 2,500 Stored Procedures with the SQL Polyglot-powered approach:
As you can see, SQL Polyglot dramatically reduces the time required to convert stored procedures - a critical and time-consuming part of database migrations. In a traditional process, human engineers would need to manually convert high-complexity stored procedures, which accounts for most of the migration time. SQL Polyglot, however, leverages GenAI to convert these high-complexity stored procedures with an 80% success rate, leaving only 20% for human intervention.
This approach reduces the time for stored procedure conversions by 75%, from 40 weeks to just 10 weeks. Additionally, unit testing time is cut by 85%, thanks to our AI-generated test stubs and automated testing system.
The impact on the overall migration process is substantial:
Normal migration process: Nearly a year
With SQL Polyglot: 3-4 months (up to 4x faster)
What would typically be a year-long migration process is compressed into just 3-4 months. By accelerating the most time-consuming aspect – stored procedure conversion – SQL Polyglot makes previously infeasible migrations not only possible but also cost-effective. This dramatic reduction in time and effort opens up new possibilities for organizations looking to modernize their database infrastructure.
To illustrate the transformative power of SQL Polyglot, let's look at a real-world example. A large SaaS company had built up a significant footprint of databases on Microsoft SQL Server over the years. Their SaaS platform had accumulated thousands of stored procedures over a decade. The complexity of their environment led the team to create a new database for each customer, resulting in massive licensing costs.
Late last year, Caylent conducted a comprehensive assessment of the client's database environment and estimated the cost of manually converting code to move from SQL Server to PostgreSQL. While the client was frustrated with the licensing costs, the projected cost of migrating to an open-source database needed to be lowered for their budget.
This is where SQL Polyglot came to the rescue. By leveraging our tool, we could automate the conversion of 90% of the stored procedures on the first pass, with only approximately 250 requiring manual intervention. Working from a replica database, the Caylent team used Generative AI to convert stored procedures and functions, test them against the source database's live data, correct errors, and iterate until they worked correctly.
The result? A migration that would have been prohibitively expensive and time-consuming was completed in just a quarter of the time it would have taken using traditional methods. This made the migration feasible, allowing the client to realize significant cost savings and reduce technical debt.
You can now unlock database migrations that you might have previously thought were financially blocked thanks to Caylent’s Database Evolution Catalyst powered by SQL Polyglot.
SQL Polyglot is transforming the landscape of database migrations, making it possible for organizations of all sizes to modernize their data infrastructure and stay competitive in today's fast-paced digital world. The results speak for themselves:
Don't let the complexity of database migrations prevent your organization from achieving its goals. Reach out to learn more, and let us help you take the first step toward a faster, more efficient database migration process!
Jorge Goldman is an Engineering Manager with over 12 years of experience in diverse areas from SRE to Data Science. Jorge is passionate about Big Data problems in the real world. He graduated with a Bachelors degree in Software Engineering and a Masters degree in Petroleum Engineering and Data Science. He is always looking for opportunities to improve existing architectures with new technologies. His mission is to deliver sophisticated technical solutions without compromising quality nor security. He enjoys contributing to the community through open-source projects, articles, and lectures, and loves to guide Caylent's customers through challenging problems.
View Jorge's articles
Ryan Gross leads Cloud Data/AI/ML delivery at Caylent. Through his 15+ years of experience, Ryan has guided over 50 clients in building tech-driven data and AI cultures across various industries. By identifying technology trends, and leading the development of asset backed consulting offerings to realize value, he builds a growth culture within his team. Ryan is also a frequent conference speaker on emerging data and AI trends.
View Ryan's articles
Caylent Catalysts™
Accelerate Your Database Modernization with AI-Driven Migrations
Caylent Catalysts™
From implementing data lakes & migrating off commercial databases to optimizing data flows between systems, turn your data into insights with AWS cloud native data services.
Caylent Services
From implementing data lakes and migrating off commercial databases to optimizing data flows between systems, turn your data into insights with AWS cloud native data services.
Jorge Goldman
Ryan Gross