Hedge fund analytics platform stabilizes workloads and fuels business growth with migration to AWS infrastructure

Solution

To minimize business disruption, the migration had to take place in 3 months. The roadmap followed AWS best practice recommendations and proprietary Caylent templates for building a secure, high-performing, resilient, and efficient landing zone and hosting environment.

Caylent outlined a multi-environment, multi-account strategy tailored to SEI Novus’ specific needs that would not take unnecessary risks with the company’s production workloads. This new model allowed for user privilege separation at a granular account level to protect high-visibility workloads from errors and external threats while empowering SEI Novus’ staff to do more creative work with fewer blockers. The approach also increased full visibility across the operational infrastructure for better debugging and integration testing purposes in AWS.

Once the critical phases of the migration were completed and BAU operations resumed, Caylent worked with SEI Novus to identify opportunities to introduce cloud native tools and services, this involved some applications being re-architected using containers; environments being templatized and built using Infrastructure-as-Code; cloud cost optimization opportunities such as Spot being leveraged; and standardizing DevOps principles within the teams.

Project Details

Caylent started the project with a Proof-of-Concept (POC) around stabilizing Novus’ Amazon Elastic Kubernetes Workload. We provided Novus with a clear analysis path and course of action that would identify the issue’s root cause and achieve better operational visibility to stabilize the applications running inside the Amazon EKS environments. The team then instantiated monitoring and observability tooling on AWS and Kubernetes, by implementing AWS CloudWatch, Prometheus Operator and RabbitMQ.

Following the POC, Caylent prepared a detailed roadmap to migrate all SEI Novus’ applications and infrastructure, including the company’s core application: Alpha, from their on-premises datacenter to AWS. The roadmap our team provided included best practice recommendations for migrating to cloud architecture, using the AWS Well-Architected Framework for realizing secure, high-performing, resilient, and efficient infrastructure.

In coordination with AWS and SEI Novus, Caylent performed a comprehensive total cost of ownership (TCO) analysis and developed a migration and modernization plan that would meet the business needs of both SEI Novus’s finance and development team: 

1. Shift CapEx costs to OpEx and reduce overall infrastructure spend
2. Improve operational efficiencies and facilitate business expansion
3. Improve developer experience and velocity with DevOps best practices through cloud-based infrastructure and automation

The initial migration work that Caylent performed was focused on designing and implementing a custom multi-account structure Landing Zone using AWS Organizations and access federation. Following the Landing Zone implementation, Caylent worked with Novus in a series of workshops to analyze the primary Alpha application to see how it could be containerized, including reducing dependencies on local files and OS-level services.

Based on the findings, Caylent developed IaC Terraform templates to automate multi-account deployment of container definitions, Dockerfiles and Helm Charts, and leveraged these templates to migrate their Alpha Application. 

During the migration process, SEI Novus’ databases related to their Alpha application were Rehosted from on-prem memSQL to a multi-AZ Amazon Relational Database Service (RDS) MySQL. In addition, Caylent refactored Novus’ Alpha Application to enable containerization of the workload.

Thorough app testing was conducted in the QA/staging account/environment as well as the production account prior to any cutover activities. Caylent performed canary testing over a course of 3 weeks between their legacy environment and new AWS EKS environment. At the time of cutover, Novus’ DNS record was updated to point at the new AWS-hosted environment, with users experiencing minimal to no downtime. 

Results

Caylent laid the groundwork for further Agile development with extensive discovery and knowledge transfer support in the form of walkthroughs and demo sessions to facilitate the company’s future cloud and DevOps initiatives. The adoption of Infrastructure as Code templates has democratized engineering and developer access to AWS tools and empowered the product teams to own and manage their applications through test to production through self-service capabilities.

Novus Partners’ continue to work with Caylent to provide thought leadership around product deployments and assist with scalable DevOps support.

Novus was able to hand over daily firefighting and performance issues, instead concentrating their internal energies on revenue-generating product development. Caylent also implemented the following best practices post-migration to further enhance the cloud footprint:

• Spot fleets which resulted in over 50% cost savings compared to traditional EC2 instances.
• Auto-scaling and elastic infrastructure configuration and optimization.
• A significant AWS cost savings plan to further reduce costs associated with on-demand EC2 instances, Spot instances for the EKS clusters, and reserved RDS instances.
• Monitoring, alerting, and observability enhancements for Alpha Platform on EKS.
• Laid the foundations for a modern, cloud-native disaster recovery strategy and environment.
• Faster application deployment which is now significantly less painful for the Novus engineering team.
• Dramatically improved confidence in deployments, enabling much quicker release cycles.