Learn how to build GenAI recommendation systems using AWS Bedrock and Amazon Personalize. Discover use cases, architecture, and hybrid approaches for better user experiences.
Generative AI has a broad variety of use cases and this is precisely why it is being adopted widely and is rapidly transforming many sectors. However, its application in many areas remains unclear.
We have worked with numerous customers across industries like retail, SaaS, finance and more to determine avenues to generate value with Generative AI. In some circumstances, Generative AI can even augment data analytics capabilities that are generally approached with traditional AI and machine learning technologies.
We will explore how GenAI enhances traditional recommendation engines, and provide an overview of how to build a GenAI recommendation engine with AWS Bedrock and Amazon Personalize.
A GenAI recommendation engine is an advanced approach to personalized recommendations that leverages generative artificial intelligence to create tailored suggestions for users. Unlike traditional recommendation engines, which primarily rely on historical data and predefined patterns, GenAI recommendation engines can generate new, context-aware recommendations by understanding and interpreting complex user preferences and behaviors.
These systems work by combining large language models (LLMs) with traditional recommendation algorithms. The GenAI component analyzes user data, preferences, and contextual information to generate personalized content, product descriptions, or even entirely new recommendations that might not exist in the current inventory. This approach allows for more nuanced, creative, and potentially more relevant suggestions to users
Traditional recommendation engines rely on analyzing user behavior and item characteristics to predict preferences. They might use algorithms that focus on past interactions, like purchases or views, to suggest similar items or those that other similar users have liked.
However, GenAI introduces a new paradigm by generating content or item features based on understanding the data context. While GenAI shows promising possibilities, it's important to balance the advantages against the costs. In many cases, traditional methods powered by services like Amazon Personalize may still be the best choices due to their scalability and cost-effectiveness. Yet, the integration of GenAI can offer unparalleled advantages in creating more personalized user experiences.
In this post we’re focused on how to build a recommendation engine using AWS Bedrock and Amazon Personalize.
We’ll pick apart our process in a case study later on in the post. But let’s start with an overview of the process.
Gather and import user, item, and interaction data into Amazon Personalize. While only interaction data is required, including user and item metadata will improve recommendation quality.
Use Amazon Personalize to train a recommender, such as the “Top picks for you” model. The service automatically configures the best model for your use case and filters out content users have already interacted with.
Query the trained recommender using the Amazon Personalize Runtime API, providing user IDs to receive top recommendations for each user.
Combine the recommended items and user demographic data with a predefined prompt template. This template will guide the generative model in crafting personalized content (e.g., an email or marketing message).
Send the enhanced prompt to a foundation model in Amazon Bedrock via its API. Bedrock will generate personalized outbound communication based on the recommendations and user data.
Use your preferred channel (email, app notification, etc.) to send the generated personalized message to each user.
GenAI recommendation engines offer innovative solutions across various industries. Here are some top use cases:
In e-commerce, GenAI can create highly personalized product recommendations by generating detailed product descriptions tailored to individual user preferences. It can also suggest product bundles or combinations that may not exist in the catalog but are likely to appeal to specific customers based on their browsing and purchase history.
For content platforms like news sites or blogs, GenAI can generate personalized article summaries, headlines, or even entire articles based on a user's reading history and interests. This can significantly enhance user engagement and time spent on the platform.
In the streaming entertainment industry, GenAI recommendation engines can significantly enhance the user experience. These systems can analyze a user's viewing history, preferences, and even contextual factors like time of day or current events to generate highly personalized recommendations.
For example, a GenAI system could create custom content descriptions that highlight aspects of a show or movie most likely to appeal to a specific user. It might also suggest unique combinations of genres or themes based on a user's eclectic tastes, potentially recommending content that traditional algorithms might overlook.
In travel and hospitality, GenAI can generate personalized itineraries or activity recommendations based on a user's preferences, past travels, and current location. It can create detailed descriptions of attractions or restaurants that cater to the user's specific interests and dietary requirements.
Using GenAI spikes the cost and diminishes the performance when used as the recommendation engine. We can either get data for each customer to personalize the promotions and prompt it to get a recommendation. Or we can get a chunk of data, but then we lose the personalization power. Both performance and cost are not reasonable.
Therefore, using Amazon Personalize, the fully managed machine learning service that uses the same data to generate item recommendations, is much more suitable. We combine it with GenAI by getting Personalize's recommendations and crafting content to present to the client, enriching the personalization experience. A high-level solution architecture diagram for using Personalize with GenAI capabilities is shown below.
Figure 2: Recommended architecture for recommendation systems
Amazon Personalize is a fully managed machine learning service provided by Amazon Web Services (AWS). It enables developers to create and deploy personalized recommendations for applications without requiring extensive machine learning expertise.
Amazon Personalize is used to build recommendation engines for various applications, including:
The service uses machine learning algorithms to analyze user behavior data and generate personalized recommendations in real-time. It can handle various types of data, including user interactions, item metadata, and user attributes, to create tailored experiences for each user.
Amazon Personalize now offers the Content Generator, a Generative AI feature designed to elevate traditional suggestions into dynamic, thematic narratives. This enhancement transforms traditional recommendations into captivating themed experiences that significantly boost user engagement and sales.
These themes can be particularly effective in promotional contexts, such as a merchandise sales campaign themed ‘Summer Adventures’ for outdoor gear and accessories, enhancing user engagement and improving the specificity of promotions.
AWS Bedrock is a fully managed service that provides access to high-performance foundation models (FMs) from leading AI companies through a single API. It offers a comprehensive suite of tools and capabilities for building and scaling generative AI applications.
AWS Bedrock is used for:
By integrating AWS Bedrock with other AWS services, developers can create powerful, scalable, and secure generative AI applications that leverage the latest advancements in language models and AI technology.
The architecture for integrating Amazon Personalize with GenAI via AWS Bedrock involves several key components working in harmony:
Let’s look at a project we conducted with a confidential client, in which we developed a proof of concept (PoC) to test the use of GenAI in recommendation engines. Spoiler alert: the concept was actually unproven, and surprisingly, that was great news.
In the current digital marketplace, the demand for personalized customer experiences has never been higher. So, the client wanted to adopt GenAI as the core of their recommendation platform to find new data patterns, improve recommendation quality, and develop new product offerings, as traditional promotional strategies often lack the targeted engagement necessary for optimizing conversion rates. This gap between customer expectations and the actual shopping experience highlights a critical need for innovation in how promotions are crafted and delivered.
The objective was to leverage AI to enhance the personalization of promotional content. By doing so, the client aimed to:
To achieve the business objectives given the context above, we started a PoC to personalize promotions for each client. The PoC goals were:
We relied on the following datasets:
Also, we used pre-trained LLMs from Amazon Bedrock, enabling us to jumpstart the personalization process. The development was divided into five phases:
The initial phase involved filtering data, selecting relevant columns, and merging tables to create a unified dataset that was ready for analysis and model training.
Amazon S3 hosted the compressed CSV files containing the data tables for customers, visits, transactions, and promotions, while SageMaker Studio Notebook accessed the data from S3, performed necessary preprocessing, and prepared the data for prompt engineering.
We added context using demographics and past transaction data, enriching the model's ability to generate personalized content.
We re-coded demographics to ensure the model would interpret them accurately and converted the aggregated data from the previous step into a natural language format that Claude v2.1 large language model (LLM) could process. Then, the LLM generated the top-k recommendations.
We set up the Claude v2.1 model in the SageMaker AI environment via the Bedrock API, to run inferences using the prompt engineered in the previous step. Utilizing Claude v2.1, we did multi-step queries to dive deep into customer individuality.
Finally, we had a model inference feeding the formatted customer data into the LLM to generate promotion recommendations.
For example, the model generates: "For customer 123, offer a 20-off promo code which gives them a 20% discount on electronics, valid for the next two weeks."
Then, we post-process it, converting the model's natural language output into a structured format for practical use. For example, parsing the model's text output to extract promotion details and map them to existing offer IDs in the database.
The final step is setting up metrics to measure the effectiveness of the recommendations (e.g., conversion rate, customer engagement, ROI, etc). We recommended testing and Iterating. This means, running tests with a control group and the recommended promotions to evaluate the performance. Based on the results, refine the model.
Figure 1: PoC Architecture
We concluded that using GenAI for recommendation engines is costly at the production scale and has limited context processing capabilities.
However, despite disproving the initial concept, we could achieve both the PoC goals by providing the client with an alternative architecture and validating that GenAI can indeed optimize personalization with the appropriate approach.
Through a small-scale implementation, we were able to pivot the architecture towards a more suitable and scalable approach, thereby avoiding investment in an unsustainable solution.
Our final suggestion was to combine GenAI while relying on Amazon Personalize for recommendation infrastructure to overcome the aforementioned challenges. And we will describe that below.
Are you exploring ways to take advantage of Analytical or Generative AI in your organization? Partnered with AWS, Caylent's data experts have extensively helped organizations implement AI solutions with a focus on strategies that generate real ROI. Get in touch with our team to learn how we can accelerate your AI endeavors.
Caylent can help update your innovation processes for recommendations using our AI Innovation Engine. Caylent’s AI Innovation Engine is an embedded, agile team that accelerates both short- and long-term generative AI initiatives. With experience delivering hundreds of AI solutions, we know how to turn ideas into production-ready results.
Too often, AI experiments stall before they create value. Without operational rigor, feasibility validation, and alignment to business goals, organizations face high costs, wasted effort, and unrealized potential. We change that by working backwards from your business priorities. Our team validates technical and business feasibility, then operationalizes proven AI through a continuous innovation framework. As your needs evolve, so does our team’s skill mix.
Generative AI can create more nuanced and creative recommendations. For example, it can generate unique product bundles or personalized content summaries. This leads to richer user experiences and higher engagement.
Caylent has the expertise to develop an AWS implementation plan for AI systems, with a strong focus on recommendation applications. We use Amazon Personalize to generate real-time, personalized recommendations. Then, we enhance these recommendations with AWS Bedrock’s generative AI capabilities.
This hybrid approach gives you the best of both worlds. Amazon Personalize ensures recommendations are accurate and scalable. Generative AI enriches these recommendations with tailored descriptions or themed promotions, making each user’s experience feel unique.
A GenAI recommendation engine creates tailored suggestions for users. Unlike traditional recommendation engines that rely on historical data and predefined patterns, GenAI systems can generate new, context-aware recommendations by understanding complex user preferences and behaviors.
These systems combine large language models (LLMs) with traditional recommendation algorithms to analyze user data and generate personalized content or entirely new recommendations that might not exist in the current inventory.
Traditional recommendation engines analyze user behavior and item characteristics to predict preferences based on past interactions. GenAI systems, however, introduce a new paradigm by generating content or item features based on understanding the data context.
While traditional methods powered by services like Amazon Personalize are often more scalable and cost-effective, GenAI can create more personalized user experiences. For example, GenAI can generate detailed product descriptions tailored to individual preferences, while traditional systems typically match existing items to user profiles.
GenAI recommendation engines can be applied across various industries, including e-commerce for personalized product recommendations and bundle suggestions, content platforms for customized article summaries or headlines, and streaming entertainment for tailored content descriptions.
They're also valuable for location-based recommendations in travel and hospitality, where they can generate personalized itineraries or activity recommendations based on user preferences and current location. For example, in streaming services, a GenAI system could highlight specific aspects of a show most likely to appeal to a particular viewer based on their unique taste profile.
A practical approach to building GenAI recommendation engines combines Amazon Personalize for the core recommendation engine with AWS Bedrock for content generation. This architecture uses Personalize's machine learning capabilities to generate personalized item recommendations based on user behavior data, then enhances these recommendations with GenAI-generated content using Bedrock's foundation models.
For example, Personalize might recommend specific products, while Bedrock generates engaging, personalized descriptions or creates themed promotional narratives around those recommendations, such as a "Summer Adventures" campaign for outdoor gear.
Cost and performance trade-offs present significant challenges. Using GenAI alone for recommendations can spike costs and diminish performance compared to traditional methods. Processing individual customer data for personalized recommendations through GenAI is resource-intensive, while batch processing sacrifices personalization power.
The recommended approach is a hybrid architecture that uses traditional recommendation engines like Amazon Personalize for the core functionality while leveraging GenAI for content enhancement and presentation.
Amazon Personalize provides several built-in algorithms (recipes) optimized for various recommendation scenarios with customizable parameters.. For even greater flexibility, AWS SageMaker allows you to build and train fully custom recommendation models using frameworks like TensorFlow or PyTorch, giving you access to model internals and weights.
Leveraging our accelerators and technical experience
Browse GenAI Offerings
As a Cloud Data Engineering Manager at Caylent, Danilo Figueiredo has been leading several different engagements in the data realm, from database migration to generative AI, ensuring they align with Caylent's standards. He also mentors architects and engineers and collaborates closely with Project Managers during the projects and with the pre-sales team during the proposal phase. In his free time, Danilo enjoys engaging in group activities with friends and family, such as games, travel, and culinary experiences, drawing energy from these shared moments.
View Danilo's articles
Sankalp is an ML Architect with a passion for building impactful machine learning solutions on AWS. He focuses on creating scalable, robust models that drive business growth and enhance operational efficiency across various industries.
View 's articles
AI agents represent the next evolution of APIs, but they also bring new security challenges and attack vectors. Examine real-world adversarial threats and learn defensive strategies in this blog.
Discover hard-earned lessons we've learned from over 200 enterprise GenAI deployments and what it really takes to move from POC to production at scale.
Explore hard-earned lessons we've learned from 200+ enterprise GenAI deployments.
Danilo Figueiredo
Sankalp Prabhakar