IdenX, an analytics-as-a-service company, leverages proprietary machine learning and artificial intelligence technology to deliver customized, data-driven insights across various organizational challenges in talent acquisition, logistics, mergers and acquisitions, and more. Its analytics provide clients with comprehensive and transparent insights quickly for strategic decision-making.
IdenX employs a meticulous approach to research and data analysis that can be time and resource-intensive. This included querying thousands of files and datasets, converting text to SQL, assessing data quality, and analyzing data, which involved repetitive tasks that significantly slowed operational efficiency. IdenX sought a solution to streamline these processes, lower the technical barrier to entry for their operational and analytics teams, and democratize access to actionable data across the organization.
In collaboration with Caylent, IdenX decided to utilize a Generative AI-powered application leveraging Claude 2, to address these challenges. The solution involves:
Data Processing Pipeline: IdenX’s solution utilizes a Large Language Model (LLM) to identify metadata such as delimiter, encoding, and headers across files of various sizes, formats, and languages. This approach aims to improve target identification and calculate a density quotient to assess the value of parsing each file. This accelerates the qualification process, ensuring IdenX’s experts only focus on files that offer value.
Amazon Bedrock Implementation: After evaluating Amazon Sagemaker AI, Amazon Textract, and AWS Comprehend, the teams chose Amazon Bedrock for its GenAI capabilities. Caylent developed scripts to extract metadata from files and measure density, streamlining the data processing workflow.
Cost-Effective Strategies: Utilizing samples from files for prompts and employing an on-demand pricing model for Bedrock allowed initial development within free tiers, optimizing costs.
The deployment of the Amazon Bedrock-powered solution enabled IdenX to:
Enhance Efficiency: Query thousands of files instantly, significantly improving the efficiency of time and resources compared to the manual processing of approximately 500 files per day per resource.
Accelerate Processing: Achieve processing times of around 25 seconds for small files and 40 seconds for wider files, with the capability to process about 20-30 files within a 15-minute window, depending on file complexity.
Seamless Integration: Easily integrate the solution with other services, further enhancing operational efficiency and data analysis capabilities.
Khobaib Zaamout
