Skip to content

Commit 6723c83

Browse files
author
Raul Cajias
committed
feat: add Nexus Gateway integration for expanded model access
This commit implements Nexus Gateway integration, allowing the chatbot to connect to additional AI models through a centralized gateway service. Key additions include: 1. NexusGatewayClient for authentication and API communication: - OAuth token management with caching - Secure API request handling - Comprehensive error handling 2. NexusModelProvider implementation: - Model discovery through the gateway - Transformation of Nexus models to application format - Support for both chat and embedding models 3. Fixed adapter structure issues: - Added missing types.py with ModelProvider abstract base class - Created direct and nexus provider implementations - Fixed import errors with proper directory structure 4. Added comprehensive test coverage: - Unit tests for client authentication and API calls - Tests for model transformation logic - Error handling test cases
1 parent 587cf63 commit 6723c83

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+7231
-3884
lines changed

.gitignore

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -461,3 +461,20 @@ lib/user-interface/react-app/src/graphql/subscriptions.ts
461461
/.project
462462
/.pydevproject
463463
/outputs.json
464+
465+
# Hatch
466+
.hatch/
467+
468+
# IDE
469+
.idea/
470+
.vscode/
471+
*.swp
472+
*.swo
473+
.env*
474+
.envrc.local
475+
.env.local
476+
node_modules
477+
.amazonq
478+
AmazonQ.md
479+
.DS_Store
480+
cdk-exports.json

.python-version

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.9.18

README.md

Lines changed: 35 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -1,82 +1,53 @@
1-
# Building RAG use cases with GenAI Chatbot on AWS
1+
# AWS GenAI LLM Chatbot
22

3-
[![Release Notes](https://img.shields.io/github/v/release/aws-samples/aws-genai-llm-chatbot)](https://github.com/aws-samples/aws-genai-llm-chatbot/releases)
4-
[![GitHub star chart](https://img.shields.io/github/stars/aws-samples/aws-genai-llm-chatbot?style=social)](https://star-history.com/#aws-samples/aws-genai-llm-chatbot)
5-
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
3+
Enterprise-ready generative AI chatbot with RAG capabilities.
64

7-
[![Deploy with GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://aws-samples.github.io/aws-genai-llm-chatbot/guide/deploy.html#deploy-with-github-codespaces)
5+
## Overview
86

9-
[![Full Documentation](https://img.shields.io/badge/Full%20Documentation-blue?style=for-the-badge&logo=Vite&logoColor=white)](https://aws-samples.github.io/aws-genai-llm-chatbot/)
7+
The AWS GenAI LLM Chatbot is a production-ready solution that enables organizations to deploy a secure, feature-rich chatbot powered by large language models (LLMs) with Retrieval Augmented Generation (RAG) capabilities.
108

11-
![sample](docs/about/assets/chabot-sample.gif "GenAI Chatbot on AWS")
9+
## Key Features
1210

11+
- **Multiple LLM Support**: Amazon Bedrock (Claude, Llama 2), SageMaker, and custom model endpoints
12+
- **Nexus Gateway Integration**: Connect to Nexus Gateway for additional model access
13+
- **Comprehensive RAG Implementation**: Connect to various data sources for context-aware responses
14+
- **Enterprise Security**: Fine-grained access controls, audit logging, and data encryption
15+
- **Conversation Memory**: Full conversation history with persistent storage
16+
- **Web UI and API Access**: Modern React interface and API endpoints for integration
17+
- **Cost Optimization**: Token usage tracking and cost management features
18+
- **Deployment Flexibility**: Multiple deployment options to fit your needs
1319

14-
## 🚀 NEW! Support for new Amazon Nova Models 🚀
15-
### Deploy this chatbot to use the recently announced [Amazon Nova models](https://aws.amazon.com/blogs/aws/introducing-amazon-nova-frontier-intelligence-and-industry-leading-price-performance/)!
16-
### These powerful models can __understand__ and __generate__ images and videos.
20+
## Getting Started
1721

18-
Deploy this chatbot to experiment with:
19-
- `Amazon Nova Micro`
20-
- `Amazon Nova Lite`
21-
- `Amazon Nova Pro`
22-
- `Amazon Nova Canvas`
23-
- `Amazon Nova Reels`
22+
This blueprint deploys the complete AWS GenAI LLM Chatbot solution in your AWS account.
2423

24+
### Prerequisites
2525

26+
- AWS Account with appropriate permissions
27+
- AWS CLI configured with credentials
28+
- Node.js 18+ and npm
29+
- Python 3.8+
2630

27-
Make sure to request access to the new models [here](https://aws-samples.github.io/aws-genai-llm-chatbot/documentation/model-requirements.html#amazon-bedrock-requirements)
31+
### Deployment
2832

29-
Read more about the new models [here](https://www.aboutamazon.com/news/aws/amazon-nova-artificial-intelligence-bedrock-aws)
33+
The deployment process is fully automated using AWS CDK and SeedFarmer.
3034

31-
---
35+
## Architecture
3236

37+
The solution architecture includes:
3338

34-
This solution provides ready-to-use code so you can start **experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts** in your own AWS account.
39+
- Amazon Bedrock for LLM access
40+
- Amazon OpenSearch for vector storage
41+
- Amazon S3 for document storage
42+
- Amazon Cognito for authentication
43+
- AWS Lambda for serverless processing
44+
- Amazon API Gateway for API access
45+
- React-based web interface
3546

36-
Supported model providers:
47+
## Documentation
3748

38-
- [Amazon Bedrock](https://aws.amazon.com/bedrock/) which supports a wide range of models from AWS, Anthropic, Cohere and Mistral including the latest models from Amazon Nova. See [Recent announcements](https://aws.amazon.com/blogs/aws/introducing-amazon-nova-frontier-intelligence-and-industry-leading-price-performance/) for more details.
39-
- [Amazon SageMaker](https://aws.amazon.com/sagemaker/) self-hosted models from Foundation, Jumpstart and HuggingFace.
40-
- Third-party providers via API such as Anthropic, Cohere, AI21 Labs, OpenAI, etc. [See available langchain integrations](https://python.langchain.com/docs/integrations/llms/) for a comprehensive list.
49+
For complete documentation, visit the [GitHub repository](https://github.com/aws-samples/aws-genai-llm-chatbot).
4150

42-
# Additional Resources
51+
## License
4352

44-
| Resource |Description|
45-
|:-------------|:-------------|
46-
| [Secure Messenger GenAI Chatbot](https://github.com/aws-samples/secure-messenger-genai-chatbot) | A messenger built on Wickr that can interface with this chatbot to provide Q&A service in tightly regulated environments (i.e. HIPAA). |
47-
| [Project Lakechain](https://github.com/awslabs/project-lakechain) | A powerful cloud-native, AI-powered, document (docs, images, audios, videos) processing framework built on top of the AWS CDK. |
48-
| [AWS Generative AI CDK Constructs](https://github.com/awslabs/generative-ai-cdk-constructs/) | Open-source library extension of the [AWS Cloud Development Kit (AWS CDK)](https://docs.aws.amazon.com/cdk/v2/guide/home.html) aimed to help developers build generative AI solutions using pattern-based definitions for their architecture. |
49-
| [Artifacts and Tools for Bedrock](https://github.com/aws-samples/artifacts-and-tools-for-bedrock) | An innovative chat-based user interface with support for tools and artifacts. It can create graphs and diagrams, analyze data, write games, create web pages, generate files, and much more. |
50-
51-
# Roadmap
52-
53-
Roadmap is available through the [GitHub Project](https://github.com/orgs/aws-samples/projects/69)
54-
55-
# Authors
56-
57-
- [Bigad Soleiman](https://www.linkedin.com/in/bigadsoleiman/)
58-
- [Sergey Pugachev](https://www.linkedin.com/in/spugachev/)
59-
60-
# Contributors
61-
[![contributors](https://contrib.rocks/image?repo=aws-samples/aws-genai-llm-chatbot&max=2000)](https://github.com/aws-samples/aws-genai-llm-chatbot/graphs/contributors)
62-
63-
# License
64-
65-
This library is licensed under the MIT-0 License. See the LICENSE file.
66-
67-
- [Changelog](CHANGELOG.md) of the project.
68-
- [License](LICENSE) of the project.
69-
- [Code of Conduct](CODE_OF_CONDUCT.md) of the project.
70-
- [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
71-
72-
Although this repository is released under the MIT-0 license, its front-end and SQL implementation use the following third party projects:
73-
- [psycopg2-binary](https://github.com/psycopg/psycopg2)
74-
- [jackspeak](https://github.com/isaacs/jackspeak)
75-
- [package-json-from-dist](https://github.com/isaacs/package-json-from-dist)
76-
- [path-scurry](https://github.com/isaacs/path-scurry)
77-
78-
These projects' licensing includes the LGPL v3 and BlueOak-1.0.0 licenses.
79-
80-
# Legal Disclaimer
81-
82-
You should consider doing your own independent assessment before using the content in this sample for production purposes. This may include (amongst other things) testing, securing, and optimizing the content provided in this sample, based on your specific quality control practices and standards.
53+
This project is licensed under the MIT-0 License.

aws-genai-llm-chatbot/README.md

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# AWS GenAI LLM Chatbot - SeedFarmer Blueprint
2+
3+
This directory contains the SeedFarmer blueprint (also known as a "capability" in Nexus terminology) for deploying the AWS GenAI LLM Chatbot solution.
4+
5+
## What is SeedFarmer?
6+
7+
[SeedFarmer](https://seed-farmer.readthedocs.io/) is a deployment framework designed to simplify the process of deploying complex AWS solutions across multiple accounts and regions. It provides a consistent way to define, deploy, and manage infrastructure using a hierarchical structure of deployments, groups, and modules.
8+
9+
> **Note:** In the Nexus ecosystem, the terms "blueprint" and "capability" are used interchangeably to refer to a SeedFarmer deployment configuration.
10+
11+
## Directory Structure
12+
13+
```
14+
aws-genai-llm-chatbot/
15+
├── assets/ # Visual assets and example configurations
16+
│ ├── chatbot-icon.svg # Icon for the catalog listing
17+
│ ├── chatbot-thumbnail.svg # Thumbnail for the catalog detail page
18+
│ └── example-input.yaml # Example input parameters
19+
├── modules/ # SeedFarmer modules
20+
│ └── chatbot/ # Main chatbot module
21+
│ ├── deployspec.yaml # Deployment instructions
22+
│ └── module.yaml # Module definition
23+
├── package/ # Package for Nexus integration
24+
│ └── package/ # Nested directory required by Nexus
25+
│ └── capability.yaml # Copy of capability.yaml for Lambda processing
26+
├── capability.yaml # Blueprint definition for Nexus catalog
27+
├── deployment.yaml # SeedFarmer deployment manifest
28+
├── README.md # This file
29+
└── seedfarmer.yaml # Project metadata
30+
```
31+
32+
## File Descriptions
33+
34+
### capability.yaml
35+
36+
This file defines the blueprint for the Nexus catalog, including:
37+
- Metadata: name, description, version
38+
- Input parameters and their validation rules
39+
- Deployment environments
40+
- Labels for categorization
41+
42+
This is the primary file that determines how the solution appears in the Nexus catalog and what configuration options are available to users.
43+
44+
### deployment.yaml
45+
46+
The main SeedFarmer deployment manifest that defines:
47+
- Deployment structure
48+
- Target accounts and regions
49+
- Module groups and their relationships
50+
- Environment variable mapping
51+
52+
This file serves as the entry point for SeedFarmer when deploying the solution.
53+
54+
### seedfarmer.yaml
55+
56+
Contains project metadata:
57+
- Project name and description
58+
- Reference to README.md for documentation
59+
60+
This file provides additional context for the SeedFarmer deployment.
61+
62+
### modules/chatbot/module.yaml
63+
64+
Defines the chatbot module:
65+
- Parameters passed from environment variables
66+
- Output definitions for deployed resources
67+
- Stack naming convention
68+
69+
This file specifies how the CDK stack should be deployed and what outputs should be collected.
70+
71+
### modules/chatbot/deployspec.yaml
72+
73+
Contains deployment instructions:
74+
- Commands to execute during deployment
75+
- Environment variable handling
76+
- CDK deployment configuration
77+
- Cleanup commands for destruction
78+
79+
This file is executed by SeedFarmer during the deployment process.
80+
81+
### assets/example-input.yaml
82+
83+
Provides an example configuration with:
84+
- Sample values for all parameters
85+
- Comments explaining each parameter
86+
- Default values and recommendations
87+
88+
This file helps users understand how to configure the solution.
89+
90+
### assets/chatbot-icon.svg and assets/chatbot-thumbnail.svg
91+
92+
Visual assets for the Nexus catalog:
93+
- Icon: Small image for the catalog listing
94+
- Thumbnail: Larger image for the catalog detail page
95+
96+
These files enhance the visual presentation of the solution in the Nexus catalog.
97+
98+
## Deployment Process
99+
100+
When deployed through SeedFarmer or the Nexus catalog, the process follows these steps:
101+
102+
1. User selects the blueprint from the Nexus catalog
103+
2. User configures parameters through the Nexus UI
104+
3. SeedFarmer creates a deployment manifest based on these parameters
105+
4. SeedFarmer deploys the modules in the specified order
106+
5. Outputs are collected and made available to the user
107+
108+
## Key Parameters
109+
110+
The blueprint supports several configuration parameters:
111+
112+
- **Core Parameters**: PREFIX, BEDROCK_REGION, TARGET_ACCOUNT
113+
- **Feature Flags**: BEDROCK_ENABLE, RAG_ENABLE, NEXUS_ENABLE
114+
- **Security Options**: CREATE_CMKS, RETAIN_ON_DELETE
115+
- **RAG Options**: RAG_OPENSEARCH_ENABLE
116+
- **Nexus Integration**: NEXUS_GATEWAY_URL, NEXUS_AUTH_CLIENT_ID, etc.
117+
118+
## Additional Resources
119+
120+
- [SeedFarmer Documentation](https://seed-farmer.readthedocs.io/)
121+
- [Nexus Blueprint Guide](https://nexus-3p-vending-d523f3.pages.aws.dev/)
122+
- [AWS GenAI LLM Chatbot Documentation](https://github.com/aws-samples/aws-genai-llm-chatbot)
Lines changed: 5 additions & 0 deletions
Loading
Lines changed: 6 additions & 0 deletions
Loading
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# AWS GenAI LLM Chatbot Configuration
2+
# --------------------------------
3+
# This file contains the configuration parameters for the AWS GenAI LLM Chatbot deployment.
4+
# Modify the values below to customize your deployment.
5+
6+
# Core parameters
7+
# ------------------------------
8+
TARGET_ACCOUNT: 123456789012 # Your AWS Account ID
9+
PREFIX: chatbot # Prefix for all resources (max 16 chars)
10+
BEDROCK_REGION: us-east-1 # AWS region for Amazon Bedrock
11+
12+
DEPLOYMENT_TYPE: default # Deployment type (default or custom)
13+
RETAIN_ON_DELETE: false # Retain resources on delete
14+
CREATE_CMKS: false # Create customer managed KMS keys
15+
16+
# Bedrock integration
17+
# ------------------------------
18+
BEDROCK_ENABLE: false # Enable Amazon Bedrock integration
19+
20+
# RAG capabilities
21+
# ------------------------------
22+
RAG_ENABLE: false # Enable RAG capabilities
23+
# RAG_OPENSEARCH_ENABLE: true # Use OpenSearch as vector store
24+
# RAG_AURORA_ENABLE: false # Enable Aurora as RAG engine
25+
# RAG_KENDRA_ENABLE: false # Enable Kendra as RAG engine
26+
# RAG_KENDRA_CREATE_INDEX: false # Create a new Kendra index
27+
# RAG_KENDRA_ENTERPRISE: false # Use Kendra Enterprise edition
28+
# RAG_KNOWLEDGE_BASE_ENABLE: false # Enable Knowledge Base as RAG engine
29+
# RAG_CROSS_ENCODING_ENABLE: false # Enable cross-encoding for RAG
30+
31+
# Nexus integration (if needed)
32+
# ------------------------------
33+
NEXUS_ENABLE: false # Enable Nexus integration
34+
# NEXUS_GATEWAY_URL: https://your-nexus-gateway.example.com # Nexus Gateway URL
35+
# NEXUS_AUTH_CLIENT_ID: client-12345 # Nexus Auth Client ID
36+
# NEXUS_AUTH_CLIENT_SECRET: secret-xyz # Nexus Auth Client Secret
37+
# NEXUS_AUTH_TOKEN_URL: https://your-token-url.example.com # Nexus Auth Token URL

0 commit comments

Comments
 (0)