NVIDIA Generative AI Examples

This repository is a starting point for developers looking to integrate with the NVIDIA software ecosystem to speed up their generative AI systems. Whether you are building RAG pipelines, agentic workflows, or fine-tuning models, this repository will help you integrate NVIDIA, seamlessly and natively, with your development stack.

Table of Contents

• What's New?
  • Knowledge Graph RAG
  • Agentic Workflows with Llama 3.1
  • RAG with Local NIM Deployment and LangChain
• Try it Now!
• RAG
  • RAG Notebooks
  • RAG Examples
  • RAG Tools
  • RAG Projects
• Documentation
  • Getting Started
  • How To's
  • Reference
• Community

What's New?

Knowledge Graph RAG

This example implements a GPU-accelerated pipeline for creating and querying knowledge graphs using RAG by leveraging NIM microservices and the RAPIDS ecosystem to process large-scale datasets efficiently.

• Knowledge Graphs for RAG with NVIDIA AI Foundation Models and Endpoints

Agentic Workflows with Llama 3.1

• Build an Agentic RAG Pipeline with Llama 3.1 and NVIDIA NeMo Retriever NIM microservices [Blog, Notebook]
• NVIDIA Morpheus, NIM microservices, and RAG pipelines integrated to create LLM-based agent pipelines

RAG with Local NIM Deployment and LangChain

• Tips for Building a RAG Pipeline with NVIDIA AI LangChain AI Endpoints by Amit Bleiweiss. [Blog, Notebook]

For more information, refer to the Generative AI Example releases.

Try it Now!

Experience NVIDIA RAG Pipelines with just a few steps!

1. Get your NVIDIA API key.

   1. Go to the NVIDIA API Catalog.
   2. Select any model.
   3. Click Get API Key.
   4. Run:

      export NVIDIA_API_KEY=nvapi-...
      

2. Clone the repository.

   git clone https://github.com/nvidia/GenerativeAIExamples.git
   

3. Build and run the basic RAG pipeline.

   cd GenerativeAIExamples/RAG/examples/basic_rag/langchain/
   docker compose up -d --build
   

4. Go to https://localhost:8090/ and submit queries to the sample RAG Playground.

5. Stop containers when done.

   docker compose down
   

RAG

RAG Notebooks

NVIDIA has first-class support for popular generative AI developer frameworks like LangChain, LlamaIndex, and Haystack. These end-to-end notebooks show how to integrate NIM microservices using your preferred generative AI development framework.

Use these notebooks to learn about the LangChain and LlamaIndex connectors.

LangChain Notebooks

• RAG
  • Basic RAG with CHATNVIDIA LangChain Integration
  • RAG using local NIM microservices for LLMs and Retrieval
  • RAG for HTML Documents
  • Chat with NVIDIA Financial Reports
• Agents
  • NIM Tool Calling 101
  • Agentic RAG with NeMo Retriever
  • Agents with Human in the Loop

LlamaIndex Notebooks

• Basic RAG with LlamaIndex Integration

RAG Examples

By default, these end-to-end examples use preview NIM endpoints on NVIDIA API Catalog. Alternatively, you can run any of the examples on premises.

Basic RAG Examples

• LangChain example
• LlamaIndex example

Advanced RAG Examples

• Multi-Turn
• Multimodal Data
• Structured Data (CSV)
• Query Decomposition

RAG Tools

Example tools and tutorials to enhance LLM development and productivity when using NVIDIA RAG pipelines.

• Evaluation
• Observability

RAG Projects

• NVIDIA Tokkio LLM-RAG: Use Tokkio to add avatar animation for RAG responses.
• Hybrid RAG Project on AI Workbench: Run an NVIDIA AI Workbench example project for RAG.

Documentation

Getting Started

• Prerequisites

How To's

• Changing the Inference or Embedded Model
• Customizing the Vector Database
• Customizing the Chain Server:
  • Chunking Strategy
  • Prompting Template Engineering
• Configuring LLM Parameters at Runtime
• Supporting Multi-Turn Conversations
• Speaking Queries and Listening to Responses with NVIDIA Riva

Reference

• Support Matrix
• Architecture
• Using the Sample Chat Web Application
• RAG Playground Web Application
• Software Component Configuration

Community

We're posting these examples on GitHub to support the NVIDIA LLM community and facilitate feedback. We invite contributions! Open a GitHub issue or pull request! See contributing Check out the community examples and notebooks.