vercel-labs/ai-gateway-embeddings-demo

A simple Next.js chatbot app to demonstrate the use of embedding models for RAG through the Vercel AI Gateway.

Paste any facts or docs into the chat and this app will chunk them, generate embeddings via the Vercel AI Gateway, and store them as vectors in Neon. When you ask a question, your query is embedded and the app performs a cosine-similarity search to pull the most relevant chunks. The model then answers using only the retrieved snippets, demonstrating semantic retrieval (RAG).

1. Clone this repository with the Deploy button above
2. Install the Vercel CLI if you don't already have it (pnpm i -g vercel)
3. Clone the repository you created above: git clone <repo-url>
4. Link it to a Vercel project: vc link or vc deploy
5. Pull environment variables with vc env pull
6. Install packages with pnpm i (or npm i or yarn i)
7. Run a database migration with pnpm db:migrate && pnpm db:push
8. Run the development server with vc dev and open http://localhost:3000 to try the chatbot

1. If you prefer running your local development server directly rather than using vc dev, you'll need to run vc env pull to fetch the project's OIDC authentication token locally
   1. the token expires every 12h, so you'll need to re-run this command periodically.
   2. if you use vc dev it will auto-refresh the token for you, so you don't need to fetch it manually
2. If you're linking to an existing, older project, you may need to enable the OIDC token feature in your project settings.
   1. visit the project settings page (rightmost tab in your project's dashboard)
   2. search for 'OIDC' in settings
   3. toggle the button under "Secure Backend Access with OIDC Federation" to Enabled and click the "Save" button
3. See this page to learn how AI Gateway billing works

This repository is maintained by the Vercel team and community contributors.

Contributions are welcome! Feel free to open issues or submit pull requests to enhance functionality or fix bugs.