Cohere: Toolkit for Developing RAG Applications

Project Introduction

Cohere is an open-source toolkit for developing RAG applications, which can be deployed to Microsoft Azure with one click or run locally.

Building and Running Locally

Clone the repository and run

make setup

Configure the model according to the instructions – AWS Sagemaker, Azure, or the Cohere platform. This can also be accomplished by running make setup (see option 2 below), which will help generate files for you, or manually create a .env file and copy the provided .env-template. Then replace the values with the correct ones.

Cohere Platform

• COHERE_API_KEY: If your application will interact with Cohere’s API, you need to provide an API key. This is not required if using AWS Sagemaker or Azure. Register at https://dashboard.cohere.com/ to create an API key.

• NEXT_PUBLIC_API_HOSTNAME: The backend URL that the frontend will communicate with. The default is http://localhost:8000

• DATABASE_URL: The PostgreSQL connection string for SQLAlchemy should follow the format postgresql+psycopg2://USER:PASSWORD@HOST:PORT.

AWS Sagemaker

To use this toolkit with AWS Sagemaker, you first need the cohere model (command version), which supports chat deployed in Sagemaker. Follow Cohere’s guides and notebook deployment commands to deploy the model and create endpoints that can then be used with the toolkit.

Then, you will need to set up authorization; see here for more details. The default toolkit setup uses a configuration file with the following environment variables (after aws configure sso):

• SAGE_MAKER_REGION_NAME: The region configured for the model.

• SAGE_MAKER_ENDPOINT_NAME: The name of the endpoint created in the notebook.

• SAGE_MAKER_PROFILE_NAME: Your AWS profile name

Hosted Tools

• PYTHON_INTERPRETER_URL: The URL pointing to the Python interpreter container. The default is http://localhost:8080.

• TAVILY_API_KEY: If you want to enable internet search, you need to provide the Tavily API key. This is not required.

Local Deployment

Once the environment variables are set up, you can deploy the Toolkit locally! Pull the Docker image from the Github Artifact registry or build files from source. Refer to the Makefile for all available commands.

Requirements:

• Docker

• Poetry

• Docker-compose >= 2.22

Option 1 – Local Installation Using Docker:

Ensure your shell is authenticated with GHCR.

Pull a single container image from Github’s Artifact Registry

docker pull ghcr.io/cohere-ai/cohere-toolkit:latest

Run the image locally:

docker run --name=cohere-toolkit -itd -e COHERE_API_KEY='Your Cohere API key here' -p 8000:8000 -p 4000:4000 ghcr.io/cohere-ai/cohere-toolkit

Option 2 – Build Locally from Scratch:

Option 2.1 – Run Everything at Once

Run make first-run to start the CLI, which will generate a .env file for you. This will also run all database migrations and start the containers

make first-run

Option 2.1 – Run Each Command Separately

Run make setup to start the CLI, which will generate a .env file:

make setup

Then run:

make migrate make dev

If you haven’t changed the default port, visit http://localhost:4000/ in your browser to chat with the model.

What Does the Toolkit Include?

The components in this repository include:

• src/interfaces/coral_web – A built-in web application with Next.js. Includes a simple SQL database out of the box for storing conversation history in the application.

• src/backend – Contains pre-configured data sources and retrieval code for setting up RAG on custom data sources (called “retrieval chains”). Users can also configure which model to use, selecting from Cohere models hosted on the Cohere platform, Azure, and AWS Sagemaker. By default, we configured a Langchain data retriever to test RAG on Wikipedia and documents you upload.

Follow the “GitHubStore” public account

Scan the following WeChat

1 Join the technical exchange group, note “Development Language-City-Nickname“