mini-rag

This is a minimal implementation of the RAG model for question answering.

The Course

This is an educational project where all of the codes where explained (step by step) via a set of Arabic youtube videos. Please check the list:

#	Title	Link	Codes
1	About the Course ماذا ولمـــاذا	Video	NA
2	What will we build ماذا سنبنى في المشروع	Video	NA
3	Setup your tools الأدوات الأساسية	Video	NA
4	Project Architecture	Video	branch
5	Welcome to FastAPI	Video	branch
6	Nested Routes + Env Values	Video	branch
7	Uploading a File	Video	branch
8	File Processing	Video	branch
9	Docker - MongoDB - Motor	Video	branch
10	Mongo Schemes and Models	Video	branch
11	Mongo Indexing	Video	branch
12	Data Pipeline Enhancements	Video	branch
13	Checkpoint-1	Video	branch
14	LLM Factory	Video	branch
15	Vector DB Factory	Video	branch
16	Semantic Search	Video	branch
17	Augmented Answers	Video	branch
18	Checkpoint-1 + Fix Issues	Video	branch
19	Ollama Local LLM Server	Video	branch
20	From Mongo to Postgres + SQLAlchemy & Alembic	Video	branch
21	The way to PgVector	Video	branch
22	App Deployments 1/2	Video	branch
22	App Deployments 2/2	Video	branch
24	Celery Workers 1/2	Video	branch
25	Celery Workers 2/2	Video	branch

Requirements

• Python 3.10

Install Dependencies

sudo apt update
sudo apt install libpq-dev gcc python3-dev

Install Python using MiniConda

1. Download and install MiniConda from here
2. Create a new environment using the following command:

$ conda create -n mini-rag python=3.10

3. Activate the environment:

$ conda activate mini-rag

(Optional) Setup you command line interface for better readability

export PS1="\[\033[01;32m\]\u@\h:\w\n\[\033[00m\]\$ "

(Optional) Run Ollama Local LLM Server using Colab + Ngrok

• Check the notebook + Video

Installation

Install the required packages

$ pip install -r requirements.txt

Setup the environment variables

$ cp .env.example .env

Run Alembic Migration

$ alembic upgrade head

Set your environment variables in the .env file. Like OPENAI_API_KEY value.

Run Docker Compose Services

$ cd docker
$ cp .env.example .env

• update .env with your credentials

$ cd docker
$ sudo docker compose up -d

Access Services

• FastAPI: http://localhost:8000
• Flower Dashboard: http://localhost:5555 (admin/password from env)
• Grafana: http://localhost:3000
• Prometheus: http://localhost:9090

Run the FastAPI server (Development Mode)

$ uvicorn main:app --reload --host 0.0.0.0 --port 5000

Celery (Development Mode)

For development, you can run Celery services manually instead of using Docker:

To Run the Celery worker, you need to run the following command in a separate terminal:

$ python -m celery -A celery_app worker --queues=default,file_processing,data_indexing --loglevel=info

To run the Beat scheduler, you can run the following command in a separate terminal:

$ python -m celery -A celery_app beat --loglevel=info

To Run Flower Dashboard, you can run the following command in a separate terminal:

$ python -m celery -A celery_app flower --conf=flowerconfig.py

open your browser and go to http://localhost:5555 to see the dashboard.

POSTMAN Collection

Download the POSTMAN collection from /assets/mini-rag-app.postman_collection.json