Lab 02 — RAG Application Demo

A hands-on lab: build a local, offline RAG (Retrieval-Augmented Generation) application on your own machine with Ollama, FAISS and sentence-transformers. No API key, and nothing leaves your laptop. Download the lab files below, then follow the steps.

⬇ Download the lab files (.zip)

Step 1: Download Ollama and install the application in your local machine.

Step 2: Open Ollama desktop application. Click on the “New Chat” Icon. After that, click on the model switcher and Search for “llama3:8b” model and click on the download link beside it. This model will run locally and is sufficient for our lab.

Step 3: Now, we need to download “all-minilm:l6-v2” model from Ollama. Open powershell and type “ollama pull all-minilm:l6-v2”. The model will be downloaded to your system. You can verify the model by typing “ollama list”.

Step 4: Download Python from the http://python.org/ and install it in your local machine.

Step 5: Verify Python installation by opening a Powershell and typing the below command. “Python –version”. You will get the installed python version on your shell.

Step 6: Download Lab 02- RAG Application Demo zip file and unzip it in a directory. For the purpose of this lab. We are going to unzip the file at C:\AI-PlayGround\Training\Done\

Step 7: Open VS Code and open the folder from the directory where we have the RAG demo lab code. Click on “Yes” in the trust Author dialog box.

Step 8: Verify the folder on the VS code and check for the code file “examples/rag_vector_example_explained.py”.

Step 9: Read the below code and refer comments to know more about the code line.

import faiss   # Vector Store + Similarity Search – Facebook AI Similarity Search
 
import numpy as np 

import ollama  # Talks to the local LLM 

from sentence_transformers import SentenceTransformer  #Embedding Model – Text into Token

DOCUMENTS = [
    "The user's name is Chris.",
    "Chris is a backend developer working with Java and Spring Boot.",
    "Chris works on APIs",
    "Chris lives in India.",
    "Chris is learning AI and RAG systems.",
    "Vectors have magnitude and direction.",
    "RAG stands for Retrieval Augmented Generation.",
    "AI embeddings are vector representations of meaning.",
    "Delhi weather is extremely hot during summer.",
    "Delhi is located in India.",
    "Current Delhi climate is hot.",
]


EMBEDDING_MODEL = "all-MiniLM-L6-v2“  # Embedding Model 
CHAT_MODEL = "llama3:8b“  # Chat Model  
TOP_K = 5    # How many docs to retrieve

model = SentenceTransformer(EMBEDDING_MODEL)  # ~90 MB, first run only
doc_embeddings = model.encode(DOCUMENTS)  #Convert documents into token



doc_embeddings = np.array(doc_embeddings).astype("float32") #FAISS demands Float 32 


faiss.normalize_L2(doc_embeddings)   #length 1 -> dot IS cosine


dimension = doc_embeddings.shape[1]   # .shape -> [11, 384], .shape[1] -> 384

index = faiss.IndexFlatIP(dimension) #FAISS need to be told the vector width 

index = faiss.IndexFlatIP(dimension)

index.add(doc_embeddings)  #load the 11 sentences

while True:

 query = input("\nAsk a question (or type exit): ").strip()   # User input
 if not query:
        continue

    if query.lower() == "exit":
        break

query_embedding = model.encode([query])  #Query Encoding

query_embedding = np.array(query_embedding).astype("float32")
 
faiss.normalize_L2(query_embedding)

  # Sending the query along with context to the olama model 

    response = ollama.chat(
        model=CHAT_MODEL,
        messages=[{"role": "user", "content": prompt}],
    )

Step 10: To execute the program, we need to download relevant packages. Open a PowerShell and navigate inside the demo directory “Lab 02 - RAG Application Demo\API-PlayGround”. Enter the following command “powershell -ExecutionPolicy Bypass -File .\setup.ps1”. You package download will commence.

Step 11: Verify the status of the package download and notice that .venv directory is created in the “Lab 02 - RAG Application Demo\API-PlayGround” directory. This is the local directory of the python packages.

Step 12: Check for the confirmation and run the lab with the following command. .\.venv\Scripts\python.exe examples\rag_vector_example_explained.py

Step 13: Query the model about Chris, and it will give you the similarity score from the documents and the retrieved context, followed by the answer from the context.

Ready to try it? Download the lab files (.zip), unzip, and run setup.ps1 to get started.