Building a Transparent RAG App for Community Guidelines: Beginner-Friendly Tutorial

Omale Happiness

9 min read

·

Mar 9, 2026

--

Listen

Share

Most community guidelines are long, and let’s be honest, community members barely read them from beginning to end. Yet they matter a lot. Developers need to know what’s allowed, what’s not, and how to stay compliant. That’s where Retrieval-Augmented Generation (RAG) comes in.

In this article, I’ll show you how to build a transparent RAG system that doesn’t just answer questions, but shows exactly where the answers came from. And by the end, we’ll wrap everything in a simple Streamlit app you can actually use.

What is RAG?

RAG stands for Retrieval-Augmented Generation. Instead of letting an AI model “guess” an answer, RAG works in two steps:

1. Retrieve relevant information from a document
2. Generate an answer based only on that information

Think of it like this, a developer asks, “Can I share API keys in public channels? A normal LLM might hallucinate. A RAG system goes into your guideline document, finds the exact section, and answers based on it.

Why Transparent RAG for Community Guidelines?

Community guidelines are one of the best use cases for RAG. Without it, people skip important sections, break rules unintentionally, waste time searching, and get unclear answers. A transparent RAG system fixes this by retrieving only relevant sections, showing the exact chunks used, and generating clear responses.

How the System Works

User Question → Retriever → Relevant Chunks → LLM → Answer + Citations → Streamlit

Let’s break that down:

• Chunks: Smaller pieces of your document
• Embeddings: Numerical meaning of each chunk
• FAISS: Fast similarity search
• Retriever: Finds relevant chunks
• LLM: Generates the answer
• Streamlit: Displays everything nicely

Press enter or click to view image in full size

Step 1: Prepare Your Data

Create a file called “guidelines.txt”, fill it with your community guidelines or any document you want the RAG system to reference. For example, mine contains rules about AI Engineering community conduct, privacy, content moderation, and technical ethics.

# AI Engineering Community Guidelines

Welcome to our AI Engineering community. Our goal is to foster an environment of 
collaborative learning, ethical development, and technical excellence. 
By participating, you agree to the following standards:

## 1. Respectful & Professional Communication
* **No Harassment:** We have zero tolerance for hate speech, personal attacks, or 
    discriminatory language based on race, gender, orientation, or background.
* **Constructive Critique:** When reviewing code or models, focus on the logic 
    and data rather than the individual. Aim to mentor, not disparage.

## 2. Technical Integrity & Ethics
* **Responsible AI:** Members are encouraged to discuss and mitigate bias, 
    safety risks, and alignment issues in their projects.
* **Academic Honesty:** Cite your sources. If you are using specific papers, 
    datasets, or pre-trained weights, provide proper attribution.
* **No Malicious Use:** Discussion or development of AI for the purpose of 
    generating deepfakes for harassment, malware, or misinformation is prohibited.

## 3. Content Moderation
* **No Spam:** Avoid excessive self-promotion, referral links, or "low-effort" 
    AI-generated spam posts.
* **Quality Control:** Ensure technical questions include enough context 
    (code snippets, error logs, or environment specs) to be answerable.
* **Misleading Info:** Do not present hallucinations or unverified AI outputs 
    as factual technical advice.

## 4. Privacy & Security
* **Data Privacy:** Never share PII (Personally Identifiable Information) or 
    proprietary company data in public channels.
* **Key Security:** Do not post screenshots or code containing active 
    API keys (OpenAI, Anthropic, AWS, etc.).

## 5. Enforcement
The moderation team monitors activity to ensure a safe environment.
* **Level 1 (Warning):** Minor first-time infractions or off-topic behavior.
* **Level 2 (Suspension):** Repeated violations or significant ethical breaches.
* **Level 3 (Permanent Ban):** Severe harassment, malicious activity, or 
    repeated disregard for community safety.

---
Last Updated: January 2026

Step 2: Install Dependencies

pip install langchain openai faiss-cpu streamlit

• Langchain: For building the RAG workflow
• OpenAI: To access LLMs for answer generation
• Faiss-cpu: To create a fast vector search store
• Streamlit: For building the interactive UI

Step 3: Set Your API Key

Open jupyter workspace and create a new notebook, write this code

import os
os.environ["OPENAI_API_KEY"] = "your-api-key"

Replace “your-api-key” with your actual OpenAI API key.

Step 4: Chunk the Guidelines

from langchain_text_splitters import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=400,
    chunk_overlap=80
)

chunks = text_splitter.create_documents([guidelines_text])

# Add chunk IDs for citation
for i, doc in enumerate(chunks):
    doc.metadata["chunk_id"] = i

print(f"Total chunks: {len(chunks)}")

• We import a tool from LangChain that breaks large text into smaller chunks because Smaller chunks make retrieval more precise.
• We create a splitter object.
• chunk_size=400: Each chunk will have about 400 characters.
• chunk_overlap=80: Each chunk overlaps with the previous one by 80 characters to avoid losing context at the edges. This splits your guidelines text into a list of small chunks.
• Each chunk becomes a “document” that can later be embedded and retrieved.
• We added a unique ID to each chunk in its metadata, so when the RAG system answers a question, it can cite exactly which chunk(s) it used.
• Finally, we print the total number of chunks to check our splitting.
• This lets you know how many pieces the document was divided into, which is useful for debugging.

Step 5: Create Embeddings And Vector Store

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()

from langchain_community.vectorstores import FAISS

vectorstore = FAISS.from_documents(
    documents=chunks,
    embedding=embeddings
)

Now your document is searchable by meaning, not just keywords. We convert each chunk into a numerical representation that the AI can understand, called an embedding. These embeddings capture the meaning of the text so the model can find relevant information later. We then store these embeddings in FAISS, a fast vector search database. FAISS allows us to quickly search for the chunks most relevant to a user’s question, which is the core of how RAG retrieves accurate answers.

Step 6: Generate Answers With LLM

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOpenAI(
    temperature=0
)

prompt = ChatPromptTemplate.from_template("""
You are answering questions strictly using the provided context.

Rules:
- Use ONLY the context below.
- Cite the chunk IDs you used.
- If the answer is not found, say:
  "I could not find this in the guidelines."

Context:
{context}

Question:
{question}

Answer format:
Answer:
<your answer>

Used Chunks:
[chunk_ids]
""")

def generate_answer(question, retrieved_docs):
    context_blocks = []

    for doc in retrieved_docs:
        chunk_id = doc.metadata["chunk_id"]
        context_blocks.append(
            f"[Chunk {chunk_id}]\n{doc.page_content}"
        )

    context = "\n\n".join(context_blocks)

    chain = prompt | llm

    response = chain.invoke({
        "context": context,
        "question": question
    })

    return response.content

We use a language model (LLM) to turn the retrieved chunks into a clear answer.

• The ChatPromptTemplate ensures the model only uses the provided chunks and cites which chunk(s) it used.
• The generate_answer function takes a question and the chunks retrieved by FAISS, then formats the context for the LLM.
• The LLM then generates an answer with transparency, showing exactly which chunks informed the response.

If the answer isn’t in the chunks, it will honestly respond: “I could not find this in the guidelines.”

Step 7: Test the RAG System

query = "Can I share API keys in public channels?"

retrieved = retrieve_chunks(query)

for doc in retrieved:
    print(f"Chunk {doc.metadata['chunk_id']}")

print("\n--- ANSWER ---\n")
print(generate_answer(query, retrieved))

Now that our RAG system is set up, it’s time to see it in action:

• In this example, we check: “Can I share API keys in public channels?”
• retrieve_chunks(query) searches the FAISS vector store and returns the most relevant sections from your guideline document.
• We print the chunk_id for each retrieved chunk so we know which parts of the document the system looked at.
• generate_answer(query, retrieved) passes these chunks to the LLM, which produces a clear, concise answer.
• The answer also includes the chunk references, making it fully transparent and trustworthy.

Press enter or click to view image in full size

Step 8: Build a Streamlit App

This final step turns our RAG system into a friendly web app so anyone can ask questions about the guidelines and see transparent answers in real time.

To turn this into an interactive application, create a new file called “app.py” in the same directory as your “guidelines.txt " file. Move the Streamlit code into this file, and make sure your environment variables and dependencies are already set.

import os
os.environ["OPENAI_API_KEY"] = "your_api_key"
import streamlit as st
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate
from pathlib import Path
import re


# -------------------------
# PAGE CONFIG
# -------------------------
st.set_page_config(
    page_title="Community RAG",
    layout="wide"
)


st.title("Community Guidelines")
st.caption("RAG over AI Engineering Community Guidelines")


# -------------------------
# LOAD & CHUNK DATA
# -------------------------
@st.cache_resource
def load_vectorstore():
    text = Path("guidelines.txt").read_text()


    splitter = RecursiveCharacterTextSplitter(
        chunk_size=400,
        chunk_overlap=80
    )


    docs = splitter.create_documents([text])


    for i, doc in enumerate(docs):
        doc.metadata["chunk_id"] = i
        doc.metadata["source"] = "community_guidelines"


    embeddings = OpenAIEmbeddings()
    return FAISS.from_documents(docs, embeddings)


vectorstore = load_vectorstore()
retriever = vectorstore.as_retriever(search_kwargs={"k": 5})


# -------------------------
# LLM + PROMPT
# -------------------------
llm = ChatOpenAI(temperature=0)


prompt = ChatPromptTemplate.from_template("""
Answer strictly using the context.


Rules:
- Use ONLY the context
- Cite chunk IDs
- If not found, say:
  "I could not find this in the guidelines."


Context:
{context}


Question:
{question}


Answer:
""")


# -------------------------
# UI
# -------------------------
query = st.text_input("Ask a question about the community guidelines")


if query:
    retrieved_docs = retriever.invoke(query)


    # -------------------------
    # Prepare context for the LLM
    # -------------------------
    context = "\n\n".join(
        f"[Chunk {doc.metadata['chunk_id']}]\n{doc.page_content}"
        for doc in retrieved_docs
    )


    # Generate answer
    response = (prompt | llm).invoke({
        "context": context,
        "question": query
    })


    # -------------------------
    # Display Chunks (clean view)
    # -------------------------
    st.subheader(" Retrieved Chunks (summary)")


    # Always show Chunk 0
    chunk0 = next(doc for doc in retrieved_docs if doc.metadata["chunk_id"] == 0)
    st.markdown("**Chunk 0 — Community Guidelines Intro**")
    st.write(chunk0.page_content)


    # Extract chunks actually cited in answer
    used_chunks = re.findall(r"Chunk (\d+)", response.content)
    used_chunks = [int(c) for c in used_chunks if int(c) != 0]  # exclude 0


    for doc in retrieved_docs:
        if doc.metadata["chunk_id"] in used_chunks:
            st.markdown(f"**Chunk {doc.metadata['chunk_id']}**")
            st.write(doc.page_content)


    # -------------------------
    # Display final answer
    # -------------------------
    st.subheader("Answer")
    st.write(response.content)

In the app.py file, the load_vectorstore() function is decorated with @st.cache_resource. This means Streamlit will cache the result of the function after the first run. Instead of rebuilding the vector store every time the script reruns, Streamlit simply reuses the cached version during the session.

This is important for performance because with caching, the vector store is created only once per session, making the app faster and more efficient.

The app will:

• Retrieve relevant chunks from the guideline document using FAISS.
• Pass the chunks to the LLM with a structured prompt for a transparent answer.
• Display results cleanly, always showing the intro (Chunk 0) and only the chunks actually cited in the answer.
• Present a clear final answer with citations, making it easy to see how the response was generated.

Once that’s done, open your terminal, navigate to the project folder, and run the app using:

streamlit run app.py

This command launches a local web server and opens the app in your browser, where you can ask questions, see the retrieved guideline chunks, and view the final cited answers in real time.

Press enter or click to view image in full size

In a world full of black-box AI, explainability is a feature.

Conclusion

Building a transparent RAG system for community guidelines changes how developers interact with documentation. Instead of searching manually, guessing answers, they can ask questions, see sources, and trust responses, and with tools like LangChain, FAISS, and Streamlit, this is no longer complex to build, it’s accessible.

My name is Happiness. Are you looking for someone to tell your story? I am a technical writer and technical video creator, exploring DevRel. Send a DM, let’s collaborate!

I’m that person who gets excited about turning complex tech jargon into something that feels like a casual conversation. Whether it’s breaking down AI concepts or writing practical tutorials, I enjoy creating content that helps people learn without feeling overwhelmed.

I’m also big on community building. I love sharing knowledge, engaging with people, and helping others grow in tech.

I would love to contribute and create content that truly connects with readers.