NoteBot is a powerful web-based tool built with Streamlit that allows users to upload PDF files containing notes or text, and interact with those notes by asking questions. It uses LangChain, OpenAI's GPT-3.5, and FAISS for semantic search to provide intelligent, context-based answers to the user's queries.
The main objective is to extract text from PDFs, break it into smaller chunks, store embeddings, and then enable users to ask questions about the content of the document.
Upload PDF: Upload a PDF file containing your notes or text.
Text Extraction: Extracts text from the uploaded PDF document.
Chunking: Breaks the extracted text into smaller, manageable chunks.
Semantic Search: Uses FAISS to perform similarity search on the document chunks.
Question Answering: Use GPT-3.5 (via OpenAI API) to answer questions based on the extracted content.
Customizable Prompt: Customize the AI's behavior with a custom prompt template for more specific interactions.
To run this project locally, you need the following Python libraries:
streamlit
PyPDF2
langchain
langchain_community
langchain_openai
faiss-cpu or faiss-gpu (depending on your setup)
openai
langchain_text_splitters
- Download the repository: Clone or download this repository to your local machine.
git clone <repository_url> cd <project_directory>
- API Key: To interact with OpenAI models, you'll need to provide your OpenAI API key. Replace the placeholder in the code with your actual OpenAI API key:
OpenAI_API_KEY = "Paste your OpenAI API key here"
- Running the Application: To run the app, simply navigate to the project directory and run the following command:
streamlit run app.py
-
Upload a PDF: Once the application is running, open it in your web browser (usually at http://localhost:8501) and upload a PDF containing your notes.
-
Ask Questions: After the PDF is uploaded and processed, you can enter a question in the provided text input field, and NoteBot will return relevant answers based on the content of the PDF.
- PDF Text Extraction
The app uses PyPDF2 to extract text from each page of the uploaded PDF.
my_pdf = PdfReader(file) text = "" for page in my_pdf.pages: text += page.extract_text()
- Text Chunking
After extracting the text, the document is split into smaller chunks using RecursiveCharacterTextSplitter from LangChain. This helps in storing the text as embeddings.
splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=50) chunks = splitter.split_text(text)
- Vector Store and Embeddings
The text chunks are converted into vector embeddings using OpenAIEmbeddings and stored in FAISS for efficient similarity search.
embeddings = OpenAIEmbeddings(api_key=OpenAI_API_KEY) vector_store = FAISS.from_texts(chunks, embeddings)
- User Query Handling
The user enters a query, which is then used to perform a semantic search through the vector store to retrieve the most relevant document chunks.
matching_chunks = vector_store.similarity_search(user_query)
- Question Answering
Using the retrieved chunks, a LangChain pipeline powered by GPT-3.5 (via the OpenAI API) is invoked to generate answers to the user's question. The system uses a custom prompt template to guide the assistant's behavior.
customized_prompt = ChatPromptTemplate.from_template( """ You are my assistant tutor. Answer the question based on the following context and if you did not get the context simply say "I don't know user" : {context} Question: {input} """ ) chain = create_stuff_documents_chain(llm, customized_prompt) output = chain.invoke({"input": user_query, "input_documents": matching_chunks})
Upload a PDF: Upload a document titled "Machine Learning Notes.pdf".
Ask a Question: "What is the difference between supervised and unsupervised learning?"
Response: The app will return a relevant answer based on the document content, like: "Supervised learning involves training a model on a labeled dataset, whereas unsupervised learning involves training a model on unlabeled data..."
The accuracy of the answers depends on the quality and clarity of the text extracted from the PDF.
Some complex PDFs with scanned images or poor formatting may not be processed correctly.
The application only supports English text at the moment.
Feel free to fork the repository, submit issues, or make pull requests if you want to improve the functionality or add new features. Any contributions are welcome!
This project is licensed under the MIT License - see the LICENSE file for details.