RAG Example with LLama-Index + Nebius
This example shows querying a PDF using llama index framework and running LLM on Nebius Token Factory
References and Acknowledgements
Features
- Minimal RAG application for querying PDF documents
- Tech Stack
- RAG framework: llamaindex
- open source embedding models: BAAI/bge-small-en (fast and good performance) , BAAI/bge-en-icl (large and more accurate) etc.
- powerful open source LLMs: meta-llama/Llama-3.3-70B-Instruct or openai/gpt-oss-20b
- Nebius Token Factory to run models and embeddings
- Run embedding models locally or on the cloud
Pre requisites
- Nebius API key. Sign up for free at Token Factory
Run the code
Option-1 (Easy!): On Google Colab
No setup required.
Option-2: Run locally
2A (Preferred): Using uv
Install dependencies
uv sync
Run notebook
uv run --with jupyter jupyter lab rag_pdf_query.ipynb
2B: Using pip
pip install -r requirements.txt
jupyter lab rag_pdf_query.ipynb
Dev Notes
How the uv project was created.
uv init .
uv add -r requirements.txt
uv add --dev ipykernel # for jupyter kernel
uv sync
# create a kernel to use uv env within vscode
source .venv/bin/activate
uv run python -m ipykernel install --user --name=$(basename $(pwd)) --display-name "$(basename $(pwd))"
Notebooks in this recipe: