RAG With llama-index + Milvus + Qwen - Part 1¶
References
Step-1: Configuration¶
In [1]:
import os
from dotenv import load_dotenv
load_dotenv()
if os.getenv('NEBIUS_API_KEY'):
print ("✅ Found NEBIUS_API_KEY in environment, using it")
else:
raise ValueError("❌ NEBIUS_API_KEY not found in environment. Please set it in .env file before running this script.")
✅ Found NEBIUS_API_KEY in environment, using it
Step-2: Read documents¶
In [2]:
%%time
from llama_index.core import SimpleDirectoryReader
import pprint
# load documents
documents = SimpleDirectoryReader(
input_dir = '../../data/10k',
).load_data()
print (f"Loaded {len(documents)} chunks")
# print("Document [0].doc_id:", documents[0].doc_id)
# pprint.pprint (documents[0], indent=4)
Loaded 545 chunks CPU times: user 12.2 s, sys: 158 ms, total: 12.4 s Wall time: 11.5 s
Step-3: Setup Embedding Model¶
We have a choice of local embedding model (fast) or running it on the cloud
If running locally:
- choose smaller models
- less accuracy but faster
If running on the cloud
- We can run large models (billions of params)
In [ ]:
import os
from llama_index.core import Settings
# Option 1: Running embedding models on Nebius cloud
from llama_index.embeddings.nebius import NebiusEmbedding
EMBEDDING_MODEL = 'Qwen/Qwen3-Embedding-8B' # 8B params
EMBEDDING_LENGTH = 4096 # Length of the embedding vector
Settings.embed_model = NebiusEmbedding(
model_name=EMBEDDING_MODEL,
embed_batch_size=50, # Batch size for embedding (default is 10)
api_key=os.getenv("NEBIUS_API_KEY") # if not specfified here, it will get taken from env variable
)
## Option 2: Running embedding models locally
# from llama_index.embeddings.huggingface import HuggingFaceEmbedding
# os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'
# Settings.embed_model = HuggingFaceEmbedding(
# # model_name = 'sentence-transformers/all-MiniLM-L6-v2' # 23 M params
# model_name = 'BAAI/bge-small-en-v1.5' # 33M params
# # model_name = 'Qwen/Qwen3-Embedding-0.6B' # 600M params
# # model_name = 'BAAI/bge-en-icl' # 7B params
# #model_name = 'intfloat/multilingual-e5-large-instruct' # 560M params
# )
Step-4: Connect to Milvus¶
In [4]:
from pymilvus import MilvusClient
DB_URI = './rag.db' # For embedded instance
COLLECTION_NAME = 'rag'
milvus_client = MilvusClient(DB_URI)
print ("✅ Connected to Milvus instance: ", DB_URI)
# if we already have a collection, clear it first
if milvus_client.has_collection(collection_name = COLLECTION_NAME):
milvus_client.drop_collection(collection_name = COLLECTION_NAME)
print ('✅ Cleared collection :', COLLECTION_NAME)
/home/sujee/my-stuff/projects/nebius/token-factory-cookbook-1/rag/rag-milvus-1/.venv/lib/python3.13/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.27.2 is exactly one major version older than the runtime version 6.31.1 at schema.proto. Please update the gencode to avoid compatibility violations in the next runtime release. warnings.warn( /home/sujee/my-stuff/projects/nebius/token-factory-cookbook-1/rag/rag-milvus-1/.venv/lib/python3.13/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.27.2 is exactly one major version older than the runtime version 6.31.1 at common.proto. Please update the gencode to avoid compatibility violations in the next runtime release. warnings.warn( /home/sujee/my-stuff/projects/nebius/token-factory-cookbook-1/rag/rag-milvus-1/.venv/lib/python3.13/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.27.2 is exactly one major version older than the runtime version 6.31.1 at milvus.proto. Please update the gencode to avoid compatibility violations in the next runtime release. warnings.warn( /home/sujee/my-stuff/projects/nebius/token-factory-cookbook-1/rag/rag-milvus-1/.venv/lib/python3.13/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.27.2 is exactly one major version older than the runtime version 6.31.1 at rg.proto. Please update the gencode to avoid compatibility violations in the next runtime release. warnings.warn( /home/sujee/my-stuff/projects/nebius/token-factory-cookbook-1/rag/rag-milvus-1/.venv/lib/python3.13/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.27.2 is exactly one major version older than the runtime version 6.31.1 at feder.proto. Please update the gencode to avoid compatibility violations in the next runtime release. warnings.warn( /home/sujee/my-stuff/projects/nebius/token-factory-cookbook-1/rag/rag-milvus-1/.venv/lib/python3.13/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.27.2 is exactly one major version older than the runtime version 6.31.1 at msg.proto. Please update the gencode to avoid compatibility violations in the next runtime release. warnings.warn( /home/sujee/my-stuff/projects/nebius/token-factory-cookbook-1/rag/rag-milvus-1/.venv/lib/python3.13/site-packages/milvus_lite/__init__.py:15: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. from pkg_resources import DistributionNotFound, get_distribution
✅ Connected to Milvus instance: ./rag.db ✅ Cleared collection : rag
In [5]:
%%time
# connect to vector db
from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.vector_stores.milvus import MilvusVectorStore
vector_store = MilvusVectorStore(
uri = DB_URI ,
dim = EMBEDDING_LENGTH ,
collection_name = COLLECTION_NAME,
overwrite=True
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
print ("✅ Connected Llama-index to Milvus instance: ", DB_URI )
✅ Connected Llama-index to Milvus instance: ./rag.db CPU times: user 17.3 ms, sys: 5.96 ms, total: 23.2 ms Wall time: 554 ms
Step-5: Create Index and Save to DB¶
In [6]:
%%time
# create an index
from llama_index.core import VectorStoreIndex
print ("⚙️ Creating index from documents...")
index = VectorStoreIndex.from_documents(
documents, storage_context=storage_context
)
print ("✅ Created index:", index )
print ("✅ Saved index to db ", DB_URI )
2025-08-13 23:20:37,445 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:20:44,626 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:20:51,232 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:20:57,390 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:02,770 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:08,161 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:12,936 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:18,637 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:25,015 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:30,860 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:36,920 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:42,771 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:48,807 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:21:55,170 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:22:01,386 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK" 2025-08-13 23:22:03,877 - INFO - HTTP Request: POST https://api.tokenfactory.nebius.com/v1/embeddings "HTTP/1.1 200 OK"
✅ Created index: <llama_index.core.indices.vector_store.base.VectorStoreIndex object at 0x705a2b6c4050> ✅ Saved index to db ./rag.db CPU times: user 3.72 s, sys: 460 ms, total: 4.18 s Wall time: 1min 33s