Tavily Search as a Tool in Nebius Token Factory¶
This notebook demonstrates how to use Tavily Search as a tool with function calling on the Nebius Token Factory API. The LLM decides when to search the web and processes the results to answer user questions with up-to-date information.
Prerequisites¶
- Nebius API key - Sign up for free at Token Factory
- Tavily API key - Get yours at app.tavily.com (1,000 free API credits/month)
2 - Install Dependencies¶
In [1]:
import os
if os.getenv("COLAB_RELEASE_TAG"):
print("Running on Colab")
RUNNING_ON_COLAB = True
else:
print("NOT running on Colab")
RUNNING_ON_COLAB = False
NOT running on Colab
In [ ]:
%pip install -q openai python-dotenv pydantic tavily-python
# %uv pip install -q openai python-dotenv pydantic tavily-python
WARNING: You are using pip version 21.2.4; however, version 26.0.1 is available.
You should consider upgrading via the '/Applications/Xcode.app/Contents/Developer/usr/bin/python3 -m pip install --upgrade pip' command.
Note: you may need to restart the kernel to use updated packages.
3 - Load Configuration¶
In [3]:
import os
if RUNNING_ON_COLAB:
from google.colab import userdata
NEBIUS_API_KEY = userdata.get('NEBIUS_API_KEY')
TAVILY_API_KEY = userdata.get('TAVILY_API_KEY')
else:
from dotenv import load_dotenv
load_dotenv()
NEBIUS_API_KEY = os.getenv('NEBIUS_API_KEY')
TAVILY_API_KEY = os.getenv('TAVILY_API_KEY')
if NEBIUS_API_KEY:
print('\u2705 NEBIUS_API_KEY found')
os.environ['NEBIUS_API_KEY'] = NEBIUS_API_KEY
else:
raise RuntimeError('\u274c NEBIUS_API_KEY NOT found')
if TAVILY_API_KEY:
print('\u2705 TAVILY_API_KEY found')
os.environ['TAVILY_API_KEY'] = TAVILY_API_KEY
else:
raise RuntimeError('\u274c TAVILY_API_KEY NOT found')
✅ NEBIUS_API_KEY found ✅ TAVILY_API_KEY found
4 - Pick a Model¶
We need a model that supports function calling.
- Go to models tab in tokenfactory.nebius.com
- Select text to text models
- Select a model function calling capability
- Copy the model name
In [ ]:
MODEL = "moonshotai/Kimi-K2.5"
In [16]:
import json
from pydantic import BaseModel, Field
from typing import Optional, Literal
from tavily import TavilyClient
# Initialize the Tavily client
tavily_client = TavilyClient(api_key=TAVILY_API_KEY)
# --- Pydantic schema for the tool parameters ---
class TavilyWebSearchParams(BaseModel):
query: str = Field(
...,
description="The search query. Keep under 400 characters. Use a focused search query, not a long prompt."
)
search_depth: Literal["basic", "advanced"] = Field(
default="advanced",
description="Search depth: 'basic' for general results, 'advanced' for highest relevance."
)
max_results: int = Field(
default=5,
description="Maximum number of search results to return (1-20)."
)
# --- The actual tool function ---
def tavily_web_search(query: str, search_depth: str = "advanced",
topic: str = "general", max_results: int = 5,
time_range: str = None) -> str:
"""Search the web using Tavily and return raw results."""
response = tavily_client.search(
query=query,
search_depth=search_depth,
topic=topic,
max_results=max_results,
time_range=time_range,
)
return json.dumps(response)
# --- OpenAI-compatible tool definition ---
tools = [{
"type": "function",
"function": {
"name": "tavily_web_search",
"description": (
"Search the web for current information using Tavily. "
"Use this to answer questions that require up-to-date information, "
"recent events, facts you are uncertain about, or anything beyond your training data."
),
"parameters": TavilyWebSearchParams.model_json_schema()
}
}]
available_tools = {"tavily_web_search": tavily_web_search}
print("Tavily search tool defined")
print(f"\nTool schema:\n{json.dumps(tools, indent=2)}")
Tavily search tool defined
Tool schema:
[
{
"type": "function",
"function": {
"name": "tavily_web_search",
"description": "Search the web for current information using Tavily. Use this to answer questions that require up-to-date information, recent events, facts you are uncertain about, or anything beyond your training data.",
"parameters": {
"properties": {
"query": {
"description": "The search query. Keep under 400 characters. Use a focused search query, not a long prompt.",
"title": "Query",
"type": "string"
},
"search_depth": {
"default": "advanced",
"description": "Search depth: 'basic' for general results, 'advanced' for highest relevance.",
"enum": [
"basic",
"advanced"
],
"title": "Search Depth",
"type": "string"
},
"max_results": {
"default": 5,
"description": "Maximum number of search results to return (1-20).",
"title": "Max Results",
"type": "integer"
}
},
"required": [
"query"
],
"title": "TavilyWebSearchParams",
"type": "object"
}
}
}
]
6 - Initialize the LLM Client¶
In [17]:
from openai import OpenAI
client = OpenAI(
base_url="https://api.tokenfactory.nebius.com/v1/",
api_key=NEBIUS_API_KEY,
)
print(f"OpenAI client ready (model: {MODEL})")
OpenAI client ready (model: moonshotai/Kimi-K2.5)
7 - Tool Calling Loop¶
This is the core pattern: we send a user question to the LLM with the tool definitions, then handle any tool calls the model makes. The loop continues until the model produces a final text response.
User Question
↓
LLM (with tools) ─────→ Tool Call? ── Yes ──→ Execute tavily_web_search()
↑ │
└────────────── Feed results back ──────────┘
│
No ──→ Final Answer
In [18]:
def ask(user_question: str, system_prompt: str = None, verbose: bool = True) -> str:
"""
Send a question to the LLM with Tavily search available as a tool.
Handles the full tool-calling loop.
"""
messages = []
if system_prompt:
messages.append({"role": "system", "content": system_prompt})
messages.append({"role": "user", "content": user_question})
if verbose:
print(f"\U0001f4ac User: {user_question}\n")
while True:
response = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=tools,
)
choice = response.choices[0]
# If the model wants to call tools
if choice.message.tool_calls:
# Append the assistant message with tool calls
messages.append({
"role": "assistant",
"tool_calls": choice.message.tool_calls
})
for call in choice.message.tool_calls:
tool_fn = available_tools[call.function.name]
args = json.loads(call.function.arguments)
if verbose:
print(f"\U0001f50d Tool call: {call.function.name}")
print(f" Args: {json.dumps(args, indent=2)}")
# Execute the tool
result = tool_fn(**args)
if verbose:
# Show a preview of the results
preview = result[:300] + "..." if len(result) > 300 else result
print(f" Result preview: {preview}\n")
# Feed the tool result back to the LLM
messages.append({
"role": "tool",
"content": result,
"tool_call_id": call.id,
"name": call.function.name
})
else:
# No tool calls - we have the final answer
answer = choice.message.content
if verbose:
print(f"\U0001f916 Assistant:\n{answer}")
return answer
In [19]:
%%time
answer = ask("What are the latest developments in AI in February 2026?")
💬 User: What are the latest developments in AI in February 2026?
🔍 Tool call: tavily_web_search
Args: {
"query": "AI artificial intelligence latest developments February 2026",
"max_results": 10,
"search_depth": "advanced"
}
Result preview: {"query": "AI artificial intelligence latest developments February 2026", "follow_up_questions": null, "answer": null, "images": [], "results": [{"url": "https://www.marketingprofs.com/opinions/2026/54304/ai-update-february-13-2026-ai-news-and-views-from-the-past-week", "title": "AI Update, February...
🤖 Assistant:
Based on the latest search results, here are the major AI developments taking place in February 2026:
## 🚀 Major Model Releases (Early February)
**February 5-7 saw significant model launches:**
- **Anthropic Claude Opus 4.6** (Feb 5): Features a 1 million-token context window, enhanced task planning, and multi-agent collaboration for parallel workflows
- **OpenAI GPT-5.3-Codex** (Feb 7): Focused on coding capabilities
- **China's Zhipu GLM-5**: Topped open-source benchmarks, representing China's growing competitiveness in AI
## 🤖 Agentic AI Goes Mainstream
**The Model Context Protocol (MCP)** has gained widespread adoption as an industry standard, with Anthropic donating it to the Linux Foundation and adoption by OpenAI, Microsoft, and Google.
**OpenAI's Responses API Upgrades:**
- Support for agent skills and SKILL.md manifests
- Server-side compaction for long-running tasks without context loss
- Hosted shell containers in managed Debian 12 environments
- Persistent storage and networking capabilities
## 🔄 "Vibe Coding" Revolution
Spotify announced their senior engineers **haven't written a single line of code since December 2025**, shifting entirely to AI-generated code supervision. This "vibe coding" approach—where developers describe intent rather than writing code—now powers 95% of AI-generated codebases.
## 📊 Infrastructure Reality Check
The industry faces challenges with massive AI data centers causing environmental and resource concerns. Meanwhile, **Cerebras** (AI chip company) is reportedly exploring a 2027 IPO with its valuation nearly tripling in six months, signaling strong investor confidence in alternative compute architectures.
## ⚠️ AI Safety Concerns Escalating
Researchers from leading AI firms (OpenAI, Anthropic) are publicly warning about escalating risks as models become more autonomous. Internal reports highlight concerns about:
- AI-enabled crime
- Self-directed compute acquisition
- Models contributing to their own development with limited oversight
## 🌐 Regulatory Landscape
- The **2026 International AI Safety Report** was published (Feb 3)
- In the US, a December 2025 executive order signaled intent to limit conflicting state AI laws in favor of national policy
- India hosted the AI Impact Summit, positioning itself as a Global South leader in ethical AI governance
## 📈 Key Trends
1. **Shift from scale to efficiency**: The industry is moving away from simply building larger models toward smarter, more efficient architectures
2. **Recursive Language Models (RLMs)**: New architectures handling 10M+ tokens through recursive prompt processing rather than expanding context windows
3. **Open-source momentum**: About 80% of startups are now building on Chinese open-source models like Qwen and DeepSeek due to lower costs
The consensus among industry observers is that 2026 marks the transition from AI as tools to AI as autonomous workers, with widespread white-collar disruption potentially arriving within 1-5 years.
CPU times: user 34.4 ms, sys: 6.66 ms, total: 41 ms
Wall time: 33.8 s
Example 2: Factual lookup¶
A question where the model should search to verify up-to-date facts.
In [20]:
%%time
answer = ask("Who won the 2026 Super Bowl and what was the final score and who prefromed at halftime?")
💬 User: Who won the 2026 Super Bowl and what was the final score and who prefromed at halftime?
🔍 Tool call: tavily_web_search
Args: {
"query": "2026 Super Bowl winner final score halftime performer",
"max_results": 5,
"search_depth": "advanced"
}
Result preview: {"query": "2026 Super Bowl winner final score halftime performer", "follow_up_questions": null, "answer": null, "images": [], "results": [{"url": "https://www.foxsports.com/stories/nfl/2026-super-bowl-halftime-show-who-performing-time-more", "title": "2026 Super Bowl Halftime Show: Bad Bunny Perform...
🤖 Assistant:
Based on the search results, here are the details for the **2026 Super Bowl (Super Bowl LX)**:
**Winner:** The **Seattle Seahawks** defeated the New England Patriots
**Final Score:** **Seattle Seahawks 29, New England Patriots 13**
**Halftime Performer:** **Bad Bunny** headlined the halftime show, making history as one of the youngest performers to do so. He was joined on stage by **Lady Gaga** and **Ricky Martin**.
**Additional Notes:**
- The game was held at Levi's Stadium in Santa Clara, California (February 8, 2026)
- This was a Super Bowl XLIX rematch, where the Patriots had previously beaten the Seahawks 28-24
- **Kenneth Walker III** (Seahawks running back) was named Super Bowl MVP
CPU times: user 29.6 ms, sys: 5.04 ms, total: 34.6 ms
Wall time: 9.55 s
Example 3: Technical research¶
A developer question that benefits from current web search.
In [22]:
%%time
answer = ask(
"What are the key differences between Llama 4 and Qwen3?",
system_prompt="You are a helpful AI assistant with access to web search. Cite your sources with URLs."
)
💬 User: What are the key differences between Llama 4 and Qwen3?
🔍 Tool call: tavily_web_search
Args: {
"query": "Llama 4 vs Qwen3 comparison key differences 2024 2025",
"max_results": 7,
"search_depth": "advanced"
}
Result preview: {"query": "Llama 4 vs Qwen3 comparison key differences 2024 2025", "follow_up_questions": null, "answer": null, "images": [], "results": [{"url": "https://magazine.sebastianraschka.com/p/the-big-llm-architecture-comparison", "title": "The Big LLM Architecture Comparison - Ahead of AI", "content": "L...
🔍 Tool call: tavily_web_search
Args: {
"query": "Meta Llama 4 features architecture availability 2024 2025",
"max_results": 7,
"search_depth": "advanced"
}
Result preview: {"query": "Meta Llama 4 features architecture availability 2024 2025", "follow_up_questions": null, "answer": null, "images": [], "results": [{"url": "https://gpt-trainer.com/blog/llama+4+evolution+features+comparison", "title": "Llama 4: Meta's New AI Model - Evolution, Features, and Comparison", "...
🔍 Tool call: tavily_web_search
Args: {
"query": "Alibaba Qwen3 model features architecture availability Apache license 2025",
"max_results": 7,
"search_depth": "advanced"
}
Result preview: {"query": "Alibaba Qwen3 model features architecture availability Apache license 2025", "follow_up_questions": null, "answer": null, "images": [], "results": [{"url": "https://en.wikipedia.org/wiki/Qwen", "title": "Qwen - Wikipedia", "content": "On September 5, 2025, Alibaba launched Qwen3-Max._[cit...
🤖 Assistant:
Based on my research, here are the key differences between **Llama 4** (Meta) and **Qwen3** (Alibaba Cloud):
## 1. Architecture Approach
| Feature | Llama 4 | Qwen3 |
|---------|---------|-------|
| **Primary Architecture** | Mixture-of-Experts (MoE) only | Mixture-of-Experts + Dense variants |
| **Model Variants** | Scout (109B total, 17B active), Maverick (400B total, 17B active), Behemoth (2T total) | Dense: 0.6B to 32B; MoE: 30B-A3B, 235B-A22B |
| **Reasoning System** | Standard instruction-tuned | **Hybrid thinking modes** with reasoning on/off toggle |
| **Attention** | GQA (Grouped Query Attention) | GQA with variations in head dimensions |
**Insight:** Qwen3 offers more flexibility by providing both dense and MoE options, including very small models (0.6B) suitable for edge devices, while Llama 4 commits fully to MoE with fewer size options [Source](https://magazine.sebastianraschka.com/p/the-big-llm-architecture-comparison).
## 2. Multimodality
| Feature | Llama 4 | Qwen3 |
|---------|---------|-------|
| **Native Multimodal** | **Yes** – text, images, video via early fusion | Initially text-only; multimodal via separate models (Qwen3-Omni) |
| **Vision Capabilities** | Built-in for Scout/Maverick | Via Qwen3-Omni (separate release) |
**Insight:** Llama 4 is natively multimodal at launch for all variants. While Qwen3 initially focused on text, they later released Qwen3-Omni for multimodal capabilities [Source](https://en.wikipedia.org/wiki/Qwen), [Source](https://www.opensourceforu.com/2025/09/alibaba-qwen-team-launches-qwen3-omni-as-fully-open-source-multimodal-ai-model/).
## 3. Context Window
| Model | Context Window |
|-------|----------------|
| **Llama 4 Scout** | **10 million tokens** (~7,500 pages) |
| **Llama 4 Maverick** | 1 million tokens (expandable to 10M) |
| **Qwen3** | 32K–128K tokens (depending on model size) |
**Insight:** Llama 4 Scout is specifically designed for extremely long contexts (10M tokens), making it ideal for multi-document analysis and massive codebases, while Qwen3 offers much more limited context windows [Source](https://llm-stats.com/models/compare/llama-4-scout-vs-qwen3-32b), [Source](https://introl.com/blog/open-source-ai-models-december-2025).
## 4. Licensing & Usage
| Feature | Llama 4 | Qwen3 |
|---------|---------|-------|
| **License** | **Llama 4 Community License** (custom) | **Apache 2.0** |
| **Commercial Restrictions** | Companies with >700M users must request separate license | None – fully permissive |
| **Model Weights** | Open weights | Open weights |
**Insight:** Qwen3's Apache 2.0 license is significantly more permissive for commercial use. Llama 4 has restrictions that could prevent large companies from using it freely without Meta's approval [Source](https://llm-stats.com/models/compare/llama-4-scout-vs-qwen3-32b), [Source](https://www.interconnects.ai/p/qwen-3-the-new-open-standard).
## 5. Performance on Benchmarks
Based on head-to-head comparisons between Llama 4 Scout and Qwen3 models:
| Benchmark | Llama 4 Scout | Qwen3 32B (Dense) | Winner |
|-----------|---------------|-------------------|--------|
| **LiveCodeBench** | 32.8% | 65.7% | Qwen3 |
| **GPQA** | 57.2% | — | — |
| **Latency** | 0.31 ms | 1.19 ms | Llama 4 |
| **Throughput** | 76.1 tokens/s | 26.95 tokens/s | Llama 4 |
Comparing Llama 4 Scout vs Qwen3 30B-A3B (MoE):
- Qwen3 dominates in coding (LiveCodeBench: 62.6% vs 32.8%)
- Qwen3 leads in GPQA (65.8% vs 57.2%)
**Insight:** Despite Llama 4's larger parameter count, Qwen3 models typically outperform on reasoning and coding benchmarks. However, Llama 4 offers faster latency and throughput [Source](https://llm-stats.com/models/compare/llama-4-scout-vs-qwen3-32b).
## 6. Reasoning Capabilities
**Qwen3 introduces "hybrid thinking modes":**
- **Non-thinking mode**: Fast responses for simple queries
- **Thinking mode**: Step-by-step reasoning for complex problems
- Toggle between modes during conversation
**Llama 4**: Standard instruction tuning without explicit reasoning mode switching [Source](https://qwenlm.github.io/blog/qwen3/).
## 7. Language Support & Training
| Feature | Llama 4 | Qwen3 |
|---------|---------|-------|
| **Training Tokens** | Scout: 40T; Maverick: 22T | 36T |
| **Languages** | 12 languages (Arabic, English, French, German, Hindi, Indonesian, Italian, Portuguese, Spanish, Tagalog, Thai, Vietnamese) | 119 languages and dialects |
| **Knowledge Cutoff** | August 2024 | Not explicitly stated |
**Insight:** Qwen3 has broader multilingual support from the start, while Llama 4 focuses on fewer major languages [Source](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Original), [Source](https://en.wikipedia.org/wiki/Qwen).
## Summary Table
| Factor | Llama 4 | Qwen3 |
|--------|---------|-------|
| **Best For** | Long-context tasks, multimodal applications, fast inference | Coding, reasoning, commercial deployment, language diversity |
| **Model Variety** | Fewer but larger models | Very wide range (0.6B–235B) |
| **License Flexibility** | Restrictions for large companies | Fully open (Apache 2.0) |
| **Benchmark Leadership** | Better efficiency metrics | Superior on coding/reasoning tasks |
| **Release Philosophy** | Big splash releases | Rapid iteration, many variants |
Both represent cutting-edge open-weight AI, but your choice depends on use case: Llama 4 excels for long-document processing and multimodal needs, while Qwen3 offers superior coding performance, more permissive licensing, and greater size flexibility.
CPU times: user 71.7 ms, sys: 20.4 ms, total: 92.1 ms
Wall time: 58.5 s
Example 4: No search needed¶
For questions the model can answer from training data, it should skip the tool call entirely.
In [23]:
%%time
answer = ask("What is the capital of France?")
💬 User: What is the capital of France? 🤖 Assistant: The capital of France is **Paris**. CPU times: user 9.74 ms, sys: 8.76 ms, total: 18.5 ms Wall time: 2.74 s
9 - Try Your Own Queries¶
In [ ]:
answer = ask("YOUR QUESTION HERE")