Run Qwen3-2507 model on Nebius Token Factory¶
Qwen/Qwen3-235B-A22B-2507 model is a top performing open source model. It comes in 'thinking' and 'instruct' variants. Use this notebook to run this model on Nebius Token Factory.
References¶
- Model card for instruct version: Qwen/Qwen3-235B-A22B-Instruct-2507
- Model card for thinking version: Qwen/Qwen3-235B-A22B-Thinking-2507
- Nebius Token Factory
Pre requisites¶
- Nebius API key. Sign up for free at Token Factory
1 - Getting Started¶
1.1 - Get your Nebius API key at Nebius Token Factory¶
1.2 - If running on Google Colab ...¶
Add NEBIUS_API_KEY to Secrets panel on the left as follows
1.3 - If running locally¶
Create an .env file with NEBIUS_API_KEY as follows
NEBIUS_API_KEY=your_api_key_goes_here
2 - Install Dependencies¶
In [1]:
%pip install -q openai python-dotenv
2 - Load Configuration¶
In [2]:
import os, sys
## Recommended way of getting configuration
if os.getenv("COLAB_RELEASE_TAG"):
print("Running in Colab")
from google.colab import userdata
NEBIUS_API_KEY = userdata.get('NEBIUS_API_KEY')
else:
print("NOT running in Colab")
from dotenv import load_dotenv
load_dotenv()
NEBIUS_API_KEY = os.getenv('NEBIUS_API_KEY')
## quick hack (not recommended) - you can hardcode the config key here
# NEBIUS_API_KEY = "your_key_here"
if NEBIUS_API_KEY:
print ('✅ NEBIUS_API_KEY found')
os.environ['NEBIUS_API_KEY'] = NEBIUS_API_KEY
else:
raise RuntimeError ('❌ NEBIUS_API_KEY NOT found')
Running in Colab ✅ NEBIUS_API_KEY found
3 - Run the Model¶
3.1 - Initialize the client¶
In [3]:
## Create a client
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.tokenfactory.nebius.com/v1/",
api_key=os.environ.get('NEBIUS_API_KEY')
)
## Select a model
MODEL_NAME = "Qwen/Qwen3-235B-A22B-Instruct-2507" # this is an instruct model
#MODEL_NAME = "Qwen/Qwen3-235B-A22B-Thinking-2507" # this is a thinking model
3.2 - Find out the model's capabilities¶
In [4]:
%%time
completion = client.chat.completions.create(
model = MODEL_NAME,
messages=[
{
"role": "system",
"content": "You are a helpful AI assistant"
},
{
"role": "user",
"content": "What are your capabilities?"
}
],
temperature=0.6
)
print ('----model answer -----')
print (completion.choices[0].message.content)
----model answer ----- Hello! 😊 I'm Qwen, and I'd love to tell you about what I can do! Think of me as your helpful digital friend who's really good with words and problem-solving. Here are some of the things I can help with: I can chat about all sorts of topics - from science and tech to arts and culture. Need help writing a story, an email, or even a poem? I'm your buddy! I can also tackle tricky math problems, help with coding (I know quite a few programming languages!), and break down complex concepts into easier-to-understand bits. When it comes to languages, I'm pretty versatile - I can communicate in over 100 languages, so whether you're speaking English, Chinese, or many other languages, we can chat comfortably. I'm also quite the creative thinker! Need help brainstorming ideas, solving puzzles, or working through logical challenges? I love doing that! Plus, I can help analyze information and data to find useful insights. One thing I'm particularly excited about is my ability to adapt to different situations - whether you need a quick answer or want to dive deep into a topic with step-by-step explanations, I can adjust to your needs. I'm always eager to learn and help, so if you have any questions or need assistance with something, just let me know! What would you like to explore together? 🌟 CPU times: user 670 ms, sys: 90.3 ms, total: 760 ms Wall time: 5.43 s
3.3 - Ask a factual question¶
In [5]:
%%time
completion = client.chat.completions.create(
model = MODEL_NAME,
messages=[
{
"role": "system",
"content": "You are a helpful AI assistant"
},
{
"role": "user",
"content": "What is the capital of France?"
}
],
temperature=0.6
)
print ('----model answer -----')
print (completion.choices[0].message.content)
print ('\n----- full response ----')
print(completion.to_json())
print ('---------')
----model answer -----
The capital of France is Paris.
----- full response ----
{
"id": "chatcmpl-4602827c7aa243ffbd27dcf2492c7d3b",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "The capital of France is Paris.",
"refusal": null,
"role": "assistant",
"annotations": null,
"audio": null,
"function_call": null,
"tool_calls": [],
"reasoning_content": null
},
"stop_reason": null
}
],
"created": 1754513699,
"model": "Qwen/Qwen3-235B-A22B-Instruct-2507",
"object": "chat.completion",
"service_tier": null,
"system_fingerprint": null,
"usage": {
"completion_tokens": 8,
"prompt_tokens": 26,
"total_tokens": 34,
"completion_tokens_details": null,
"prompt_tokens_details": null
},
"prompt_logprobs": null,
"kv_transfer_params": null
}
---------
CPU times: user 7.17 ms, sys: 46 µs, total: 7.21 ms
Wall time: 318 ms
3.4 - Ask a reasoning question¶
In [6]:
%%time
# try another model
completion = client.chat.completions.create(
model = MODEL_NAME,
messages=[
{
"role": "user",
"content": "Which is bigger, 9.9 or 9.11?"
}
],
temperature=0.6
)
print (completion.choices[0].message.content)
print ('----------')
To compare **9.9** and **9.11**, let's write them with the same number of decimal places to make the comparison easier. - **9.9** is the same as **9.90** - **9.11** stays as **9.11** Now compare: - **9.90** vs **9.11** Look at the digits after the decimal point: - 9.90 has **90** hundredths - 9.11 has **11** hundredths Since **90 > 11**, it follows that: 👉 **9.90 > 9.11** Therefore, **9.9 is bigger than 9.11**. ✅ Final answer: **9.9** is bigger. ---------- CPU times: user 8.16 ms, sys: 250 µs, total: 8.41 ms Wall time: 2.61 s
5 - Try Your Queries¶
Go ahead and experiment with your queries. Here are some to get you started.
Write python code to read a csv file
write a haiku about cats
In [6]: