Run GLM-4.5 model on Nebius Token Factory¶
zai-org/GLM-4.5 model is a top performing open source model. It comes in 'thinking' and 'instruct' variants. Use this notebook to run this model on Nebius Token Factory.
References¶
- Model card : zai-org/GLM-4.5
- Model card for 'air' version: zai-org/GLM-4.5-Air
- Nebius Token Factory
Pre requisites¶
- Nebius API key. Sign up for free at Token Factory
1 - Getting Started¶
1.1 - Get your Nebius API key at Nebius Token Factory¶
1.2 - If running on Google Colab ...¶
Add NEBIUS_API_KEY to Secrets panel on the left as follows
1.3 - If running locally¶
Create an .env file with NEBIUS_API_KEY as follows
NEBIUS_API_KEY=your_api_key_goes_here
2 - Install Dependencies¶
In [7]:
%pip install -q openai python-dotenv
2 - Load Configuration¶
In [8]:
import os, sys
## Recommended way of getting configuration
if os.getenv("COLAB_RELEASE_TAG"):
print("Running in Colab")
from google.colab import userdata
NEBIUS_API_KEY = userdata.get('NEBIUS_API_KEY')
else:
print("NOT running in Colab")
from dotenv import load_dotenv
load_dotenv()
NEBIUS_API_KEY = os.getenv('NEBIUS_API_KEY')
## quick hack (not recommended) - you can hardcode the config key here
# NEBIUS_API_KEY = "your_key_here"
if NEBIUS_API_KEY:
print ('✅ NEBIUS_API_KEY found')
os.environ['NEBIUS_API_KEY'] = NEBIUS_API_KEY
else:
raise RuntimeError ('❌ NEBIUS_API_KEY NOT found')
Running in Colab ✅ NEBIUS_API_KEY found
3 - Run the Model¶
3.1 - Initialize the client¶
In [9]:
## Create a client
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.tokenfactory.nebius.com/v1/",
api_key=os.environ.get('NEBIUS_API_KEY')
)
## Select a model
MODEL_NAME = "zai-org/GLM-4.5" #
#MODEL_NAME = "zai-org/GLM-4.5-Air" # lighter model
3.2 - Find out the model's capabilities¶
In [10]:
%%time
completion = client.chat.completions.create(
model = MODEL_NAME,
messages=[
{
"role": "system",
"content": "You are a helpful AI assistant"
},
{
"role": "user",
"content": "What are your capabilities?"
}
],
temperature=0.6
)
print ('----model answer -----')
print (completion.choices[0].message.content)
----model answer ----- I'm an AI assistant with a range of capabilities designed to help with various tasks: **Information & Knowledge:** - Answer questions on diverse topics including science, history, technology, and more - Explain complex concepts in simpler terms - Provide summaries of information on subjects you're interested in **Writing & Communication:** - Help draft, edit, and refine written content - Assist with creative writing, emails, reports, and other documents - Brainstorm ideas for projects or content - Improve grammar, style, and clarity in writing **Problem-Solving & Analysis:** - Break down complex problems - Compare and analyze information - Help with decision-making by presenting options - Assist with basic math and logical reasoning **Technical Assistance:** - Help with coding questions and debugging - Explain programming concepts - Assist with technical documentation **Language & Learning:** - Aid with language translation and learning - Explain vocabulary and grammar - Help create study materials and learning strategies I should note that I have limitations too - I don't have real-time information unless browsing is specifically enabled, I can't perform physical tasks, and I shouldn't replace professional advice in specialized fields like medicine or law. What would you like help with today? CPU times: user 10.1 ms, sys: 104 µs, total: 10.2 ms Wall time: 10.6 s
3.3 - Ask a factual question¶
In [11]:
%%time
completion = client.chat.completions.create(
model = MODEL_NAME,
messages=[
{
"role": "system",
"content": "You are a helpful AI assistant"
},
{
"role": "user",
"content": "What is the capital of France?"
}
],
temperature=0.6
)
print ('----model answer -----')
print (completion.choices[0].message.content)
print ('\n----- full response ----')
print(completion.to_json())
print ('---------')
----model answer -----
The capital of France is Paris.
----- full response ----
{
"id": "chatcmpl-66476152779940f8894157b05a09a948",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "The capital of France is Paris.",
"refusal": null,
"role": "assistant",
"annotations": null,
"audio": null,
"function_call": null,
"tool_calls": [],
"reasoning_content": "This is a straightforward factual question about the capital of France. The capital of France is Paris, which has been the capital city for many centuries and is the country's largest city. I'll provide a clear and direct answer."
},
"stop_reason": 151336
}
],
"created": 1754687831,
"model": "zai-org/GLM-4.5",
"object": "chat.completion",
"service_tier": null,
"system_fingerprint": null,
"usage": {
"completion_tokens": 56,
"prompt_tokens": 20,
"total_tokens": 76,
"completion_tokens_details": null,
"prompt_tokens_details": null
},
"prompt_logprobs": null,
"kv_transfer_params": null
}
---------
CPU times: user 3.57 ms, sys: 1.77 ms, total: 5.34 ms
Wall time: 1.28 s
3.4 - Ask a reasoning question¶
In [12]:
%%time
# try another model
completion = client.chat.completions.create(
model = MODEL_NAME,
messages=[
{
"role": "user",
"content": "Which is bigger, 9.9 or 9.11?"
}
],
temperature=0.6
)
print (completion.choices[0].message.content)
print ('----------')
To determine which number is bigger between 9.9 and 9.11, we need to compare their decimal parts carefully, as both share the same whole number part (9). - 9.9 can be written as 9.90 (adding a zero to make it easier to compare with two decimal places). - 9.11 is already expressed with two decimal places. Now, compare the decimal parts: - 9.90 has a decimal part of 0.90 (which is 90 hundredths). - 9.11 has a decimal part of 0.11 (which is 11 hundredths). Since 90 hundredths is greater than 11 hundredths (0.90 > 0.11), it follows that 9.90 > 9.11, and therefore **9.9 is bigger than 9.11**. ### Why this might be confusing: Sometimes people mistakenly think that 9.11 is larger because "11" is greater than "9" when ignoring the decimal point. However, in decimal notation, the place value matters: - In 9.9, the digit 9 is in the tenths place, meaning it represents 9/10 or 0.9. - In 9.11, the digit 1 is in the tenths place (representing 1/10 or 0.1), and the second 1 is in the hundredths place (representing 1/100 or 0.01). Since the tenths place has a higher value than the hundredths place, 9.9 (with 9 tenths) is larger than 9.11 (with only 1 tenth and 1 hundredth). Even if we consider the hundredths, 9 tenths (90 hundredths) is still greater than 11 hundredths. ### Additional verification: - **Subtraction method**: 9.9 - 9.11 = 0.79, which is positive, confirming that 9.9 is larger. - **Number line**: On a number line, 9.9 (or 9.90) is closer to 10.0 than 9.11, so it is to the right and thus larger. In summary, **9.9 is bigger than 9.11**. ---------- CPU times: user 7.64 ms, sys: 82 µs, total: 7.72 ms Wall time: 29.7 s
5 - Try Your Queries¶
Go ahead and experiment with your queries. Here are some to get you started.
Write python code to read a csv file
write a haiku about cats
In [12]: