LLMs in (March)2025
Open source is the new black
Alexandre Strube
March 26th, 2025
Website
https://helmholtz-blablador.fz-juelich.de
EVERYTHING CHANGED
- OpenAI is no longer the only game in town
- The GPT-4 barrier was broken by Open Source (The
2023 version of GPT-4 is #63 on LM Arena)
- Training costs are WAY down
- Inference costs, too
The rise of China 🇨🇳
- China is now a major player in AI
- And they do a lot of open source!
- 2020: USA 11 LLMs, China 2 LLMs
- 2021: 31 LLMs on both countries (I need more recent
data)
Diagram of thought
Trying to overcome limitations of
Chain-Of-Thought (Andrew Chi-Chih Yao from Tsinghua
University)
Huawei
- Probably the most sanctioned company in the
world
- Selling AI Chips
- Ascend 910C: claimed to be on par with Nvidia’s
H100
- (In practice, their chips are closed to A100)
- Already made by SMIC (previous models were made in
Taiwan by TSMC)
- Has LLMs since 2023 on Huawei Cloud
- Can’t download, but can fine-tune and download the
finetuned models
- Servers in EU comply with EU regulations/AI
act
China Telecom
- Already has a 1 trillion parameter model using
Huawei chips
- TeleChat2-115b
- Meanwhile, Deutsche Telekom gives me 16mbps in
Jülich, like I had in 2005 in Brazil 😥
- Using copper, like the ancient
aztecs or whatever 🗿𓂀𓋹𓁈𓃠𓆃𓅓𓆣
Baichuan AI
- Was the first model on Blablador after Llama/Vicuna
in 2023
- Has a 1 trillion parameter model for a year
already
- Baichuan 2 is Llama 2-level, Baichuan 4 (closed) is
ChatGPT-4 level (china-only)
Baidu
- Used to be their AI leader, playing catch-up
- Fell behind BECAUSE THEY ARE CLOSED SOURCE
- Doing a lot of AI research
- Ernie 4.5 from March 2025 (version 3 is open) -
They swear they’ll open them in June 30th(?)
- Ernie-Health
- Miaoda (no-code dev)
- I-Rag and Wenxin Yige (text to image)
- Ernie X1 is 50% of DeepSeek’s price, same
reasoning
- 100,000+ GPU clusters
Yi (01.ai)
- Yi 1.5 pre-trained on 3.5T tokens and then produces
it’s own data and re-train on it
- Was the best model at the beginning of 2024 on
benchmarks
- IMHO was not so good in practice 😵💫
- Yi Lightning is #12 on LmArena as of
26.01.2025
- Yi VL, multimodal in June (biggest vision model
available, 34b)
- Yi Coder was the best code model until 09.2024
- Went quiet after that (probably busy making money,
last update 11.2024)
- https://01.ai
StepFun
- Step-2 is #8 on LLM Arena (Higher than Claude,
Llama, Grok, Qwen etc)
- Step-1.5V is a strong multimodal model
- Step-1V generates images
- No open source LLM available
Alibaba
- Qwen2.5-1M: 1m tokens for context
- Qwen2.5-VL: (21.03.2025) 32b parameters video
model: can chat through camera, play games, control your phone etc
- BLABLADOR: alias-code for Qwen2.5-coder
- Better than Llama
- Open weights on HuggingFace, modelscope and free
inference too
- 28.01.2025: Qwen 2.5 MAX released (only on their
website and api)
- QwQ-32 is a reasoning model, available on Blablador
(#13 on LM Arena)
- LHM: From picture to animated 3d in seconds
- CEO was warning yesterday about “data center
bubble”
InternLM
- InternLM-3 is an 8B model with 4x less training
time than Llama 3
- Released in 2025.01.15
- Strube’s opinion: Previous versions were bad, need
to check again
Tencent
- Hunyuan-DiT LLM and can create images too
- Hunyuan-T1 (24.03.2025) is a Mamba-like model with
reasoning, beating o1
- The Five: Home Robot
Zhipu
- Real-time video conversations in commercial
products, scientific paper reader, etc
- Open source:
- CodeGeex (runs directly on VSCode),
- LLM: GLM4, GLM-130B
- Visual model: CogVLM 17b/CogAgent
18b/GLM-4v-Plus
- Image generator: GofView-3-Plus
- CogVideoX: Video generator with 5b parameters
- Auto Agent: AutoGLM (does things for you on your
phone or web)
- https://zhipuai.cn/en/
ByteDance: Doubao
- Multi-modal
- 200x cheaper than OpenAI’s GPT-4o
- Video generation diffusion models:
Doubao-PixelDance and Doubao-Seaweed
OpenBMB
- MiniCPM-o 2.6: A GPT-4o-level multimodal model with
8B parameters
- MiniCPM-V 2.6: A image/video/ocr model with 8B
parameters.
You ain’t
here to hear about ModelScope or OpenBMB
DeepSeek R1
![]()