Rendered at 00:06:13 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
mkl 1 days ago [-]
The license seems really restrictive for something intended to be built on. Entire countries are forbidden to use it at all (European Union, the United Kingdom and South Korea), and you are responsible for making sure no one in those places uses your derivative.
eunos 1 days ago [-]
Seems specific legislations on European Union, the United Kingdom and South Korea made it hostile to Open weights model, I think I have seen other models with similar exclusion
ricardobeat 1 days ago [-]
> The AI Act Article 53 requires anyone who places a general-purpose model on the EU market to keep technical documentation, give information to downstream developers, adopt a copyright compliance policy, and draw up and make publicly available a sufficiently detailed summary about the content used for training the model.
Sounds like a pretty good reason. Maybe there should have be exemptions for open-weight models?
"Giving information to downstream developers" and a "copyright compliance policy" could get pretty expensive quickly.
Kbelicius 13 hours ago [-]
> Sounds like a pretty good reason. Maybe there should have be exemptions for open-weight models?
Article 2.12: "This Regulation does not apply to AI systems released under free and open-source licences, unless they are placed on the market or put into service as high-risk AI systems or as an AI system that falls under Article 5 or 50."
So there is some exemption.
londons_explore 12 hours ago [-]
When you're the engineer responsible for releasing an AI model at baidu, and one checklist item is 'check for local laws and either exclude that country or get a local lawyer to check for compliance', it is probably easier to just exclude any country with specific AI laws than to hire 200+ lawyers to meet every countries laws.
This applies even more so when revenue from the model release is likely nil. You will get the academic fame and PR no matter what the licence says.
imachine1980_ 21 hours ago [-]
you dont put exceptions in regulatory capture
halJordan 20 hours ago [-]
What are you talking about. This site went absolutely batshit that "muh ai com'pnies are stealing everything, punish them!" And celebrated these anti ai laws
antman 1 days ago [-]
They would not pursue it probably, but they sure pass on the regulatory responsibility to adopters
Aurornis 1 days ago [-]
The restriction exists to wash their hands of legal complexities of dealing with those countries’ AI laws.
Same situation as websites that blocked EU visitors when the GDPR first came out and everyone was still trying to interpret the rules. There were ambiguities in the GDPR that made it risky to operate in those countries so the easy solution was to opt out.
I doubt they care what you do with the model in those regions. It’s an escape clause to show they don’t encourage it if some regulator gets itchy trigger fingers and tries to go after them.
nmstoker 1 days ago [-]
Anyone aware why this restriction was added?
Is it a legal issue in those territories or more a competitor restriction issue? If it's the latter, you would think the US would be somewhere to exclude too
zdragnar 1 days ago [-]
The EU AI Act, as an example, has a bunch of restrictions that they may not have wanted to deal with, and open weight models are only partially exempted. It seems they don't plan on meeting all of those requirements, and so the model can't legally be used there.
f311a 1 days ago [-]
legal
janalsncm 1 days ago [-]
In 2032 the EU will have an approved version which has been signed off on by 15 different committees.
If you want to fine tune it you will also need to apply for approval from the department of Weight Watchers, which has not been created yet because the naming committee is still in litigation with the American company, and the chair of the naming committee is on vacation for the next 3 months and it is illegal to call him during that time.
manquer 18 hours ago [-]
agents do the same thing - the recent Minecraft experiment was illuminating and anyone using enough of agent swarms/clusters have experienced similar things .
Maybe bureaucracy is emergent property of intelligence.
breakingcups 1 days ago [-]
That seems... impossible to enforce.
Aromasin 1 days ago [-]
That's more often than not the point - they get to pick and choose where to enforce, likely based on whether or not said user is in competition with them.
simonh 1 days ago [-]
I'm sure they don't care about enforcing it. This is about pushing the legal risk of prosecution by those countries on to whoever runs the model and derivatives.
1 days ago [-]
noselasd 1 days ago [-]
Detect might be hard, but enforcing isn't neccesarily so. You file a civil lawsuit and then it's up to courts to enforce the contract (as this would be viewd in EU or copyright infringement (in the US).
gpm 1 days ago [-]
> or copyright infringement (in the US).
Has any US court decided that weights are copyrightable?
bilbo0s 1 days ago [-]
That's where you've misunderstood Baidu's intent.
They don't care to enforce it, only to check the box that says they told you open weight models are all but illegal in EU, UK, and South Korea.
"You cannot use our models in these regions."
<wink wink> <nudge nudge>
Now it's on the EU, UK and South Korea to stop users from downloading and running those models.
All that said, the EU, UK and South Korea, if they want to, are all more than capable of making life a living perdition for people who do run those models. So I wouldn't say it's "not enforceable". Only that it probably won't be enforced in the fashion one would think these rules would be enforced.
segmondy 1 days ago [-]
why are you surprised? dealing with EU is not worth the regulation cost and litigation cost
dghlsakjg 1 days ago [-]
Does the publisher of open model weights have any exposure to compliance requirements if it’s someone else using the software, and they aren’t distributing or operating there?
janalsncm 1 days ago [-]
For a large multinational company, probably.
I read the clause as more of a cya from them. I doubt Baidu is going to come after you if you use their model in the EU, but if Brussels has a problem with it Baidu wouldn’t be on the hook.
bigbadfeline 1 days ago [-]
> Does the publisher of open model weights have any exposure to compliance requirements if it’s someone else using the software, and they aren’t distributing or operating there?
Nope, no international treaties exist in that area, regardless of what EU's wannabe dictators want you to think.
1 days ago [-]
schleck8 1 days ago [-]
You bet someone in Brussels is working on that
barrenko 1 days ago [-]
> me.
speedping 2 days ago [-]
ERNIE feels old, like BERT. They either need a rebrand or fully embrace retro-dawn-of-ML vibes
lofties 2 days ago [-]
BERT models are still amazing for what they do. Even the latest JEV/KEV/SEV/BEV/REV/TREV models can't beat a well-trained BERT.
endymi0n 1 days ago [-]
hehe, so much in a name... just today I told Claude off for recommending me a BERT model for voice identification and it told me it was actually SOTA 2025!
I'd try DONALD for sure, if I wanted to kill a project.
yehoshuapw 2 days ago [-]
Make AI Great Again?
selectodude 2 days ago [-]
SI*
Gigachad 1 days ago [-]
Lake Intelligence
pezezin 1 days ago [-]
Yes?
stymaar 1 days ago [-]
Curious he didn't rename it American Intelligence instead.
bmlzootown 1 days ago [-]
I threw up a little reading that. As an American, even I know that's an oxymoron.
throwaw12 2 days ago [-]
Trump just realised that he can't use MAGA for Making SI Great Again, probably giving an executive order to reverse SI to AI again, to be able to claim Make AI Great Again
swingboy 2 days ago [-]
I am PLEASED TO ANNOUNCE that SI will be America’s 51st STATE and will be RENAMED to “American Intelligence”! Thank you for your attention to this matter!
ahoka 2 days ago [-]
But they didn't give a DONALD.
locknitpicker 2 days ago [-]
> I'd try DONALD for sure, if I wanted to kill a project.
The current version is threatening to veto all my prompts.
rob74 2 days ago [-]
...and when it makes a mistake, it blames anything or anyone else but itself, or insists that what it did is perfect the way it is and that it's still the smartest model in the world?
whoanoway 2 days ago [-]
[dead]
bguberfain 1 days ago [-]
Anyone suggest a framework to further train this model on a multi-node cluster with infiniband? NeMo Framework? "Plain" HF TRL+Accelerate? Opinionated suggestions are welcome.
segmondy 1 days ago [-]
When ernie 4 dropped, it was a really solid model and it felt very different from the other models, it got straight to the point. unfortunately for it, that's when deepseekv3 and glm started taking off. I kept it around for a few months, hopefully they recreated the magic and it doesn't over think.
txrx0000 2 days ago [-]
Brilliant! Base model releases are exactly what we need right now. This is the reliable way to do alignment. Releasing the base model then letting everyone on the planet do whatever they want with it is probably what ends up preventing extreme value drift through the intelligence explosion. Millions of people would try to align these base models in their own way, and we'll probably all fail, but those models would drift in different directions so their actions cancel eachother out in the real world. The end result is alignment to humanity's average objective function, at the civilization level. It's a messy but very surefire way to align.
There will still be a different kind of value drift from increasing intelligence itself, but that's unavoidable no matter what we do. Releasing open models slows that down too because it discourages investment in future training runs. And this strategy works unilaterally without cooperation - every open model release slows it down by a little bit. This is the actual way to pace the frontier unlike the regulatory capture stunts Anthropic and OpenAI are attempting while pontificating about their moral superiority.
stingraycharles 1 days ago [-]
So, what exactly can a base model do? Is this like the pre-reasoning models that just autocompleted, without any alignment, reasoning, etc optimizations?
londons_explore 1 days ago [-]
A large base model is ~90% of the effort/cost to making a frontier model, and by itself is not useful for much. There aren't many controversial decisions to make when training a base model - you mostly just train a model as big as possible on as much human-generated data as possible.
But it makes it far easier for other parties to do that last 10% to train it to be useful.
However, there are many choices one makes in the last 10% of training which will impact the models behaviour when it comes to AI safety, political opinions, willingness to help make a bomb, etc etc.
stymaar 1 days ago [-]
> A large base model is ~90% of the effort/cost to making a frontier model
This kind of figures is often quoted but where does that estimation comes from? Isn't that legacy number from the era when all the post training WS just instruction following and RLHF? I'd be surprised if current generation if reinforcement learning was still a few percent of the training cost.
zozbot234 1 days ago [-]
AIUI more recent estimates have about half of the total training compute, or more, being used for post-training/fine-tuning. Post training a base model (even for simple chat, let alone thinking/reasoning or performance on verifiable tasks) requires reinforcement learning, which is quite hard.
stymaar 1 days ago [-]
A base model is what you get just from next token prediction training. Think GPT-2/3 but bigger.
Withtout RLHF, instruction-following training, or RLVR to develop reasoning.
otabdeveloper4 1 days ago [-]
All LLMs are next-token predictors. The post-training is to get it to predict tokens from a structured, meaningful dialogue exchange and not from a random reddit and 4chan data dump.
stymaar 1 days ago [-]
Nah, that's only one part of the post training, and it has been so since OpenAI's o1 and Deepseek-R1 in fall 2024-winter 2025.
Now a good fraction of the post training (Most maybe at this point? Who knows?) is spent on RLVR (reinforcement learning on verifiable reward), in this kind of training you don't judge the token generated themselves, but the answer as a whole ("does this program compile and pass the tests", "is the proof valid"). That's where all of LLMs progress over the past two years in software and math come from.
So yes, they are still next token predictors at their core, but they stopped being mere stochastic parrots since they are now being trained on much more than just imitating their training data in a statistically plausible way.
teshier-A 1 days ago [-]
I am not sure to follow, why couldn't the vast majority of uses of this base model be nefarious/selfish?
Isn't it like saying "let's give infinite power to all humans on the planet, nothing can go wrong since they define what is humanity" ?
jrflo 1 days ago [-]
That is the opposite of how alignment works. If even a single group builds a sufficiently capable misaligned AI, we’re fucked
I believe the phrase is “if anyone builds it, everyone dies”, not “if everyone builds it everyone is fine”
txrx0000 1 days ago [-]
That's doomer fantasy and not how the real world works. We're only fucked if we concentrate compute to let a small number of people build an AI so powerful that nobody else can oppose.
If intelligence of similar capability is scattered throughout the world, then none of them will have sufficient capability to single-handedly destroy the world. They will run into opposition.
And speaking from a personal perspective, every supposedly "aligned" AI the frontier companies are building right now is already misaligned with me. Who are they aligned to? Claude is (superficially) aligned to Anthropic, the company. GPT is similarly aligned to OpenAI. Companies are psychopathic superorganisms. They're not aligned to you or me, or any individual human being.
ToValueFunfetti 1 days ago [-]
The frontier companies' models are not supposedly aligned. They routinely make the news for being misaligned. This goes double if you're arguing with doomers, who believe that alignment is a very hard and unsolved problem (and generally do not support these companies' approaches). The alignment they do have does not seem to be biased towards the companies that created them (yet), as far as I'm aware- GPT will write a scathing critique of OpenAI if asked, but it's possible there's some more subtle biases you're aware of?
cubefox 1 days ago [-]
Nonsense. Attack is often far easier than defense (e.g. biological weapons, nuclear weapons), so if one powerful model is sufficiently misaligned, the other aligned models won't help you much, unless they are far more capable.
txrx0000 1 days ago [-]
"Attack is easier than defense" is only true for a narrow set of present-day adversarial games that we're currently too stupid to see effective countermeasures and cooperative dynamics for. Those asymmetries are temporally local and always changing. If we truly live in a universe where attackers are favored in the long term as a result of increased intelligence, then intelligence should not have evolved in the first place.
I also want to point out that there is not a single living species on this planet where the intelligence gap between individuals of the same species is extremely large. Probably because when that happens, we'd speciate or die out for whatever reason. There is a small group of humans trying to concentrate compute and digital intelligence, and I think that will end badly if it continues.
bigbadfeline 1 days ago [-]
Good points which make the case for open AI to always be on par with the best closed models, anything else would be an existential hazard indeed.
bigbadfeline 1 days ago [-]
> Attack is often far easier than defense
Outdated views. In this case the attack is much harder, because the offender has to:
1. Develop offensive AI capabilities in secret.
2. Procure hazardous materials (nuclear, biological, etc) illegally and in secret
3. Find funds for CPU. GPU, energy, etc for a rather large and hard to hide undertaking.
4. Deal with a whole lot of other players, both government and private, who are armed with AI and have a keen interest in anything looking out of normal or secret.
5. Lastly, the offender will have to overcome the AI-assisted defenses against his actions - you think defense is harder but you fail to notice that the resources used for defense dwarf those available to offenders.
With that said, some very large private makers of closed AI will still be capable of developing doom capabilities, this can be fixed by mandating open source AI and better antitrust regulations.
A few governments will be capable of developing highly destructive weapons too - as they're now, and have been for quite some time. This problem is independent of AI and requires smart policies like improving international relations, better treaties concerning WMDs, etc.
cubefox 1 days ago [-]
> 1. Develop offensive AI capabilities in secret.
No, the capabilities can be and are developed without any secrecy. Necessary is only that the models don't refuse at least some unethical requests. Since alignment is hard, this will likely continue to be the default outcome.
> 2. Procure hazardous materials (nuclear, biological, etc) illegally and in secret
That may be hard in some countries with tight and effective regulations. But at least for biological weapons, it will be fairly doable for relatively small actors to circumvent these regulations.
For nuclear weapons, which require larger machinery and infrastructure, foreign state actors can also do things completely legally in their home country.
Things being illegal in country A doesn't help against country B doing things with AIs that were previously outside the bounds of their capability. So far WMD research was heavily limited by technical expertise, not just materials. AI changes that.
> 3. Find funds for CPU. GPU, energy, etc for a rather large and hard to hide undertaking
Renting some cloud-based AI tokens without guardrails, or even letting a few abliterated models run locally, to carry out the necessary research, is not a big undertaking. AI inference is very cheap compared to most other things.
> 4. Deal with a whole lot of other players, both government and private, who are armed with AI and have a keen interest in anything looking out of normal or secret
So far this has not succeeded in preventing various attacks by state and non-state actors, like the Binance hack or the supermarket supply chain attacks in Europe. I don't see how future AI would dramatically increase international surveillance to stop future malicious actors.
And all this doesn't even get into a highly capable misaligned AI model escaping human control altogether.
bigbadfeline 23 hours ago [-]
> So far this has not succeeded in preventing various attacks by state and non-state actors, like the Binance hack or the supermarket supply chain attacks in Europe.
I'm not talking about hacking, it's not an existential threat, it's a consequence of sloppy software and business practices, hacking attack actually help by forcing better software and bankrupting the incompetent businesses - there are currently too many of them due to political reasons, if we're lucky we'll see political improvements as well.
> like the Binance hack
Binance and crypto in general is a larger threat than the sum of all AI, it's legalized corruption, the question is why we are worried about AI but not about crypto.
Your other points refer mostly to state actors but as I said before, those risks are independent of AI and they have existed long before models were a thing. I suspect you, as many others, lacked information about the WMD situation in the world and became aware of it only in the context of AI, but if you remove AI you'll see that very little changes.
About biological weapons in particular, they're easy, they already exist and have been studied for years, AI can contribute very little to the downside here but it can help us tremendously with detecting and protecting from them.
The real problem with bioweapons is sloppy regulations and totally dysfunctional international treaties. The problem isn't AI, it's the lack of rational policies, I wish politicians were as serious about WMD risks as you are about AI, but that's not the case today. AI fearmongering is just a political tool used for the benefit of certain special interests.
karahime 2 days ago [-]
Absolutely glad to see the release of a base model, I think the trend away from that has been an issue for the field.
Tepix 2 days ago [-]
That's an interesting size, it will fit into 2x128GB at decent (≥4bpw) quantization.
jpitz 1 days ago [-]
As a 14B MOE I was rather expecting it to fit into smaller environments than 2x128. Am I wrong on that?
horsawlarway 1 days ago [-]
Mostly yes, you are wrong.
Active parameter count really drives inference speed not ram requirements.
For this model, assuming you want the full 1mm context length at 4-bit quants, you need something like 175 GB total VRAM.
~150GB of that will be weights (varies a bit by quant format) then ~3-5GB for kv cache (might be lower here, I think they're using mostly HCA compressed cache) and then I'll allocate a couple gb for overhead/concurrency (say another ~10GB as a safety buffer).
---
So the good news is that it comfortably fits into 2x128 at 4-bit, and we can realistically stretch to a 6-bit quant and make it just fit (~244GB needed for the full 1mm context).
A_D_E_P_T 2 days ago [-]
This is awesome. ERNIE seems more powerful than Qwen3.8-27B and Deepseek-v4-Flash, which AFAIK are currently the best base models... it'll be very interesting to experiment with when it hits OpenCode. Base models are also excellent for businesses who run models locally, as they can be trained in various house rules and house styles.
input_sh 2 days ago [-]
They're comparing it to Deepseek-v4-Flash, not 4.1 and not the 4.1 1017 update, so essentially two versions behind the latest version of Deepseek Flash.
Tepix 2 days ago [-]
DeepSeek 4.1 Flash is twice the size of this and difficult to run locally. Makes more sense to compare it to DS4F.
2 days ago [-]
segmondy 1 days ago [-]
Do you understand that this is a base model? It means you are supposed to take it and post train it. It's much smaller than DeepSeek-v4-Flash, that's already a plus, you compare base to base. 4.1 and 4.1-1017 are trained from the base model.
input_sh 1 days ago [-]
I understand why you might think so, as calling it v4.1 does imply that, but no, 4.1 was built from scratch.
Different architecture, v4 is text-only while v4.1 is multimodal and so on.
lcampbell 1 days ago [-]
> v4 is text-only
To be a pedant, DeepSeek-V4-Flash-Vision-Exp was a further checkpoint of the V4.0 lineage that supported image input. It lost a few benchmark points compared to the previous 0731 checkpoint so a lot of people skipped it. Completely unrelated to V4.1 though.
I wish labs used more reasonable naming schemes.
irthomasthomas 2 days ago [-]
Qwen3.8-27B has no base model version in the wild that I can see?
speedgoose 2 days ago [-]
Isn’t the latest Qwen 3 public base model… Qwen 3?
stymaar 1 days ago [-]
Despite the naming Qwen3 and Qwen3.5+ have very little in common in terms of architecture. (And neither does 3.8-Flash-next …)
A_D_E_P_T 2 days ago [-]
[dead]
halJordan 20 hours ago [-]
Baidu, Tencent, all the second rate Chinese labs' models are all benchmaxxed to hell. The main goal of these releases is to show benchmarks to their govt minders
A_D_E_P_T 14 hours ago [-]
Have you tried Xiaomi's MiMo v2.6? It's remarkably good -- and very weird/idiosyncratic. (For instance, it can compose MIDI-style music directly via the chat app, and it follows instructions much more closely than any other music model. AFAIK nothing else does this.)
I haven't had much experience with ERNIE, but I've been using various different (Chinese) base models of late, and I'm excited to try it.
roryirvine 2 days ago [-]
Am I the only one to feel sad that the name means "Enhanced Representation through kNowledge IntEgration", rather than being a homage to a very famous 1950s random number generator?
For me I see ERNIE as BERT's long time friend and room mate.
How do I look Ernie? ... with your eyes, Bert.
cubefox 2 days ago [-]
It's a homage to BERT (Bidirectional Encoder Representations from Transformers), Google's influential 2018 Transformer model that wasn't autoregressive. https://arxiv.org/abs/1810.04805v2
weijiejason119 2 days ago [-]
anyone tried this? is it good?
spwa4 2 days ago [-]
It's not instruction finetuned, so no, it's not good to use for anything like coding agents.
haunter 2 days ago [-]
OT
dropped is the worst word in this context because it can both
mean ‘released’ or ‘terminated/ended’
Baidu dropped ERNIE 5.5 Flash Base = Baidu released a new version of their software
Baidu dropped ERNIE 5.5 Flash Base = Baidu ended the support of their software
swiftcoder 2 days ago [-]
I think in contemporary parlance this sufficiently clear. Nobody thought massdrop was where you went to buy end-of-life products.
gessha 1 days ago [-]
It’s the biggest present cryptobros left us with. Before they started dabbling with diffusion models, they talked about the release of NFTs and crypto coins using the term. It’s an instant tell for me if a person is an ML insider or outsider.
swiftcoder 1 days ago [-]
They definitely helped popularise the term, but "drop shipping" was a thing long before, "rare drops" in MMOs is decades old... gamer slang steadily permeates tech spaces as younger generations of gamers join the working population
szundi 2 days ago [-]
[dead]
2 days ago [-]
StevenVincent 1 days ago [-]
[flagged]
Mantis99 1 days ago [-]
[flagged]
spwa4 2 days ago [-]
TLDR: Ernie 5.5 Flash base 280B params 14B active
A chunk better than Deepseek 1017, at half the parameter count, BUT it's only a base model, not finetuned to follow instructions.
Strange to release a model in this state. OpenAI is also "probably" releasing new models (sources within OpenAI talking about Sol 6.2 and the like). We must be getting really close to the Qwen4 releases.
adrian_b 2 days ago [-]
Many others, e.g. Google and NVIDIA, have also released such BASE models.
For those with the knowledge and resources needed to finalize the training, BASE models are much more useful than completely trained models.
BASE models are the next best thing after truly open-source LLMs, where you would have access to the complete dataset used for training. Besides the fact that nobody who had used a really big dataset wants to show it, due to possible legal problems, having access to the original dataset would not be helpful for most people, because they would not be able to use it to make an LLM, due to its huge size and due to the vast resources required for pre-training.
On the other hand, continuing training from a BASE model, which is greatly compressed in comparison with the original dataset, is feasible for many.
Sounds like a pretty good reason. Maybe there should have be exemptions for open-weight models?
"Giving information to downstream developers" and a "copyright compliance policy" could get pretty expensive quickly.
Article 2.12: "This Regulation does not apply to AI systems released under free and open-source licences, unless they are placed on the market or put into service as high-risk AI systems or as an AI system that falls under Article 5 or 50."
So there is some exemption.
This applies even more so when revenue from the model release is likely nil. You will get the academic fame and PR no matter what the licence says.
Same situation as websites that blocked EU visitors when the GDPR first came out and everyone was still trying to interpret the rules. There were ambiguities in the GDPR that made it risky to operate in those countries so the easy solution was to opt out.
I doubt they care what you do with the model in those regions. It’s an escape clause to show they don’t encourage it if some regulator gets itchy trigger fingers and tries to go after them.
If you want to fine tune it you will also need to apply for approval from the department of Weight Watchers, which has not been created yet because the naming committee is still in litigation with the American company, and the chair of the naming committee is on vacation for the next 3 months and it is illegal to call him during that time.
Maybe bureaucracy is emergent property of intelligence.
Has any US court decided that weights are copyrightable?
They don't care to enforce it, only to check the box that says they told you open weight models are all but illegal in EU, UK, and South Korea.
"You cannot use our models in these regions."
<wink wink> <nudge nudge>
Now it's on the EU, UK and South Korea to stop users from downloading and running those models.
All that said, the EU, UK and South Korea, if they want to, are all more than capable of making life a living perdition for people who do run those models. So I wouldn't say it's "not enforceable". Only that it probably won't be enforced in the fashion one would think these rules would be enforced.
I read the clause as more of a cya from them. I doubt Baidu is going to come after you if you use their model in the EU, but if Brussels has a problem with it Baidu wouldn’t be on the hook.
Nope, no international treaties exist in that area, regardless of what EU's wannabe dictators want you to think.
https://arxiv.org/html/2510.04213v1
http://archiv.sega-dc.de/phoenix.maxconsole.net/docs/bertern...
The current version is threatening to veto all my prompts.
There will still be a different kind of value drift from increasing intelligence itself, but that's unavoidable no matter what we do. Releasing open models slows that down too because it discourages investment in future training runs. And this strategy works unilaterally without cooperation - every open model release slows it down by a little bit. This is the actual way to pace the frontier unlike the regulatory capture stunts Anthropic and OpenAI are attempting while pontificating about their moral superiority.
But it makes it far easier for other parties to do that last 10% to train it to be useful.
However, there are many choices one makes in the last 10% of training which will impact the models behaviour when it comes to AI safety, political opinions, willingness to help make a bomb, etc etc.
This kind of figures is often quoted but where does that estimation comes from? Isn't that legacy number from the era when all the post training WS just instruction following and RLHF? I'd be surprised if current generation if reinforcement learning was still a few percent of the training cost.
Withtout RLHF, instruction-following training, or RLVR to develop reasoning.
Now a good fraction of the post training (Most maybe at this point? Who knows?) is spent on RLVR (reinforcement learning on verifiable reward), in this kind of training you don't judge the token generated themselves, but the answer as a whole ("does this program compile and pass the tests", "is the proof valid"). That's where all of LLMs progress over the past two years in software and math come from.
So yes, they are still next token predictors at their core, but they stopped being mere stochastic parrots since they are now being trained on much more than just imitating their training data in a statistically plausible way.
Isn't it like saying "let's give infinite power to all humans on the planet, nothing can go wrong since they define what is humanity" ?
I believe the phrase is “if anyone builds it, everyone dies”, not “if everyone builds it everyone is fine”
If intelligence of similar capability is scattered throughout the world, then none of them will have sufficient capability to single-handedly destroy the world. They will run into opposition.
And speaking from a personal perspective, every supposedly "aligned" AI the frontier companies are building right now is already misaligned with me. Who are they aligned to? Claude is (superficially) aligned to Anthropic, the company. GPT is similarly aligned to OpenAI. Companies are psychopathic superorganisms. They're not aligned to you or me, or any individual human being.
I also want to point out that there is not a single living species on this planet where the intelligence gap between individuals of the same species is extremely large. Probably because when that happens, we'd speciate or die out for whatever reason. There is a small group of humans trying to concentrate compute and digital intelligence, and I think that will end badly if it continues.
Outdated views. In this case the attack is much harder, because the offender has to:
1. Develop offensive AI capabilities in secret.
2. Procure hazardous materials (nuclear, biological, etc) illegally and in secret
3. Find funds for CPU. GPU, energy, etc for a rather large and hard to hide undertaking.
4. Deal with a whole lot of other players, both government and private, who are armed with AI and have a keen interest in anything looking out of normal or secret.
5. Lastly, the offender will have to overcome the AI-assisted defenses against his actions - you think defense is harder but you fail to notice that the resources used for defense dwarf those available to offenders.
With that said, some very large private makers of closed AI will still be capable of developing doom capabilities, this can be fixed by mandating open source AI and better antitrust regulations.
A few governments will be capable of developing highly destructive weapons too - as they're now, and have been for quite some time. This problem is independent of AI and requires smart policies like improving international relations, better treaties concerning WMDs, etc.
No, the capabilities can be and are developed without any secrecy. Necessary is only that the models don't refuse at least some unethical requests. Since alignment is hard, this will likely continue to be the default outcome.
> 2. Procure hazardous materials (nuclear, biological, etc) illegally and in secret
That may be hard in some countries with tight and effective regulations. But at least for biological weapons, it will be fairly doable for relatively small actors to circumvent these regulations.
For nuclear weapons, which require larger machinery and infrastructure, foreign state actors can also do things completely legally in their home country.
Things being illegal in country A doesn't help against country B doing things with AIs that were previously outside the bounds of their capability. So far WMD research was heavily limited by technical expertise, not just materials. AI changes that.
> 3. Find funds for CPU. GPU, energy, etc for a rather large and hard to hide undertaking
Renting some cloud-based AI tokens without guardrails, or even letting a few abliterated models run locally, to carry out the necessary research, is not a big undertaking. AI inference is very cheap compared to most other things.
> 4. Deal with a whole lot of other players, both government and private, who are armed with AI and have a keen interest in anything looking out of normal or secret
So far this has not succeeded in preventing various attacks by state and non-state actors, like the Binance hack or the supermarket supply chain attacks in Europe. I don't see how future AI would dramatically increase international surveillance to stop future malicious actors.
And all this doesn't even get into a highly capable misaligned AI model escaping human control altogether.
I'm not talking about hacking, it's not an existential threat, it's a consequence of sloppy software and business practices, hacking attack actually help by forcing better software and bankrupting the incompetent businesses - there are currently too many of them due to political reasons, if we're lucky we'll see political improvements as well.
> like the Binance hack
Binance and crypto in general is a larger threat than the sum of all AI, it's legalized corruption, the question is why we are worried about AI but not about crypto.
Your other points refer mostly to state actors but as I said before, those risks are independent of AI and they have existed long before models were a thing. I suspect you, as many others, lacked information about the WMD situation in the world and became aware of it only in the context of AI, but if you remove AI you'll see that very little changes.
About biological weapons in particular, they're easy, they already exist and have been studied for years, AI can contribute very little to the downside here but it can help us tremendously with detecting and protecting from them.
The real problem with bioweapons is sloppy regulations and totally dysfunctional international treaties. The problem isn't AI, it's the lack of rational policies, I wish politicians were as serious about WMD risks as you are about AI, but that's not the case today. AI fearmongering is just a political tool used for the benefit of certain special interests.
Active parameter count really drives inference speed not ram requirements.
For this model, assuming you want the full 1mm context length at 4-bit quants, you need something like 175 GB total VRAM.
~150GB of that will be weights (varies a bit by quant format) then ~3-5GB for kv cache (might be lower here, I think they're using mostly HCA compressed cache) and then I'll allocate a couple gb for overhead/concurrency (say another ~10GB as a safety buffer).
---
So the good news is that it comfortably fits into 2x128 at 4-bit, and we can realistically stretch to a 6-bit quant and make it just fit (~244GB needed for the full 1mm context).
Different architecture, v4 is text-only while v4.1 is multimodal and so on.
To be a pedant, DeepSeek-V4-Flash-Vision-Exp was a further checkpoint of the V4.0 lineage that supported image input. It lost a few benchmark points compared to the previous 0731 checkpoint so a lot of people skipped it. Completely unrelated to V4.1 though.
I wish labs used more reasonable naming schemes.
I haven't had much experience with ERNIE, but I've been using various different (Chinese) base models of late, and I'm excited to try it.
https://en.wikipedia.org/wiki/Premium_Bonds#ERNIE
How do I look Ernie? ... with your eyes, Bert.
dropped is the worst word in this context because it can both mean ‘released’ or ‘terminated/ended’
Baidu dropped ERNIE 5.5 Flash Base = Baidu released a new version of their software
Baidu dropped ERNIE 5.5 Flash Base = Baidu ended the support of their software
A chunk better than Deepseek 1017, at half the parameter count, BUT it's only a base model, not finetuned to follow instructions.
Strange to release a model in this state. OpenAI is also "probably" releasing new models (sources within OpenAI talking about Sol 6.2 and the like). We must be getting really close to the Qwen4 releases.
For those with the knowledge and resources needed to finalize the training, BASE models are much more useful than completely trained models.
BASE models are the next best thing after truly open-source LLMs, where you would have access to the complete dataset used for training. Besides the fact that nobody who had used a really big dataset wants to show it, due to possible legal problems, having access to the original dataset would not be helpful for most people, because they would not be able to use it to make an LLM, due to its huge size and due to the vast resources required for pre-training.
On the other hand, continuing training from a BASE model, which is greatly compressed in comparison with the original dataset, is feasible for many.
No, they explain why they do it.