Isn’t parameters just the number of virtual dials that can be changed, essentially the number of neurons, I don’t think it’s got anything to do with the actual amount of data.
After all distilling works (somehow), by taking all the knowledge of the large AI with a lot of parameters and putting it into a smaller AI with fewer parameters. The amount of data hasn’t changed only the number of parameters. Presumably the inverse is also true, you can have a lot of parameters but not a lot of data.
I’m wondering what training data they are using. The same pirated and scraped data the others are using as well?
The answer is most probably yes. This is genuinely so tiring.
They could use self-play and existing ethical models, like Apertus.
That said, they don’t disclose afaict. Haven’t trained it, so they probably don’t know themselves.
If they don’t disclose wouldn’t that be against the AI Act 2026?
I suspect no, if they haven’t actually trained a model yet?
Presumably you mean after they actually train the thing. Then I assume yes, but I’m unfamiliar with the law.
Yeah 1 trillion params has to be pretty much all of human written creation
Isn’t parameters just the number of virtual dials that can be changed, essentially the number of neurons, I don’t think it’s got anything to do with the actual amount of data.
After all distilling works (somehow), by taking all the knowledge of the large AI with a lot of parameters and putting it into a smaller AI with fewer parameters. The amount of data hasn’t changed only the number of parameters. Presumably the inverse is also true, you can have a lot of parameters but not a lot of data.