Today Google has presented a major update to its flagship SOTA model - Gemini 2.0.
What caught my attention was the fact that only one size was presented—the smallest one—Flash. Apparently, it's way cheaper and faster than Gemini 1.5 Pro (the bigger model, a competitor to GPT-4o and Claude 3.5). Besides, this model matches the performance of 1.5 Pro.
Why is there no larger size? If/when the Pro model size follows, will it be a reasonable improvement? Or, as with the Orion/GPT-4.5/GPT-5 rumors, the improvements might be so marginal that they wouldn't be worth the increased cost.
Employees who tested the new model, code-named Orion, reportedly found that even though its performance exceeds OpenAI’s existing models, there was less improvement than they’d seen in the jump from GPT-3 to GPT-4.
Let me also remind you of the GPT-4o update that came in November. It showed a significant bump in speed (from 80 tok/s to 180 tok/s) and showed worse performance across benches. That is right, the newer OpenAI model has actually degraded in its smarts.
Recently there's a lot of talk about Gen AI hitting a wall, plateuing, showing diminishing returns. Gemini 2.0 and GPT-4o (Nov) do present more arguments supporting the statements.
Back in July, I talked about convergence of LLMs, so far there's no indication that newer models get smarter or better reasoners, they are converging to some base level of performance. And this level seems quite mediocre, given the hype and inflated expectations we had since ChatGPT.
Top comments (1)
Until these products are capable of web based deep dives and social policy analysis in an historical context I find them of little utility.
For an undergrad student to copypasta there's I suppose a certain limited utility but as yet I don't think they can replace a good graduate level research assistant
Please! Prove me wrong about provide me a tool that can FOLLOW THE MONEY!
We social scientists really want to know!