Disclaimer: Don't take my word because I'm not an AI engineer but a developer for fun. This article aims to provide information about the "disaster" happening because of something known as Devin. This title is also not a clickbait, this tweet was actually posted by Cognition Labs; click here.
The Disaster πβ οΈ
So the day before yesterday(12th March 2024) there was an announcement that shocked the world that has been going pretty viral about Devin, the first AI software engineer.
Let me be honest for a moment, this caused a disaster on Tech Twitter. (you know it if you are active on the platform)
Cognition Labs(its creator) describe Devin as "a tireless, skilled teammate, equally ready to build alongside you or independently complete tasks for you to review."
Sounds promising?
He's pretty good and by pretty good I mean, he's as good as being able to pass actual engineering interviews.
Just imagine you're conducting the interview, you're the interviewer, and then this AI named Devin joins in and wants to do the interview.
Can take on a Side Hustle? π±
According to the announcement he can also do real jobs on Upwork.
This is some crazy stuff.
Congnition Labs πΈ
I don't know about you but I'd never heard of that company before this announcement, they seem like a super new thing.
It seems like they just kind of locked themselves in an underground thing for years just to launch the step in AI into the market.
Well, to be honest, there's pretty much some information here: https://www.cognition-labs.com/blog
Talking about numbers, they are well funded, including a $21 million Series A led by the Founders Fund.
Devin: Performance & Comparison with Other Models
This image shows the impressive performance of Devin on the Benchmark.
The SWE BENCH Benchmark evaluates large language models on real-world issues from GitHub repositories.
The model is given a random open-source code base and tasked with solving issues. The model's task is to generate fixes for problems in the code that pass unit tests.
If all unit tests run successfully, the model gets a score of one. If the model's code fails any unit test, it receives a score of zero.
Devin can solve about three times as many problems unassisted as any other model.
Devin correctly resolves 13.86%* of the issues end-to-end, far exceeding the previous state-of-the-art of 1.96%. Even when given the exact files to edit, the best previous models can only resolve 4.80% of issues.
While 13.86% might seem low, it marks significant progress compared to previous models.
Things You Need to Know π
This thing can:
β
Learn how to use unfamiliar technologies.
β
Build and deploy apps end to end.
β
Autonomously find and fix bugs in codebases.
β
Train and fine-tune its own AI models.
β
Address bugs and feature requests in open-source repositories.
β
Contribute to mature production repositories.
Final Thoughts π€
We, humans, have the power to do anything but something which holds this capacity back is our slow upload(learning) speed.
This is a fact for sure that an AI cannot truly replicate human intelligence.
Human brain is the most complex kilo in the universe, connected by trillions of connections.
The final question still remains: Will this completely replace us?
Drop your insights on this topic.
According to me - Haha, not even close.
Yes, it'll replace bad programmers with good or exceptional programmers or simply, programmers who leverage the use of Artificial Intelligence to efficiently complete work.
By bad, I mean programmers with less than 80% knowledge of a core language/framework and not constantly learning new things.
They also said, "With Devin, engineers can focus on more interesting problems and engineering teams can strive for more ambitious goals." That clearly says that they are not replacing the good developers, just providing a way to the top for them; to build something better.
These insights are greatly influenced by @mehulmpt. (Founder, CEO @ codedamn | I watched his video on YouTube)
I hope you liked the article! β€οΈ
Connect with me: linktree
Happy Coding! π
Thanks for 20866! π€
Top comments (24)
Just like any other AI Bubble, Devin will be dead in few days lol
haha
I guess not.
It's marketing strategy is something we should learn from.
The disaster didn't happen because of a new model but the headline that said "the first AI Software Engineer"
Cognition business is to sell an AI tool, and they are doing an amazing job at marketing their product.
I'm personally excited about all those new tools, and I'm on the Devin waiting list and really looking forward to trying it.
Now, that being said based on the information available, they say that Devin scores 13.86% on the SWE BENCH Benchmark. What I hear is that it still fails 86.14 % of the time. When I visit Cognition's website, I can see that they are hiring Software Engineers.
I don't feel like we are even close to AI replacing Human Software Engineers. I'm personally not afraid, but I keep a close eye on progress and make sure I stay up-to-date and relevant no matter which direction things go.
Thanks for the article Arjun
Same here! My pleasure, bud!
We dont need to be feared from replacement , just we need to adopt it in our life so that we can maintain a balanace with it
yes we can improve our skills and make ourselves more efficient with these new technologies
Agreed! This is something we have to take in mind rather than living in denial.
Just remember that software Engineering isn't coding 100%. Coding is just a small part of the entire Software Development Lifecycle.
AI will and always be a tool for us to iterate & search things faster.
Improve your hard skills, knowledge & experience. Don't let the media drive the fear to your head π
Of course, Al will always, be a tool for us. Thanks for the comment!
I thoughts programmers were last to be replaced but it seems we are first in line ππππ
Don't worry, the Developers know how to create a bug and work on them for their living, lol
That's the funniest thing I have read in a while about coding and stuff. Hats Off!
I too had that thought for a while.
Great post! π
Thanks! Glad you found it helpful.
The day an AI model can understand the actual development requirements of my clients, I'll search for a new job.
I'm ready to bet this will not happen before I retire
The true value of AI will show itself when profit margins start getting cut due to AI caused bugs that AI canβt fix.
I discuss this a lot in my post here:
Will AI Replace Us? π€ π«¨
Best Codes γ» Feb 17
And this guy sums up a lot of this nicely:
Actually, to be lazy is bad, not AI. AI is just a tool. People are not ready to use this kind of tool, but this is another discussion.
When you say that AI can destroy our future, is a statement that was said by other people in other contexts, with Google, calculator, and other tools.
Also, I do not believe that AI will do the same job that programmers do today as some CEOs are trying to sell This belief comes from ideology, not science. I do not think that AI will ever replace humanity. It can not be trained by its own output, so AI will always depend on humans to work.
In my perception, there is no such thing as "war against AIs", we don't fight tools, we just use them.
Exactly!
Great post!
i need improve skill AI.
Yes, we all do.
You can check out courses on YouTube. It's a great place to begin if you have zero knowledge about a field. And after that, you can switch to some paid resource or continue with YouTube.
Believe me, it's literally a free university.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.