What is the difference between the models writing code “well” and their performance in this context? Are we referring to readability?
Genuine question. If we use agents to read, edit, and review code, why do we care about readability? That’s a human constraint. Unless attempting to do those three is not effective and thus requires human attention to correct issues which would justify readable code. If that’s the case; why use the agent to edit the code in the first place?
To quote the research paper conclusion…
Conclusion We evaluate the impact of context files on coding agent performance for four common coding agents on SWE-BENCH and the novel CTXBENCH, built from recent GitHub issues and less popular repositories containing developer-written context files. We find that all context files consistently increase the cost and number of steps required to complete tasks. LLM-generated context files have a marginal negative effect on task success rates, while developer-written ones provide a marginal performance gain, neither statistically significant. Our trace analyses show that instructions in context files are generally followed and lead to more test- ing and broader exploration; however, they do not function as effective repository overviews. Over- all, our results suggest that context files don’t improve coding agent performance, and should only contain specific additional instructions beyond what is already available in the codebase. This high- lights a concrete gap between current agent-developer recommendations and observed outcomes, and motivates future work on principled ways to automatically generate concise, task-relevant guid- ance for coding agents.
That sounds like “Agents.MD doesn’t work” to me.
Am I missing something?
This. This 1000%.
Software is not a mass-produced commodity that has to be distributed as fast as possible. People already have a limited attention span for the features of already existing software so speed running even more does not create value.
At some point we’re going to have to rekon with the fact that creating, reviewing, and provisioning aren’t the real bottlenecks.
Humans using the software is the bottle neck. Humans trusting the software is the bottleneck. Humans building communities and cultures around specific tools and methodologies is the bottleneck.
At the end of the day you’re user base isn’t going to be “AI” agents. It’s going to be people.
This might be one of the best summaries of LLMs in their current state I’ve seen here. People claiming they’re useless are stuck in 23’ but to say they’re at AGI/ASI is accelerationist techo-paganism.
Based and levelheaded take
Totally unrelated but another post on here is of a news article that uses precisely the same thumbnail as this one to describe how Ukraine is considering taxing porn to fund defense.
Guix.
Try Guix. It’s great.
Okay I’m seeing a lot of your replies here this is a really weird argument to make. You’re literally arguing from symantics.
Hi! I use FreeBSD as a daily driver.
I browse the web, code, and game on it. Works great for me.
My current wishlist is for FreeBSD to have a more robust process supervision system (Dinit I’m looking at you) and for pkg to include some kind of declarative way to build.
It’s great, I love it, join devil Linux.
It’s always refreshing to read something that I can be 100% sure was not written by an AI.
A few of the erdos problems have been independently verified. It’s legit.
I’m not a rust programmer, can anyone with experience in systems programming explain a legitimate use case for the uutils other than weakening the GPL? The auther even admits that several coreutils are still GNU in 26.04 which begs one to ask “why the rewrite?”
So I actually got my BS CompSci from WGU so I probably fall in this category. Did 2.5 years at community college for a math associates, ran out of money and joined the military, then finished the degree online in my last year in. I suppose all together it came out to about 4 years and it’s accredited so {shrug}
I have mixed feelings about the degree, it got me the job I have now working as a Linux Sysadmin for a robotics company and working towards a role with the robotics Dev team but the education was thin.
Strictly speaking, if you did all the supplemental material you were given the classes were actually dense as hell but the problem was it was way easier to cram for each test.
That being said, I know a lot of CS grads that don’t know what an array is so honestly I think I’m on the side of “maybe cramming all your education into 4 years is worse than just slowly picking at it over a lifetime”.
I think I’d like to see a system like that. Like IT certs but not complete shit.
The US needs to fall at this point. American hegemony is a threat to global democracy.
Get out fed
Okay but not really? Systemd does not really provide that much usability that any other init system + elogind doesn’t. The benefit to the FOSS environment is that we, as users and developers, can starve out bad actors.
Want Nix but don’t like systemd? Guix. Want Arch but don’t like systemd? Void. You know what “compile” means and don’t want systemd? Gentoo.
I genuinely question if they’re thinking very far into the future. Billions of agents doing what? We didn’t need a rogue AI to turn us into paper clips; we’re going to do that ourselves. Is he thinking local AI at everyone’s computer? People barely use the features of the machines they have now.