I think the quality of Codex CLI (and Claude Code) is an honest reflection of the status quo of LLM-driven development. The future doesn’t look really promising.
To be fair we've gone from barely capable at writing code to what we have today in barely 3 years. I wouldn't put money on the counter bet that they won't get beyond this hump.
agreed. the thing i reach for most is its ability to coordinate my agent graph, e.g. seeing what sub-agents are doing, messaging agents, adding new capacity. claude code really is a delight to use here
It has pretty rapidly reached the point where I do not think any acceptable LLM-harness is widely available. Every single one I've tried has broken in bizarre ways on even minor updates.
I used codex the first time yesterday and quickly decided I need to sandbox it and came up with a pretty easy and straightforward solution (I think).
I use nixos and use a little wrapper that wraps Codex CLI in Docker. Run codex-box /path/to/project and it starts or reuses a container with only that project bind-mounted, keeping login and history in a separate volume. It uses tmux for detaching and reattaching
There is actually even worse code quality written by humans, is that small sample also a reflection of human-driven development? And therefore, the future doesn't look promising that way either?
The difference is the source of the sample. This is a flagship product by a frontier lab. It should be reflective of the best that is possible. It should be biased in the most favorable possible way. It was designed by people without the normal limitations and quotas of outsiders in building it, as well as in learning experiences in driving it. This is not a random cherry-pick of bad code.
I've used the CLIs for Codex, Gemini and Claude and they all give off the feeling of being on an very rickety, very fast runaway train.
You install an update. Oops, they added a sandbox that doesn't work inside a docker container, it's stopped working.
You install an update. Oops, you can't log in inside a docker container any more, it's stopped working.
You install an update. Oops, they started catching the copy+paste shortcut and replaced it with one that doesn't work.
You revert a broken update. Oops, the update changed the format of your login details, the old version can't read them, gotta log in again.
You install an update. Oops, can't scroll with the mouse wheel any more.
Not showstopping bugs (if you're used to persuading broken oauth logins to work) but it certainly makes updating an "ugh, what have they broken this time?" experience.
Solar power has become a real transformative technology and instead of trying to recreate a better world we tariffed them and the execs of this country spent all our coins on power hungry data-centers that write mid code.
Seems like its just a bunch of young engineers making bad architecture decisions. Not sure I would blame the culture of vibe coding everything. It might just be bad taste.
Codex is very badly structured. I was an early user with codex. But it mixes state and configuration data; which is why I can't really declaratively manage it in my nix-darwin environment; so abandoned it a while ago. This issue being raised, and the way that the codex development responds back to it makes me not trust their harness ever again. I only use codex in opencode; which seems to be doing OK. At least openai doesn't gate their product to their harness like anthropic.
Not trying to sing praises; i think opencode is doing a bunch of weird stuff too but it's not codex level.
There aren't any documented settings to disable this, but there is a feature flag it currently sits behind. You can flip it in your ~/.codex/config.toml[1]:
[features]
daemon_auto_start = false
There are also some files that it writes to in ~/.codex at startup: pid files, lock files, things like that. You can delete them and replace them with symlinks to /dev/null, then mark those unwritable with `chattr +i` or similar. Presumably then the background daemon will crash on startup, if the feature flag ever goes away.
What makes ACP deficient for headless uses of LLM agents? Maybe fixing that or standardizing on another protocol designed for pluggable LLM agents (I guess ACP is more oriented towards pair programming in an IDE, at least originally) is a better idea imo.
For whatever reason, the daemon has been refusing to start and Codex has been grumbling to me. I was wondering where the daemon had come from, and thought maybe I missed something previously and that it had just broken.
Yep, the real issue is that it was something they didn’t have before, and then with Codex 0.157.0 enabled, they installed this persistent nonsense [1] that survives after the last instance of Codex has exited on your machine, which is just gross.
The daemon has many bugs. For example, when auto-updating, it would randomly ruin the permissions and block the goal as .git became read-only. Generally, it is extremely annoying how many bugs Codex has. I basically can't trust it to execute on any goal since it would often get blocked every few hours due to random bugs.
It also fills up my HD with old versions.
I investigated and it took up something like 12 gigabytes in total on my system; which seems insane to me, it’s just a harness?
People moan about how badly made Claude code is but Codex seems much worse on every metric
All of the harnesses I've used are terrible at cleaning up after themselves. They leave all kind of clutter behind, to the point where on my dev VM, I now have an LLM run once a day to just look through the disk and delete everything that looks like it's leftover garbage.
Can't wait until we get really well designed harnesses written in a compiled language that doesn't bloat my computer. Will probably have to be written by a company other than OpenAI because codex is like the exact opposite of everything I just listed
It's because they statically linked it, and probably rely on a trillion pointless crates (as is standard in Rust development). A good statically linked harness can easily be ~5MB MAX
it's minimal and somewhat opinionated, not integrated well with memory subsystems (although that's kind of the point)
my only real criticism is that last I played with it it deals with slow provider responses very poorly , so lots of locally hosted llms require a lot of tuning to get it to handle streaming/non-streaming/timers well.
way less minimal (but still pretty minimal) I prefer Pi; but different markets I think.
hax fit my openbsd laptop well, espescially since Pi and related derivatives are kind of un-spokenly incompatible with the BSDs.
(author passing by) Could you please report the issues you've experienced on the repo? I did quite a bit of testing myself on local models (Qwen, Ornith, Gemma, Muse Glimmer) with llama.cpp, and they all worked fine. Details would be helpful.
Also one brief note for Codex users specifically. hax can work out-of-the-box as codex-cli drop-in replacement (since it can use the already authenticated codex token), so it's super easy to try it - basically it will be a zero-config experience.
I did, and the weird things you have to do to make work at least decent. I’ve been doing evals and I got some things better than opencode but not as good as pi. Though my goal is to optimize for a browser and browser based llm so it can create stuff in a web container or at least generate ui by requesting json schema (a2ui style).
I suspect a lot of people wouldn't use AI if they had to pay per token at current rates. Yes, it's technically a different discussion, but "just write your own harness" isn't really an answer for the vast majority of us riding on the investor-subsidized subscription plans.
> People moan about how badly made Claude code is but Codex seems much worse on every metric
Also, it feels that chatgpt.com and aistudio.google.com are the buggiest websites I have seen for long time. Why it is always something related to LLMs?
for some reason I had 11 separate codex and codex-code-mode-host binaries in my ~/.bun/install/cache/@openai . each weighing about ~300 MB. might be worth a look to anyone running low on disk space.
Yes, my DEV VM tended to fill up with VSCode backups as well. I finally expanded the disk so it's less of an issue today, but for a while ENOSPC was resolved by cleaning up ~/.vscode-server/cli/servers.
I’ve been using GPT models via hax[1], a bare bones harness with minimal tools on my system and have the models rely on bash for the most part the last few months. It’s been a breath of fresh air amidst all these overengineered harnesses.
If I need special actions, I just make them CLI tools on my Mac and tell the AI model that such a tool exists(via a single line in its system prompt) and that’s it. It’s been very nice.
You seem to care, so I wanna point out a semantic nit; "such a tool exists" could be interpreted as a harness provided tool, rather than a command line tool. (Edit, on a re-read its pretty obvious you mean cli tool, but keeping this statment here as im sure it'll open someone's eyes to the difference)
I've gotta give hax a go, as a pi user who came on-board due to the no frills direction pi takes. I say takes rather than took, because regardless of the (heated) discussion surrounding mcp and code mode support, I cannot fight the numbers- augmenting the harness provided tools is a notable boost to execution round trip performance.
I may need to take a second look at Pi again if it has any niceties like that notable boost in round trip performance compared to hax.
I mostly stuck around with Hax because of a few design choices. The sessions are merely plain text files indexed by a dirpath slug. This means I can just start a hax session in that directory and expect sessions history to be available in any model.. (I occasionally dabble in GLM for security checks, Gemini for writing stuff, Deepseek for tightly scoped tests etc).
Off-topic: I think the front-end quality of ChatGPT/Codex Desktop and Android app is simply dreadful—hard to imagine how such a large number of frequent failures made it into production. Do they not use human testing at all?
User experience only gets better with focused taste consistently applied over long periods of time and software evolution. It's easy to backslide and you kind of need a person with authority to tell people to go back and do it again if it isn't up to snuff.
LLMs might, maybe, be able to help you exercise your taste in a focused way over long periods of time and software evolution, but if no one is taking the time, the UX will continue to suck. And no one seems to be.
It's right about now that having a operating system distro model is great - you get folks who will package the software & test it out, make generally sensible defaults, and generally keep shenanigans like this from going on
Ordinary package managers are totally capable of updating thousands of programs without manual effort, including automatically and in the background if that's a thing you want. But they do it without violating users' sovereignty over their own computers or turning every app into a special snowflake that gets to install software (namely itself) in its own bespoke way.
Yeah this seems like a decision yielded by a model. One of countless examples of the model’s outputs careening into individually defensible but holistically nonsensical results
A human seems less likely to embark upon a path that most humans know leads to something a human is unlikely to want
LLMs produce amazing things, but only one token at a time. Anything beyond that is retrospectively emergent at best
Firefox does not only not prompt me for updates (it only updates if I manually run an update), even if I do update with Firefox running, it does not stop me from continuing to use Firefox without restarting it. All I did was install Firefox from flathub (packaged by Mozilla (a default configuration provided by Mozilla)).
I am running Firefox Developer Edition on macOS, where it is the default option. Have to set "Check for updates, but choose when to install" to get the similar behavior.
I've never seen that behaviour, it downloads updates in the background and then gives me a little notification that I should restart it to update, but it's dismiss-able.
Being able to turn auto-updates off is okay, but that being the default would certainly not be okay in Codex, giving the reach, what it can do and how immature it is.
Serious tools shouldn't do this. Sure, you can offer an opt-in rapid release beta update option, but if you can't go more than a month without forcing me into a "stable" update that changes the UI and behavior I'm getting used to, then I'm not going to consider your software stable and reliable enough to be worth developing expertise in using.
Software Engineering clearly is solved, that's why Google has a tiny animated banner on the AI Studio sidebar (advertising subscription upgrade), which shows a simple animated border around it and utilizing exactly 100% of the GPU to render just that banner. And there is a browser extension removing that styling element (without banner GPU load is obviously under 0.1% as it should be) released way back in May. So the almighty AI didn't manage to fix a major UI blocker for a flagship AI product in half a year now. :)
When I had a Claude account (cancelled ~May time) I vividly remember the web chat having broken editing for backtick-quoted monospace text. If you typed <space>`foobar`<space> the last space would move the cursor backward into the closing quote, and the space would be emitted inside the monospaced region. It was like this for several months (on Firefox). I always found this baffling for a product marketed to programmers.
Products moving fast doesn't mean that they're moving in the direction I want, or that I want them to move underneath me while I'm in the middle of a project.
Auto-update systems have value but are not a suitable default for professional tools.
You can just copy the executable then run it with --no-daemon, or edit the config equivalently. No autoupdates and no background process.
The recent terminal UI changes were more fastidious, but even those are easily disabled with CODEX_TUI_DISABLE_KEYBOARD_ENHANCEMENT=1 and the /tui command.
My 2-cents: never install those things into your host. A locked down Docker container is the bare minimum. Pin your versions and never update them without checking the changelog.
Is anyone using the vscode extension of Codex? Its insane how brittle it is.
Just a couple of days ago, it was completely broken for about a week before it got somewhat fixed again. When I say fixed, I mean basic functionality is working again but bugs and regressions are still left. There are multiple issues tracking the same problems. The community have even started to patch the extension themselves as a workaround until its adressed.
You know whats even crazier? There are like a new release every other day with lots of commits and changes, yet it still in a bad shape. I have turned off autoupdate for Codex extension for this reason. The only extension I’ve had to.
I mean, it doesn’t take that much work to have some tests in the pipeline to make sure the extension continue to work, it almost feels like they are purposely doing this.
I have now experienced the consequences of ”move fast and break things”. I realize how much software quality means to me. Its not fun being on the receiving end, especially when it affects my daily workflow.
If any of you, see anyone hype up LLMs again, please refer them to the state of Codex vscode extension.
not to mention the daemon mode breaks a bunch of assumptions one might have about about codex CLI e.g. that codex CLI inherits the env vars of the shell that invokes it.
—no-daemon is the way, but still feels like the noose is tightening …
It's not necessarily hostility. We considered different updating approaches with our legal IDE where security, stability, etc. are paramount, and having a buggy application out in the wild is user-hostile. Sure, in a dream world we'd say "don't ship until it's perfect", but that's just not realistic.
We considered the daemon thing, but ended up with a hard-update check on every boot. It's thus a user choice: if you don't like the automatic updating, you can just not run the application anymore.
And even with all that, it's hard to stay on the right side of AV algorithms, etc.
> It's thus a user choice: if you don't like the automatic updating, you can just not run the application anymore.
That's not a choice. It s just what most apps do these days, but it's the first time I've seen anyone stating this so directly, or trying to frame it as a choice.
(And behaviorally, this makes your product just a SaaS website with extra steps.)
I guess I mean it's a choice in the sense that it's a transparent part of the product that one chooses how and when to use (or not). That's opposed to something running in the background on its own schedule, with its own agenda whether the user is running the application or not.
[Edit: yes, that is kind of the idea. Native version has better features because it can access them.]
This is the same operational model as Claude Code.
ls ~/.local/share/claude/versions/
ps auxww | grep "claude bg"
I see a few prior versions of the Claude Code binary and several running background tasks, without any copy of Claude Code in the foreground. I regularly get a "Update available" message in my Claude Code UI too.
This kind of stuff often breaks for applications packaged with Nix, so it's often removed from Nix packages for compatibility reasons. Both behaviors you mention are helpfully patched out of the Claude Code package I use: https://github.com/numtide/llm-agents.nix/blob/dc285bfd94ec7...
As a result, I never knew that Claude Code does a nasty thing like this, either. :-\
Fwiw, packages in that repo get version bumped each day, so if that's a fast enough pace for you it may be a good alternative.
I use it at work. On Windows it also launches a new console window every it executes (?) a command. You should definitely disabled this using --no-daemon.
I don't know how and why they ruined this CLI with needless "features". It was perfectly usable two weeks ago.
I discovered this while benchmarking the tool I'm developing. I realized Codex was surprisingly fast after the initial run. Some other harnesses do similar things, so I decided to use bwrap around these tools for accurate benchmarking.
Refusal to adhere to the XDG specification is a big signal that the app is going to be a general bull in a china shop. It's not that hard to do OS- and distribution-appropriate package management.
A well-behaved application should not only use XDG on Linux, but it should use platform-native conventions on macOS as well: "~/Library/Application Support", etc.
Yeah, and for the contrary I spotted Claude app being ugly. When I turned off Claude, then put my computer to sleep and later woke it up, it made my fans spin loudly, and after I checked, there were like 2 dozen Claude processes and a lot of Python. What was it doing - I have no idea, but it was crazy. I'm setting up a VM with nothing but Claude and necessary tools. I feel Claude is not honest with everything it is doing.
Can you share some more detail of how you would implement this? Are you thinking something like VirtualBox (?), running a virtual machine, and you're using - for example, OpenCode terminal in the VM...?
We all know all the valuations are fake. Elon taught us this lesson well. Let’s not pretend they don’t have a gorillion users though, including every enterprise on earth.
I think the quality of Codex CLI (and Claude Code) is an honest reflection of the status quo of LLM-driven development. The future doesn’t look really promising.
To be fair we've gone from barely capable at writing code to what we have today in barely 3 years. I wouldn't put money on the counter bet that they won't get beyond this hump.
The product and UX of the tool is distinct from the raw ability of the tool.
Claude code has great UX.
agreed. the thing i reach for most is its ability to coordinate my agent graph, e.g. seeing what sub-agents are doing, messaging agents, adding new capacity. claude code really is a delight to use here
No, the tooling I make has better UX very quickly. It is built for their financial goals, not for the end user.
If we’re learning anything from the past 15 years it’s probably not to extrapolate AI progress in a straight line.
https://finance.yahoo.com/sectors/healthcare/articles/godfat...
Progress is spiky and not guaranteed. The capital costs have also ballooned to the point where the rate has to level out
Time doesn't solve bad designs. We still have lots of buffer overflows in C.
We don't see any indication that the LLM companies are slowing down and go for better code.
Instead it just looks like a cash grab until the AI bubble bursts.
Both CLIs are getting worse. I think it's because they keep bolting in unconsidered "features", not because of the code quality.
But maybe being able to code up a feature at the drop of a hat has turbo-charged their ability to make bad UX decisions.
I say this about 30 times a day, to nobody. "How the FUCK is this all so bad? They have unlimited tokens"
I think it’s a new spin on Brook’s Law. Adding tokens to a project makes it less efficient not more.
It has pretty rapidly reached the point where I do not think any acceptable LLM-harness is widely available. Every single one I've tried has broken in bizarre ways on even minor updates.
I used codex the first time yesterday and quickly decided I need to sandbox it and came up with a pretty easy and straightforward solution (I think).
I use nixos and use a little wrapper that wraps Codex CLI in Docker. Run codex-box /path/to/project and it starts or reuses a container with only that project bind-mounted, keeping login and history in a separate volume. It uses tmux for detaching and reattaching
https://gist.github.com/entropie/8f30b9e33e9412fda3adf2f7507...
Build your own, using the broken harness
Maybe thats just the tool nerd in me that just wants the perfect tool for every job.
There is actually even worse code quality written by humans, is that small sample also a reflection of human-driven development? And therefore, the future doesn't look promising that way either?
The difference is the source of the sample. This is a flagship product by a frontier lab. It should be reflective of the best that is possible. It should be biased in the most favorable possible way. It was designed by people without the normal limitations and quotas of outsiders in building it, as well as in learning experiences in driving it. This is not a random cherry-pick of bad code.
I find codex CLI to be a pretty high quality application. Is the existence of a backend daemon what you're referring to ?
I've used the CLIs for Codex, Gemini and Claude and they all give off the feeling of being on an very rickety, very fast runaway train.
You install an update. Oops, they added a sandbox that doesn't work inside a docker container, it's stopped working.
You install an update. Oops, you can't log in inside a docker container any more, it's stopped working.
You install an update. Oops, they started catching the copy+paste shortcut and replaced it with one that doesn't work.
You revert a broken update. Oops, the update changed the format of your login details, the old version can't read them, gotta log in again.
You install an update. Oops, can't scroll with the mouse wheel any more.
Not showstopping bugs (if you're used to persuading broken oauth logins to work) but it certainly makes updating an "ugh, what have they broken this time?" experience.
To be fair, how many products this old even have a revert update mechanism?
It's buggy like hell once you do not run the default settings. Whole permissions system is completely broken, for example.
It’s kinda vibe coding in a nutshell. Lots of cool and varied features, that often just don’t quite work right.
Totally.
The problem is hardly anyone gives a shit if they can pump out stuff that makes them look like they are good at stuff.
Vanity before engineering.
https://www.jerrypournelle.com/reports/jerryp/iron.html
Solar power has become a real transformative technology and instead of trying to recreate a better world we tariffed them and the execs of this country spent all our coins on power hungry data-centers that write mid code.
Seems like its just a bunch of young engineers making bad architecture decisions. Not sure I would blame the culture of vibe coding everything. It might just be bad taste.
I'm sure if you file a bug on the 15k backlog they'll fix it eventually /s
Legit bugs still open 6 months on.. It's a bit ironic
Codex is very badly structured. I was an early user with codex. But it mixes state and configuration data; which is why I can't really declaratively manage it in my nix-darwin environment; so abandoned it a while ago. This issue being raised, and the way that the codex development responds back to it makes me not trust their harness ever again. I only use codex in opencode; which seems to be doing OK. At least openai doesn't gate their product to their harness like anthropic.
Not trying to sing praises; i think opencode is doing a bunch of weird stuff too but it's not codex level.
There aren't any documented settings to disable this, but there is a feature flag it currently sits behind. You can flip it in your ~/.codex/config.toml[1]:
There are also some files that it writes to in ~/.codex at startup: pid files, lock files, things like that. You can delete them and replace them with symlinks to /dev/null, then mark those unwritable with `chattr +i` or similar. Presumably then the background daemon will crash on startup, if the feature flag ever goes away.--
1: https://github.com/openai/codex/blob/0c329ad2c9d5a2592128a1d...
Yes this should be turned off by default. The daemon is useful when building apps on top of the daemon server, e.g. pi or pi-durable custom harnesses.
What makes ACP deficient for headless uses of LLM agents? Maybe fixing that or standardizing on another protocol designed for pluggable LLM agents (I guess ACP is more oriented towards pair programming in an IDE, at least originally) is a better idea imo.
For whatever reason, the daemon has been refusing to start and Codex has been grumbling to me. I was wondering where the daemon had come from, and thought maybe I missed something previously and that it had just broken.
Yep, the real issue is that it was something they didn’t have before, and then with Codex 0.157.0 enabled, they installed this persistent nonsense [1] that survives after the last instance of Codex has exited on your machine, which is just gross.
[1] - https://github.com/openai/codex/issues/48195The daemon has many bugs. For example, when auto-updating, it would randomly ruin the permissions and block the goal as .git became read-only. Generally, it is extremely annoying how many bugs Codex has. I basically can't trust it to execute on any goal since it would often get blocked every few hours due to random bugs.
https://github.com/openai/codex/issues/52251#issuecomment-60...
Also, in Codex, one randomly can't click "New Chat" or "Scheduled Tasks". You know, the absolute two basic functions.
The CLI has been pretty good on this score, way better than the App. Has been…
It also fills up my HD with old versions. I investigated and it took up something like 12 gigabytes in total on my system; which seems insane to me, it’s just a harness?
People moan about how badly made Claude code is but Codex seems much worse on every metric
All of the harnesses I've used are terrible at cleaning up after themselves. They leave all kind of clutter behind, to the point where on my dev VM, I now have an LLM run once a day to just look through the disk and delete everything that looks like it's leftover garbage.
It's like fighting fire by employing an arsonist.
I just erased 20GB of Cursor’s cruft.
Is that cruft or past conversation history / subagent history and observability logs?
Just garbage. Temp files, worktrees that weren't removed after subagents finished, build files... all kinds of things.
But that's agent behavior, and not harness responsibility...?
Harness problem, or model problem?
Let's be real, what do you think wrote the harness?
> I now have an LLM run once a day to just look through the disk and delete everything that looks like it's leftover garbage
...are you sure it wouldn't consider your personal files garbage?
"In my dev VM". Everything of value in there is in a git repo.
Can't wait until we get really well designed harnesses written in a compiled language that doesn't bloat my computer. Will probably have to be written by a company other than OpenAI because codex is like the exact opposite of everything I just listed
Codex is written in rust, fyi... Not that I disagree.
Oh, I haven't been keeping up. I assumed it was still TS since it prompts me to run npm all the time.
They just use npm as a binary delivery mechanism, for whatever reason (I guess they expect all their users to have npm?)
Which generates unnecessarily huge binaries.
It's because they statically linked it, and probably rely on a trillion pointless crates (as is standard in Rust development). A good statically linked harness can easily be ~5MB MAX
Found this one: https://usehax.dev/
Anyone experiences with it?
it's okay.
it's minimal and somewhat opinionated, not integrated well with memory subsystems (although that's kind of the point)
my only real criticism is that last I played with it it deals with slow provider responses very poorly , so lots of locally hosted llms require a lot of tuning to get it to handle streaming/non-streaming/timers well.
way less minimal (but still pretty minimal) I prefer Pi; but different markets I think.
hax fit my openbsd laptop well, espescially since Pi and related derivatives are kind of un-spokenly incompatible with the BSDs.
(author passing by) Could you please report the issues you've experienced on the repo? I did quite a bit of testing myself on local models (Qwen, Ornith, Gemma, Muse Glimmer) with llama.cpp, and they all worked fine. Details would be helpful.
Also one brief note for Codex users specifically. hax can work out-of-the-box as codex-cli drop-in replacement (since it can use the already authenticated codex token), so it's super easy to try it - basically it will be a zero-config experience.
Omp
Just write your own harness. Ironically Codex/Claude are both abysmal.
I did, and the weird things you have to do to make work at least decent. I’ve been doing evals and I got some things better than opencode but not as good as pi. Though my goal is to optimize for a browser and browser based llm so it can create stuff in a web container or at least generate ui by requesting json schema (a2ui style).
How do you even write a harness that can use a Claude subscription? I thought it was not allowed by Anthropic's ToS and can get you banned?
If you're using the sub, only OpenAI (via the OAuth token) is viable. Claude is API_KEY only
That’s a different discussion.
I suspect a lot of people wouldn't use AI if they had to pay per token at current rates. Yes, it's technically a different discussion, but "just write your own harness" isn't really an answer for the vast majority of us riding on the investor-subsidized subscription plans.
It's allowed, but you get bumped to per token pricing.
That defeats the purpose of using the Claude subscription.
> People moan about how badly made Claude code is but Codex seems much worse on every metric
Also, it feels that chatgpt.com and aistudio.google.com are the buggiest websites I have seen for long time. Why it is always something related to LLMs?
All the LLM turds we've curled out at work are the same.
It’s actually insane how buggy the Gemini web version is (or at least was a few months ago, since I stopped using it).
ChatGPT.com is dreadful. The whole thing chokes when the conversation gets moderately long. It's amazing that it's been allowed to stay this way.
Because they're the most likely to be 100% vibe coded. And as much as some people still deny it, AI code = buggy code.
My preferred Nix package for this patches at least that behavior out, afaict: https://github.com/numtide/llm-agents.nix/blob/dc285bfd94ec7...
https://github.com/numtide/llm-agents.nix/
It’s extremely ironic that the software that’s meant to end software is itself pretty terrible software.
for some reason I had 11 separate codex and codex-code-mode-host binaries in my ~/.bun/install/cache/@openai . each weighing about ~300 MB. might be worth a look to anyone running low on disk space.
I haven't worked with the update subsystem before but that seems like an oversight to me. I'm gonna fix it.
what's the lazy way to check? can you share where/how you cleaned up?
Yep, .codex/packages/standalone/releases keeps all the old versions. Also each codex bin is somehow ~300MB.
thanks, I just removed all the old stuff.
Remember: VSCode runs a complete headless Eclipse instance as its Java LSP, so I don't expect quality code from any of the big companies anymore.
Are you referring to something built into VS Code, or an extension? It looks you're talking about https://github.com/redhat-developer/vscode-java, which is RedHat's doing.
Yes, my DEV VM tended to fill up with VSCode backups as well. I finally expanded the disk so it's less of an issue today, but for a while ENOSPC was resolved by cleaning up ~/.vscode-server/cli/servers.
Even if you installed codex with brew?
I’ve been using GPT models via hax[1], a bare bones harness with minimal tools on my system and have the models rely on bash for the most part the last few months. It’s been a breath of fresh air amidst all these overengineered harnesses.
If I need special actions, I just make them CLI tools on my Mac and tell the AI model that such a tool exists(via a single line in its system prompt) and that’s it. It’s been very nice.
[1] - https://usehax.dev/
No affiliation whatsoever. Just a happy user.
You seem to care, so I wanna point out a semantic nit; "such a tool exists" could be interpreted as a harness provided tool, rather than a command line tool. (Edit, on a re-read its pretty obvious you mean cli tool, but keeping this statment here as im sure it'll open someone's eyes to the difference)
I've gotta give hax a go, as a pi user who came on-board due to the no frills direction pi takes. I say takes rather than took, because regardless of the (heated) discussion surrounding mcp and code mode support, I cannot fight the numbers- augmenting the harness provided tools is a notable boost to execution round trip performance.
I may need to take a second look at Pi again if it has any niceties like that notable boost in round trip performance compared to hax.
I mostly stuck around with Hax because of a few design choices. The sessions are merely plain text files indexed by a dirpath slug. This means I can just start a hax session in that directory and expect sessions history to be available in any model.. (I occasionally dabble in GLM for security checks, Gemini for writing stuff, Deepseek for tightly scoped tests etc).
Off-topic: I think the front-end quality of ChatGPT/Codex Desktop and Android app is simply dreadful—hard to imagine how such a large number of frequent failures made it into production. Do they not use human testing at all?
User experience only gets better with focused taste consistently applied over long periods of time and software evolution. It's easy to backslide and you kind of need a person with authority to tell people to go back and do it again if it isn't up to snuff.
LLMs might, maybe, be able to help you exercise your taste in a focused way over long periods of time and software evolution, but if no one is taking the time, the UX will continue to suck. And no one seems to be.
It's right about now that having a operating system distro model is great - you get folks who will package the software & test it out, make generally sensible defaults, and generally keep shenanigans like this from going on
Nix/NixOS takes it a step further and makes you have to go out of the way to enable self-updating behavior when packaging something :)
What are the top ones you recommend? ( I am running Mint Debian Linux )
This stuff is moving fast, should update automatically. This is kind of standard now. Browsers do it. No reason for harnesses not to do it as well.
Ordinary package managers are totally capable of updating thousands of programs without manual effort, including automatically and in the background if that's a thing you want. But they do it without violating users' sovereignty over their own computers or turning every app into a special snowflake that gets to install software (namely itself) in its own bespoke way.
You can do this without a daemon, and without storing n previous versions as noted by another user.
Yeah this seems like a decision yielded by a model. One of countless examples of the model’s outputs careening into individually defensible but holistically nonsensical results
A human seems less likely to embark upon a path that most humans know leads to something a human is unlikely to want
LLMs produce amazing things, but only one token at a time. Anything beyond that is retrospectively emergent at best
I guarantee the prompt “add auto update feature” was the first and last human input applied to that feature.
My browser doesnt update automatically, i have to approve it.
Maybe this kind of stuff is standard for some people but I absolutely loathe auto updating software for many reasons
One practical reason - system stability for important events.
I travel for work, and often have a tech demo, solution, tool, or something to show - or am running a workshop.
I usually do NOT update things right before my trip, as I want to minimize the potential for disruption.
Isn’t this what docker is for?
Jokes aside I do the same I tend not to upgrade my Mac, my phone unless I really have to.
The “update” button on the top of my brave install has now turned from green to orange and I’ll likely wait until it goes red
> Isn’t this what docker is for?
What? Pinning a browser?
First thing it'll want to do after starting is update itself to newest version anyway.
For example, Firefox updates by default and forces you to restart it, disabling using websites or seeing any site content before doing so.
On your setup.
Firefox does not only not prompt me for updates (it only updates if I manually run an update), even if I do update with Firefox running, it does not stop me from continuing to use Firefox without restarting it. All I did was install Firefox from flathub (packaged by Mozilla (a default configuration provided by Mozilla)).
I am running Firefox Developer Edition on macOS, where it is the default option. Have to set "Check for updates, but choose when to install" to get the similar behavior.
I've been using Firefox for years and don't think I've ever seen this. Are you running some custom configuration?
Standard on Linux, I believe. One of many reasons I got back to Windows as my main OS (with Linux needs fulfilled by WSL).
Firefox Developer Edition on macOS.
I've never seen that behaviour, it downloads updates in the background and then gives me a little notification that I should restart it to update, but it's dismiss-able.
I think this is the old default option. What options you see in the update options?
Are you sure this isn't your organization's update policy kicking in here?
Iirc, this is a pattern of the flatpack version of Firefox. I do not believe the .deb does this.
Being able to turn auto-updates off is okay, but that being the default would certainly not be okay in Codex, giving the reach, what it can do and how immature it is.
Serious tools shouldn't do this. Sure, you can offer an opt-in rapid release beta update option, but if you can't go more than a month without forcing me into a "stable" update that changes the UI and behavior I'm getting used to, then I'm not going to consider your software stable and reliable enough to be worth developing expertise in using.
The entire class of software (harnesses) you're talking about was literally invented like 2 years ago...
If software engineering is solved they should be able to get it right the first time
Software Engineering clearly is solved, that's why Google has a tiny animated banner on the AI Studio sidebar (advertising subscription upgrade), which shows a simple animated border around it and utilizing exactly 100% of the GPU to render just that banner. And there is a browser extension removing that styling element (without banner GPU load is obviously under 0.1% as it should be) released way back in May. So the almighty AI didn't manage to fix a major UI blocker for a flagship AI product in half a year now. :)
When I had a Claude account (cancelled ~May time) I vividly remember the web chat having broken editing for backtick-quoted monospace text. If you typed <space>`foobar`<space> the last space would move the cursor backward into the closing quote, and the space would be emitted inside the monospaced region. It was like this for several months (on Firefox). I always found this baffling for a product marketed to programmers.
Coding is solved, software engineering still TBD
CODEX WRITES ITSELF! DONT YOU PEOPLE UNDERSTAND!!!!!
Nah, you're confusing it with Autolith.
So you imply that Codex has a hoarding issue?
It loves hording issues. Over 15 THOUSAND on GH.
It gets very attached to itself
Products moving fast doesn't mean that they're moving in the direction I want, or that I want them to move underneath me while I'm in the middle of a project.
Auto-update systems have value but are not a suitable default for professional tools.
Then use the package manager, or windows store on windows.
Sure yea just let sam altman install a rootkit on your box.
That's not odd whatsoever. /s
you run it on the bare metal?!! Balls. I run it in a container
You can just copy the executable then run it with --no-daemon, or edit the config equivalently. No autoupdates and no background process.
The recent terminal UI changes were more fastidious, but even those are easily disabled with CODEX_TUI_DISABLE_KEYBOARD_ENHANCEMENT=1 and the /tui command.
My 2-cents: never install those things into your host. A locked down Docker container is the bare minimum. Pin your versions and never update them without checking the changelog.
Is anyone using the vscode extension of Codex? Its insane how brittle it is.
Just a couple of days ago, it was completely broken for about a week before it got somewhat fixed again. When I say fixed, I mean basic functionality is working again but bugs and regressions are still left. There are multiple issues tracking the same problems. The community have even started to patch the extension themselves as a workaround until its adressed.
You know whats even crazier? There are like a new release every other day with lots of commits and changes, yet it still in a bad shape. I have turned off autoupdate for Codex extension for this reason. The only extension I’ve had to.
I mean, it doesn’t take that much work to have some tests in the pipeline to make sure the extension continue to work, it almost feels like they are purposely doing this.
I have now experienced the consequences of ”move fast and break things”. I realize how much software quality means to me. Its not fun being on the receiving end, especially when it affects my daily workflow.
If any of you, see anyone hype up LLMs again, please refer them to the state of Codex vscode extension.
It's as if there was a reason the whole industry was obsessed with quality up to like 3 years ago
not to mention the daemon mode breaks a bunch of assumptions one might have about about codex CLI e.g. that codex CLI inherits the env vars of the shell that invokes it.
—no-daemon is the way, but still feels like the noose is tightening …
It's not necessarily hostility. We considered different updating approaches with our legal IDE where security, stability, etc. are paramount, and having a buggy application out in the wild is user-hostile. Sure, in a dream world we'd say "don't ship until it's perfect", but that's just not realistic.
We considered the daemon thing, but ended up with a hard-update check on every boot. It's thus a user choice: if you don't like the automatic updating, you can just not run the application anymore.
And even with all that, it's hard to stay on the right side of AV algorithms, etc.
We wrote about it here: https://tritium.legal/blog/update
> It's thus a user choice: if you don't like the automatic updating, you can just not run the application anymore.
That's not a choice. It s just what most apps do these days, but it's the first time I've seen anyone stating this so directly, or trying to frame it as a choice.
(And behaviorally, this makes your product just a SaaS website with extra steps.)
I guess I mean it's a choice in the sense that it's a transparent part of the product that one chooses how and when to use (or not). That's opposed to something running in the background on its own schedule, with its own agenda whether the user is running the application or not.
[Edit: yes, that is kind of the idea. Native version has better features because it can access them.]
This is the same operational model as Claude Code.
I see a few prior versions of the Claude Code binary and several running background tasks, without any copy of Claude Code in the foreground. I regularly get a "Update available" message in my Claude Code UI too.This kind of stuff often breaks for applications packaged with Nix, so it's often removed from Nix packages for compatibility reasons. Both behaviors you mention are helpfully patched out of the Claude Code package I use: https://github.com/numtide/llm-agents.nix/blob/dc285bfd94ec7...
As a result, I never knew that Claude Code does a nasty thing like this, either. :-\
Fwiw, packages in that repo get version bumped each day, so if that's a fast enough pace for you it may be a good alternative.
And here is the issue raised in github: https://github.com/anthropics/claude-code/issues/14980
TLDR: They don't care.
I use it at work. On Windows it also launches a new console window every it executes (?) a command. You should definitely disabled this using --no-daemon.
I don't know how and why they ruined this CLI with needless "features". It was perfectly usable two weeks ago.
I discovered this while benchmarking the tool I'm developing. I realized Codex was surprisingly fast after the initial run. Some other harnesses do similar things, so I decided to use bwrap around these tools for accurate benchmarking.
while this points it out, i'll show you what I had in place:
~/.codex/config.toml
hell, I also have this redundancy configured in bash:All I'd want is a CLI with a few tools AND a built-in, properly configurable and working sandbox. Seems that's too much too ask.
Just be the 10th person to make one this week.
Sandbox everything if the OS is not doing what it’s supposed to do
Claude also now hacks your MacOS to put an "Ask Claude" item into every right-click menu. You can remove it through the system's keyboard preferences.
In some circles this is called a backdoor.
Refusal to adhere to the XDG specification is a big signal that the app is going to be a general bull in a china shop. It's not that hard to do OS- and distribution-appropriate package management.
A well-behaved application should not only use XDG on Linux, but it should use platform-native conventions on macOS as well: "~/Library/Application Support", etc.
Yeah, and for the contrary I spotted Claude app being ugly. When I turned off Claude, then put my computer to sleep and later woke it up, it made my fans spin loudly, and after I checked, there were like 2 dozen Claude processes and a lot of Python. What was it doing - I have no idea, but it was crazy. I'm setting up a VM with nothing but Claude and necessary tools. I feel Claude is not honest with everything it is doing.
Can you share some more detail of how you would implement this? Are you thinking something like VirtualBox (?), running a virtual machine, and you're using - for example, OpenCode terminal in the VM...?
I've used pretty much this type of setup:
https://michael.stapelberg.ch/posts/2026-02-01-coding-agent-...
If you want something not NixOS-related, smolmachines.com (I haven't tried this yet) gets mentioned here a lot.
[flagged]
[dead]
[dead]
[flagged]
this is specifically about a "trillion" dollar company and the boot it needs you to lick to justify it's existence.
We all know all the valuations are fake. Elon taught us this lesson well. Let’s not pretend they don’t have a gorillion users though, including every enterprise on earth.