The conversation

sarchertechHNSource author
Lessons learnedExperience excerpt · English

What kinds of things are you building? This is not my experience at all

What kinds of things are you building? This is not my experience at all. Just today I asked Claude using opus 4.6 to build out a test harness for a new dynamic database diff tool. Everything seemed to be fine but it built a test suite for an existing diff tool. It set everything up in the new directory, but it was actually testing code and logic from a preexisting directory despite the plan being correct before I told it to execute. I started over and wrote out a few skeleton functions myself then asked it write tests for those to test for some new functionality. Then my plan was to the ask it to add that functionality using the tests as guardrails. Well the tests didn’t actually call any of the functions under test. They just directly implemented the logic I asked for in the tests. After $50 and 2 hours I finally got something working only to realize that instead of creating a new pg database to test against, it found a dev database I had lying around and started adding tables to it. …

Original discussion1M context is now generally available for Opus 4.6 and Sonnet 4.6519 source comments · at import
5

What was this responding to?

Parent context · original excerpt

FR
FrannkyHN

Opus 4.6 is nuts. Everything I throw at it works. Frontend, backend, algorithms—it does not matter. I start with a PRD, ask for a step-by-step plan, and just execute on each step at a time. Sometimes ideas are dumb, but checking and guiding step by step helps it ship working things in hours. It was also the first AI I felt, "Damn, this thing is smarter than me." The other crazy thing is that with today's tech, these things can be made to work at 1k tokens/sec with multiple agents working at the same time, each at that speed.

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 5

Direct replies available in this import. Original-language excerpts.

DC
dcreHN

Curious what language and stack. And have people at your company had marginally more success with greenfield projects like prototypes? I guess that’s what you’re describing, though it sounds like it’s a directory in a monorepo maybe?

Full original
JH
jhatemyjobHN

Similar experience. I use these AI tools on a daily basis. I have tons of examples like yours. In one recent instance I explicitly told it in the prompt to not use memcpy, and it used memcpy anyway, and generated a 30-line diff after thinking for 20 minutes. In that amount of time I created a 10-line diff that didn't use memcpy. I think it's the big investors' extremely powerful incentives manifesting in the form of internet comments. The pace of improvement peaked at GPT-4. There is value in autocomplete-as-a-service, and the "harnesses" like Codex take it a lot farther. But the people who are blown away by these new releases either don't spend a lot of time writing code, or are being paid to be blown away. This is not a hockey stick curve. It's a log curve. Bigger context windows are a welcome addition. And stuff like JSON inputs is nice too. But these things aren't gonna like, take yo…

Full original
EX
extrHN

You probably just don't have the hang of it yet. It's very good but it's not a mind reader and if you have something specific you want, it's best to just articulate that exactly as best you can ("I want a test harness for <specific_tool>, which you can find <here>"). You need to explain that you want tests that assert on observable outcomes and state, not internal structure, use real objects not mocks, property based testing for invariants, etc. It's a feedback loop between yourself and the agent that you must develop a bit before you start seeing "magic" results. A typical session for me looks like: - I ask for something highly general and claude explores a bit and responds. - We go back and forth a bit on precisely what I'm asking for. Maybe I correct it a few times and maybe it has a few ideas I didn't know about/think of. - It writes some kind of plan to a markdown file. In a fresh s…

Full original
JO
JoeMerchantHN

Try https://github.com/gsd-build/get-shit-done . It's been a game changer for me.

Full original
GO
godd2HN

> After about 4 hours and $75 Huh? The max plan is $200/month. How are you spending $75 in 4 hrs?

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.