The conversation

chubotHNSource author
Lessons learnedExperience excerpt · English

First I tried it on a bunch of programming topics -- it definitely gets in the right ballpark, but …

… First I tried it on a bunch of programming topics -- it definitely gets in the right ballpark, but then the details are frequently wrong. I asked which languages have the F# |> operator, and it told me that OCaml, Elixir, Haskell and Scala do. 2 of them are correct, and 2 appeared wrong, so I googled it. After Googling, then I asked it specifically if Haskell does, and it says no. I ask it if Scala does, and it apologizes for its earlier mistake. It gave me a good although slightly shallow answer about how PyPy works, but then fell down hard for a prolog type checker, and a few other programming queries. It doesn't understand the question. --- It is generally very mealy-mouthed and vague. It often doesn't answer the question directly, but just tells you what it happens to know in the ballpark. I like another framing I saw on Twitter -- it's "mansplaining as a service". If you want someone to confidently explain things you didn't really ask about, and be frequently wrong, that's kinda what it does!! …

Original discussionProgrammer Salaries in the Age of LLMs152 source comments · at import
5

What was this responding to?

Parent context · original excerpt

GE
geraneumHN

There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term. As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. Maybe some other architecture will prove more useful. LLMs can generate BS authoritatively which is easy to detect for toy examples and forgivable by the AI hype train but can be very costly or hard to fix in a complex system with a lot of moving parts. I’m actually optimistic about the future of AI. At the same time I believe we should know the capabilities and understand the limitations of our tools in order to use them more effectively. Unfortunately people are misunderstanding tools like ChatGPT, etc. for we are human and vulnerable to anthropomorphism. …

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 5

Direct replies available in this import. Original-language excerpts.

DI
disconcisionHN

expectations are interesting things... i was impressed with chatgpt, though not blown away as it's simply the latest in a chain of increasingly impressive research demos. substantially though my experience was the same as yours, though perhaps i'm more forgiving (haskell does have a reverse function application operator, just using a different symbol, and you can get |> in scala via a commonly-used library). i guess one difference might be that i also tried a variety of tasks where confabulation was called for, like making up games and getting it to play them with me. all of this certainly does seem farther from productizability than the optimists admit... it tantalizingly 'almost works' in so many domains, but despite the rate of progress, its not obvious at all to me that it will soon converging on the reliability which is necessary for robust application. playing with the tech in more…

Full original
EL
elcritchHN

That vibes with the images from what I saw with Dall-E. The rendering/styling was superb, however, the logic and scene understanding were toddler-like. Still impressive, but if you had a picture with dogs sometimes a dog head would be pasted on top at the wrong angle. Confabulation is a great term for it IMHO.

Full original
RI
rightbyteHN

I tried it without reading anything about it and I freaked out about losing my job. Then I tried it for any practical programming and realised it was no immediate threat at least and I calmed down abit.

Full original
EL
elevaetHN

Yes, I too have found it to be completely unreliable when it comes to specifics. However, I've been finding it really helpful in a few areas: - taking point form, or messy notes, and converting it to full sentences/paragraphs. Useful for things like writing blurbs, longer emails, proposals. Naturally, it does veer towards cliche, so you kind of have to massage it back after. - getting a bash script started in the right direction. I'm always fuzzy on bash syntax because I don't use it all the time. ChatGPT will usually screw up some details, but those can be fixed - getting a general overview of a new topic area like a new software library, or say a historical period

Full original
BA
barrenkoHN

This is not what it's good at at all. Not sure why people expect this to be it's forte. It's like a really smart sheep dog or a junior junior carpenter. It's currently helping me refactor a .NET app to a trending pattern, not because it know how to do that exactly or which pattern is better, but because it doesn't tire out at the "boring" parts like I do, and it knows the difference (most of time) between IQueryable and ISomething that I just can't bother to google for the Nth time.

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.