The conversation

Lessons learnedExperience excerpt · English

GPT-4 knew to use linear programming and acknowledged the constraints, even without me formatting …

… GPT-4 knew to use linear programming and acknowledged the constraints, even without me formatting the tabular data so the labels were with the values and properly separated! It also ran all of the 2-3 digit integer multiplications/divisions/subtractions/additions correctly. It still failed to "put it all together" in the final step and forgot some constraints. I prompted it "won't I run out of time?" and it acknowledged it then redid it forgetting a different constraint. I wasn't able to get it to come to the right conclusion. It feels like it has learned a pattern for solving these types of questions but hasn't really gained any actual reasoning about whether it's applying the pattern in a way that makes sense. It confidently announces that it followed all of the constraints when the pattern it chose to follow didn't involve one of the constraints. It then acknowledges it was wrong but doesn't apply reason as much as knows to apply a different pattern that fixes that specific issue. …

Original discussionGPT-42,507 source comments · at import
2

What was this responding to?

Parent context · original excerpt

BU

What I don't understand is how GPT-4 is able to do reasonably well on tests like the AMC12: Many of the AMC12 questions require a number of logical/deductive steps. If GPT-4 is simply trained on a large corpus of text, how is it able to do this? Does this imply that there is some emergent deductive ability that you get simply by learning "language?" Or am I missing something? Obviously, I'm assuming that GPT-4 wasn't trained on the exams that it was tested against.

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 2

Direct replies available in this import. Original-language excerpts.

PR

Asking a LLM trained on the internet, full of computer specialists blogging and posting data non-stop for decades to perform something that can be found in a textbooks is like asking a human to flex a muscle.

Full original
DA

You might find this useful: https://www.jonstokes.com/p/chatgpt-explained-a-guide-for-no...

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.