this post was submitted on 11 Aug 2023
39 points (91.5% liked)

BecomeMe

753 readers
1 users here now

Social Experiment. Become Me. What I see, you see.

founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 2 points 1 year ago (3 children)

Title feels misleading, it gets stack overflow questions wrong 52% of the time

However it got 77% of easy Leetcode questions correct. Also I believe that's first try, which is not generally how chatgpt should be used.

Also also, you should probably be using a coding specific model if you want good coding results

[–] [email protected] 5 points 1 year ago

Every leetcode question has been answered a billion times and you train it on those billions of answers, it should get those right.

[–] OskarAxolotl 3 points 1 year ago

Probably because the model has seen thousands of possible solutions to those exact Leetcode problems. Actual questions people ask on StackOverflow tend to be much more specialized.

[–] [email protected] 2 points 1 year ago

But it confidently explains the wrong answers.

I just hope politicians don't find out how to use it. It'll be our doom.