https://x.com/leothecurious/status/2036812581076119869#:~:text=davinci,Views
yeah,
is one of the few who really get it. he sees through the loud hype. he knows we’re not there yet, he agrees with sutton’s recent and highly controversial take on LLMs being anti-bitter-pilled. the way he explains it is delightfully simple as well. essentially, just because u smuggled in human knowledge via decades of crowdsourced human annotations pertaining to all interesting aspects of reality, doesn’t mean that u can naively advertise the product as scalable in the sense that sutton originally envisioned. LLMs rely very heavily on human knowledge to bootstrap their impressive competence, and once u take that away, the intelligence that remains is severely lacking. tl;dr: just because it wasn’t u who annotated reality by hand for a few decades, doesn’t mean that ur model requires no human annotations.
