Doing the Lord’s work in the Devil’s basement

  • 0 Posts
  • 48 Comments
Joined 5 months ago
cake
Cake day: May 8th, 2024

help-circle







  • I suspect a lot comes from the ingredients being mediocre when you buy them at high demand periods.

    I come from the French country side, my father raises poultry and makes his own foie gras and deli meats. When I see the shit they sell at Christmas, which most of my fellow countrymen eat every year… I wouldn’t be surprised reading a comment similar to yours about French Christmas food.

    Maybe your grandma can’t afford the good stuff, or doesn’t have access to it ?













  • Then these models are stupid

    Yup that is kind of the point. They are math functions designed to approximate human tasks.

    These models should start out with basics of language, so they don’t have to learn it from the ground up. That’s the next step. Right now they’re just well read idiots.

    I’m not sure what you’re pointing at here. How they do it right now, simplified, is you have a small model designed to cut text into tokens (“knowledge of syllables”), which are fed into a larger model which turns tokens into semantic information (“knowledge of language”), which is fed to a ridiculously fat model which “accomplishes the task” (“knowledge of things”).

    The first two models are small enough that they can be trained on the kind of data you describe, classic books, movie scripts etc… A couple hundred billion words maybe. But the last one requires orders of magnitude more data, in the trillions.