• BadRS@lemmy.world
    link
    fedilink
    arrow-up
    14
    arrow-down
    2
    ·
    1 year ago

    It cannot read. It doesn’t see words or letters. It works with Tokens which words are converted into. It cant count the number of letters in a word because it can’t see them. OpenAI has a Tokenizer you can plug a prompt into to see how its broken up, but youre asking a fish to fly.

    • ParkingPsychology@kbin.social
      link
      fedilink
      arrow-up
      2
      ·
      1 year ago

      And a single “S” is also a token. Which has vectors to all other words that start with an S.

      One thing to point out here is that the word sentences is severely mistyped as “sententences”. That’s not going to help.