• HidingCat@kbin.social
    link
    fedilink
    arrow-up
    5
    ·
    1 year ago

    Not familiar with the tech, but wouldn’t server-side LLMs still have an advantage regardless because of the greater power available on tap? Anything that improves local LLM will also benefit server-side LLMs, wouldn’t it?

    • my_hat_stinks@programming.dev
      link
      fedilink
      English
      arrow-up
      10
      ·
      1 year ago

      Not necessarily, as it gets faster the latency between your local and remote machines becomes a bigger fraction of the time taken to process anything. If your local machine processes in 50ms and the remote machine in 5s, a latency of just 45ms would make your machine faster.

      Running locally also cuts out a lot of potential security issues inherent to sending data over a network, and not sending your data to a third party is a bonus too.

    • bamboo@lemm.ee
      link
      fedilink
      English
      arrow-up
      10
      ·
      1 year ago

      Possibly, but given the choice between paying $20/m for a marginally better version of something that’s free and probably built in to your editor at that point, most people would probably take the free thing. At that point paid llms will need to find new niches beyond simply existing.