Theoretically we could slow down training and coast on fine-tuning existing models. Once the AI’s trained they don’t take that much energy to run.
Everyone was racing towards “bigger is better” because it worked up to GPT4, but word on the street is that raw training is giving diminishing returns so the massive spending on compute is just a waste now.
Issue is, we’re reaching the limits of what GPT technologies can do, so we have to retrain them for the new ones, and currently available data have been already poisoned by AI generated garbage, which will make the adaptation of new technologies harder.
Theoretically we could slow down training and coast on fine-tuning existing models. Once the AI’s trained they don’t take that much energy to run.
Everyone was racing towards “bigger is better” because it worked up to GPT4, but word on the street is that raw training is giving diminishing returns so the massive spending on compute is just a waste now.
Issue is, we’re reaching the limits of what GPT technologies can do, so we have to retrain them for the new ones, and currently available data have been already poisoned by AI generated garbage, which will make the adaptation of new technologies harder.