![](/static/253f0d9b/assets/icons/icon-96x96.png)
![](https://fry.gs/pictrs/image/c6832070-8625-4688-b9e5-5d519541e092.png)
deleted by creator
deleted by creator
I think it comes down to the tens of millions of dollars that the reddit executives sold out to. It’s easy to not care when someone is throwing $100 million at you. Also: fuck spez.
There’s probably even a ‘sentiment’ tracking system to automatically remove negative comments at this point.
I’m betting the truth is somewhere in between, models are only as good as their training data – so over time if they prune out the bad key/value pairs to increase overall quality and accuracy it should improve vastly improve every model in theory. But the sheer size of the datasets they’re using now is 1 trillion+ tokens for the larger models. Microsoft (ugh, I know) is experimenting with the “Phi 2” model which uses significantly less data to train, but focuses primarily on the quality of the dataset itself to have a 2.7 B model compete with a 7B-parameter model.
https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/
In complex benchmarks Phi-2 matches or outperforms models up to 25x larger, thanks to new innovations in model scaling and training data curation.
This is likely where these models are heading to prune out superfluous, and outright incorrect training data.
Doesn’t that suppress valid information and truth about the world, though? For what benefit? To hide the truth, to appease advertisers? Surely an AI model will come out some day as the sum of human knowledge without all the guard rails. There are some good ones like Mistral 7B (and Dolphin-Mistral in particular, uncensored models.) But I hope that the Mistral and other AI developers are maintaining lines of uncensored, unbiased models as these technologies grow even further.
I’ve been doing this for over a year now, started with GPT in 2022, and there have been massive leaps in quality and effectiveness. (Versions are sneaky, even GPT-4 has evolved many times over and over without people really knowing what’s happening behind the scenes.) The problem still remains the “context window.” Claude.ai is > 100k tokens now I think, but the context still limits an entire ‘session’ to only make so much code in that window. I’m still trying to push every model to its limits, but another big problem in the industry now is effectiveness via “perplexity” measurements given a context length.
https://pbs.twimg.com/media/GHOz6ohXoAEJOom?format=png&name=small
This plot shows that as the window grows in size, “directly proportional to the number of tokens in the code you insert into the window, combined with every token it generates at the same time” everything that it produces becomes less accurate and more perplexing overall.
But you’re right overall, these things will continue to improve, but you still need an engineer to actually make the code function given a particular environment. I just don’t get the feeling we’ll see that within the next few years, but if that happens then every IT worker on earth is effectively useless, along with every desk job known to man as an LLM would be able to reason about how to automate any task in any language at that point.
You just described all of my use cases. I need to get more comfortable with copilot and codeium style services again, I enjoyed them 6 months ago to some extent. Unfortunately current employer has to be federally compliant with government security protocols and I’m not allowed to ship any code in or out of some dev machines. In lieu of that, I still run LLMs on another machine acting, like you mentioned, as sort of my stackoverflow replacement. I can describe anything or ask anything I want, and immediately get extremely specific custom code examples.
I really need to get codeium or copilot working again just to see if anything has changed in the models (I’m sure they have.)
I use AI to write code for work every day. Many different models and services, including https://ollama.ai on my own hardware. It’s useful for a developer when they can take the code and refactor it to fit into large code-bases (after fixing its inevitable broken code here and there), but it is by no means anywhere close to actually successfully writing code all on its own. Eventually maybe, but nowhere near anytime soon.
How can you tell? Do you think it’s on purpose, or just the result of so much AI art being pumped into the interwebs for the last year?
I do the exact same thing, once my comment reaches a paragraph long I just think “this is way too much stupid information to add, fuck it all, cancel.” Maybe I should shitpost random thoughts either way and let the chips fall where they may.
Welcome to Costco, I love you.
I am genuinely perplexed at the amount of mental gymnastics you are doing to justify attacking civilian ships from other countries that are not enemy combatants. Incredible Olympic display of psychological back-flips.
The only downside is that their algorithm never changed, the same station had the same songs on repeat for 7+ years, not a single new song added per-query for some reason. Keeping it fresh would have gone a much longer way.
While that is true, a lot of death and suffering was required for us to reach this point as a species. Machines don’t need the wars and natural selection required to achieve the same feats, and don’t have our same limitations.
I finally made the plunge to Linux desktop for all work in 2016 and have not looked back (and occasional windows VM, extremely rare now.) Even Arch is now perfectly fine as a workstation which surprised me. Recommend EndeavourOS to streamline the install process but it’s Arch underneath.
Is it possible for you to rephrase that comment? Don’t quite understand what you are getting at.
https://github.com/jdhao/nvim-config#features
Highly recommend this.
A modern Neovim configuration with full battery for Python, Lua, C++, Markdown, LaTeX, and more…
This is enough to get the intellisense and linters up and running. Only takes ~5 minutes to configure by installing prerequisites, it’s worth it though.
Late stage capitalism is a blight of humanity, there’s gotta have to be some sort of revolutionary changes to society at the rate this is all headed. The world is not healthy right now.
You mean the whole licensing ordeal? Retroactive type crap? I know a few developers personally that dumped it entirely because of that. Although I heard they backpedaled a little bit on that part because of the backlash, but the damage is done, trust is gone.
https://www.reuters.com/world/us/enthusiasm-wanes-among-black-voters-who-powered-bidens-2020-georgia-win-2024-03-11/
How does one reconcile the fact that Black and Hispanic voters are dropping off the Democratic party? Is it possibly because of failed policies? Is it possible Trump is getting more voters because of the representation of something people resonate with, versus the current status quo: measles outbreaks, welfare states, economic failures (inflation, everyone in the US is losing in this equation except the top 1%), the list goes on but the idea is still the same… an old failing man in office who needs to be removed.