1·
23 days agoNo, all sizes of llama 3.1 should be able to handle the same size context. The difference would be in the “smarts” of the model. Bigger models are better at reading between the lines and higher level understanding and reasoning.
Terrasque
- 0 Posts
- 56 Comments
Joined 1 year ago
Cake day: June 11th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
Wow, that’s an old model. Great that it works for you, but have you tried some more modern ones? They’re generally considered a lot more capable at the same size
Increase context length, probably enable flash attention in ollama too. Llama3.1 support up to 128k context length, for example. That’s in tokens and a token is on average a bit under 4 letters.
Note that higher context length requires more ram and it’s slower, so you ideally want to find a sweet spot for your use and hardware. Flash attention makes this more efficient
Oh, and the model needs to have been trained at larger contexts, otherwise it tends to handle it poorly. So you should check what max length the model you want to use was trained to handle
0·2 months agoAny moment now…
The only issue I see with targeting Linux is the sheer variety of Desktop setups. Finding one keyboard shortcut and payload that will work on even just the majority of distros would be a challenge.
I still use http a lot for internal stuff running in my own network. There’s no spying there… I hope … And ssl for local network only services is a total pita.
So I really hope browsers won’t adapt https only
I doubt the disk will bottleneck at 40mb/s when doing sequential write. Torrent downloads are usually heavy random writes, which is the worst you can do to a HDD.
I remember back in the day this automated downloader program… the links had a limit of one download at a time and you had to solve a captcha to start each download.
So the downloader had built in “solve other’s captcha” system, where you could build up credit.
So when you had say 20 links to download you spent some minutes solving other’s captchas and get some credit, then the program would use that crowdsourcing to solve yours as they popped up.
Llama3 8b can be run at 6gb vram, and it’s fairly competent. Gemma has a 9b I think, which would also be worth looking into.
That’s like saying car crash is just a fancy word for accident, or cat is just a fancy term for animal.
Hallucination is a technical term for this type of AI, and it’s inherent to how it works at it’s core.
And now I’ll let you get back to your hating.
Better background backups
Rework background backups to be more reliable
Hilarious for a system which main point / feature is photo backup
1·5 months agoI worked on one where the columns were datanasename_tablename_column
They said it makes things “less confusing”
0·7 months agoI’ve seen Skype do that. It was a weird folder name, but gallery found it and displayed the images.
Which is how I noticed it in the first place
Hah as if. In the early 00s the mods were in maybe once or twice a day and there was tons of CP being posted.
Worst I saw was a little girl chopped into pieces, and a many -page discussion / argument if it should be sorted as CP or Necro porn. That was the old 4chan.
Even for 4chan that’s fucked up.
Oh, sweet summer child…
61·9 months agoThey had a trial run with bleach already
On occasion their strategy has been “if we send in enough people, they’ll eventually run out of bullets”
They out-Zapp Brannigan’ed Zapp Brannigan. That should terrify you on multiple levels
Koboldcpp
I generally agree with your comment, but not on this part:
They’re quite capable of following instructions over data where neither the instruction nor the data was anywhere in the training data.
Critical thought, generally no. Basic reasoning, that they’re somewhat capable of. And chain of thought amplifies what little is there.