Tl;dr
I have no idea what I’m doing, and the desire for a NAS and local LLM has spun me down a rabbit hole. Pls send help.
Failed Attempt at a Tl;dr
Sorry for the long post! Brand new to home servers, but am thinking about building out the setup below (Machine 1 to be on 24/7, Machine 2 to be spun up only when needed for energy efficiency); target budget cap ~ USD 4,000; would appreciate any tips, suggestions, pitfalls, flags for where I’m being a total idiot and have missed something basic:
Machine 1: TrueNAS Scale with Jellyfin, Syncthing/Nextcloud + Immich, Collabora Office, SearXNG if possible, and potentially the *arr apps
On the drive front, I’m considering 6x Seagate Ironwolf 8TB in RAIDz2 for 32TB usable space (waaay more than I think I’ll need, but I know it’s a PITA to upgrade a vdev so trying to future-proof), and I am thinking also want to add in an L2ARC cache (which I think should be something like 500GB-1TB m.2 NVMe SSD); I’d read somewhere that back of the envelope RAM requirements were 1GB RAM to 1TB storage (though the TrueNAS Scale hardware guide definitely does not say this, but with the L2ARC cache and all of the other things I’m trying to run I probably get to the same number), so I’d be looking for around 48GB (though I am under the impression that using an odd number of DIMMs isn’t great for performance, so that might bump up to 64GB across 4x16GB?); I’m ambivalent on DDR4 vs. 5 (and unless there’s a good reason not to, would be inclined to just use DDR4 for cost), but am leaning ECC, even though it may not be strictly necessary
Machine 2: Proxmox with LXC for Llama 3.3, Stable Diffusion, Whisper, OpenWebUI; I’d also like to be able to host a heavily modded Minecraft server (something like All The Mods 9 for 4 to 5 players) likely using Pterodactyl
I am struggling with what to do about GPUs here; I’d love to be able to run the 70b Llama 3.3, it seems like that will require something like 40-50GB VRAM to run comfortably at a minimum, but I’m not sure the best way to get there; I’ve seen some folks suggest 2x3090s is the right balance of value and performance, but plenty of other folks seem to advocate for sticking with the newer 4000 architecture (especially with the 5000 series around the corner and the expectation prices might finally come down); on the other end of the spectrum, I’ve also seen people advocate for going back to P40s
Am I overcomplicating this? Making any dumb rookie mistakes? Does 2 machines seems right for my use cases vs. 1 (or more than 2?)? Any glaring issues with the hardware I mentioned or suggestions for a better setup? Ways to better prioritize energy efficiency (even at the risk of more cost up front)? I was targeting something like USD 4,000 as a soft price cap across both machines, but does that seem reasonable? How much of a headache is all of this going to be to manage? Is there a light at the end of the tunnel?
Very grateful for any advice or tips you all have!
Hi all,
So sorry again for the long post. Just including a little bit of extra context here in case it’s useful about what I am trying to do (I feel like this is the annoying part of an online recipe where you get a life story instead of the actual ingredient list; I at least tried to put that first in this post.) Essentially I am a total noob, but have spent the past several months lurking on forums, old Reddit and Lemmy threads, and have watched many hours of YouTube videos just to wrap my head around some of the basics of home networking, and I still feel like I know basically nothing. But I felt like I finally got to the point where I felt that I could try to articulate what I am trying to do with enough specificity to not be completely wasting all of your time (I’m very cognizant of Help Vampires and definitely do not want to be one!)
Basically my motivation is to move away from non-privacy respecting services and bring as much in-house as possible, but (as is frequently the case), my ambition has far outpaced my skill. So I am hopeful that I can tap into all of your collective knowledge to make sure I can avoid any catastrophic mistakes I am likely to blithely walk myself into.
Here are the basic things I am trying to accomplish with this setup:
• A NAS with a built in media server and associated apps
• Phone backups (including photos)
• Collaborative document editing
• A local ChatGPT 4 replacement
• Locally hosted metasearch
• A place to run a modded Minecraft server for myself and a few friends
The list in the tl;dr represent my best guesses for the write software and (partial) hardware to get all of these done. Based on some of my reading, it seemed that a number of folks recommend running TrueNAS baremetal as opposed to in ProxMox for when there is an inevitable stability issue, and that got me thinking more about how it might be valuable to split out these functions across two machines, one to hand heavier workloads when needed but to be turned off when not (e.g. game server, all local AI), and a second machine to function as a NAS with all the associated apps that would hopefully be more power efficient and run 24/7.
There are two things that I think would be very helpful to me at this point:
- High level feedback on whether this strategy sounds right given what I am trying to accomplish. I feel like I am breaking the fundamental Keep It Simple Stupid rule and will likely come to regret it.
- Any specific feedback on the right hardware for this setup.
- Any thoughts about how to best select hardware to maximize energy efficiency/minimize ongoing costs while still accomplishing these goals.
Also, above I mentioned that I am targeted around USD 4,000, but I am willing to be flexible on that if spending more up front will help keep ongoing costs down, or if spending a bit more will lead to markedly better performance.
Ultimately, I feel like I just need to get my hands on something and start screwing things up to learn, but I’d love to avoid any major costly screw ups before I just start ordering parts, thus writing up this post as a reality check before I do just that.
Thanks so much if you read this far down the post, and for all of you who share any thoughts you might have. I don’t really have folks IRL I can talk to about these sorts of things, so I am extremely grateful to be able to reach out to this community. -------
Edit: Just wanted to say a huge thank you to everyone who shared their thoughts! I posted this fully expecting to get no responses and figured it was still worth doing just to write out my plan as it stood. I am so grateful for all of your thoughtful and generous responses sharing your experience and advice. I have to hop offline now, but look forward to responding to any comments I haven’t had a chance to turn to tomorrow. Thanks again! :)
Reading the title and looking at the thumbnail, I was thinking, “sure I’ll do a good deed and help out a noob.” Then I read your post and I realized you know what you’re doing better than me.
HomerInBushes.gif
OP sharing decent DD tbh
source: i am regarded
Thank you for this! Honestly maybe it’s just been all of the Youtubers I watch but I constantly feel like I have no idea about how to make things work (and also, to be fair, basically everything I wrote is just me reading what other people who seem to know what they’re talking about think and then trying to fit all the pieces together. I sort of feel like a money at a typewriter in that way.) Really appreciate you commenting though! It’s given me a little more confidence :)
It’s easy to feel like you know nothing when A) there’s seemingly infinite depth to a skill and B) there are so many options.
You’re letting perfection stop you from starting my dude. Dive in!
Thank you! I think I am just at the “Valley of Despair” portion of the Dunning-Kruger effect lol, but the good news is that it’s hopefully mostly up from here (and as you say, a good finished product is infinitely better than a perfect idea).
Honestly why not just use an old laptop you have laying around to test 1 or 2 of your many project/ideas and see how it goes, before going 4000 $ deep.
This is definitely good advice. I tend to run my laptops into the ground before I replace them, but a lot of the feedback here has made me think experimenting with something much less expensive first is probably the right move instead of trying to do everything all at once (so that when I inevitably screw up, it at least won’t be a $4k screw up.) But thanks for the sanity check!
$4,000 seems like a lot to me. Then again, my budget was like $200.
I would start by setting yourself a smaller budget. Learn with cheaper investments before you screw up big. Obviously $200 is probably a bit low but you could build something simple for around $500. Focus on upgrade ability. Once you have a stable system up skill and reflect on what you learned. Once you have a bit more knowledge build a second and third system and then complete a Proxmox cluster. It might be overkill but having three nodes gives a lot of flexibility.
One thing I will add. Make sure you get quality enterprise storage. Don’t cheap out since the lower tier drives will have performance issues with heavier workloads. Ideally you should get enterprise SSD’s.
I did a double take at that $4000 budget as well! Glad I wasn’t the only one.
You are both totally right. I think I anchored high here just because of the LLM stuff I am trying to get running at around a GPT4 level (which is what I think it will take for folks in my family to actually use it vs. continuing to pass all their data to OpenAI) and it felt like it was tough to get there without spending an arm and a leg on GPUs alone. But I think my plan is now to start with the NAS build, which I should be able to accomplish without spending a crazy amount and then building out iteratively from there. As you say, I’d prefer to screw up and make a $500 mistake vs. a multiple thousand dollar one. Thanks for the sanity check!
ZFS Raid Expansion has been released days ago in OpenZFS 2.3.0 : https://www.cyberciti.biz/linux-news/zfs-raidz-expansion-finally-here-in-version-2-3-0/
It might help you with deciding how much storage you want
Woah, this is big news!! I’d been following some of the older articles talking about this being pending, but had no idea it just released, thanks for sharing! Will just need to figure out how much of a datahoarder I’m likely to become, but it might be nice to start with fewer than 6 of the 8TB drives and expand up (though I think 4 drives is the minimum that makes sense; my understanding is also that energy consumption is roughly linear with number of drives, though that could be very wrong, so maybe I’ve even start with 4x a 10-12TB drive if I can find them for a reasonable price). But thanks for flagging this!
I know most of the less expensive used hardware is going to be server-shaped/rackmount. Don’t go for it unless you have a garage or shed that you can stuff them in. They put out jet-engine levels of noise and require god tier soundproofing in order to quiet them. The ones that are advertised as quiet are quiet as compared to other server hardware.
You can grab an epyc motherboard that is ATX and will do all you want, and can then move it to a rackmount later if you end up going that way.
The NVIDIA launch has been a bit of a paper one. I don’t expect the prices of anything else to adjust down, rather the 5090 may just end up adjusting itself up. This may change over time, but the next couple of months aren’t likely to have major deals worth holding out for.
Thanks for this! The jet engine sound level and higher power draw were both what made me a little wary of used enterprise stuff (plus jumping from never having a home server straight to rack mounted felt like flying a little too close to the sun). And thanks also for the epyc rec; based on other comments it sounds like maybe pairing that with dual 3090s is the most cost effective option (especially because I fear you’re right on prices not being adjusted downward; not sure if the big hit Nvidia took this morning because of DeepSeek might change things but I suppose that ultimately unless underlying demand drops, why would they drop their prices?) Thanks again for taking the time to respond!
check out serverpartdeals
Thanks, will do!
They’re the best site around for high quality/capacity drives that don’t cost an arm and a leg. Another great resource for tools n’ stuff is awesomeselfhosted
Website: https://awesome-selfhosted.net/
Github: https://github.com/awesome-selfhosted/awesome-selfhosted
Thanks so much for sharing! I just poked around for the Ironwolf 8TB drives I was thinking of an it unfortunately looks like they’re sold out for now (as are the 8TB WD Reds it looks like), but I’ll definitely keep an eye out for them here (and honestly maybe explore some different size options honestly; the drive costs I was seeing on other sites was more than I expected, but wasn’t sure if that was just the new normal; glad to have another option!) And thanks so much for the awesomeselfhosted list!! I don’t think I’d seen everything collected in one place like that before, that will be super helpful!
Their prices lately have been very unimpressive.
as with all things. prepare for it to get worse in every aspect
Am I overcomplicating this?
I fear that you may be overthinking things a bit. For a home server I wouldn’t worry about things like min/maxing memory to storage sizes. If you’re new to this then sizing can be tricky.
For a point of reference - I’m running a MD RAID5 with 4TiB x 4 disks (12TiB usable) on an old Dell PowerEdge T110 with 8GiB of RAM. It’s a file server (NFS) that does little else (just a bind9 server and dhcpd). I’ve had disks fail in the RAID but I’ve never had a 2 disk failure in 10+ years. I always keep my fileserver separate so that I can keep it simple and stable since everything else depends on it. I also do my backups to and from it so it’s a central place for all storage.
That’s just a file-server. I have 3 proxmox servers of widely variable stats from acquired machines… An old System76 laptop with 64GiB RAM (and NVidia 1070 GTX that is used by Jellyfin), a Lenovo Thinkserver with 16GiB RAM, and an old Dell Z740 with 128GiB RAM (long story).
None of these servers are speed demons by any current standards, but they support a variety of VMs comfortably (home assistant, jellyfin, web sever, DNS, DHCP, a 3 node microk8s cluster running searxng, subsonic, a docker registry etc.)
RAM has always mattered more to me for servers. The laptop is the most recent and has 8 cores, the Lenovo only has 4.
Could things be faster? Sure. Do they perform “well enough for me?” Absolutely. I’m not as worried about low-power as you seem to be but my point is that you can get away with pretty modest hardware for MOST of the types of things you’re looking to do.
The AI one is the thing to worry about - that could be your entire budget. VRAM is king for LLMs and gets pricey quick. My personal laptop’s NVidia 3070 with 8GiB VRAM runs models that fit in that low amount of memory just fine. But I’m restricted to models that fit…
Thanks so much for all of this info! You’re almost certainly correct that I’m overthinking this (it’s definitely a talent of mine). I had been leaning z2 on the NAS only because I’d heard that the resilvering process can be somewhat intensive on the drives, especially when they’re larger, but I had also seen folks say that this was probably overkill for most home settings and so I’m glad someone with experience on it could chime in. I think my biggest takeaway from what you shared is that it sounds like keeping the file system baremetal and fiddling with it as little as possible is the strategy (vs. virtualizing it and running it out of one large Proxmox machine, for instance). And I think you’re totally right on the LLMs being the real sticking point; I’d had no idea just how resource intensive they were not just to train but even to operate until I started looking into running one locally. It’s honestly making me think that maybe trying to roll this out in phases starting with the NAS (while also doing some other infrastructure upgrades like looking at running cat 6a and swapping out my router from the ISP all-in-one to something that can run OPNSense paired with some WAPs), might be a better place to start. Then, if I can get some early successes under my belt, I can move onto the LLM arena and see how much time, money, and tears I want to spend getting that up and running. Oh, and thanks also for mentioning TiB; it sent me down a very interesting rabbit hole on the base 10 vs. base 2 byte measurement and how drive companies use the difference to pump up the number they get to advertise; I had no idea that accounting for the discrepancy in drive size, but is definitely not surprising.
I would definitely scale things out slowly. While the NAS will eventually be the cornerstone of your setup it will be an investment. You could also try setting up a cheap server as a stand-alone to get the feel for running applications. Maybe even as cheap as a Raspberry PI or small single-board system. Some of them have pretty decent specs at very affordable costs. Such a system could go on to serve simple services in your final architecture (DHCP, DNS, etc.).
There are sometimes ways to upgrade a RAID later. In one scenario I replaced the drives one at a time with larger drives and created a second RAID on the same disks (in a second partition). Wasn’t a great idea perhaps - but it worked! I just expanded my LVM pool to the new RAID and was off to the races. I’m sure performance was hit with two RAIDs on the same disks - but it did the job and worked well enough for me. 😉
I’m not as familiar with zfs to know what options it has for expansion. With MD these days I think you can just fail and replace each disk one-by-one and expand the raid to the new size once they’re all replaced. MD can be pretty picky about drives having exactly the same number of sectors though so care must be taken to use the same disks or partition a bit smaller than the drive… Waiting for each disk to sync can take ages but it’s possible. There may be other options for ZFS (scaling with more disks maybe?).
Good luck with your project!
Pretty sure truenas scale can host everything you want so you might only want one server. Use Epyc for the pcie lanes, and a fractal design r7 XL and you could even escape needing a rack mount if you wanted. Use a pcie to m.2 adapter and you could easily host apps on them on a mirrored pool and use a special vdev to speed up the HDD storage pool
The role of the proxmox server would essentially be filled by apps and/or VM you could turn on or off as needed.
you seem pretty on track, and being broke i haven’t looked at the expensive stuff you’re considering, so i can’t give you any value tips.
However, i would like to point out that if you’re just going to be hosting minecraft game servers crafty controller is a much easier to setup&use tool than pterodactyl
Thank you! Honestly I am probably going way overboard myself (I think I’ve tried to convince myself that it might make sense given the likelihood of tariffs around the corner, but honestly might still end up downscaling to more like 10-20TB of storage and radically reduce my LLM expectations). And thanks also for the Crafty Controller rec, I hadn’t heard of them and will definitely check them out!
for high vram ai stuff it might be worth waiting and seeing how the 24gb b580 variant is
Intel has a bunch of translation layer sort of stuff though that I think generally makes it easy to run most CUDA ai things on it, but I’m not sure if common ai software supports multi gpu with it though
IDK how cash limited you are but if it’s just the vram you need and not necessarily the tokens/sec it should be a much better deal when it releases
Not entirely related but I have a full half hourly shapshotted computer backup going to a large HDD in my home server using Kopia, its very convenient and you don’t need to install anything on the server except a large drive and the ability to use ssh/sftp (or another method, it supports several). It supports many compression formats and also avoids storing duplicate data. I haven’t needed to use it yet, but I imagine it could become very useful in the future. I also have the same set up in the cli on the server, largely so I can roll back in case some random person happens upon it and decides to destroy everything in my Minecraft server (which is public and doesn’t have a whitelist…). It’s pretty easy to set up and since it can back up over the internet, its something you could easily use for a whole family.
My home server (with a bunch of used parts plus a computer from the local university surplus store) was probably about ~170$ in total (i7 6700, 16gb ddr4, 256gb ssd, 8tb hdd) and is enough to host all of the stuff I have (very light modded MC with geyser, a gitlab instance, and the backup) very easily, but it is very much not expandable (the case is quite literally tiny and I don’t have space to leave it open, I could get a pcie storage controller but the psu is weak and there aren’t many sata ports), probably not all that future proof either, and definitely isn’t something I would trust to perform well with AI models.
this (sold out now) is the hdd I got, I did a lot of research and they’re supposed to be super reliable. I was worried about noise, but after getting one I can say that as long as it isn’t within 4 feet of you you’ll probably never hear it.
Anyways, it’s always nice to really do something the proper way and have something fully future proof, but if you just need to host a few light things you can probably cheap out on the hardware and still get a great experience. It’s worth noting that a normal Minecraft server, backups, and a document editor for example are all things that you can run on a Raspberry Pi if you really wanted to. I have absolutely no experience using a NAS, metasearch, or heavy mods however, those might be a lot harder to get fast for all I know.
Thank you so much for all of this! I think you’re definitely right that probably starting smaller and trying a few things out is more sensible. At least for now I think I am going to focus on putting something together for the lower-hanging fruit by focusing on the NAS build first and then build up to local AI once I have something stable (but I’ll definitely be keeping an eye out for GPU deals in the meantime, so thanks for mentioning the B580 variant, it wasn’t on my radar at all as an option). But I think the thread has definitely given me confidence that splitting things out that way makes sense as a strategy (I had been concerned when I first wrote it out that not planning out everything all at once was going to cause me to miss some major efficiency, but I feel like it turns out that self-hosting is more like gardening than I thought in that it sort of seems to grow organically with one’s interest and resources over time; sort of sounds obvious in retrospect, but I was definitely approaching this more rigidly initially). And thank you for the HDD rec! I think the Exos are the level above the Ironwolf Pro I mentioned, so will definitely consider them (especially if they come back online for a reasonable price at serverpartdeals or elsewhere). Just out of curiosity, what are you using for admin on your MC server? I had heard of Pterodactyl previously, but another commenter mentioned CraftyController as a bit easier to work with. Thank you again for writing all of this up, it’s super helpful!
I’m just using basic fabric stuff running through a systemd service for my MC server. It also basically just has every single performance mod I could find and nothing else (as well as geyser+floodgate) so there isn’t all that much admin stuff to do. I set up RCON (I think it’s called) to send commands from my computer but I just set up everything through ssh. I haven’t heard of either pterodactyl or crafty controller, I’ll check those out!
