Home Server/NAS build

ED E · 08-12-2017 12:44am

Are you sure you shouldnt be using BSD so? Thats the masochists OS.

This (workstation, primary) is 2016. Belter of a box 99% of the time. Win10 that M$ wont dare f'ck with. My main task for the NAS is to keep plex going, its become a staple of the household so I hear about it when unavailable. Really just want it to work without intervention.

Currently the laptops arent backed up as they error out writing to Flex. Even a WD shelf unit manages that fine. Silly crap.

Serephucus · 08-12-2017 12:49am

Yiiiiich, nah. I'll stick with Debian as much as I can.

Mine does a few other bits. Plex is the main one, but also hosts a few game servers. MineOS in Docker, the rest in VMs. Then I have a few other VMs on the go for work projects.

I've been using Macrium Reflect for machine backups. Seems to do things pretty nicely. I've heard good things about Duplicati for files, but haven't given it a shot myself yet.

ED E · 08-12-2017 1:00am

It itself uses QBackup to push to B2, for now anyways. That seems ok. FTTH would be nice though to really blast that out of it.

Serephucus · 08-12-2017 1:08am

Oh, stop. Server's on Cat.6, but my desktop is on homeplugs. 9MB/s max. Sad.

ED E · 08-12-2017 1:20am

Both my server and desktop are on em, 140Mb usable (680 link rate). So 17MB/s but 120MB/s between each other (3 NICs each).

ED E · 31-01-2018 8:24pm

3.10 1/31/2018 BIOS Instant Flash 3.09MB Update MC to M01406D812A for Intel Spectre & Meltdown bug Asia/USA Europe China

Now is this the bad MC or the good MC? Time to go check.

ED E · 05-09-2019 12:10am

Small update, Flexraid is dead.

https://www.reddit.com/r/DataHoarder/comments/cetqis/the_flexraid_site_is_down_now/

Serephucus · 05-09-2019 12:21am

Your box still on Flex?

ED E · 05-09-2019 12:26am

Embarrassingly, yes.

Parity data fails to val/ver so its a whole hot mess. And 2008R2 is corrupt as hell and needs replacing (it doesnt support AES-NI even with reg hacks which is a big hit).

Wonder where I can rob an AWS outpost. That'd do the trick.

Serephucus · 05-09-2019 10:47pm

You know what I'm going to suggest you migrate to, but you'd need empty disks for the move. (and probably anything else you moved to)

ED E · 05-09-2019 11:00pm

Snapraid would migrate in place. Everything else wouldn't.

I'm mulling over a few bits and window shopping at what's out there. Other than the LPC flaw which was Intel's fault the Asrock board is a nice enough machine. Had a look at lunch today to see what's it's contemporary offering. Well there's Xeon D which I expected but also Epyc. They all have fairly meagre SATA counts but do have PCIE or more sata(not both). Think it'd end up being a case of a beastly HBA to make em viable which would drive the cost through the roof. Have looked at SMs offerings yet. Not that I'm upgrading.

With the 2750:
External 3 drive Jbod to sit beside it
3x PCI bracket SATA adapter to externalise 3 ports
1 X SSD
8+3 HDD

With only PCI there's no hope of using a Raid controller and dropping software and thats out for expandibility reasons anyways. So Freenas, Unraid, Debian btrfs etc are the options. Believe ZFS expansion is a problem, and RAM allocation is notable. BTRFS stability is a whole thing.

I need to spend a week reading r/datahoarder :pac:

Serephucus · 05-09-2019 11:15pm

You can actually get some pretty damn cheap HBAs that are decent:
https://www.ebay.ie/itm/LSI-OEM-SAS-9201-16I-PCI-Express-2-0-x8-SATA-SAS-Host-Bus-Adapter-Card/131994740667?hash=item1ebb7fe7bb:g:xXwAAOSwC7xdHbj-

You can go with a 9305, which gets you 12GB/s SAS and PCIE3, but it's a bunch of money (about €220, IIRC)

I've had a look at those ITX EPYC boards as well for a backup server, and they're hella-spensive, so I'd probably go older Intel or just Ryzen 3000 or something. Depends how much you want IPMI. I'm probably just going to keep my C2750 for that until it dies.

For what it's worth, I really like my 4U case. Cables were a pain in the arse to find, but that's just because of the connectors on my HBA and backplane (reverse breakout 8087->8643 is not common, as it turns out)

ED E · 05-09-2019 11:53pm

The real selling point of Epyc would be higher NVME speeds but I don't need that so no benefit. Also I've done some work on Epyc boxes (32c x2 style) and problem free they have not been so maybe thats something....

Talking about migration. Yeah, its a bummer. I need a lend of a DAS or something. Need to find somebody retiring a SAN box I can steal for a week and image over twice at 2Gbps (17 hrs each way + array build time...... ). Eugh.

Sidenote, just out:

Serephucus · 06-09-2019 12:08am

I saw that. He's a madman.

Oh, as far as BTRFS, so far as I know, it's only RAID 5/6 that has issues. 0/1 is fine.

ED E · 06-09-2019 12:18am

0/1 arent very practical unfortunately.

Looking at LTO drives at the minute.... Ive probably lost it.

Serephucus · 06-09-2019 12:25am

ED E wrote: »

LTO

Yup. You have.

Go to US and grab some 10TB Easystores. You buy enough, and even with the flights it's cheaper.

z0oT · 06-09-2019 9:03am

In case anyone hadn't seen this before - ASRock have a Ryzen X470 board with IPMI. 8 SATA ports & M.2 aswell.
https://www.asrockrack.com/general/productdetail.asp?Model=X470D4U

Makes me wish I'd got that instead of the B450 board I went with for my server in the end.

ED E · 06-09-2019 7:37pm

12 to 8 seems like such a downgrade. Or 9 if you wanna call it that.

Johnboy1951 · 06-09-2019 7:55pm

z0oT wrote: »

In case anyone hadn't seen this before - ASRock have a Ryzen X470 board with IPMI. 8 SATA ports & M.2 aswell.
https://www.asrockrack.com/general/productdetail.asp?Model=X470D4U

Makes me wish I'd got that instead of the B450 board I went with for my server in the end.

OS	Microsoft&#174; Windows:
-Windows&#174; 10 (64 bit)

Linux:
- UBuntu 16.04.4 (64 bit)
- RedHat Enterprise Linux Server 7.3 (64bit) / 7.4 (64bit)

* Please refer to our website for the latest OS support list.
* [B][COLOR="Red"]On Linux system doesn't support Raid mode[/COLOR][/B]
* Support UEFI BOOT only

No Linux RAID would not be acceptable to me.

Coyler · 06-09-2019 10:47pm

I'm planning out my 3 GPU build. Not going to happen that soon but soon enough I'd hope. So, I want 3 separate desktops for the kids as well as various containers running plex and other items. The kids are at LEGO game and Minecraft level so 4 cores for them would be loads but perhaps budget for higher. If I want hardware transcoding for plex, I'm really looking at another GPU like the 1050 so that's 4 full PCI-E slots. Take for granted I've considered all the usuals like usb controllers, and cache drives.

So, is Threadripper my only real option here if I don't want to spend eye watering money? And do I hold out for the next gen? Discuss.

ED E · 07-09-2019 7:22am

Intel ES/QS is great. Ive got 40PICE lanes and up to a TB of RAM if I want to populate it.

Serephucus · 07-09-2019 10:30am

Coyler wrote: »

I'm planning out my 3 GPU build. Not going to happen that soon but soon enough I'd hope. So, I want 3 separate desktops for the kids as well as various containers running plex and other items. The kids are at LEGO game and Minecraft level so 4 cores for them would be loads but perhaps budget for higher. If I want hardware transcoding for plex, I'm really looking at another GPU like the 1050 so that's 4 full PCI-E slots. Take for granted I've considered all the usuals like usb controllers, and cache drives.

So, is Threadripper my only real option here if I don't want to spend eye watering money? And do I hold out for the next gen? Discuss.

Bear in mind that consumer GPUs have a limited imposed by NVIDIA of 2 transcodes at once. If you want full hardware transcoding, a Quadro P2000 is your minimum, which is €4-500. It is more power efficient than the equivalent CPU though.

3 GPU passthrough will be a doable, but it means you'll need 4 GPUs in your system. (5, if you do end up going for a P2000). If we say you want a minimum of x8 for the gaming GPUs, then that limits your board choices a little.

Something like this could work: https://www.asus.com/Motherboards/Pro-WS-X570-ACE/
It's X570, so you could throw a 3900X in there (and for reference, that's a +50% improvement over my server's 1950X, in single and multi-threaded perf. It's got your 3 x8 PCIE slots, and an x1 for the unRAID GPU, and IPMI which is a god-send for remote management.

If you want to look at older stuff, 1st/2nd gen TR is massively discounted (and probably will be more so in the next month or two with TR3), or a ED says you can look at older Xeon stuff.

Coyler · 07-09-2019 5:58pm

ED E wrote: »

Intel ES/QS is great. Ive got 40PICE lanes and up to a TB of RAM if I want to populate it.

Have you got some examples I can gander?

Coyler · 07-09-2019 6:06pm

Serephucus wrote: »

Bear in mind that consumer GPUs have a limited imposed by NVIDIA of 2 transcodes at once. If you want full hardware transcoding, a Quadro P2000 is your minimum, which is €4-500. It is more power efficient than the equivalent CPU though.

3 GPU passthrough will be a doable, but it means you'll need 4 GPUs in your system. (5, if you do end up going for a P2000). If we say you want a minimum of x8 for the gaming GPUs, then that limits your board choices a little.

Something like this could work: https://www.asus.com/Motherboards/Pro-WS-X570-ACE/
It's X570, so you could throw a 3900X in there (and for reference, that's a +50% improvement over my server's 1950X, in single and multi-threaded perf. It's got your 3 x8 PCIE slots, and an x1 for the unRAID GPU, and IPMI which is a god-send for remote management.

If you want to look at older stuff, 1st/2nd gen TR is massively discounted (and probably will be more so in the next month or two with TR3), or a ED says you can look at older Xeon stuff.

Surely my best bet is X399 or whatever (X599?) the boards will be for TR3?

Are you suggesting a P2000 for the unlimited streams alone? I’m aware of the limitations of the consumer cards but not sure I’ll ever need more than 2 for the foreseeable. Even at that, I can hack that limit. But sell it to me.

IPMI is a nice to have but this is a box sitting out on the floor with three monitors attached to it. One more cable and a quick source change on one of the monitors gives me the same. I love new toys but couldn’t justify it if it was an extra cost.

Serephucus · 07-09-2019 6:11pm

Coyler wrote: »

Surely my best bet is X399 or whatever (X599?) the boards will be for TR3?

Are you suggesting a P2000 for the unlimited streams alone? I’m aware of the limitations of the consumer cards but not sure I’ll ever need more than 2 for the foreseeable. Even at that, I can hack that limit. But sell it to me.

IPMI is a nice to have but this is a box sitting out on the floor with three monitors attached to it. One more cable and a quick source change on one of the monitors gives me the same. I love new toys but couldn’t justify it if it was an extra cost.

X399 should support TR3, yeah. It's up to you though; there should be more than enough CPU power in a 3900X to fulfil your needs, so if you can get by with the PCIE limitations, then you're good.

P2000 is for unlimited, yeah. If you're only doing a couple then a few cores from your CPU will be loads.

Fair enough on IPMI. It sounds like X570, X399 or Xeons would all work for you. Depends on where it's going to be and WAF then as well I guess.

ED E · 07-09-2019 6:20pm

Coyler wrote: »

Have you got some examples I can gander?

https://www.ebay.co.uk/itm/Intel-Xeon-E5-2698-V4-ES-QHUZ-2GHz-20-Core-40-Threads-LGA-2011-3-CPU-Processor/264432959899?hash=item3d916ec19b:g:v28AAOSwm5ZdVtLs

Serephucus · 07-09-2019 6:32pm

ED E wrote: »

https://www.ebay.co.uk/itm/Intel-Xeon-E5-2698-V4-ES-QHUZ-2GHz-20-Core-40-Threads-LGA-2011-3-CPU-Processor/264432959899?hash=item3d916ec19b:g:v28AAOSwm5ZdVtLs

As a point of comparison, my 1950X gets almost the same MC Passmark score, but quite a bit higher in single-thread. The 3700X would be your newest option, and get very similar performance to both, but with even faster single-thread perf. and much lower power consumption (but costs a good bit more, about €350).

So, we could extrapolate that TR3 would be similar performance, but power consumption would be a bit higher, as the IO die, interconnect, and chipset will pull more.

Depends what you're looking for with this, in terms of the single/multi/efficiency balance.

ED E · 07-09-2019 6:36pm

Your 1950 is a single processor. Im suggesting buying TWO. No PCIE chokepoint :pac:

Serephucus · 07-09-2019 7:14pm

RE PCIE, I was talking X570. Obviously C610 won't have any issues. :P

Coyler · 07-09-2019 8:09pm

What this boils down to is if I’m doing software transcoding are a few cores from a 3900X going to be as efficient as a GTX 1050. Or at least efficient enough to rule out a X399/599 board and a another GPU.

Thinking about it, most of my media doesn’t require much transcoding. And if swap the my 360 for a One that should reduce the need even more. I’m losing streaming services anyway through basic functionality so I’d kill a few birds with that move.

I just don’t want to go the X570 route and need a PCIE slot in the future but I may never need it. Of course if I do I could just sell the part I have and upgrade. Decisions, decisions.

Home Server/NAS build

Comments