Home server build - RAID questions

vibe666

Sparks wrote: »

The problem is that consumers buy disks for their RAID arrays at the same time; so the disks are usually from the same batch, and statistically, are more likely to fail close together than disks from different batches.

In other words, Joe Average's RAID is more likely to get a double failure than Average, Inc.

It's all probabilities, but that's the same deal as with backups...

i actually got mine in 2 lots of 4 about 2 months apart from different suppliers, first lot was from elara and the 2nd was the ones from overclockers in the uk so i reckon i'll probably be okay, altho like you say it's all down to probabilities, although i do feel that using raid-z2 is going to greatly reduce the possibility of data corruption (due to the inherent reliability of the zfs filesystem) and increase my chances of a successful rebuild in the event of s single or double physical disk failure.

ronkmonster wrote: »

File Server 5.5TB (40%to 50% but a few drives are close to full)
NAS 2 Bay 4TB used for tv+documentaries (maybe half full)
Media Box 2TB (half full)
NAS 2 Bay 500GB (not used at the moment)
External 1TB connected to netbook server 80% full

that's a lot of different things doing a job that could be done with one big NAS.

you could get the hp microserver (there's a long running BA thread about them, a great little box for next to nothing with the cashback offer) running freenas and 4x 3tb drives to take care of all that and keep one of the other NAS's for rsync mirroring of your critical data and sell on the rest of your unneeded hardware and disks.

ronkmonster

vibe666 wrote: »

that's a lot of different things doing a job that could be done with one big NAS.

you could get the hp microserver (there's a long running BA thread about them, a great little box for next to nothing with the cashback offer) running freenas and 4x 3tb drives to take care of all that and keep one of the other NAS's for rsync mirroring of your critical data and sell on the rest of your unneeded hardware and disks.

That would require me to buy a dedicated nas when I have a perfectly good PC already (I recycle my PC when I upgrade) that I can use for other tasks if I want to. I also can turn the file server on and off remotely. The netbook is my 24/7 server and is never off (application server). I like spreading out over a few devices. If my file server is down for a few days, I'll still have my movies and TV series as they would be unaffected. Just my personal preference though.

vibe666

you could run freenas on your pc or netbook, no need for a dedicated nas, or even raid if you don't want to and your data would be safer from disk corruption with zfs (even without raid) than with other filesystems.

Serephucus

Meant to ask, what sort of processing power is required for FreeNAS? Probably looking at a 5/6x3TB RAID-Z2 array. Are we talking P4, or i5, i7 territory?

vibe666

CPU power isn't going to be as big an issue as RAM, so whatever you can manage with for CPU (as long as it supports 64bit) and as much RAM as possible with the rule of thumb being a bare minimum of 6gb with the ideal being as many gigabytes of ram as you have terabytes of storage, so you'd be looking at at least 12gb of ram for 6x 3tb disks with raid-z2, but ideally 16gb to get the most out of it.

looking on the bright side of that, RAM is as cheap as chips these days and certainly cheaper than a good hardware raid controller, so all you'll need is a cheap 8 port controller with no raid and freenas will do the rest.

people tend to shy away from soft-raid as inferior to hardware raid because it uses the system cpu and ram instead of a dedicated raid chip, but with freenas and zfs the tables are very much turned and you are much better off giving freenas direct access to your disks instead of letting a separate raid controller handle things.

zfs was designed from the ground up around bulletproof reliability and to prevent or fix disk errors on the fly, but it needs direct access to the disks to do that properly so no need for expensive raid cards so what you save on that you can spend on more ram.

Serephucus

Yeah, 16GB of RAM won't be an issue at all. Dirt cheap. Bearing in mind I'd be running another VM of either MineOS, Linux, or Win7, handling Dropbox, uTorrent, (if 8.2 isn't out by the time I build this) and three or four MC servers (3-4 in each, max.) I was thinking of something along the lines of an i3 2120.

Ideally, I'd like to stick everything in a Fractal R3 case, with an 80+ Gold PSU, and a motherboard with enough SATA ports.

I'd then have the 5x3TB array in Z2, which all of my media would be backed up to, and another 2TB drive stand-alone, backing up my computer's OS and application drives. Maybe a second 2TB for mine and my GF's laptop as well.

vibe666

just bear in mind that with raid-z2 you're effectively going to be sacrificing the space of 2 of your disks to parity, so be sure you still have enough storage for your needs.

if you plan on running quite a few other vm's i'd be tempted to go with a quad core i5 rather than an i3 if your budget can stretch to it.

Serephucus

Even 5x3 would give me 9TB (about 8.2 usable or something) and that should be enough for at least three years, I'd think. By then I can just upgrade to 5x8TB drives or something.

Not sure about budget at all yet. Priced up a rough build yesterday, and it came to about €1,150, and to be honest I'm not sure I have the sort of money to justify spending a grand on what's basically storage. As I've said though it'll be a couple of months. This thread was mostly to brainstorm ideas / theory.

vibe666

yeah, hard drive prices are still killing you unfortunately.

Serephucus

2TB drives are still around the €100-110 mark, and 3TB drives are actually slightly better value comparatively (unlike before the floods) at around €145-150. If 3TB drives come down to around €100 by the end of the Summer, that alone would save me about €250.

mr kr0nik

Hi everyone,
What are peoples thoughts on forgetting about RAID and just croning an rsync job every night between a couple of disks?

Are there any potential pitfalls here (I know I need to take backups as well).

Sparks

mr kr0nik wrote: »

Hi everyone,
What are peoples thoughts on forgetting about RAID and just croning an rsync job every night between a couple of disks?

Are there any potential pitfalls here (I know I need to take backups as well).

What you're describing is a backup solution; you couldn't use it online. You'd have to use only the disk you're rsyncing from; the disk you rsync to would be a snapshot backup and if you used it, you're talking about out-of-date data or writing data that'll be overwritten that night.

mr kr0nik

Thanks for the clarification. I do it just with data disk (movies, music, docs photos etc.) I really consider it as a "once I'm happy with something it goes there and I know it will be copied at the cron run time".

I ahve a different disking running the OS (Ubuntu) and I have coies of passwd routing hosts etc copited to the data disk regularly.

Sparks

If it's static, read-only data, then you won't have issues over data corruption; but you won't get the possible speed benefits of RAID1 either...

It's a good idea to have backups, don't get me wrong, but RAID and backups are like cruise control and seatbelts - one makes your life easier and may help avoid getting into a crash; the other saves your life if you do crash.

vibe666

i think it all comes down to what is an acceptable level of risk for your data.

if you're an enterprise or smb, you'll have most or all of your data backed up regularly and going off-site to make sure it's as safe as possible to minimise the risk of losing anything critical.

if you're a home user, you'll have a few gigabytes of photo's, home movies and documents you'll 100% want to keep permanently, but mostly it will likely be downloaded movies, music and tv shows that you want to keep, but wouldn't be the end of the world if you lost as you can download again and in reality, you're never likely to watch again anyway and is only there because deleting it would be a waste of the download even after watching it.

personally, i like having everything in one place, so a big NAS with a lot of storage is perfect for my needs. one network drive mapped on my PC's with everything easily accessible in it.

for me, freenas and raid-z2 is going to keep all that as safe as i want it to be, but the critical stuff is also mirrored onto another drive on my own PC and from there goes to dropbox or youtube, vimeo or picasa, depending on what it is and where there is room for it and i'm happy with that.

sure some of it is done manually whenever i get the time, but it's enough to keep me happy.

other people do it differently and that keeps them happy and i'm sure we'll all keep doing it the way we want to until something changes our minds somewhere down the line.

vibe666

Sparks wrote: »

RAID and backups are like cruise control and seatbelts - one makes your life easier and may help avoid getting into a crash; the other saves your life if you do crash.

nice analogy, i like it.

Serephucus

Qeustions (more out of curiosity than anything at this point) for you Vibe:

What sort of read/write speeds do you typically get from your array?
What's the processor usage like, in terms of typical, and spike?
How much power does it consume?

As I say, just curious. Don't really mind speeds too much, as long as they're at least comparable to a single drive, which I have to assume they would be, even factoring in calculations.

Thanks!

vibe666

it's doing about 30MB/s write and maybe 40MB/s read at the moment, but i'm convinced if i could give it more RAM to play with it would be a lot quicker.

by comparison, I could get up to 100MB/s out of my old thecus NAS copying to/from my desktop PC SSD.

ESX idle cpu usage is only a couple of hundred Mhz and that's with the NAS and a windows 2003 vm running 24/7. spikes up to around 3500Mhz (2x 2.8Ghz CPU cores) if i'm copying stuff to or from the NAS.

Not actually sure about overall power usage, I've never actually checked. there's an onboard iLO/RSA yoke in the ML110 G6 (altho no remote control option without a paid upgrade) but i've never actually connected the NIC to check it out other than when i first bought it to see what was there.

Serephucus

Wow, that's a lot lower than I was expecting... Still fine for just media and stuff I suppose. I'd be interested to hear what happens if/when it gets more RAM stuck in it.

From the sounds of it then an i5 is definitely in order, considering the Minecraft servers as well.

Power isn't a massive issue, though obviously it would be nice to keep it down. Was just wondering to get an idea of cost, etc.

vibe666

yeah, i think it's the dual parity really kills it without a shedload of ram dedicated to the NAS. of course i may well have something set up wrong, i just haven't had the time to spend tinkering with it. parenthood is the single biggest consumer of free time known to man.

i'm on the lookout for a cheap Xeon X3480 which is the fastest socket 1156 CPU i can get my hands on (and the fastest CPU my ML110 G6 will handle) and to upgrade to 16gb of RAM too, so we'll see how those 2 things and my finances (and free time) intersect in the next couple of months and if it makes any difference.

i also periodically keep getting an error of some sort on the NAS which is think is one of the disks acting hinky (it's possible i missed the samsung firmware update on one) or it might be failing. not geting any s.m.a.r.t. warnings on any of them, but who knows. again, it all comes down to the free time thing.

Surfn

from the freenas server, can you stream video and music to LAN tv's and android phones aswell as just to other pc's. this is a very interesting thread here. freenas zfs raid-z2 seems to be a good option rather than going for hardware raid 5 or 6. what motherboard do you intend to use for your build Serephucus

Serephucus

Haven't look into compatibility, but I'm thinking the ASRock P67 Extreme6. It's a relatively cheap board (about €120 I think), but offers no less than ten SATA ports.

Surfn

Serephucus wrote: »

Haven't look into compatibility, but I'm thinking the ASRock P67 Extreme6. It's a relatively cheap board (about €120 I think), but offers no less than ten SATA ports.

sounds like a good board, probably need 10 minimum sata ports for raid-z2.
i really need to do something along them same lines, ive lots of hard drives laying around full of movies etc.

Serephucus

I'm probably going to try for either five or six 3TB drives, so either 9 or 12TB of storage with Z2. Depends on HDD prices and stuff at the time. I hope I'll have the money for this. I've needed something bigger for a long time at this point, and any more hard drives in my gaming rig would make it too noisy.

Surfn

Serephucus wrote: »

I'm probably going to try for either five or six 3TB drives, so either 9 or 12TB of storage with Z2. Depends on HDD prices and stuff at the time. I hope I'll have the money for this. I've needed something bigger for a long time at this point, and any more hard drives in my gaming rig would make it too noisy.

when you start your build be sure to keep us updated. to me this is very interesting. i wonder what sort of read and write speeds ya get with z2 using the onboard sata controllers and when streaming a full bluray movie. i wonder actually how many 1080p blurays ya could stream at the same time before it starts to bootleneck using the onboard nic

Serephucus

Well it'll be the end of the Summer at the earliest, just so yas know, but I'll post up a build log when I get going either way.

Surfn

vibe666 can you access a freenas server from say an android mobile fone

vibe666

yep, if you could be arsed setting it up.

Ctrl Alt Del

My fresh,recent experience with NAS devices...

One device had issues after power cut in the area.
All went down,no way to access it using web based or application that came with ! Logged call with supplier,told to upgrade the firmware,the reset to defaults ... no joy !

They gave me some internal only software to reset the unit to defaults,then logged as default username,upgraded the firmware and managed to get my data back.

Copied on a USB drive,connected to a Windows box and put the NAS in storage...never ever touch it again !

vibe666

the same thing could just as easily happen with any other kind of storage if you don't have a UPS or keep backups, it's not something exclusive to NAS's.

Serephucus wrote: »

Wow, that's a lot lower than I was expecting... Still fine for just media and stuff I suppose. I'd be interested to hear what happens if/when it gets more RAM stuck in it.

i actually just copied something from my SSD's to the NAS and it touched 90MB/s and then averaged about 85MB/s so i guess whatever the speed issue was went away on it's own.