Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.user > #225505 > unrolled thread
| Started by | Patrick Bartek <patrick.bartek@gmail.com> |
|---|---|
| First post | 2020-07-29 22:00 +0200 |
| Last post | 2020-08-04 06:30 +0200 |
| Articles | 20 on this page of 114 — 18 participants |
Back to article view | Back to linux.debian.user
Homebuilt NAS Advice Patrick Bartek <patrick.bartek@gmail.com> - 2020-07-29 22:00 +0200
Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-07-29 22:20 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-07-29 22:30 +0200
Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-07-29 23:40 +0200
Re: Homebuilt NAS Advice Patrick Bartek <patrick.bartek@gmail.com> - 2020-07-30 02:50 +0200
Re: Homebuilt NAS Advice Tom Dial <tddial@comcast.net> - 2020-07-30 22:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 01:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-02 03:10 +0200
Re: Homebuilt NAS Advice Keith Bainbridge <ke1th3216@gmail.com> - 2020-08-02 04:50 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-02 06:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 21:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 22:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 06:10 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-03 17:30 +0200
Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-08-03 17:40 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 18:40 +0200
Re: Homebuilt NAS Advice <tomas@tuxteam.de> - 2020-08-03 19:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-03 22:00 +0200
Re: Homebuilt NAS Advice <tomas@tuxteam.de> - 2020-08-03 22:20 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 23:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:40 +0200
Re: Homebuilt NAS Advice Anssi Saari <as@sci.fi> - 2020-07-30 10:00 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-07-30 13:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-07-30 01:50 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-07-30 23:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-08 01:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 03:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 14:40 +0200
Re: Homebuilt NAS Advice Peter Ehlert <peter@sdi-baja.com> - 2020-07-30 19:00 +0200
Re: Homebuilt NAS Advice Charles Curley <charlescurley@charlescurley.com> - 2020-07-30 21:50 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-07-31 13:10 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-07-30 23:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 01:50 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-08-02 17:50 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-03 17:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:10 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-04 01:20 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-04 10:40 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-04 11:30 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-04 16:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-04 16:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 05:50 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-05 16:20 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 17:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-05 17:40 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 12:10 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 16:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 17:10 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 18:10 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-07 18:30 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-07 19:40 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-08 00:10 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 04:30 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-09 11:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 17:00 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 18:30 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 19:20 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-09 23:50 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-10 00:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:50 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 18:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 23:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 05:30 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-07 18:30 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-06 17:20 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Andrei POPESCU <andreimpopescu@gmail.com> - 2020-08-04 17:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 06:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 04:30 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 09:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 11:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 21:00 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 03:50 +0200
Re: Homebuilt NAS Advice Kushal Kumaran <kushal@locationd.net> - 2020-08-06 06:20 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 04:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 05:50 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 12:30 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 12:30 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 21:10 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 23:30 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 00:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 01:20 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-08 09:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 04:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 04:50 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 06:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 07:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 07:30 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 18:10 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 03:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-07 03:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 05:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 05:40 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 08:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-07 08:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 09:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 17:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 01:40 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-08 04:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 22:30 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 00:20 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-09 01:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 22:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-10 01:30 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-08-04 06:30 +0200
Page 2 of 6 — ← Prev page 1 [2] 3 4 5 6 Next page →
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-04 00:50 +0200 |
| Message-ID | <AzRrk-mN-7@gated-at.bofh.it> |
| In reply to | #225793 |
On 8/3/2020 4:45 PM, Stefan Monnier wrote: >> A BPi can stand as a media server, but I don't recommend it in >> general. A media server to a very limited number of clients does >> not need much I/O power, but if it needs to recode a video on the >> fly, a BPi is going to be really hard pressed to keep up, >> especially if it is attempting to serve multiple streams. > > In my case the videos aren't recoded on the fly by the media server, so That frees up a lot of breathing room > this is not an issue. In theory, recoding on the fly should be possible > (at least for some combinations of input/output formats), tho, using > the A20'a VPU. I never tried, tho. > > Still, recoding doesn't seem to justify 15GB in addition to the 1GB of > my server. Memory is pretty cheap these days, although single board computers have very limited memory expansion options. I have quite a few Raspberry Pi hosts and a couple of Banana Pi hosts, but none of them are deployed as media servers. > I'm not saying 16GB can't be useful in a NAS (I don't assume my use case > to be the same as the OP's, for example), I'm just pointing out that it > can be massively overkill for some NAS scenarios, so it's a bad idea to > recommend it without first knowing what the usage will look like. Absolutely, but it is not expensive overkill. Desktop memory can cost as little as $30 for 16G.
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-04 00:40 +0200 |
| Message-ID | <AzRhE-jq-7@gated-at.bofh.it> |
| In reply to | #225767 |
On 8/2/2020 11:07 PM, Stefan Monnier wrote: >> I absolutely agree, no matter what the file system, I would >> definitely up the memory to the 16GB max, especially if this is to >> be a media server. > > Unless you're serving some quite demanding clients, I'd expect 16GB in > an NAS to be a complete waste of money. There is the rub! A flat file server needs a solid IO system, but not much computing horsepower. In this respect, media files are no different than any other files. The twist comes in if the server is offering up video files in formats that are not native to the clients and must recode the files on the fly. My main server has an Asus Sabertooth 990FX motherboard with an 8 core AMD FX-8350 processor and 16G of memory. When doing simple file operations, especially primarily reads, the CPU frequency usually hovers around 2.4GHz with at most two or three cores running at moderate load and plenty of free memory. When a request comes along to recode an HD MPEG-II file to h.264 or vice-versa, the CPU clock jumps to over 4.0 GHz with all 8 cores slamming away at over 80% utilization and the memory usage pegs. > IOW it all depends about what kind of NAS you're designing, and with > what kind of connectivity and software. Absolutely.
[toc] | [prev] | [next] | [standalone]
| From | Anssi Saari <as@sci.fi> |
|---|---|
| Date | 2020-07-30 10:00 +0200 |
| Message-ID | <AybDQ-2vA-7@gated-at.bofh.it> |
| In reply to | #225507 |
Dan Ritter <dsr@randomstring.org> writes: > samba - if you need to share files with Windows That depends, Windows 10 Pro these days has NFS support. Not that good since it's from Microsoft but gets the job done.
[toc] | [prev] | [next] | [standalone]
| From | rhkramer@gmail.com |
|---|---|
| Date | 2020-07-30 13:20 +0200 |
| Message-ID | <AyeLo-4yV-17@gated-at.bofh.it> |
| In reply to | #225523 |
On Thursday, July 30, 2020 03:57:42 AM Anssi Saari wrote: > Dan Ritter <dsr@randomstring.org> writes: > > samba - if you need to share files with Windows > > That depends, Windows 10 Pro these days has NFS support. Not that good > since it's from Microsoft but gets the job done. I don't have much recent experience with networking between local computers (back in the day (15 to 25 years ago, I used SAMBA) (e.g., NAS), but recently started digging into it again. For my current purposes, I'm finding ssh (sshfs) pretty convenient, partly because I recently set up ssh for other reasons and then the leap to sshfs is quite small. I've always had a "learning disability" with respect to NFS, and sort of a religious objection to SAMBA (and also some problems with it, but I forget what those were, they were annoying things, and just to make up something, maybe it had to losing connectivity without being notified or something similarly aggravating). sshfs seems to be outperformed (speedwise) by SAMBA and NFS, but it is hard to imagine that being a real problem with modern (fast) computers and 100MB or better Ethernet. If you have sshfs set up, you have nothing more to do on the server. On the client, you install the sshfs client.
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-07-30 01:50 +0200 |
| Message-ID | <Ay3ZD-6a5-1@gated-at.bofh.it> |
| In reply to | #225505 |
I run a pair of Debian servers. One is essentially a NAS, and the
other is a backup system. Both have 30TB (soon to be 48TB) arrays. I
am running XFS, rather than ZFS on the RAID arrays. ZFS is definitely
nice, but is not supported directly under Debian. I don't find the
comparative deficencies of XFS vs. ZFS to be an issue. If these were
enterprise systems, I probably would go with ZFS, directly supported or
not. As it is, these systems work just fine for me, and have for well
over a decade. Here is my layout for my main system:
/dev/md1:
Version : 0.90
Creation Time : Thu Jan 30 20:51:21 2020
Raid Level : raid1
Array Size : 204736 (199.94 MiB 209.65 MB)
Used Dev Size : 204736 (199.94 MiB 209.65 MB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Wed Jul 22 16:32:48 2020
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : bitmap
UUID : 5a6b8565:1c89ed91:fa72b0ce:3d8e1057 (local to host
RAID-Server)
Events : 0.78
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
/dev/md2:
Version : 1.2
Creation Time : Thu Jan 30 20:52:18 2020
Raid Level : raid1
Array Size : 104791040 (99.94 GiB 107.31 GB)
Used Dev Size : 104791040 (99.94 GiB 107.31 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Wed Jul 29 18:31:46 2020
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : bitmap
Name : RAID-Server:2 (local to host RAID-Server)
UUID : 30344eb6:91b2c63b:c7db2e28:80c2d6fe
Events : 1013
Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
1 8 18 1 active sync /dev/sdb2
/dev/md0:
Version : 1.2
Creation Time : Fri Oct 3 20:06:55 2014
Raid Level : raid6
Array Size : 29301835776 (27944.41 GiB 30005.08 GB)
Used Dev Size : 4883639296 (4657.40 GiB 5000.85 GB)
Raid Devices : 8
Total Devices : 9
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Wed Jul 29 18:31:43 2020
State : clean
Active Devices : 8
Working Devices : 9
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 1024K
Consistency Policy : bitmap
Name : RAID-Server:0 (local to host RAID-Server)
UUID : d26e92db:8bd207bb:db9bec69:4117ed57
Events : 7426903
Number Major Minor RaidDevice State
10 8 32 0 active sync /dev/sdc
12 8 48 1 active sync /dev/sdd
8 8 64 2 active sync /dev/sde
11 8 80 3 active sync /dev/sdf
9 8 96 4 active sync /dev/sdg
15 8 112 5 active sync /dev/sdh
16 8 128 6 active sync /dev/sdi
14 8 144 7 active sync /dev/sdj
13 8 160 - spare /dev/sdk
Filesystem Size Used Avail Use% Mounted on
udev 7.8G 0 7.8G 0% /dev
tmpfs 1.6G 42M 1.6G 3% /run
/dev/md2 98G 6.7G 87G 8% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/md1 194M 57M 128M 31% /boot
/dev/md0 28T 22T 6.0T 79% /RAID
Backup:/Backup 44T 44T 512K 100% /Backup
tmpfs 1.6G 16K 1.6G 1% /run/user/0
The boot drives are a pair of 128G SSDs with three partitions on each.
Two of the partitions are employed in RAID 1 arrays formatted as ext2
(/boot) and ext4 (/), with the rest employed as swap space. The data
arrays consist of nine 8T unpartitioned hard drives, 8 as members of a
RAID 6 array formatted as XFS and 1 as a hot spare.
On 7/29/2020 2:40 PM, Patrick Bartek wrote:
> Hi! all,
>
> Thought putting an old, retired system to good use would be better than
> letting it gather dust in a closet. And by old, I mean OOOOOLD! I
> built it 13 years ago. However, it's been upgraded many times since,
> and was still my main box running Stretch until last year. Its current
> specs: ASRock A770DE+ AM3 MB, AMD Phenom II x4 @ 3.0 GHZ, 8GB DDR2 RAM
> (max 16GB), 6 - SATA II & 1 - IDE connections, USB2.0.
>
> The problem I've run into is finding a NAS OS to run on it. They all
> seem to require UEFI. which this MB does not support. (I said it was
> old.) However, in my search I did come across OpenMediaVault which is
> a simple, lightweight NAS OS based on Debian Jessie that will work with
> either MBR or UEFI. One nice feature OMV has is it can be installed as
> a service on top of any Debian OS. So, I can use something more
> contemporary and still supported.
>
> Anyone currently using OpenMediaVault, or have recommendations for
> another package, or advice, in general, on homebuilt NAS?
>
> My plan is to use it for backups, Dropbox-like storage, and possibly
> home media server.
>
> Thanks
>
> B
>
[toc] | [prev] | [next] | [standalone]
| From | Patrick Bartek <nemommxiv@gmail.com> |
|---|---|
| Date | 2020-07-30 23:40 +0200 |
| Message-ID | <Ayoro-1SG-3@gated-at.bofh.it> |
| In reply to | #225510 |
On Wed, 29 Jul 2020 18:41:46 -0500 Leslie Rhorer <lesrhorer@att.net> wrote: > I run a pair of Debian servers. One is essentially a NAS, and the > other is a backup system. Both have 30TB (soon to be 48TB) arrays. I > am running XFS, rather than ZFS on the RAID arrays. ZFS is definitely > nice, but is not supported directly under Debian. I don't find the > comparative deficencies of XFS vs. ZFS to be an issue. If these were > enterprise systems, I probably would go with ZFS, directly supported or > not. As it is, these systems work just fine for me, and have for well > over a decade. Here is my layout for my main system: > > /dev/md1: > [BIG snip] Thanks for the recommendations and set up details. B
[toc] | [prev] | [next] | [standalone]
| From | David Christensen <dpchrist@holgerdanske.com> |
|---|---|
| Date | 2020-08-08 01:30 +0200 |
| Message-ID | <ABjYd-53U-3@gated-at.bofh.it> |
| In reply to | #225510 |
On 2020-07-29 16:41, Leslie Rhorer wrote: > I run a pair of Debian servers. One is essentially a NAS, and the > other is a backup system. Both have 30TB (soon to be 48TB) arrays. I > am running XFS > Filesystem Size Used Avail Use% Mounted on > /dev/md0 28T 22T 6.0T 79% /RAID > Backup:/Backup 44T 44T 512K 100% /Backup On 2020-08-02 13:09, Leslie Rhorer wrote: > servers can easily pump out more than 4Gbps to clients On 2020-08-05 18:38, Leslie Rhorer wrote: > A full backup of my main array to hard drive media will take a minimum > of 15 days. Transferring across the 10G link to my backup server > would take a minimum of 2.8 days. > The backup server is an exact mirror of the main server, The NAS array is 8 @ 5 TB live drives and 1 @ 5 TB hot spare? The backup system array is 8 @ 8 TB data drives and 1 @ 8 TB hot spare? No LVM? AIUI you are running desktop motherboards without ECC memory and XFS does not protect against bit rot. Are you concerned? I agree that the 79% usage on the NAS array means action is required. As I understand md RAID6, the only way to add capacity is to backup, rebuild the array with additional and/or larger drives, and restore (?). Are you concerned about 100% usage on the backup server array? > plus several T of additional files I don't need on the main server. 44 TB total - 22 TB backup = 22 TB additional. That explains the 100% usage. Have you considered putting the additional files on another server that is not backed up, only archived? On 2020-08-06 18:58, Leslie Rhorer wrote: > The servers have 10G optical links between them. A full backup to the > RAID 6 array takes several days. One 10 Gbps network connection per server? 22E+12 bytes in 2.8 days is ~90 MB/s. That is a fraction of 4 Gbps and an even smaller fraction of 10 Gbps. Have you identified the bottleneck? > A full backup to single drives takes > 2 weeks, because single drives are limited to about 800Mbps, while the > array can gulp down nerly 4Gbps. Nightly backups (via rsync) take a > few minutes, at most. 800 Mbps network throughput should be ~88 MB/s HDD throughput. 2 to 4 TB drives should be faster. Have you identified the bottleneck? 44E+12 bytes in 15 days is ~34 MB/s. Is this due to a DAR manual workflow that limits you to one or two archive drives per day? Are you using hot-swap for the archive drives? What make and model rack? What HBA/RAID card? Same for hot spares and HBA? Same for the 16 bay rack, HBA, port replicators? If you have two HDD hot-swap bays, can DAR leap-frog destination media? E.g. You insert two archive drives and have DAR begin writing to the first. When the first is full, DAR begins writing to the second and notifies you. You pull the first drive, insert the third drive, and notify DAR. When the second drive is full, DAR begins writing to the third, and notifies you. Etc.? If you have many HDD hot-swap bays, can DAR write in parallel? With leap-frog? In my experience, HDD's that are stored for long periods have the bad habit of failing within hours of being put back into service. Does this concern you? What is your data destruction policy? One design pattern for ZFS is a pool of striped virtual devices (VDEV), each VDEV being two or more mirrored drives of the same size and type (e.g. SSD, SAS, SATA, etc.). Cache, intent log, and spare devices can be added for performance and/or reliability. To add capacity, you insert another pair of drives and add them into the pool as a VDEV mirror. The top-level file system is automatically resized. File systems without size restrictions can use the additional capacity. Performance increases. For backup, choices include replication to another pool and mirror tricks (add one drive to each VDEV mirror, allow it to resilver, remove one drive from each mirror in rotation). David
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-08 03:30 +0200 |
| Message-ID | <ABlQl-6bJ-9@gated-at.bofh.it> |
| In reply to | #225994 |
On 8/7/2020 6:23 PM, David Christensen wrote: >> ??Filesystem?????????? Size?? Used Avail Use% Mounted on > >> ??/dev/md0???????????????? 28T???? 22T?? 6.0T?? 79% /RAID >> ??Backup:/Backup???? 44T???? 44T?? 512K 100% /Backup > > The NAS array is 8 @ 5 TB live drives and 1 @ 5 TB hot spare? It was. I was in the process of upgrading. Now it is 8 x 8 + 8 > The backup system array is 8 @ 8 TB data drives and 1 @ 8 TB hot spare? Yep. I always upgrade the backup before I upgrade the main array. Well, wait a second. To be clear, that is 6 x 8T of data, plus 2 x 8T of parity, plus 1 x 8T of spare. > No LVM? No. I don't feel a need for LVM on the data arrays. I use the entire, unpartitioned drive for /RAID. > AIUI you are running desktop motherboards without ECC memory and XFS > does not protect against bit rot.?? Are you concerned? Yes. I have routines that compare the data on the main array and the backup array via checksum. When needed, the backups supply a third vote. The odds of two bits flipping at the very same spot are astronomically low. There has been some bit rot, but so far it has been manageable. > I agree that the 79% usage on the NAS array means action is required. Uh-huh. > As I understand md RAID6, the only way to add capacity is to backup, > rebuild the array with additional and/or larger drives, and restore (?). No, not at all. To add a drive: `mdadm /dev/md0 --add /dev/sdX` `mdadm -v /dev/md0 --grow --raid-devices=Y` Note if an internal bitmap is set, it must be removed prior to growing the array. It can be added back once the grow operation is complete. To increase the drive size, replace any smaller drives with larger drives one at a time: `mdadm /dev/md0 --add /dev/sdX` `mdadm /dev/md0 --fail /dev/sdY` Once all the drives are larger than the current device size used by the array: `mdadm /dev/md0 --grow` This will set the device size based upon the smallest device in the array. The device size can be set to a smaller value using the -z parameter. Once the array is grown, the filesystem needs to be expanded via the tool used for that purpose for the given file system. > Are you concerned about 100% usage on the backup server array? Some, yes. I am going to fix it by removing some very large but unnecessary files. It has only been at 100% for a few days. > > plus several T of additional files I don't need on the main server. > > 44 TB total - 22 TB backup = 22 TB additional.?? That explains the 100% > usage. Actually, no. There are not two backup copies on the file system. Believe it or not, there are 22T of files from other sources. > Have you considered putting the additional files on another server that > is not backed up, only archived? They should no longer be needed. Once I confirm that (in a few minutes from now, actually), they will be deleted. If any of the files in question turn out to be necessary, I will do that very thing. > On 2020-08-06 18:58, Leslie Rhorer wrote: > > The servers have 10G optical links between them.?? A full backup to the > > RAID 6 array takes several days. > > > One 10 Gbps network connection per server? Yes. I don't have slots for additional NIC boards, and my boards only have one port. > 22E+12 bytes in 2.8 days is ~90 MB/s.?? That is a fraction of 4 Gbps and > an even smaller fraction of 10 Gbps.?? Have you identified the bottleneck? That was a calculated number. Did I make a mistake? ... Oops. That should have been about 15 hours or so. The transfer rate for a large file is close to 4Gbps, which is about the best I would expect from this hardware. It's good enough. > > A full backup to single drives takes > > 2 weeks, because single drives are limited to about 800Mbps, while the > > array can gulp down nerly 4Gbps.?? Nightly backups (via rsync) take a > > few minutes, at most. > > 800 Mbps network throughput should be ~88 MB/s HDD throughput.?? 2 to 4 > TB drives should be faster.?? Have you identified the bottleneck? It's probbably the internal SATA controller on this old motherboard. I'm not using a high-dollar controller for external drives. Again, since I don't do this sort of thing daily, I am not worried about it. I start the backup and walk away. When I come back, it's done. Differential backups are small, so I only very rarely need a second drive. > 44E+12 bytes in 15 days is ~34 MB/s.?? Is this due to a DAR manual > workflow that limits you to one or two archive drives per day? No, that's about what I get on average transfers to external drives. > Are you using hot-swap for the archive drives??? What make and model > rack??? What HBA/RAID card??? Same for hot spares and HBA??? Same for the > 16 bay rack, HBA, port replicators? Yes on the hot swap. I just use a little eSATA docking station attached to an eSATA port on the motherboard. 'Definitely a poor man's solution. > If you have two HDD hot-swap bays, can DAR leap-frog destination media? I believe it can, yes. A script to handle that should be pretty simple. I have never done so. > ??E.g. You insert two archive drives and have DAR begin writing to the > first.?? When the first is full, DAR begins writing to the second and > notifies you.?? You pull the first drive, insert the third drive, and > notify DAR.?? When the second drive is full, DAR begins writing to the > third, and notifies you.?? Etc.? Right. I just use the device ID (rather than the name) to write the files and pause when the drive is full. It should be possible to do it with multiple device ID targets. In fact, I know it would be. The script I use right now pauses and waits for the user to replace the drive and press <Enter>. It would be trivial to have the script continue with a different device ID instead of pausing. Iterating through a list of IDs is hardly any more difficult. Hmm. You have given me an idea. Thanks! > If you have many HDD hot-swap bays, can DAR write in parallel??? With > leap-frog? No, I don't think so, at least not in general. I suppose one could create a front-end process which divides up the source and passes the individual chunks to multiple DAR processes. A Python script should be able to handle it pretty well. > In my experience, HDD's that are stored for long periods have the bad > habit of failing within hours of being put back into service.?? Does this > concern you? No, not really. If a target drive fails during a backup, I can just isolate the existing portion and then start a new backup on the isolate. A failed drive during a restore could be a bitch, but that's pretty unlikely. Something like dd_rescue could be a great help. > What is your data destruction policy? You mean for live data? I don't have one. Do you mean for the backups? There is no formal one. > One design pattern for ZFS is a pool of striped virtual devices (VDEV), > each VDEV being two or more mirrored drives of the same size and type > (e.g. SSD, SAS, SATA, etc.).?? Cache, intent log, and spare devices can > be added for performance and/or reliability.?? To add capacity, you > insert another pair of drives and add them into the pool as a VDEV > mirror.?? The top-level file system is automatically resized.?? File > systems without size restrictions can use the additional capacity. > Performance increases.?? For backup, choices include replication to > another pool and mirror tricks (add one drive to each VDEV mirror, allow > it to resilver, remove one drive from each mirror in rotation). Oh, yes. For an enterprise system, ZFS is the top contender, in my book. These are for my own use, and my business is small, however. If I ever get to the point where I have more than 10 employees, I will no doubt switch to ZFS. Let me put it this way: if a business has the need for a separate IT manager, his filesystem of choice for the file server(s) is pretty much without question ZFS. For a small business or for personal use the learning curve may be a bit more than the non-IT user might want to tackle. Or not. I certainly would not discourage anyone who wants to take on the challenge.
[toc] | [prev] | [next] | [standalone]
| From | David Christensen <dpchrist@holgerdanske.com> |
|---|---|
| Date | 2020-08-09 14:40 +0200 |
| Message-ID | <ABSMi-Ud-19@gated-at.bofh.it> |
| In reply to | #225997 |
On 8/7/20 6:24 PM, Leslie Rhorer wrote: > On 8/7/2020 6:23 PM, David Christensen wrote: >>> ??Filesystem?????????? Size?? Used Avail Use% Mounted on Your editor seems to replace multiple spaces with two question marks for each leading space (?). Please disable the feature if you can. >> The NAS array > Now it is 8 x 8 + 8 > >> The backup system array is 8 @ 8 TB data drives and 1 @ 8 TB hot spare I assume you filled your 16 drive rack with 8 TB drives (?). Is there a reason why you did not use a smaller number of larger drives, partially fill the rack, and leave open bays for future expansion and/or additional servers? > I don't feel a need for LVM on the data arrays. I use the > entire, unpartitioned drive for /RAID. I was leading in to LVM's ability to add capacity, but you seem to have solved this with mdadm (see below). >> Are you concerned [about bit rot]? > > Yes. I have routines that compare the data on the main array and > the backup array via checksum. When needed, the backups supply a third > vote. The odds of two bits flipping at the very same spot are > astronomically low. There has been some bit rot, but so far it has been > manageable. I had similar experiences and used similar methods in the past. BSD's mtree(8) is built for this purpose, but lacks a cache. The Debian version is behind FreeBSD (even when built from Sid source) and lacks key features. I resorted to writing a Perl script with caching. ZFS and replication made all of that unnecessary. > To add a drive: > > `mdadm /dev/md0 --add /dev/sdX` > `mdadm -v /dev/md0 --grow --raid-devices=Y` > > Note if an internal bitmap is set, it must be removed prior to > growing the array. It can be added back once the grow operation is > complete. > > To increase the drive size, replace any smaller drives with larger > drives one at a time: > > `mdadm /dev/md0 --add /dev/sdX` > `mdadm /dev/md0 --fail /dev/sdY` > > Once all the drives are larger than the current device size used by > the array: > > `mdadm /dev/md0 --grow` Nice. :-) >> Have you considered putting the additional files on another server >> that is not backed up, only archived? > > They should no longer be needed. Once I confirm that (in a few > minutes from now, actually), they will be deleted. If any of the files > in question turn out to be necessary, I will do that very thing. If DAR maintains a catalog of archive media and the files they contain, this would facilitate a data retention policy of "some files only exist on archive media". >> 22E+12 bytes in 2.8 days is ~90 MB/s.?? That is a fraction of 4 Gbps >> and an even smaller fraction of 10 Gbps.?? Have you identified the >> bottleneck? > That should have been about 15 hours or so. The transfer > rate for a large file is close to 4Gbps, which is about the best I would > expect from this hardware. It's good enough. 22E+12 bytes in 15 hours is ~408 MB/s. That makes more sense. >> Are you using hot-swap for the archive drives??? > Yes on the hot swap. I just use a little eSATA docking station > attached to an eSATA port on the motherboard. 'Definitely a poor man's > solution. My 2011 desktop motherboard with dual eSATA ports (150 Mbps?) gives very satisfactory performance. >> If you have two HDD hot-swap bays, can DAR leap-frog destination media? > > I believe it can, yes. A script to handle that should be pretty > simple. I have never done so. > The > script I use right now pauses and waits for the user to replace the > drive and press <Enter>. It would be trivial to have the script > continue with a different device ID instead of pausing. Iterating > through a list of IDs is hardly any more difficult. > > Hmm. You have given me an idea. Thanks! YW. :-) Let us know if you can reduce the time to create an archive set. >> If you have many HDD hot-swap bays, can DAR write in parallel??? With >> leap-frog? > > No, I don't think so, at least not in general. I suppose one could > create a front-end process which divides up the source and passes the > individual chunks to multiple DAR processes. A Python script should be > able to handle it pretty well. I have pondered writing a script to read a directory and create a set of hard link trees, each tree of size N bytes or less; filtered, sorted, and grouped by configurable parameters. If anyone knows of a such a utility, please reply. >> In my experience, HDD's that are stored for long periods have the bad >> habit of failing within hours of being put back into service.?? Does >> this concern you? > > No, not really. If a target drive fails during a backup, I can > just isolate the existing portion and then start a new backup on the > isolate. A failed drive during a restore could be a bitch, but that's > pretty unlikely. Something like dd_rescue could be a great help. As I understand ddrescue, it is designed for multiple copies of some content (e.g. a file or a raw device) that were originally identical, each copy was damaged in a different area, and none of the damaged areas overlap. ddrescue can then scan all the copies, identify the undamaged areas, and assemble a correct version. As I understand DAR, it uses a specialized binary format with compression, hashing, encryption, etc.. If you burn one archive media set using DAR, retain a few previous archive media sets, later need to do a restore, and one drive from the most recent archive media set is bad, I am uncertain if ddrescue will be of any help. >> What is your data destruction policy? > > You mean for live data? I don't have one. Do you mean for the > backups? There is no formal one. Likewise. It's a conundrum. > For an enterprise system, ZFS is the top contender, in my > book. These are for my own use, and my business is small, however. If > I ever get to the point where I have more than 10 employees, I will no > doubt switch to ZFS. > > Let me put it this way: if a business has the need for a separate > IT manager, his filesystem of choice for the file server(s) is pretty > much without question ZFS. For a small business or for personal use the > learning curve may be a bit more than the non-IT user might want to tackle. > > Or not. I certainly would not discourage anyone who wants to take > on the challenge. Migrating my SOHO servers from Linux, md, LVM, ext4, and btrfs to FreeBSD and ZFS has been a non-trivial undertaking. I've learn a lot and I think my data is better protected, but I still have more work to do for disaster preparedness. You have an order of magnitude more data, backups, and archives than I do. If and when you decide to try ZFS, I suggest that you break off a piece and work with that. David
[toc] | [prev] | [next] | [standalone]
| From | Peter Ehlert <peter@sdi-baja.com> |
|---|---|
| Date | 2020-07-30 19:00 +0200 |
| Message-ID | <Ayk4q-7BV-5@gated-at.bofh.it> |
| In reply to | #225505 |
This whole conversation is a bit over my head. I suggest you look into Syncthing. It's not in the Debian repos, but it is open source and it just works. https://syncthing.net/ It works for me, 750 GB, 7 machines. On 7/29/20 12:40 PM, Patrick Bartek wrote: > Hi! all, > > Thought putting an old, retired system to good use would be better than > letting it gather dust in a closet. And by old, I mean OOOOOLD! I > built it 13 years ago. However, it's been upgraded many times since, > and was still my main box running Stretch until last year. Its current > specs: ASRock A770DE+ AM3 MB, AMD Phenom II x4 @ 3.0 GHZ, 8GB DDR2 RAM > (max 16GB), 6 - SATA II & 1 - IDE connections, USB2.0. > > The problem I've run into is finding a NAS OS to run on it. They all > seem to require UEFI. which this MB does not support. (I said it was > old.) However, in my search I did come across OpenMediaVault which is > a simple, lightweight NAS OS based on Debian Jessie that will work with > either MBR or UEFI. One nice feature OMV has is it can be installed as > a service on top of any Debian OS. So, I can use something more > contemporary and still supported. > > Anyone currently using OpenMediaVault, or have recommendations for > another package, or advice, in general, on homebuilt NAS? > > My plan is to use it for backups, Dropbox-like storage, and possibly > home media server. > > Thanks > > B > >
[toc] | [prev] | [next] | [standalone]
| From | Charles Curley <charlescurley@charlescurley.com> |
|---|---|
| Date | 2020-07-30 21:50 +0200 |
| Message-ID | <AymIW-NO-9@gated-at.bofh.it> |
| In reply to | #225566 |
On Thu, 30 Jul 2020 09:40:29 -0700 Peter Ehlert <peter@sdi-baja.com> wrote: > This whole conversation is a bit over my head. > I suggest you look into Syncthing. > > It's not in the Debian repos, but it is open source and it just works. > https://syncthing.net/ Actually, it is in Buster. However, the version in buster is 1.0.0~ds1-1+b11. The version in the syncthing repo is 1.7.1. > > It works for me, 750 GB, 7 machines. I have a somewhat smaller setup, which Just Works (tm) for me. I recommend it. -- Does anybody read signatures any more? https://charlescurley.com https://charlescurley.com/blog/
[toc] | [prev] | [next] | [standalone]
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-07-31 13:10 +0200 |
| Message-ID | <AyB5f-1hq-5@gated-at.bofh.it> |
| In reply to | #225570 |
On Thu, 30 Jul 2020 13:21:00 -0600 Charles Curley <charlescurley@charlescurley.com> wrote: > On Thu, 30 Jul 2020 09:40:29 -0700 > Peter Ehlert <peter@sdi-baja.com> wrote: > > > This whole conversation is a bit over my head. > > I suggest you look into Syncthing. > > > > It's not in the Debian repos, but it is open source and it just works. > > https://syncthing.net/ > > Actually, it is in Buster. However, the version in buster is > 1.0.0~ds1-1+b11. The version in the syncthing repo is 1.7.1. FWIW, the version in Sid is 1.1.4~ds1-5. Still behind, but slightly less behind ;) Celejar
[toc] | [prev] | [next] | [standalone]
| From | Patrick Bartek <nemommxiv@gmail.com> |
|---|---|
| Date | 2020-07-30 23:30 +0200 |
| Message-ID | <AyohK-1Pv-25@gated-at.bofh.it> |
| In reply to | #225566 |
On Thu, 30 Jul 2020 09:40:29 -0700 Peter Ehlert <peter@sdi-baja.com> wrote: > This whole conversation is a bit over my head. > I suggest you look into Syncthing. > > It's not in the Debian repos, but it is open source and it just works. > https://syncthing.net/ > > It works for me, 750 GB, 7 machines. Thanks. I'll look into it. B > On 7/29/20 12:40 PM, Patrick Bartek wrote: > > Hi! all, > > > > Thought putting an old, retired system to good use would be better than > > letting it gather dust in a closet. And by old, I mean OOOOOLD! I > > built it 13 years ago. However, it's been upgraded many times since, > > and was still my main box running Stretch until last year. Its current > > specs: ASRock A770DE+ AM3 MB, AMD Phenom II x4 @ 3.0 GHZ, 8GB DDR2 RAM > > (max 16GB), 6 - SATA II & 1 - IDE connections, USB2.0. > > > > The problem I've run into is finding a NAS OS to run on it. They all > > seem to require UEFI. which this MB does not support. (I said it was > > old.) However, in my search I did come across OpenMediaVault which is > > a simple, lightweight NAS OS based on Debian Jessie that will work with > > either MBR or UEFI. One nice feature OMV has is it can be installed as > > a service on top of any Debian OS. So, I can use something more > > contemporary and still supported. > > > > Anyone currently using OpenMediaVault, or have recommendations for > > another package, or advice, in general, on homebuilt NAS? > > > > My plan is to use it for backups, Dropbox-like storage, and possibly > > home media server. > > > > Thanks > > > > B > > > > >
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-02 01:50 +0200 |
| Message-ID | <Az9qh-6Tt-5@gated-at.bofh.it> |
| In reply to | #225578 |
I don't think SyncThing is what you want, per your stated requirements. SyncThing synchronizes data among multiple hosts. You said you wanted a NAS, which implies a solitary host. On 7/30/2020 4:28 PM, Patrick Bartek wrote: > On Thu, 30 Jul 2020 09:40:29 -0700 > Peter Ehlert <peter@sdi-baja.com> wrote: > >> This whole conversation is a bit over my head. >> I suggest you look into Syncthing. >> >> It's not in the Debian repos, but it is open source and it just works. >> https://syncthing.net/ >> >> It works for me, 750 GB, 7 machines. > > Thanks. I'll look into it. > > B > >> On 7/29/20 12:40 PM, Patrick Bartek wrote: >>> Hi! all, >>> >>> Thought putting an old, retired system to good use would be better than >>> letting it gather dust in a closet. And by old, I mean OOOOOLD! I >>> built it 13 years ago. However, it's been upgraded many times since, >>> and was still my main box running Stretch until last year. Its current >>> specs: ASRock A770DE+ AM3 MB, AMD Phenom II x4 @ 3.0 GHZ, 8GB DDR2 RAM >>> (max 16GB), 6 - SATA II & 1 - IDE connections, USB2.0. >>> >>> The problem I've run into is finding a NAS OS to run on it. They all >>> seem to require UEFI. which this MB does not support. (I said it was >>> old.) However, in my search I did come across OpenMediaVault which is >>> a simple, lightweight NAS OS based on Debian Jessie that will work with >>> either MBR or UEFI. One nice feature OMV has is it can be installed as >>> a service on top of any Debian OS. So, I can use something more >>> contemporary and still supported. >>> >>> Anyone currently using OpenMediaVault, or have recommendations for >>> another package, or advice, in general, on homebuilt NAS? >>> >>> My plan is to use it for backups, Dropbox-like storage, and possibly >>> home media server. >>> >>> Thanks >>> >>> B >>> >>> >>
[toc] | [prev] | [next] | [standalone]
| From | Patrick Bartek <nemommxiv@gmail.com> |
|---|---|
| Date | 2020-08-02 17:50 +0200 |
| Message-ID | <Azopk-7K0-7@gated-at.bofh.it> |
| In reply to | #225718 |
On Sat, 1 Aug 2020 18:39:49 -0500 Leslie Rhorer <lesrhorer@att.net> wrote: > I don't think SyncThing is what you want, per your stated requirements. > SyncThing synchronizes data among multiple hosts. You said you wanted > a NAS, which implies a solitary host. After a little purusing of the docs on SyncThing, I agree. It's more of a personal Dropbox for your devices. Not what I was looking for. Besides most NAS software have "Dropbox-like" plugins, if such a feature is needed. Thanks for your feedback. B > On 7/30/2020 4:28 PM, Patrick Bartek wrote: > > On Thu, 30 Jul 2020 09:40:29 -0700 > > Peter Ehlert <peter@sdi-baja.com> wrote: > > > >> This whole conversation is a bit over my head. > >> I suggest you look into Syncthing. > >> > >> It's not in the Debian repos, but it is open source and it just works. > >> https://syncthing.net/ > >> > >> It works for me, 750 GB, 7 machines. > > > > Thanks. I'll look into it. > > > > B > > > >> On 7/29/20 12:40 PM, Patrick Bartek wrote: > >>> Hi! all, > >>> > >>> Thought putting an old, retired system to good use would be better than > >>> letting it gather dust in a closet. And by old, I mean OOOOOLD! I > >>> built it 13 years ago. However, it's been upgraded many times since, > >>> and was still my main box running Stretch until last year. Its current > >>> specs: ASRock A770DE+ AM3 MB, AMD Phenom II x4 @ 3.0 GHZ, 8GB DDR2 RAM > >>> (max 16GB), 6 - SATA II & 1 - IDE connections, USB2.0. > >>> > >>> The problem I've run into is finding a NAS OS to run on it. They all > >>> seem to require UEFI. which this MB does not support. (I said it was > >>> old.) However, in my search I did come across OpenMediaVault which is > >>> a simple, lightweight NAS OS based on Debian Jessie that will work with > >>> either MBR or UEFI. One nice feature OMV has is it can be installed as > >>> a service on top of any Debian OS. So, I can use something more > >>> contemporary and still supported. > >>> > >>> Anyone currently using OpenMediaVault, or have recommendations for > >>> another package, or advice, in general, on homebuilt NAS? > >>> > >>> My plan is to use it for backups, Dropbox-like storage, and possibly > >>> home media server. > >>> > >>> Thanks > >>> > >>> B > >>> > >>> > >> >
[toc] | [prev] | [next] | [standalone]
| From | Jonathan Dowland <jon+debian-user@dow.land> |
|---|---|
| Date | 2020-08-03 17:20 +0200 |
| Message-ID | <AzKpQ-4D3-19@gated-at.bofh.it> |
| In reply to | #225505 |
On Wed, Jul 29, 2020 at 12:40:03PM -0700, Patrick Bartek wrote: >Anyone currently using OpenMediaVault, or have recommendations for >another package, or advice, in general, on homebuilt NAS? My advice is to keep it boring and mundane, and avoid the temptation to try some flashy technology Just Because. (I put ZFS in that category, personally; and BTRFS). My NAS runs stock Debian and I've documented it here: <https://jmtd.net/hardware/phobos/> -- Please do not CC me, I am subscribed to the list. 👱🏻 Jonathan Dowland ✎ jmtd@debian.org 🔗 https://jmtd.net
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-04 00:10 +0200 |
| Message-ID | <AzQOB-95-1@gated-at.bofh.it> |
| In reply to | #225775 |
On 8/3/2020 10:14 AM, Jonathan Dowland wrote: > On Wed, Jul 29, 2020 at 12:40:03PM -0700, Patrick Bartek wrote: >> Anyone currently using OpenMediaVault, or have recommendations for >> another package, or advice, in general, on homebuilt NAS? > > My advice is to keep it boring and mundane, and avoid the temptation > to try some flashy technology Just Because. (I put ZFS in that category, > personally; and BTRFS). My NAS runs stock Debian and I've documented it > here: <https://jmtd.net/hardware/phobos/> I concur. One area in wich I disagree is your decision not to employ RAID. As you very properly comment in the linked page, RAID is not a backup. They are *VERY* different things. Therefore both,in my estimation, are absolutely required. Of course, any time a data target exceeds the readily availabe maximum spindle size, the question is moot. RAID then must be deployed in order to obtain the necessary repository size. I would never deploy any server or workstation, however, without both a RAID solution (usually several) and a backup solution (again, usually several). My main server's data is backed up every morning by a nearly identical backup server using rsync, which in turn is backed up in its entirety on a multi-volume offline, off-site hard drive set using DAR every few months. All the boot systems employ two disk RAID 1 mirrors and all the data repositories consist of 5 - 8 spindle RAID 6 arrays.
[toc] | [prev] | [next] | [standalone]
| From | deloptes <deloptes@gmail.com> |
|---|---|
| Date | 2020-08-04 01:20 +0200 |
| Message-ID | <AzRUl-LM-3@gated-at.bofh.it> |
| In reply to | #225794 |
Leslie Rhorer wrote: > My main server's data is backed up every morning by a > nearly identical backup server using rsync, which in turn is backed up > in its entirety on a multi-volume offline, off-site hard drive set using > DAR every few months. All the boot systems employ two disk RAID 1 > mirrors and all the data repositories consist of 5 - 8 spindle RAID 6 > arrays. any thoughts on using deduplication? For example I started using borg some time ago. It saves a lot of space and makes it possible to have multiple backups and longer retention. I don't backup music and video - too big and not changing. The raid is enough for that.
[toc] | [prev] | [next] | [standalone]
| From | David Christensen <dpchrist@holgerdanske.com> |
|---|---|
| Date | 2020-08-04 10:40 +0200 |
| Message-ID | <AA0Eh-62G-9@gated-at.bofh.it> |
| In reply to | #225798 |
On 2020-08-03 16:17, deloptes wrote: > any thoughts on using deduplication? For example I started using borg some > time ago. It saves a lot of space and makes it possible to have multiple > backups and longer retention. ZFS supports de-duplication, but the documents warn about enabling it. So, of course I enabled de-duplication on my ZFS SOHO file server. ;-) Everything was groovy when utilization was ~30%, but performance for bulk writes degraded precipitously as the pool filled. This includes backup replication jobs. I am fairly certain de-duplication is a major contributing factor. The only way to test this hypothesis is to create a fresh pool using similar hardware, replicate the data without de-duplication, and benchmark. jdupes looks interesting, and should work on any file system that supports hard links. I expect BorgBackup either calls jdupes or implements similar functionality: https://linuxcommandlibrary.com/man/jdupes.html David
[toc] | [prev] | [next] | [standalone]
| From | Jonathan Dowland <jon+debian-user@dow.land> |
|---|---|
| Date | 2020-08-04 11:30 +0200 |
| Message-ID | <AA1qG-6zE-21@gated-at.bofh.it> |
| In reply to | #225811 |
On Tue, Aug 04, 2020 at 01:32:24AM -0700, David Christensen wrote: >jdupes looks interesting, and should work on any file system that >supports hard links. I expect BorgBackup either calls jdupes or >implements similar functionality: > >https://linuxcommandlibrary.com/man/jdupes.html I'm fairly sure Borg stores files by a hash of their contents and maintains a separate index of names→hashes for each generation. Thanks for mentioning jdupes, I have a problem it might help me solve for which fdupes is not up to the task. -- Please do not CC me, I am subscribed to the list. 👱🏻 Jonathan Dowland ✎ jmtd@debian.org 🔗 https://jmtd.net
[toc] | [prev] | [next] | [standalone]
Page 2 of 6 — ← Prev page 1 [2] 3 4 5 6 Next page →
Back to top | Article view | linux.debian.user
csiph-web