NAS4Free 9

Install of NAS4Free 9.0.0.1.43 on to a flash stuck in PATA was uneventful. But unlike Solaris, OpenIndiana and FreeNAS the web GUI showed me exactly what I wanted to know - real time status of NAS appliance: CPU frequency and load, SMART status of hard drives.

Power management worked and CPU frequency was adjusting according to load! On Solaris and OpenIndiana smartctl never worked for me. On FreeNAS smartctl did work but powerd did not and GUI was lacking this info.
I imported a ZFS hard drive created on FreeNAS8 and jumped to network throughput testing. Writing to NAS (I copied a 5.08GB DVD from win7 PC) went at around 40MB/sec.
Reading from NAS begins but then fails. Windows shows "Network Error" dialog "There is a problem accessing \\nas\zfs".
dmesg tells me "pid 7239 (smbd), uid 0: exited on signal 6"
log.nmbd and log.smbd contain nothing but startup messages.
Houston, we have a problem.

I will try to re-create ZFS vdev and pool and to reproduce.
More from the GUI diagnostics log:

Mar 31 22:25:05 nas smbd[7102]: [2012/03/31 22:25:05.441422, 0] lib/fault.c:51(fault_report)
Mar 31 22:25:05 nas smbd[7102]: ===============================================================
Mar 31 22:25:05 nas smbd[7102]: [2012/03/31 22:25:05.441703, 0] lib/fault.c:52(fault_report)
Mar 31 22:25:05 nas smbd[7102]: INTERNAL ERROR: Signal 11 in pid 7102 (3.6.3)
Mar 31 22:25:05 nas smbd[7102]: Please read the Trouble-Shooting section of the Samba3-HOWTO
Mar 31 22:25:05 nas smbd[7102]: [2012/03/31 22:25:05.441749, 0] lib/fault.c:54(fault_report)
Mar 31 22:25:05 nas smbd[7102]:
Mar 31 22:25:05 nas smbd[7102]: From: http://www.samba.org/samba/docs/Samba3-HOWTO.pdf
Mar 31 22:25:05 nas smbd[7102]: [2012/03/31 22:25:05.441786, 0] lib/fault.c:55(fault_report)
Mar 31 22:25:05 nas kernel: pid 7102 (smbd), uid 0: exited on signal 6
Mar 31 22:25:05 nas smbd[7102]: ===============================================================
Mar 31 22:25:05 nas smbd[7102]: [2012/03/31 22:25:05.441813, 0] lib/util.c:1117(smb_panic)
Mar 31 22:25:05 nas smbd[7102]: PANIC (pid 7102): internal error
Mar 31 22:25:05 nas smbd[7102]: [2012/03/31 22:25:05.441883, 0] lib/util.c:1221(log_stack_trace)
Mar 31 22:25:05 nas smbd[7102]: BACKTRACE: 0 stack frames:
Mar 31 22:25:05 nas smbd[7102]: [2012/03/31 22:25:05.441976, 0] lib/fault.c:416(dump_core)
Mar 31 22:25:05 nas smbd[7102]: dumping core in /var/log/samba/cores/smbd
Mar 31 22:25:05 nas smbd[7102]:

Off to read Trouble-Shooting section of the Samba3-HOWTO

FreeNAS 8

I am impressed!  The installation lasted 10min 2sec.  FreeNAS 8.0.4 (FreeBSD 8.2, ZFS file system v4, ZFS storage pool v15) was installed on a 2GB flash card plugged into the PATA port. Reboot.  Reconfigured NIC for a static address.  Launched web GUI.  The GUI is kinda strange but totally usable.  Created a user.  SSHed into NAS to poke around. Left top running.  The rest was done solely through web GUI.
Imported NTFS volume, shared it through CIFS, measured network throughput by copying a 5.08GB DVD image.  NTFS is not really writable, but who cares about NTFS on NAS!
Created a UFS volume on a HD I used for OpenIndiana install - it was not recognized as ZFS and no option to import was offered.  Had to fiddle with CIFS service for the right permissions to become visible to Windows.  Measured network throughput. Pretty good!
Removed CIFS share and volume and created a simple ZFS volume forcing a 4KB sector.Again restarted CIFS to propagate the changes.  Measured network throughput.  Good!

FreeNAS 8.0.4 network (1Gbit, default frames) throughput as measured from a Windows 7 pc.  On the NAS side the following hard drive was used: WDC WD5000AAKS-00V1A0.

File System	Read (MB/s)	Write(MB/s)
NTFS	40	<1
UFS	75	35
ZFS	76	39

These are very solid results!  The biggest (and still unresolved) problem I had was in finding an SMTP server to accept reports e-mailed by NAS into my public mailbox.  So far none!
Update: resolved! Used google with application-specific password!
Yes, the versions of ZFS and even FreeBSD used in FreeNAS8 are not the latest and greatest.
There is no real time SMART status in web GUI. But smartctl works as expected which means you can hope that the GUI will catch up.
Overall it looks promising.  Good job, FreeNAS8 team!
I will check out FreeNAS 0.7.5.9496 tomorrow - look forward to FreeBSD 9.0-Release, ZFS storage pool v28, ZFS filesystem v5.
Scratch that! Instead I'll try NAS4Free 9.0.0.1.43
Improve benchmarking: use iperf.

OpenIndiana with Napp-it

napp-it site is a good source of information on Solaris flavors.  Decided to go with its advise and try OpenIndiana in hope that SMART information can be retrieved from the hard drives connected to motherboard.  This time used live CD.  It IS more user friendly than text install.  No problems with network setup this time!

Here are my notes:

• Still no SMART info.  Major bummer.  A storage appliance which can not tell the temperature of the hard drive or do SMART diagnostics is a joke.
• X kernel panics when intel video driver is used. So I am forced to use vesa driver instead.  Which does not support resolution higher than 800x600.  Which, in turn, makes some configuration GUI unusable.  Will try to follow these recommendations. 
  Tried it! Got nothing but kernel panic. So I am stuck with 800x600 resolution.
• Spent some time learning how to force boot into single user mode from GRUB prompt.
• disabled graphical login:
  svcadm disable svc:/application/graphical-login/gdm 
  Did not figure out how to start gnome desktop for those cases when GUI is desired, so later enabled GDM back.
• verified that CPU SpeedStep capability is recognized:
  
  

  alex@nas:/etc/X11# kstat -m cpu_info -s supported_frequencies_Hz
  module: cpu_info                        instance: 0
  name:   cpu_info0                       class:    misc
          supported_frequencies_Hz        1203000000:1803000000
  
  module: cpu_info                        instance: 1
  name:   cpu_info1                       class:    misc
          supported_frequencies_Hz        1203000000:1803000000
  

  
  Too bad this capability is never used and I always have:
  

  alex@nas:/etc/X11# kstat -m cpu_info -s current_clock_Hz
  module: cpu_info                        instance: 0
  name:   cpu_info0                       class:    misc
          current_clock_Hz                1803000000
  
  module: cpu_info                        instance: 1
  name:   cpu_info1                       class:    misc
          current_clock_Hz                1803000000
  

I am really disappointed with my inability to take advantage of SMART and feel this is a must have.  So I decided to jump the gun.  Investigated HBA controller options.  Done!  Ordered adapter and cables.
Finalized the plan for hard drives.  I will start with a mirror of 2x3.5" drives for performance testing and will take it from there.  OS will be stored on a mirror of 2x2.5" drives.  Got it!  Not to forget the cage!  I grew to like these!  So convenient for fast changes!  Looks like I can't move OS to a new smaller drive - another reinstall is in order.
Update: after trying FreeNAS, I decided to stick with it.  Or NAS4Free.  At least for now.  Booting from flash is soooo attractive.  So I cancelled an order for 2.5" drive and the cage.
Wooof.

Installing Solaris with Napp-it

Downloaded and burned text installer CD.  Text installation reminded me of good old slackware days.  Anyway.  On the first run the installer hung after the network settings page - I chose automatic config there.  Could it be the installer was looking for some LDAP/NIS server on my home LAN?  On the second run I chose to provide network settings manually and that helped.  I understand network layer went through substantial changes.

Reboot!  I have an ascetic login prompt.  Oh and sendmail complains it can't qualify my own domain.  Boy!  Some things never change
I type a single line to install napp-it, and after a sanity check (it confirms that Solaris 11 is supported!) off it goes to download stuff.  Lotsa stuff.  Now it loads gcc.  BTW, no progress bars.  These were so 20th century..
It worked!  I can now point my browser to nas and get to the napp-it GUI!  Also ssh works!  Set putty to use utf8 so that mc draws lines properly.
Did you notice how fast it reboots?  The fist time I thought something went wrong.  It does not go to BIOS!
Still for me to do:

• Solaris: set up e-mail delivery 
• Solaris: configure SSH to use public keys.
• Solaris: play with console setup - make sure mc can draw lines, maybe adjust # of cols/lines
• Solaris: verify that power management /speedstep works
• Solaris: verify that HDs SMART info is accessible
• Solaris: make rpool a mirror
• Solaris: CIFS server - is it configured or not?
• napp-it: verify HDs temps can be monitored

Software Options

I am excited about ZFS.  This entire project is about giving it a shot.  The promise of a reliable and easy to use file system is too big to pass.  I am excited to learn new things done in my neighborhood - it's not like new file systems appear that often.

FreeNAS

The first contender I considered was FreeNAS.  I love, love, love the idea of booting the system from flash and having a web-based administration GUI.  As a matter of fact, I made these my requirements.  Unfortunately I hear FreeNAS8 lacks in functionality and all versions of FreeNAS, by virtue of being based on FreeBSD, lag behind in the version of ZFS.  I also hear ZFS performance is lacking.  I do plan to give FreeNAS a shot!

Solaris 11 + napp-it

ZFS being native to Solaris/Illumian/NexentaCore/OpenIndiana I hope to find here the best features and performance.  Hope that napp-it will offer a functional and friendly GUI.  I understand Solaris has kernel-based CIFS server promising (unlike Samba) a good interoperability with Windows.

We shall see!

NAS Hardware

Here is an old desktop, once built for my parents to surf internet and look at their grandchildren pictures.

• CPU: Intel Core 2 Duo 65nm E4300 Allendale 1.8GHz 800MHz FSB LGA 775 65W
• Motherboard: MSI G31TM-P21 LGA 775 Intel G31 Micro ATX.  Onboard video, 4x SATA ports on ICH7.

It is now souped up:  RAM is maxed out to 8MB.  New hard drive cage is installed. It proved to be loud enough to do something about it.  Just disconnecting the cage fan results in HD temps raising to 50C within an hour, and that's with just 2 of them installed.  Placed an order for a fan controller - hope that will help.  The cage is also designed so that the connectors - all of them - are on the farther side so that after installation none of these are accessible in my case.  Hence I installed it upside down.

Rationale

At home the (only) "desktop" I use is an HTPC.  Noisy at that, because of the need to keep and cool large storage for media playback and editing.  I wonder whether something could be done.
Main requirements are performance, ease of use, reliability and ease of growth.
Copying between internal SATA HDs is being done at about 70-100MB/s.  This sets performance goal.

Reliability is now achieved by keeping multiple backup HDs.  The backup copies are frequently obsolete, but you can't beat the simplicity of sticking a bare HD into a docking station (the HTPC needs to be off) and launching a synctoy.  Growth is straightforward.  Every other year I buy a new HD, copy media to it, replace the media HD in the HTPC with a new one and relegate an old one to backup duties.
For a couple of years I was running a Linux box (I don't think term NAS was widely used then) with a software RAID5.  I verified a procedure (based on mdadm) to introduce new hard drives followed by volume expansion.  But boy, every time the thing would boot, watching it fschk'ing the drives was nerve wracking.  And write performance was underwhelming to say the least.