02-20-2013 02:32 PM
I have an XW8600 running dual X5440 processors and 6 GB of ram. I want to turn this system into a home media NAS server runing FreeNas or Open Media Vault and have a few questions.
First off, I have read in a few places this system cannot handle 6 Gb/s SATA drives on its 3 Gbp/s ports. I have upgraded the bios firmware of the board to the lastest version, and currently have a WD RED 2TB drive connected to one of the SATA ports and its running fine. Windows Vista was on it for testing for a week, and Open Media Vault is currently on it and running for testing before jumping into the project head first. Are these 6Gbp/s ddriver compatable as of yet? Has the drive firmware issue been resovled ont hese newer drives? I read here: http://h20000.www2.hp.com/bizsupport/TechSupport/D
My second question is - Can I run SATA drives in the SAS ports as well? I know SAS is backwards comptable with SATA, but I am wondering if mixing SATA drives in the SATA ports and SATA drives in the SAS ports is going to cause problems. I plan on creating two virtual disks in software RAID 5 - one for the SATA ports and a seperate VD for the SAS ports. Will this be an issue?
Solved! Go to Solution.
02-20-2013 05:29 PM - edited 02-20-2013 06:47 PM
The HP information on that is simple..... buy the drives from HP and you'll be OK. I wish they would release their breakthrough, because the prior advisory indicated things would not work for sure. I have seen no hint that they will release the firmware fix their engineers came up with to certify their SATA generation 3 drives to run reliably on the SATA generation 2 HP workstations. It is not a BIOS update. I can't blame HP for needing to get return on their engineering investment.
Having said that, you do have an alternative, which is to buy new SATA generation 2 drives. The xw8600 won't run any faster on SATA generation 3 drives.
Make sure to add in a front fan if you don't have one already, for extra cooling if you load up on hard drives. For the xw8400 and xw8600 the fans are almost identical throughout, and the supplemental front "PCI" fan is a 80x25 mm fan for both. I have figured out a way to cheaply add in a nice Noctua non-PMW 92x25 mm front fan to the xw6400/xw6600 workstations without the expensive and hard to find front fan holders, and bet that will work on the xw8600. Let me know if you're interested in lower noise.
I don't have any experience with the SAS to SATA question.......
Another tip..... the xw6600 has two PCIe x16 slots which are PCIe generation 2 slots unlike those in the xw6400 and xw8400. I bet those two PCIe x16 slots on the xw8600 are generation 2 also. I'm working on a project to get the latest HP USB 3.0 card running at full bandwidth on the lower of the two PCIe x16 generation 2 slot in my xw6600. The same idea would work with a PCIe generation 2-to-SATA generation 3 card, I bet. I do think that would get you around the SATA generation 3 hard drives on a SATA generation 2 HP workstation issue, because the controller chip would be on the card, not the motherboard. There are some good articles out there on these type of cards.
Do you need 3.5" drives? 2.5" drives draw less power and you can fit more in, cooler. There is a lot of room in there, as you know. Try to stay away from the 2.1 TB limit... it is a bear to move above that but can be done if really necessary, as has been posted about here.
Keep us updated, please.... sounds interesting!
02-21-2013 12:14 AM
After the above was complete I placed all the drives on SAS ports and did the same in both stand alone and in RAID 0. No issues came out of this either. It looks as if (at the very least) Western Digital has figured out how to get their 3rd gen drives to work on this board.
In the end I installed Open Media Vault on the system and placed the drives in a RAID 5 array, and mounted them within the OS (Debian based). They are currently up and running. No smart errors as of yet, none of the drives have dropped out of the array, and I'm currently transferring around 1.5 TB of data from an external drive onto them. I'll have to check it in the morning to see how everything is running. Going to be very picky about backups over the next while to ensure it keeps this up. I'm guessing because of the previously discussed issues with 3rd gen SATA drives that I'll be keeping this in the testing phase for a few weeks to a month before I'm comfortable with it.
02-21-2013 12:17 AM
02-21-2013 06:10 AM
That is great news on the WD Reds..... would you post the exact model number you are using for future reference here?
The front fan: You may already have the black plastic front fan holder that a 80x25mm fan will snap into. From HP those are PWM speed controlled, and they run slower by default than if they received full 12v without PWM throttle. Why they did not put in 92x25mm fans is beyond me.... there is more than enough room.
My favorite 92mm fan, if you choose to not use or don't have the fan holder is to mount a Noctua 92mm NF-B9 case fan up front with 1.5" stainless wedge head philips drive stainless #6 screws thru the front case perforation case holes. I use a #10 fender washer on the outside of the holes, and the #6 wedge head settles nicely down into that. You can also use a metric fender washer. I get this stuff stainless from Ace Hardware. Nylock nut on the inside, atop a regular #10 washer, atop a chromed wire fan grill from the inside. Air flow is from the out to in. For the xw6400/6600 cases I can only get 3 of the 4 fan holes to line up, and so I only use 3 screws total. I don't overtighten..... just snug. Those Noctua come with in-line rpm reducers..... I use the LNA, or a ULNA if I really want things quiet. You may wish to use none.... the lead is long enough without.
4-pin motherboard header is right there for you....... just shave off the thin lip on the yellow wire side of the fan plug and the plug will slip onto pins 1-3 of the motherboard header just fine. It will run at the Noctua stated speeds for 12v depending on what adapter you may or may not choose to use. They do make a PWM 92mm fan, but the HP PWM control makes that run quite slow by default.
This is much cheaper than trying to track down one of the black plastic fan holder inserts if you don't have one in the case already.
02-21-2013 09:39 AM - edited 02-21-2013 09:45 AM
I'll look into the fans today. The model for the WD Red drives im using is WD20EFRX. They're specificly made for NAS servers. I'll keep this thread updated with my progress on it and any issues that arrise.
To summarized - WD RED SATA III 6 Gbp/s HDD's work in the xw8600 on both SATA and SAS ports, as stand alone drives and in the xw8600 onboard RAID arrays. Mixed use SAS and SATA has not been tested. I highly doubt they will work when in mixed SAS and SATA ports as I have yet to find a system that will allow this.
So far from what I can gather these are the only 3rd gen SATA drives that arent from HP that will actually work in this system
03-12-2013 03:23 PM
Well I just want to update all in the future that find this thread and have a similar question. I've been running the three WD Red 2TB drives in RAID 5 (software) now for three weeks using FreeNAS. The system hasnt had any issues, no SMART errors or warnings and has just under 1 TB of data spread across the drives. I'm going to say the its a win for me.
03-13-2013 05:37 AM
Thanks much for that followup. I've attached the WD Red Drive PDF datasheet.
The 1 and 2 TB drives wil be great to have available in the xw6400, xw8400, xw6600 and xw8600 workstations. Moving up to the 3 TB drives in those can be quite the hassle. I did finally figure out how to get full 3TB drives to be recognized as a single partition in these workstations, and posted here on that.
One idea for owners of xw6600 and xw8600 workstations: The primary (video card) and secondary (lower) PCIe x16 slots in those are both PCIe generation 2 slots, unlike those in the xw6400 and xw8400 (which are both generation 1 slots). There are PCIe SATA cards that can be plugged into the generation 2 slots that provide full speed data access to SATA generation 3 hard drives.
Using one of those cards in your lower PCIe x16 slot would be a great way to get access to the full bandwidth of those drives. It also may be possible that the 3TB drives would work easily on those workstations through that pathway.
I'm using this basic concept to get full bandwidth use of a PCIe gen 2 USB 3.0 card in one of my xw6600's, and the speed benefit is amazing.
See attached PDF for the Red Drive datasheet.
03-27-2013 04:20 AM - edited 03-27-2013 04:36 AM
Update on the Western Digital Red drives. I went ahead and got one of the 1TB WD Red drives to have in one of my xw workstatons (this one a xw6600) with a boot/application Intel SSD (older one) and the Red as the storage drive for files, etc. W7Pro 64-bit install.
I had read that WD has a hidden partition of about 230 MB in there for software that has caused some in the Mac world problems. The Reds are SATA Generation 3 drives, and the xw workstatons are
SATA Generation 2 workstations, and HP has cautioned that only their SATA Generation 3 drives can be reliably used (they apply some form of firmware fix that HP developed which lets the G3 drives be cross compatible with SATA G2 workstations).
The WD Red was recognized as a raw drive out of the box by the Windows 7 Disk Management utility, and as a drive under Device Manager. I created a single MBR partition on it, did a long version NTFS default format, and the drive appeared normal under Disk Management. However, on cold reboot it would not mount on the desktop (it never had initially after the format, either). Still present in Device Manager and Disk Management. Just not showing up on the desktop.
Having read about the Mac problems with this drive related to the reported hidden WD partition I got out my bootable DBAN CD. This does a very low level reformat and clears areas of hard drives that I have not been able to clear otherwise. The .iso to burn this CD from is easy to find as a free download under google search for DBAN or Dukes Boot and Nuke, and I've used that for years. Booted into that CD, choose the interactive mode, selected that 1TB hard drive (be careful at this step, of course, because it will see all attached drives and you're one mistake away from nuking the wrong one), and let it run for the multiple hours it takes. Then, rebooted into W7 off the SSD, and the WD Red drive showed up in Disk Management, again raw. Did the long version default NTFS format (MBR again chosen, not GPT type drive selected at the very outset).
On reboot after that was finished the drive finally shows up properly on the desktop, and is working fine. This seems to support the Apple guys reports of problems related to the hidden WD partition, and that DBAN can wipe that.
This is early in the experiment, but things look good using this technique for preparing a WD Red to use as a single file storage drive above a SSD in xw workstations. I'll be backing up frequently.
If anyone else tries this process it woudl be valuable to post your results here for help in confirming the problem, and its solution, on the xw workstations.
06-13-2013 03:30 PM
FWIW I have an HP XW 8600 latest BIOS drivers etc, running Windows Server 2008 R2 with this Hitachi Drive HDS723020BLA642 which is a 2 TB SATA 6.0Gb/s running fine, about to replace this 2 TB with a 4 TB WD4001FAEX will post if there are issues. Using GPT BTW
06-13-2013 09:41 PM - edited 06-14-2013 06:08 AM
Good luck, and keep us posted.
The problem here is the complexity of getting hard drives over 2.1 TB to run normally with the specific hardware (chipset)/software (Intel drivers for the ESB2 chipset) for the particular SATA controller that is built into the xw6400, 8400, 6600, 8600 workstations. There is that long post here where I did pull that off, but concluded it was not really worth it for my needs. I've personally settled in on the 2 TB max hard drives for these workstations. Here's that how-to-do-it link:
Having said that, there may be breakthroughs yet to come, and it would be very worthwhile to hear how you do one way or another. If you can borrow that 4TB drive from a friend to experiment to prove it will work for you that would be ideal.
There is another post in the forum here about how I needed to use the free bootable DBAN CD (Dukes Boot and Nuke) for low level reformatting to initially get a 1TB Western Digital Red SATA generation 3 drive to show up on the desktop in Windows 7. That trick is worth knowing about, and that drive has been working great since. Now I wish I had bought the 2TB version......
06-20-2013 03:26 PM
The WD4001FAEX installed without issue and is fully available
I also installed a Seagate STCA4000100 USB External drive to back that one up
No issues so far, about 500 GB copied back from the 2 TB drive as of now
If there are any developments I will post back
06-21-2013 11:21 AM
Just as been posted elsewhere....
After 2.1 GB of data copied to drive everything was corrupted
What a waste of time
I guess it is back to the 2 TB drive or add another controller
06-22-2013 05:08 PM - edited 12-07-2013 07:46 PM
Generally I'm sticking with SATA generation II SSD or spindle drives at or less than 2TB size for my needs, but for cards I note that HP has liked SIIG cards in the past for adding more SATA II ports, and eSATA ports. HP also has a PCIe SATA generation III card from StarTech on their site for sale, and that adds two internal and two external SATA III ports:
So, I can't give you any PCIe SATA generation III card suggestions from personal experience. I can tell you that the Western Digital Red 2TB SATA generation III drives fully low level reformatted by DBAN and then NTFS long type reformatted have within Windows have held up very nicely on these SATA generation II workstations.
I do have experience with another HP PCIe card you may be interested in. I now have an xw6600 running the latest version of the HP USB 3.0 PCIe card, the "2x2" card, with a total of 4 outputs, running on a Texas Instrument controller chip. It is in the bottom PCIe x16 generation 2 slot (the lower of the two video card PCIe x16 slots..... both video slots are generation 2 in the xw6600 and the xw8600's, for the faster interface speeds than a PCIe generation 1 slot would give you). It is the TI chip version, not the earlier NEC/Renasus chip version, and it has two external and two internal USB 3.0 ports. HP is using the same TI chip soldered on the motherboards of the latest Z series workstations. Hence, easy to find the latest drivers. That has been working great, and I'll post on all that sometime.
06-25-2013 05:58 PM - edited 06-25-2013 05:59 PM
Any particular reason that you opted for that 1x PCIe card?
The concerns that I have of that card is that there's at best, 5.0Gbps theoretical bandwidth with a PCIe 2.0 x1 slot.
I just recently purchased a Dell PERC H310 off of ebay. I'm hoping that I'll be able to crossflash this as a LSI card. It is a PCIe 2.0 x8 card, with 8 ports of SAS on it.
07-01-2013 01:36 PM - edited 07-01-2013 01:38 PM
Remember that the USB 3.0 spec has that PCIe generation 2 speed of 5 Gb/sec, tops. So, using the HP PCIe x1 USB 3.0 card in one of the two PCIe generation 2 slots in the xw6600 or xw8600 gets me maximum current USB 3.0 speeds. Those slots are the upper and the lower PCIe x16 "video" slots, and the double speed of those slots is usable even through PCIe x1 cards. It can get confusing......
If I used the same HP USB 3.0 card in the lower PCIe x16 slot in an xw6400 or xw8400 (those are PCIe generation 1 slots) then I'd max out at 1/2 of the USB 3.0 limit..... I'd get a theoritical maximum 2.5 Gb/sec bandwidth instead of 5.0 that is possible with the generation 2 PCIe slots.
The point is that if you want to put a PCIe USB 3.0 or SATA generation III interface card into the fastest possible available PCIe slot in an xw6600 or an xw8600 you should choose the lower of the two video card slots. The upper one is for your video card. The other PCI slots in the those workstations (and all in the xw6400/xw8400 workstations) are PCIe generation one slots (2.5 Gb/sec).
07-08-2013 10:02 AM
Also had to remove the Seagate 4 TB drive from this machine as BE 2010 R3 would fail with V-79-10000-11226 - VSS Snapshot error
Oddly enough installing the Seagate on the machine running the backup did not cause an error
12-07-2013 12:03 PM
Anyone know if the LSI adapter has the 2.1 GB limitation?
According to this document "SATA device configuration guidelines"
you can use the LSI SAS controller for SATA drives
12-09-2013 09:02 PM - edited 12-09-2013 09:17 PM
Well I can see this thread has been resurected a few times from the dead with other people trying to get large drives ( > 2TB) to work. I have some more updates on it.
So since my initial post, I have upgraded from three 2TB WD RED drives to 4 2TB WD RED Drives to 6 WD RED 2 TB drives. At present my setup is as follows.
Processor: Dual E5440
Ram: 24 GB DDR2 5300F ECC
1 x 160 GB OS Drive
6 x 2TB WD RED Drives
Total Data Storage: 12TB
Usable: 9.1 TB
OS: FreeNAS 9.1.1
All in all I have 5.2 TB of space used across this RAID array. I've tried using Windows 7, Server 2008, Ubuntu and CentOS to do the same thing, but the system doesnt recognize anything over 2TB of space. For some reason, unknown to me, FreeNAS and only FreeNAS is able to see all space available on the drives even though its well over the 2TB limit.
I'm about to add four more drives (dont ask how, I'm not sure yet where they will sit within the chassis) to the system, all 2TB WD RED drives. I have tried multiple times to connect these SATA drives to the SAS plugins but every time I do the system completly craps out and results in a kernel panic. You CAN use the SATA drives in the SAS plugs so long as you dont use any of the SATA plugs, and vice versa. But you cannot mix them.
So I have managed to get my hands on three IBM M1115 PCI-E RAID cards (Not to be confused with the M1015 cards - different but almost identical and can be flashed into IT mode just like the M1015). One of them will be put into the system and flashed to IT mode to allow support for up to 8 additional drives. Heres to hoping this works. This old workhorse of a work station seems to suprise me at every turn. I've been able to do things with it I've been told arent possible.
01-23-2014 06:24 PM
I have found the HP Smart Array p400 controller to work absolutely fantastic in my xw8600. It recognizes it as HP hardware and happily goes on about its business (it complains if you try to put it in a white box machine). Even the configuration & management software from HP works great (ACU).
This is a server grade controller (stock in proliant servers) and provides RAID 0/1/5/10 along with battery backed write cache and extensive configuration via ACU CLI. It is on the VMWare hardware compatibility list as a supported device. Best of all, it can be found on ebay for around $20 with the battery & 512mb cache.
You'd need 2 SFF-8484 32 Pin to 4 SATA 7 Pin fan-out cables as this is an 8-port card but those may also be found on ebay for around $8 each.
These saved me when I ran into that pesky 2 active array limit in the onboard LSI controller. That is a ridiculous limit (especially with 8 available ports) but then again, the 1068e is the bottom of the barrel LSI chip.
I have another p400 on the way to add more storage (I think they can even be chained to produce one massive volume if needed but I don't plan on using them that way).
This is probably the best option for hardware raid at a really good price for these workstations. And it's native HP hardware (someone mentioned above using a Dell perc - I wouldn't bother with that, especially when this works so well).
Anyway, just my 2 cents...
01-23-2014 06:52 PM
(if my post was helpful, please "mark as a solution" or give "kudos" - thanks)
01-23-2014 06:59 PM
I tend to specialize in 'budget' stuff...