11-13-2012 02:07 PM - edited 03-20-2013 07:59 AM
It was not easy or intuitive, but it is working.... see attached screen captures.
I bought a 3TB Hitachi 7200k RPM HP drive, new, off eBay for a little over $100.00, the MB3000EBUCH, aka 628059-B21. HP QuickSpec on this is attached (see page 3 of it for that B21 number). There also is a pic of its label from the eBay purchase, next post below. It is a SATA generation 2 drive with specs from HP stating "3G". Note that the "generation 3" faster SATA drives new in that HP document are termed "6G" by HP. So, my new hard drive should be compatible related to the generation 2 versus generation 3 issue with my xw6400 /xw6600 /xw8400 /xw8600 workstations (which work best with with generation 2 drives per HP). These specific workstations all run on the ESB2 SATA RAID/AHCI drivers.
I plugged it as a secondary data drive into my testbed xw6600 running W7Pro64-bit with BIOS settings for SATA emulation to RAID + AHCI, running the Intel ESB2 SATA RAID controller driver from 3/30/10 version 184.108.40.2064 based on a HP document recommending at least that level, related to their white paper on Advanced Level Drives .... the drive was not recognized at all. This is a HP generation II SATA server hard drive with who-knows-what OS on it... so I then did a very low level format using DBAN bootable linux CD. DBAN could see the model number and size correctly. Ran the DBAN default settings. DBAN always takes quite some time, and this was a 36 hour investment, but it converts a drive to its fully raw status.
I have used DBAN for years now, and it has saved many disks and resolved many strange behaviors by drives simply by getting them fully back to a raw status. I'm using DBAN 2.2.7 Beta ("beta" for years now, and I run it in "interactive" mode), and you'll need to burn the .iso onto a CD:
After that 36 hour low level reformat was completed I shutdown, rebooted, and the W7 Disk Management console could now see the drive, but reported a size of only 746 GB. At that early stage I made what was visible into a GPT partition table type drive (not the default MBR type, which allows only too-small sectors) from the W7 Disk Managmement console. I then tried the Intel 10.x RAID/AHCI driver for these ESB2 based workstations that I have posted about in the past..... the drive vanished. You can go back and forth inside Device Manager updating to the different drivers once they are loaded initially (as I have described before). So, I went back to the 220.127.116.114 driver version initially used, and it came back. Back and forth a few times, same deal, so the 18.104.22.1684 driver seems to be a key component right now.
Tried many tricks, but could not get the missing >2TB to show up. So, I then formatted the 746 GB GPT partition in NTFS, and it seemed key to not leave the sector size at "Default" during that process. Rather, I shifted that up to 4096 from the dropdown list provided. (EDIT..... see next 2 posts..... this actually does not seem necessary in retrospect. Default would have been fine, I now believe).
The breakthrough came when I tried to see if the partition would still show up if I set SATA emulation to IDE Separate in BIOS. It did show up, still formatted, and the missing >2TB did also, unformatted. So, still in IDE Separate SATA emulation, I used the "quick" option and formatted the newly visible bigger partition the same way (NTFS with sector set to 4096). That worked, so I now had two NTFS 4096 sector size formatted partitions on the 3TB GPT drive. Next, I deleted both of those partitions, so now I had only one partition, unformatted, but full size. Formatted that, quick method again, and now had one partition, full size. Shifted BIOS back to RAID + AHCI, and the full sized 3TB single formatted partition persisted after reboot. These changes in BIOS require some F1 acceptances on early boot, both ways, and that took a bit of time, but all now is perfect.
Now I have a fully functioning 3TB single partition drive (plus the 160GB boot generation 2 Intel SSD) you can see from the screen captures... very fast. The offsets tested out to be properly aligned, for both... see posts below for the tools. The 3TB cannot be a boot drive due to the BIOS limitations, but I'd not want that anyway.
Again, this type of experimenting is not officially supported by HP, but some of us may benefit from this experimentation and even put together a better how-to-do-it knowing that it in fact can be done.
Solved! Go to Solution.
11-13-2012 02:23 PM - edited 11-27-2012 08:46 AM
Some added attachments and info:
If you have kept your W764 install up to date you won't have to load KB982018.... it will already be installed from the past. Reportedly the fsutil tool from MS built into the W7 install gets updated from this KB (and other improvements).
I don't know why my 3TB hard drive does not show up as an Advanced Format Hard drive using the HP tool noted..... If I should have selected a smaller sector size than 4096 during the NTFS format process I'd like to hear that. (EDIT..... it turns out one cannot turn a conventional hard drive into a AF hard drive. There are small hardware and firmware differences that preclude that, but some of the AF hard drive software tools are handy to have for probing conventional hard drives as you'll see later).
Intel source for the 22.214.171.1244 driver referred to above (and some of the HP service packs for RST use this version also):
I've attached the zip version of that driver's install folder..... it is the 64-bit version, and you'll want to read up on how to update your driver from within Device Manager from my posts earlier. You want to navigate to the X64 driver folder and select isStor.inf as I have detailed in the past how-to-do-its on this. Once you load it properly it will stay available to select from via Device Manager in the future if you experiment with other versions, unless you specifically delete it.
Remember there is a MS FixIt to get an older version of this driver which allows BIOS to be set to "RAID + AHCI" after the fact in W764, if you did not have that BIOS setting properly set during a clean install of W764 with these older workstations, explained in my prior posts here.
11-15-2012 07:42 AM
Thank you for the informative post Scott...
I haven't ordered my 3TB drive as of yet as I was unsure as to whether it would work, I'll read your post agian to see if I can do this..
in regards to the low level format, does this really need to be done, or is this just a step you always take with your drives?
11-15-2012 10:39 PM - edited 05-10-2013 06:42 AM
I have some updates..... the most important is that it appears that one cannot create an "Advanced Format" hard drive out of a non-AF hard drive. That is not what this project is doing. From my reading there are specific small hardware and firmware differences that prevent such a conversion, and you don't need an AF drive to be able to see/use a full 3TB SATA hard drive internally in these workstations, with a single partition. Here are some new points about this project:
One common reason that many internal 3TB hard drives don't show up correctly is that many come out of the box with a MBR type of partition table, formatted as 2.1 TB drive, so that they show up when the consumer simply plugs them in. In this situation one needs to delete that partition, and convert the drive to a GPT type of partition table, and then NTFS-reformat it to its full size. However, the whole process is not as easy as it sounds.
The low level reformat I mentioned in the post above is a long haul..... but it does let me get down to basics. You can use a free bootable Linux .iso, GParted Live, to boot into Linux and delete the partition table of the drive, and create a new GPT partition table much more quickly. Use of that program is beyond the scope of this thread, and it is a bit tricky..... be very careful to subselect your 3TB drive rather than your boot drive from the top right corner of the GUI that boots from the CD. I experimented with that this evening, and it took about 15 minutes from boot to finish to fully delete everything, and create a new unformatted GPT partition on the 3TB drive. NTFS format, "default" setting, while in IDE Separate SATA emulation and using the Quick format option takes only about a minute. Then reboot and change BIOS to RAID + AHCI SATA emulation setting.... another minute or two.
Over the last few days, as I experimented I found that I could take the 3TB drive, after I got it into a single 3TB GPT format mode, and insert it into any of my other xw 6400, 6600, 8400, and 8600 workstations and see/work with it as long as I had a correct SATA/RAID driver loaded. These are the 64-bit driver versions, on W764, and as an example I discovered that 3 of the 4 I have on my xw8400 testbed system worked properly (see attached list).
As I dug deeper, it seems another key to success is to go from the 750GB state to the full 3TB being available was to shift from RAID + AHCI to IDE Separate SATA emulation mode in BIOS, and do the creation of a single 3TB partition then, and only thereafter shift back to "RAID + AHCI" BIOS mode. That single partition will be created, and will persist, using this trick.
The AF drive research was worthwhile given that it uncovered some tools to check alignment of the sectors, and type of drive, and those tools confirmed that my functioning 3TB drive is truly not an Advanced Format drive. Rather, it is a "512-byte native" drive. Take a brief look at this, from MS:
Note that the MS fsutil (included in W7) gives the attached output of 512 both for bytes per sector and bytes per physical sector for this drive in these workstations (after my work on it). The fsutil output image was captured after GParted deletion of the partition down to RAW status, and thereafter with the drive finally formatted in NTFS with sector size left at default. My setting of the sector size to 4096 during the NTFS format mentioned in the earlier post produced the exact same results (512). So, that "4096" turns out to not be a useful setting. Stick with default. The most recent version of fsutil from MS is what you want.... it is already on your hard drive, from W7 updates, assuming you keep up to date.
Added reading, and tools, are available from: http://en.wikipedia.org/wiki/Advanced_Format Again, the AF stuff is interesting, but that is not really what we are doing here.... we are actually figuring out how to use the GPT partition technique and some IDE versus AHCI capabilities to accomplish our goal.
Dell has another tool that also probes for Advanced Format, and checks for alignment..... Dellafdt.exe. I have attached a pic of the output of that, also. I tracked that one down via google.... and the pic shows how to use an elevated command prompt pathway to navigate to the .exe, plus the output. I have attached the little Dell installer .exe because it is hard to find at Dell.... it works fine on non-Dell workstations. That .exe will place a Dell folder containing the program on root C, and again you need to run Command Prompt as administrator to navigate to it, as shown. The pic "latest one.png" shows you how, plus the results.
Finally, attached is a pic of the set of drivers I have loaded into this xw8400 for the SATA RAID emulation part of things.... I can shift back and forth between any of these, via Device Manager. The only one that will not work is 10.1.0.1008, and I now am using the latest one that will allow the 3TB drive to be seen (10.0.0.1046). Here is an Intel link that lets you get the whole download that includes the driver install folder. You unzip that download, navigate to the Drivers folder, copy out the X64 folder that is in there, and put that on your desktop. You can turn on the option in the Folders view tab in Control Panel to not hide extensions of known file type, and thus easily navigate to the isStor.inf file inside that X64 folder during a Disk Manager update of the SATA + AHCI driver. Once you update that and restart you'll be running on the 10.0.0.1046 driver, and the older driver(s) for that device still are on your hard drive, but not active. You can navigate back to using them if you wish via Device Manager. Here's that Intel link:
So, it can be done, but is it worth it? Seems like a lot of work to me. Sticking with an off-the-shelf SATA generation 2 (3Gb/sec) internal 2TB hard drive with MBR type of partition table would be a lot easier as a dedicated storage drive inside of one of these workstations.
EDIT: Fixed link to Intel download source for 10.0.0.1046..... thank you Peter.
11-18-2012 08:30 AM - edited 05-10-2013 06:44 AM
I edited that last post to include a few added details, and here is the Intel link to download the latest driver that includes ESB2 southbridge functionality that also allows the 3TB drive to be seen via My Computer in W764:
This 10.0.0.1046 driver version is what I'm now using on all of my ESB2 workstations (xw6400, xw6600, xw8400, and xw8600. EDIT: I checked on a Z600 I have access to for its particular SATA/RAID controller driver (under Storage Controllers in Windows 7 Device Manager) and it is not an ESB2 version. Instead, it is "Intel ICH8R/ICH9R/ICH10R/DO.....". This represents presence of a newer southbridge version in these Z series workstations. The ESB2 is in the Intel ICH6 family (I/O Controller Hub), instead. You can read all about that via a google search of ESB2 Wiki.
That ESB I/O controller hub is actually a "device" that is seen by the W7 installer, and if you have the BIOS settings for SATA emulation set to IDE Separate or IDE Combined instead of "RAID + AHCI" during your W7 initial install the installer will not include the critical driver components that let you easily switch back and forth between all of the three available BIOS settings (Separate or Combined, which are "legacy", or RAID + AHCI, which is "native").
So, if the BIOS is set to RAID + AHCI during W7 install you'll automatically get all of the necessary drivers included, but if you had BIOS set to one of those two legacy settings during W7 install the ESB2 device is seen differently by the installer, and only the legacy driver will get installed. In that case you can only switch back and forth between Separate or Combined.
You really do want to be able to run with BIOS set to "RAID + AHCI" to get all of the modern benefits of AHCI (and RAID if you ever plan to use that). Intel recommends to simply install the driver that includes RAID even if you don't plan to ever use RAID (for greatest future flexibility), and that is what the W7 installer will do automatically. I do that also, despite never using RAID.
Assuming that has been properly installed from the beginning it is easy to update the driver from Device Manager, as has been posted about here in the past. You want to select the iaStor.inf (.inf means initialization file), and that little file will direct the installer to pick correct parts from within the X64 folder to update your driver with. Your original driver for that device is not deleted by this process.... you always can go back to that from within Device Manager. Before any updating of the driver you can document what you are currently using from within Device Manager/Properties for that driver.
The W7 installer DVD has the "starter version" of this driver in it, and if you have your BIOS set properly before a clean install of W7 you'll get that automatically. Not so with XP installs..... you need to use the F6Floppy method to add that driver in if you're doing a clean install of XP. The HP XP clone installs include that driver, from the factory, and your Restore disk method from HP would include that also.
EDIT: Corrected Intel link to properly get to the 10.0.0.1046 drivers, thanks to Peter.
05-09-2013 03:48 PM
Scott, first thanks to you for this and your other XW series posts regarding updates to the Intel drivers on the ESB2 controller. They've been extremely helpful.
Were you ever able to update the Intel RST drivers beyond 10.1.0.1008?
When I try to install the latest version that the Intel Driver Update Utility recommends (126.96.36.1990), the install errors out with a "This system does not meet the minimum requirements." I never did try to point Device Manager to the 11.x driver out of caution.
The newest driver I've gotten to work is 10.0.0.1046 in Win7x64. This one at least allows the full 4TB drive to be seen by the system. The problem I had under the 8.x version was drive corruption. I'm doing some long tests now to verify that this version of the driver is going to preserve the data.
Like you, I found that the 10.1.0.1008 driver just wouldn't show the 4TB drive at all, even in Device Manager.
BTW I notice that the Intel download center links you provided all go to the 188.8.131.520 download, instead of the one for 10.0.0.1046.
The link I hunted down is:
05-10-2013 01:43 PM - edited 05-10-2013 01:52 PM
Those are kind words from you. You have the latest well-functioning ones that I have found, also (the 10.0.0.1046 set). That .1046 installer package you link to (and I fixed my links above - thanks for that catch) is dated very recently, but when one goes inside the contents the older release date is listed there.
I figured out how to tell if an ESB2 chipset driver is in an installer package.... you unzip it, go to Drivers folder, go to the X64 folder (because the assumption is that we're using 64-bit Windows 7), and find the iaStor.inf (.inf = initialization file). You may not see the extension ".inf" part if Control Panel/Folders/ View options does not have the show extensions of known file type activated. Inside that file (which you can view with Notepad) go down to the bottom area and you'll see the chipsets that the drivers in the package will accomodate (in your .1046 link's download there are 5 separate ones, including ESB2). If you do the same probe with the 11.x installer you refer to the ESB2 chipset is not listed in there and thus that driver package is of no use for this family of workstations. The xw6400, 8400, 6600 and 8600 all use the same ESB2 drivers.
I keep dredging for a newer well-functioning one, but no luck.
By the way, not sure if you saw my post on the Western Digital Red hard drives....... very nice drives but out of the box they would not show up on my xw workstations. Found I needed to do my usual very low level reformat via free DBAN, and then my usual Windows 7 NTFS long type format. Thereafter they show up and function fine. These are SATA generation III drives. I finally settled in on using 2TB Reds as storage drive (plus an Intel SATA generation II boot SSD) rather than hassling with the 3TB issue discussed in the posts above..
Please do keep us posted on your testing..... very interesting information.
05-13-2013 12:51 PM
Sure Scott, credit where it's due. You've doen a great job documenting your vey helpful work.
Sadly, my testing of the 10.0.0.1046 driver ended in failure. The drive corrupted after 2.7TB of data were copied onto it via TeraCopy. Same scenario as before driver updates.
Win7x64 now simply shows the drive as RAW. The drive is fully recoverable- on a Z800 that's also well outside the bounds of HP's officially supported BIOS and driver packs.
To be clear, there are some differences here in our configuration:
1. My host machine is an xw8600.
2. I'm using an eSata bracket going to an external G-Tech 4TB hard drive.
3. xw8600 BIOS is at 1.46 (no change in behavior from previous 1.36 version).
4. I'm using the Intel installers, not pointing Device Manager driver updater to the x64 ini files.
Deadline and limited time on the production machine requires a solution asap, so I'll be installing/testing an eSATA card from Startech tomorrow:
05-13-2013 09:54 PM - edited 05-13-2013 10:01 PM
While researching the use of HP USB 3.0 cards in the xw workstations I noted that the two PCIe x16 video slots on the xw6600 and the xw8600 are PCIe Generation II slots.... the rest of the PCIe slots are Generation I slots. In contrast, all the PCIe slots on the xw6400 and the xw8400 are Generation I type slots.
It may seem excessive but if you put your anticipated SATA PCIe card into your second PCIe x16 Generation II video slot you'll get the fastest possible SATA transfer rates through that card into the PCI bus.... twice the bandwith. They work fine in those video slots...... these type of cards are generally designed for the Generation II type of PCIe slots.
This all can get confusing, with PCIe Generation I, II (and even more recent type III) slots, and SATA Generation I, II and III drives and cables.
So, if that second video slot is available to you on your xw8600 then use it first for that card.
02-06-2014 11:42 AM
Can anyone please advise me on installing mirror onto my XW6600. I wish to put a 2 x 3TB raid mirror on my system. Can anyone advise on the most reliable HDD's for the job? I have been told WD are pretty good, but from my searches all I can see are 6Gb/s HDD's and my understanding is that they are not compatable.
I'm open to any suggestions on what to do. I just want a fail safe backup system for my other laptops and pc's too.
I'm not as upto speed as you guys on the hardware side of things, Sorry.
02-07-2014 01:37 AM - edited 02-07-2014 03:05 PM
I'll assume you plan to run the RAID internally in your xw6600, and I'd advise to not use 3TB drives in the xw HP workstations. I'd get two 2TB Western Digital Reds, and first sequentially do a low level reformat of each using DBAN. The 3TB machinations detailed in this thread above are not worth it, and potentially risky for the long haul. 2TB drives do not get into those issues. The Reds, I have found, have some things deep in their boot blocks that inhibit them from working properly with these xw workstations unless they have had the DBAN low level reformat treatment first. DBAN is easy to find, open source, no cost, and I've used it for years with zero problems and reliable benefit.
Here's the safest way to go if you're inexperienced: You'll need to burn the bootable DBAN CD off the downloadable .iso, and I'd place the Red in your xw6600 hooked up to the SATA 0 (blue) motherboard port for the low level format. Don't have any other drive attached, and you'll be sure to not mess up another drive unintentionally. Each low level reformat takes a long time on a 2TB drive, and I have a utility xw6400 set aside for such projects. I just start it and come back in a day.
If you're more experienced just have the Red as a second drive hooked up to the motherboard, and you can navigate to it after booting into DBAN for the low level reformat. Do both Reds as your first step.
Then, do a standard NTFS long-type format of both using the OS's Disk Management utility..... this is while booted off your normal boot drive.
Now both are ready for you to RAID.... and that is something I personally don't have experience with. Instead, I image my SSD installs using Clonezilla Live, and can burn another clone install using Clonezilla in about 15 minutes from my backup image onto a fresh SSD. I use the same size target SSD as the source SSD that the image was made from. The images are backed up. The Intel SSDs I'm using hold up very well for a long time, and I have not had one fail, ever. This has worked out quite nicely where our needs are not for large amounts of storage, but rather for fastest function from the boot SSD, and a second spindle drive (with regular backup) if someone needs more bulk storage than the SSD provides.
For what it is worth, I have the habit of doing this DBAN low level format/NTFS long-type format treatment on every drive I put into service now, regardless of whether it is new, used, spindle drive, or SSD.
02-09-2015 07:12 AM
Thank you for the interesting information regarding the above.
I was looking to RAID 2 x 2TB drives (with an SSD OS boot drive seperate) on an XW4600, but the supposed limit for these is 2TB and apparently there are also issues with Sata III drives on thsese units?
In those circumestances is this a total no go or will the above help me somewhat?
Thanks in advance.
02-09-2015 09:07 AM - edited 02-09-2015 09:12 AM
I don't have any experience to offer on RAID..... I use a single SSD for my boot/programs drive and a larger "documents" drive with the standard methods of targeting that drive for autostore of documents of the various subtypes, under W7.
Regarding the WD Reds..... you are correct in that these are SATA generation III drives, rather than SATA generation II drives. HP advises on their xw workstations (SATA generation II chipsets) to not use gen III drives unless they come from HP (the indications are that there is a special firmware fix HP applies to their SATA III drives to ensure SATA II compatibility). However, it is not a sure thing that all non-HP SATA gen III drives will have problems with the HP SATA gen II xw workstations. There have been references in this forum by people using the non-HP WD Reds, for example.
I have been using a non-HP WD Red 1TB documents drive for several years on a xw6600 running W7, zero problems. I just got a 2TB Red new, non-HP, for a new build, but it has some bad sectors (RMA in process... I think drive manufacturers now have us test their new drives for them in the field). This was after I did the full DBAN low level erase on this 2TB drive, and I'll warn you...... that took about 3 full days nonstop. I'm going to try just a normal NTFS long type format with the replacement Red when it comes to see if that is enough.
You can also find HP 2TB SATA gen II hard drives on eBay if you wish.
02-09-2015 12:12 PM
I'll probably take the risk on the WD Reds, the thing that concerns me the most now is HP saying these workstations can't take more than 2TB storage in total, that seems very low.