Reclaiming Space from a thin provisioned netword raid-5 volume (5739 Views)
Reply
Occasional Visitor
Anthony B Hall
Posts: 1
Registered: ‎05-21-2011
Message 1 of 43 (5,739 Views)
Accepted Solution

Reclaiming Space from a thin provisioned netword raid-5 volume

We have Windows 2003 Enterprise connected to a large 30 TB volume and are trying to shrink the volume by moving files from it into a smaller volume i.e. eventually deleting the files from the large volume. I had expected the space to be returned once the snapshot had processed. What do I have to do to recover this space ? Would the result have been different if I had thick provisioned the storage ?

thanks in advance,

Anthony.
Respected Contributor
teledata
Posts: 187
Registered: ‎06-07-2009
Message 2 of 43 (5,739 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

This is always one of the challenges (and limitations) to thin provisioning.

The technology used to provide thin provisioning in SAN/iQ is really more of a "high water mark".

Once a block has been marked as "written" you cannot recover unused space by deleting data from the volume. There is no communication facility in which the OS would "tell" the SAN that,
"Hey that block of data we were using yesterday, is now empty and you can have it back."

Since there is no way to "tell" the SAN the space is empty, those blocks of data, once written to cannot be reclaimed.

Your only option is to create a NEW volume, and migrate the data to the new volume, and then delete the old volume.

This can be challenging with direct native iSCSI mounted volumes, but if you are using a virtual machine (with virtual disks) you can reclaim storage by creating a new VMFS datastore, using sdelete to zero out unused space (within the Windows OS), then performing a storage migration and choosing "thin" provisioning on the virtual disk.

While still requiring a new (VMFS) volume, the virtualized disk can be left intact avoiding any reconfiguration within the Windows server itself.


The result would NOT be different if you had chosen thick vs thin. The blocks are still marked as used and a "high water mark" is still maintained. The only difference is when you mark it "thick" SAN/iQ reserves the entire space, and it cannot be used to provision to other volumes/snapshots.

This is why you can dynamically switch between thin and thick provisioning within the CMC.

http://www.tdonline.com
Frequent Advisor
Matthew D
Posts: 32
Registered: ‎05-06-2010
Message 3 of 43 (5,665 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

teledata - I'm curious, is this still the case with VAAI and 9.0?  I was under the impression that vSphere + VAAI + 9.0 would allow block zeroing and reclaim free space when VM's were destroyed.  I really have no clue - couldn't find any definitive information out there about it.

HP Pro
Jitun
Posts: 199
Registered: ‎03-28-2008
Message 4 of 43 (3,798 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Matthew for now space reclamation is not supported by LeftHand OS.
But we can expect that in the future releases.
--------------------------------------------------------------
How to assign points? Click the KUDOS! star!
Honored Contributor
oikjn
Posts: 770
Registered: ‎07-20-2011
Message 5 of 43 (3,781 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

matthew, you are correct, VAAI supports space reclaimation and there are new SCSI commands that support this feature, but as Jitun said, StoreVirtual doesn't support this feature yet so we are S.O.L. until they decide to.

 

This feature was "comming soon" along with SMI-S support since 9.5.  Its 10.5 now and nobody from HP has said anything about those features ETA.

 

IMO, "We can expect that in the future release" is a pretty good statement in that these are now more considered basic features and if HP isn't going to include them in future releases this product line is dead and I"m sure we all hope that isn't the case.

Frequent Advisor
Patrick Neuner
Posts: 85
Registered: ‎09-05-2007
Message 6 of 43 (2,547 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Its 2014 - but still no change when talking about this behaviour. On Linux with Mysql Databases and Dumps, this is also a huge problem. A Volume with 170 GB but a LUN that has 500 GB assigned, it will take weeks, until 500 GB is used up. For Hosting Business and sales very bad - as thin provisioning on good storage is a must to be able to make good pricing. 

 

Also Remote Replication is a hazzle this way, we transfer about 300 GB of data for those 170 GB, where less than 10 GB change. With Rsync it would save lots of bandwidth, but it's not a snapshot then. 

 

We also heard from support - it will be in a future release. But that was with OS 10.0. Now we had 10.5, 11, not sure what future release they mean, I hope 11.5 and not 20. Otherwise our Lefthand will be out of warranty in 1 year - we will switch to another system. I like the Lefthand approach though, but price compared to features a no go. 

 

I still believe they don't add this feature to sell 3PAR, or - because they know, once the feature is active, lot's of people don't need to extend existing lefthands as suddenly lots of free space is available. 

 

Anyway - this feature should not be promoted as it's not a real thin provisioning, we found out the hard way - after purchase. 

 

Patrick

Honored Contributor
Bart_Heungens
Posts: 687
Registered: ‎10-26-2005
Message 7 of 43 (2,545 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Hi,

 

Cannot tell you anything in detail but 2014 will be a good year! ;-)

 

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Frequent Advisor
Gediminas Vilutis
Posts: 53
Registered: ‎02-23-2011
Message 8 of 43 (2,518 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume


Bart_Heungens wrote:

Hi,

 

Cannot tell you anything in detail but 2014 will be a good year! ;-)

 

 

Kr,

Bart


And after missing Nth deadline for LH OS v11 online upgrades, do you expect us to believe into these nice HP roadmaps? :)

 

I've seen T10 unmap promises probably for 2 years already. As always, it is somewhere 'not in next release, but after that...'. Unfortunately, when next LH OS release is out, roadmaps also get shifted :(

 

 

Honored Contributor
oikjn
Posts: 770
Registered: ‎07-20-2011
Message 9 of 43 (2,507 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

lol. Ged you are so right.
Honored Contributor
Bart_Heungens
Posts: 687
Registered: ‎10-26-2005
Message 10 of 43 (2,496 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Same question the other way around:

Why do you want online update to 11.0?

Biggest change in 11.0 is Adaptive Optimization.

Your current physical appliances are not capable to use this feature, only the new 4335 appliance, besides the new VSA 2014.

So why is everyone asking for online update of appliances that cannot use the new features?

 

And yes the code is ready, there is only an issue found in 1 particular update case with 1 particular G2 model with 10Gb interface. And HP prefers to release the code which is 100% perfect in all cases... I follow them in their opinion...

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Honored Contributor
oikjn
Posts: 770
Registered: ‎07-20-2011
Message 11 of 43 (2,467 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Valid question...  v11 was a relatively minor update over v10.5 considering its classified as a new version.  I wanted to upgrade to support windows server 2012R2 hosts?  Or SMI-S support (not that I found it really worked with MS VMM)?

 

Here is the running joke of reasons courtesy of the CMC notifications page:

 

 

             In October 2013, all currently supported HP StoreVirtual platforms will receive an update to the best version of LeftHand OS yet. LeftHand OS 11.0 will offer a variety of improvements for the existing features as well as introduce new capabilities to the platform.

The upgrade to LeftHand OS 11.0 is available for the following platforms:

  • HP StoreVirtual VSA/HP LeftHand VSA

  • HP StoreVirtual 4000 Storage

  • HP P4000 G2 Storage

LeftHand OS 11.0 comes with a large number of new features and enhancements including:

  • Sub-volume auto-tiering with Adaptive Optimization (Exclusive feature for HP StoreVirtual VSA)

  • Support for Microsoft System Center Virtual Machine Manager

  • Smarter Centralized Management Console and Online Upgrades

  • Easier configuration and more granular control of application-managed snapshots with Application Aware Snapshot Manager for Microsoft and VMware environments

The benefit to you, if you are selected to participate in the Controlled Release program, is early access to the new production-ready software. You will receive direct access to HP Engineering Support (Level 3) for any issues or assistance you need during the Controlled Release Program. By participating, you are able to help HP meet your needs and determine what is most important to you as a customer for future releases. We want to work more closely with our customers to understand what their needs are so that we can drive our products to meet those needs in a more timely manner. We hope that you will choose to participate in this program.

If you are interested in joining our 11.0 Controlled Release Program, please click the link below and you will be taken to a survey site where you will be asked a few questions about your StoreVirtual environment. Submit those question and we will get back to you sometime after September 1 with a response.

HP StoreVirtual 11.0 CR Survey

Thank you for continuing to be a valued HP customer.

HP StoreVirtual Team           

 

Regular Advisor
a_o
Posts: 69
Registered: ‎02-22-2012
Message 12 of 43 (2,456 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

[ Edited ]

As someone who applied to be included in the Controlled Release Program in August of 2013, it was frustrating to see v.11 just show up one morning on the StoreVirtual download pages without support for existing NSMs and VSAs.

I was really looking forward to Windows 2012 R2 support and taking advantange of SCVMM integration - whatever that entailed.

What's really bothersome is that there is no published roadmap from HP regarding UNMAP/TRIM support.
My recollection was that my survey responses put that feature at the top of my list of features.

 

I dare say that I will take UNMAP/TRIM over Adaptive optimization. This is because I've personally implemented all SSD VSAs with very good performance. (Hint it's all in the hardware specs -i.e 10GBe).

 

With all due respect to Bart, it's been years since I saw the first hints about this feature coming 'soon' on this boards.

We chose HP Lefthand because they 'promised' Thin Provisioning. But we have now is not Thin Provisioning.

It's some facsimile of it, but it's not it. 

Case in point, I have a 8TB Cluster (used for SQL backups) that I have to 'purge' every now and then because a LUN that was Thin Provisioned for 2TB, uses up to 6-7TB within months even though there's never more than 2TB of data on the LUN at any given time.

Honored Contributor
Bart_Heungens
Posts: 687
Registered: ‎10-26-2005
Message 13 of 43 (2,451 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Hi,

 

I am running VSA's in my 'datacenter' from the week before HP acquired Lefthand... It must have been version 7.5 or 8.0...

Meanwhile I upgraded to every new version including somewhere around 9.5 or 10.0 where I was also in the Beta group...

Also I upgraded my underlying VMware environment to all versions from 3.x to 5.5 now...

Well I can tell you that my SAN volume on top of the VSA's have an uptime of 100%. No not 99,999 something but always online.

And that is why I love the Lefthand technology... Because it allows to do some nice things with it.

 

Well I have already some 2012 R2 servers connected to my Lefthand, don't need even 11.0 for that...

And yes I upgraded my VSA's already to 11.0, only not through the online update process that everyone prefers. But with a little workaround it works. You can also contact HP Support so they will provide you a FTP link so that you can do the online update. But I understand HP (not that I feel OK with all their decisions but that is outside this topic - I do not work for HP BTW) in this particular case that they do not release the update code specific for updates, since there is 1 known issue somewhere that they want to have removed.

 

Just my idea, it remains nice and easy technology, knowing that the unmap will be there soon... Too late but it will be there...

 

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Honored Contributor
oikjn
Posts: 770
Registered: ‎07-20-2011
Message 14 of 43 (2,442 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Bart, for the money, the VSA line definitely provides the best uptime at the lowest upfront cost.  I've been running since v8.5 and other than shutting down my server room for sandy, we haven't had any measurable downtime either.  That said, I've yet to see any official roadmap or timeline or even just an employee blog post or anything other than the page that gives a v11 release date that just keeps extending backwards and as far as availability goes, the VSA solution is losing its advantage as other options are increasing their availability capabilities.  Give it another year and there will be little reason to stick with this line unless they get back on par with the basic features other SANs are including these days.

Honored Contributor
Bart_Heungens
Posts: 687
Registered: ‎10-26-2005
Message 15 of 43 (2,432 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Well I have seen the roadmap but I am not allowed to talk about it since I have signed a NDA for that...
I know there are HP people on these forums but they are not allowed either to talk about this on this place either...
If you really need the roadmap, contact the HP product manager in your country.
BTW the 11.0 code is released... Since November downloadable and delivered on all new appliances. Like I said HP discovered an issue after an upgrade of a G2 appliance with a 10Gb card in it, which resulted in 'not-normal' network performance... HP wants to have that one cleared out before the release to anyone... The 11.0 code as such is there and is OK...
--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Honored Contributor
oikjn
Posts: 770
Registered: ‎07-20-2011
Message 16 of 43 (2,422 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Who cares about a confidential roadmap?  Why should one be confidential unless its just a goal list and not a real roadmap that will be lived to (or close to at least).  And I get that HP might not want every employee posting on this forum, but how can they not have a rep or someone with a connection to the inside here?  The only "person" they have is Prakash Singh_1 http://h30499.www3.hp.com/t5/user/viewprofilepage/user-id/550815 who I"m not sure is either an automated Bot they turn on every once and a while to post a link to the help page or a sweatshop employee with about the same level of knowledge of that bot.  

 

That information about the problem is more information that anybody else has stated, but if thats really the case and they have now delayed the relase of upgrades for everybody else because of one model w/ an optional card on it, it seems like they should at least find a way to update people who's systems don't fall into that narrow bug category.  Its only been about four months since the start of november....  how long are they going to take?

 

Internal Roadmap or not, secret watchers of the forums or not, there is virtually no communication from the product team to the general public at all and that is never a good thing.

 

 

 

 

 

Honored Contributor
Bart_Heungens
Posts: 687
Registered: ‎10-26-2005
Message 17 of 43 (2,419 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Everyone is allowed to have his opinion here, I have mine too... Get your point but it is not that simple as you present.

Since I am not representing HP here (just try to help people in my free time) I cannot tell you more... Check with your local HP representative.

 

I have heard about some cases where a customer wanted 11.0, not having these G2 appliciances with 10Gb cards, where HP delivered a special FTP link to use inside CMC to help that customer upgrade to 11.0.

Maybe you can try that way...

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Honored Contributor
oikjn
Posts: 770
Registered: ‎07-20-2011
Message 18 of 43 (2,412 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

I'm running all VSAs. I upgraded around new-years simply by setting up new v11 nodes and doing a bunch of node swaps. Once the last node (and FOM) was exchanged I got the v11 features. Everything there is working fine, but its bit coocoo for those who are paying for support and upgrades to see a new release but not have access to it for potentially the entire extent of your support contract without having to call up support and ask for special permission to get access to an update you should have access to. Anyway... I'll quit on the subject because I feel like I've beaten a dead horse, but IF anybody from HP actually follows this forum it would be nice if SOMEONE official was able to post here to answer some questions or give some inside advice/information. I'm sure most people would understand if there were times where they respond with "I can't say here, open a support request for more information".
Regular Advisor
a_o
Posts: 69
Registered: ‎02-22-2012
Message 19 of 43 (2,406 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

[ Edited ]

I also am running mostly VSAs @ V11. However, I have a bunch of NSMs that I would love to bring up to date.

 

WRT G2 NSMs with 10GBe cards and V11, IMO there must be more to this issue than what has been mentioned.

Which card excatly is this problem with: the NC550SFP cards, the NC552SFP or the original CX4 based cards?

 

Regardless, I don't understand why they would just not update the upgrade scripts to detect the specific combination of G2s and 10GbE, and refuse to run the upgrade. The've already updated the iso a few times already since V11 was originally released.

 

Also, there are tons of other cases like this where HP put out advisories right away, but the offending software/firmware is still available for downloading.

 

I'm seriously contemplating re-installing the OS on one of my G2s just to see how it would go.

 

Does anyone have any experience using HP_StoreVirtual_Quick_Restore_DVD_AX694-11011.iso to install V11 on a G2?

Per HP instructions, using the offline (rompaq) upgrade process, it should be done in the follwoing steps:

•Motherboard Firmware

•ILO Firmware

•10gbE Card Firmware
• LeftHand OS

 

Has anyone done this?

 

Frequent Advisor
Patrick Neuner
Posts: 85
Registered: ‎09-05-2007
Message 20 of 43 (2,177 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume


Bart_Heungens wrote:

Hi,

 

Cannot tell you anything in detail but 2014 will be a good year! ;-)

 

Kr,

Bart


Hi Bart,

 

I'm curious where you got that ominous Info from. I just had a Phone Call from HP Support (DAMIAN IVANOV) because of a Support Request regarding DESCARD Functionality on P4000 System. (Case ID: #4647850281)

 

HP Support Rep. (DAMIAN IVANOV) told me clearly that Lefthand Engeneering !!! IS CURRENTLY NOT !!! working on DISCARD funtionality nor will it come in the foreseeable future ... !!!

 

He logged this Line in the Case:

---------
*** PHONE LOG 3/24/2014 (M/D/Y) 2:23 PM CET-1CEST AT [UTC +1:00] WIDEN_DAMIAN_IVANOV Action Type: Outgoing call

Talked to the customer. There is no official info on this if he got that info from 3rd party / sales team he should contact them.
---------

He suggested to move to 3PAR Storage ...

Seems like we're left out in the rain!!

Andreas Schnederle-Wagner

Frequent Advisor
Patrick Neuner
Posts: 85
Registered: ‎09-05-2007
Message 21 of 43 (2,167 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

alright - seems like the Support Guy was not knowing what he was speaking about ...

I had the chance to speak with Brad Katz (HP WorldWide Product Manager StoreVirtual) ... sounds way better than the Support Rep. statement ...

------------------------------------------
Hi Andreas,

The support person was incorrect if they said no one is working on this. The UNMAP feature is being actively worked as a high priority by our development team and we expect to deliver this feature to our customers this year.

Thanks,
Brad
------------------------------------------

Looking forward for the UNMAP Feature ... ;-)

Andreas Schnederle-Wagner
Honored Contributor
Bart_Heungens
Posts: 687
Registered: ‎10-26-2005
Message 22 of 43 (2,162 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Personally I would believe Brad (great guy!) more than the support guy regarding features of StoreVirtual... ;-)

 

I keep my statement: 2014 will be a good year... ;-)

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Occasional Visitor
RayWhite
Posts: 1
Registered: ‎09-21-2012
Message 23 of 43 (2,071 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Hi All,

 

Received an email from Adam in support that trim/unmap will be in LHOS 11.5 due in April or May. I sure as hell hope so!! :)

Occasional Advisor
PavelOn
Posts: 8
Registered: ‎03-29-2012
Message 24 of 43 (1,533 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

Hi! 

 

Have somebody news about 11.5 rls. date?

Thank you


Pavel

Frequent Advisor
Patrick Neuner
Posts: 85
Registered: ‎09-05-2007
Message 25 of 43 (1,531 Views)

Re: Reclaiming Space from a thin provisioned netword raid-5 volume

heard nothing, I thought anything will be anounced on  HP Discover, as they sent ous Newsletters about important announcements, say yes to change and more... well, I didn't find a note about Lefthand at all, just EOL of current VSA.

 

Patrick

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.