What to pack in your ALM snakebite kit
Things to consider when archiving and backing up projects
If we considered the Y-axis in a spreadsheet, is what we discussed in “The snakebite prevention kit for backups” article then the X-axis of the same spreadsheet would be considered ancillary information that we discuss in this article. This information can impact your company and projects such as project size, criticality, activity, regulatory and especially the cost-effectiveness of backing up and restoring projects within ALM.
Here are a few considerations to keep in mind:
- Project size – If a project size becomes too large the concept completing a full backup is so time-consuming that one can only be done once a month. In this case you must rely on incremental or differentials greater than five at a time; this may be a consideration of creating subprojects which should be discussed in another article. If your projects are too small and you can’t run a full back-up on a daily basis and you must consider that your projects may not be as effective as they could be and archiving information for given project.
- Criticality and Activity – Evaluate your project’s impact to the company or amount of information that would be lost due to an outage or corruption of data. This is probably the single most important consideration when scheduling backups to both the database and repository.
- Regulatory requirements – Look at the project’s relationship to all the legal aspects of the company. This should only impact the number of backups taken but also the length of time that information should be stored either on-site or off-site. When setting up a project a list of regulatory groups that may be impacted should be a part of your questionnaire. Examples:
- PCI: Payment Card Industry (retail)
- SOX: Sarbanes-Oxley Act (publicly traded companies)
- HIPPA: Health Insurance Portability And Accountability Act (health and insurance)
- IRS: Internal Revenue Service (financial)
- SDI: Strategic Defense Initiative (federal employees and contractors)
- EEOC: Equal Employment Opportunity Commission (human resources)
- PPA: Code of fair information practices (human resources)
Note: this is only a sample of regulatory groups that could impact your projects. For more information you should contact your company’s legal team and project leads. Some of the regulatory groups can require IT groups to establish record retention greater than seven years.
- Cost-effectiveness – If you’re not impacted by regulatory groups your major concern may be the cost-effectiveness of storing information over long periods of time. When considering the cost of any given project the SA should include the price of storing information on-site, off-site, storage space, number and the types of media needed for storage.
Note: when discussing the costs of storage with your system, warehouse, and/or database administrators, it may behoove you to remind them of your company’s regulatory obligations and how ALM has a great track record when it comes to external audits when used properly.
- Frequency – The number of backups and types of backups that you used during any given day, week, month, or year directly correlates with the amount of information stored in the system. This data fluctuates in a given hour and/or reflects how accurate your restores are in relation to a given system failure or data corruption. In other words, if a person is only modifying or changing data in a project once or twice a day, then it wouldn’t be logical to save transactional logs every hour on the hour. But, running full backups nightly for the sake of a quick restore may be beneficial.
However if you have more than five people working around the clock in a given project, the transaction backups may be easier than scheduling daily outages for the full or differential backup that are required. But when it comes to restoring a project of that size, it could take hours or days to execute all the transactional logs to rebuild the project.
Tip: For SA’s with over 10 projects scheduling and editing project restores can be difficult. This is why I recommend grouping projects with similar backup schedules by domain this will allow you to convey information based on directory structure.
How to handle inactive projects
One way to lower the cost of storing large amounts of data is to monitor the activity of projects and either deactivate or archive inactive projects. This can save a great deal of money and time. It can also protect SA from what I call “the bite you in the rear” snake. This scenario typically plays out when a project has been inactive for quite some time. The data gets corrupted if too many full backups are taken, and with corrupted data there may be no way to restore the original project.
Typically at this point a user or group of users request access to the original project and you come to find out that the data has been corrupted and you have no way to restore it. A rule of thumb that I like to follow is to monitor all user activity by project and when the activity drops to zero for more than a month, I deactivate the project. I will allow the system to complete full backups; however, I recommend that you turn off any incremental or transactional backups.
After three months I recommend that you mark the project as archived and turn off all full backups. Keep the last backup on the database drive and repository, and create one external backup for off-site or storage. This simple documented step is equivalent to carrying around snakebite medicine kit for those tragedies that you didn’t see coming.
Sample Kit
I’ve included some typical samples that I’ve used in the past to aid in one of common overlooked areas in the site administrators job. These have the highest risk to one’s career if not planned correctly.
Common project scheduling chart: (sample)
Backup Name | Type of Backup | Frequency | Schedule | Backup File Repository | # of on-site | # stored off-site |
Yearly | Full | Yearly (use the last monthly of the company’s physical year) | Last day of the year | Normal | 1 | 1 – 15 (based on regulatory requirements) |
Monthly | Full | Once a month | Last weekend of every month | Normal | 18 | N/A |
Weekly | Full | Once a week | Scheduled during weekend outages | Normal | 8 | N/A |
Daily | Differentials/ incremental level 1 | Once a day | Scheduled during daily outages (excluding days were full backup is taken) | Incremental | 6 | N/A |
T-logs | Transactional backups | Hourly | Scheduled every hour on the hour during weekdays (no outages required) | Optional differential | 24 | N/A |
Setting up a questionnaire for the creation of ALM projects: (A partial sample)
Question | Description or types of choices | Recommendation |
Requested domain | List all the domains currently in ALM | Only lists this question if the domain areas are optional |
Project name | Requested project name | Best practice recommends that you define what a project means in ALM before allowing users to request specific names for projects |
Number of users | The number of concurrent users that could logged in at any given time | This should be based on the concurrent number of users in a project and not total number of users (if you need help estimating the number of concurrent users please contact me and I’ll be happy to help you develop a formula for your company). |
Project criticality | High, medium, or low | This should be based on corporate initiatives and cost. It can also give you an idea how large the intervals can be between backups. This directly correlates with how much data could be lost between intervals. HP’s best practices recommend that backup interval not to exceed 24 hours. |
Regulatory Entities | Create a list of all the regulatory groups that may impact either your company or industry | I would recommend contacting your legal department and/or IT leadership to create a comprehensive list of regulatory groups that may impact your software development groups |
Modules | A list of ALM modules that the group plans to use within the project. This this question should either be listed out by module or allow for multiple selections | Know in advance how many of the ALM modules the user is planning to use, then it is easier to estimate your disk space and database utilization forecasts (which will be covered in a separate article). |
Requested backup frequency | Client expectations or recommendations | While this question must be determined by the ALM SA, it may befit you to know the client’s expectations and willingness to share the extra cost that may occur for frequency of backups. You also need to know the time needed to complete a full restore of a project. |
Restore SLA’s if needed | Client expectations or recommendations | |
Requested maintenance windows | This should be based on how often backups are taken and patches are applied to ALM (optional question and subject to SA) | This is a multi-part question which should be broken into daily, weekly, and monthly Windows (typically a daily can be 1 to 2 hours, weekly 2 to 4 hours and monthly 8 to 24 hours). We also recommend not scheduling all projects at the same time. |
When information is lost, it’s no joking matter. This is especially true when you’re talking about regulatory audits, man-hours, artifacts or just the corporate bottom line. The odds are whether you’re a snake wrangler or ALM site administrator that you will get bitten at least once in your career if you’re serious about what you do. My father always told me that we all make mistakes and accidents will happen. How we react and our actions can be either perceived as heroic or pathetic. The outcome has nothing to do with luck but sheer tenacity and a well-drawn out plan.
You can also contact my colleagues and HP professional services for an in-depth assessment and solution of your current ALM environment.
In this very article I didn’t push the envelope of creative writing because of the subject matter. I still want to hear your snakebite stories and their outcomes, so others can learn not to play with King cobras or the infamous black mamba snake. These snakes can strike their victims multiple times and snakebite is known as one of the most agonizing ways to die.
Thanks
Wh4tsup_Doc
