Culture of Preparedness and Accountability
CDASH is a treasure chest that accumulates knowledge collected and organized over a long period of time. Once items are initially loaded into the Production CDASH Instance, information about places and documents may be corrected or enriched, and before too long, CDASH becomes the only repository for the most authoritative representation of the Architectural Survey.
The CHC Omeka Owner and the Administrator understand that the information in the CHC Omeka installation would be nearly impossible to regenerate should some sort of disaster strike or more importantly, if some simple mistake results in information disappearing or becoming corrupt.
The prospect of noticing that some number of CDASH items has disappeared or been damaged, without knowing the extent of the damage or when and why the information was lost should make us very anxious. The Good News is that our web host, Microsoft Azure, contains facilities for recovering the necessary components to rebuild the Production CDASH instance and its contents. Nevertheless, the recoverability of CDASH is not without limits and requirements for specialized knowledge, which are discussed in detail, below.
THe key to a happy life is to have a regular routine of verifying that everything is OK and or for catching any problems without delay.
Topics
- The Importance of Regular Audits
- What Could Go Wrong
- Establish a Culture of Routine Data Safety
- Preparedness Level 1: Verify Backup and Restore Capabilities
- Preparedness Level 2: Check Basic Functionality
- Preparedness Level 3: Count your Chickens
- The Critical Importance of Giving a Shit
- To-Do: Automated Audit Dashboard
The Importance of Regular Audits
Most computer users have experienced the sinking feeling that accompanies the discovery of mysterious lost or corrupted data. Even if you have a great backup system, one of the trickiest problems is figuring out the extent of the problem and when the problem occurred. Since the backup and restore accommodations allow an instance andd its contents to be restored to a specific date in the past, it is best to be able to restore to the day before the problem occurred. Finding out about the issue sooner is much better, since any additions or changes that were made after the restore date will be lost and have to be repeated. There is also the possibility that more than one overlay of problems has occurred, which can lead to very confusing situations.
The less time that has elapsed between the problem and the discovery of it, the greater will be the hope that a complete restoration can be made without problems.
What could Go Wrong?
Here is a list of the sorts of things that can go wrong.
- Place or Document Items disappear.
- Media (scannned pages) associated with Document Items may disappear.
- Item properties can become messed up due to errors with bulk editing.
- Relationships between Places, Documents or Folders could become discombobulated, leading to orphan documents and places or folders that seem to have no related information.
- Other
Beware the Unexpected!
The worst sort of trouble is the sort that is unexpected and undetected. As an example, did you know that if you remove a property from the resource template for a a type if item (Place or Document) that all of the values for that property will disappear without any warning or easy recovery method? This is the sort of a trap that may not be detected for a very long time after it has sprung.
Establish a Culture of Routine Data Safety
When it comes to protecting assets, a sense of anxiety is healthy -- especially if it motivates the administrator and the owner to establish regular routines of validating the integrity of the collection. We recommend that the following checks be carried out every week by the Omeka Administrator and reported to the CHC Omeka Owner.
Preparedness Level 1: Verify Backup and Restore Capabilities
The cloud host for the CHC Omeka installation (Microsoft Azure) has facilities for taking daily backups of the Omeka installation. Part of your safety culture should be to check each week that these backups are being saved and accessible. This procedure is covered in the page CHC Omeka in Azure Overview, Integrity Check, and Restoration. Check the backups for the MySQL database the CHCPersist and CHCCans file shares. These are saved for 30 days.
Preparedness Level 2: Check Basic Functionality
A simple way of assuring theat your Omeka assets are safe is to simply use the application and pay attention to the following:
- Does the map of place items look right?
- Use the View button to look at the full set of properties for Place and Document Items.
- Verify that jumping from document to document, from a list of folder contents causes the locations of cicled on the map to move around appropriately.
Preparedness Level 3: Count your Chickens
A very simple way to assure yourself and the CHC Omeka Owner that everything is OK is to regularly check the number of items, item-sets and media files.
If you update and double check the expected counts each time you add places, documents or folders, you can rest easy in the knowledge that your changes have been registered. Even when you have not been knowingly addng or deleting things, repeating this count every week can provide you a timely assourance that everything is OK or provide a warning that some unexpected loss has occurred.
For now, our audit process consists of a spreadsheet that provides a running list of upload events. For each event there is a number of items, item-sets (CDASH Folders), and media files (scans) that are affected. The spreadsheet calculates cumulative counts that have been checked against the counts of actual items in Omeka. At the time this is being written, we are involved with the initial population of CDASH, so these numbers are checked each time files are uploaded. This simple procedure is one way of checking to make sure that the expected numbers of items and folders have been created, and that they persist.
The Critical Importance of Giving a Shit
It should now be clear that detecting and recovering from management mistakes is a lot of trouble. By far the easiest way to live is to ignore problems -- especially ones that probably nobody is going to notice anyway. This would be an understandable approach for many people. The Historical Commission's Omeka Owner should be aware of the importance of having an Omeka administrator who has the archival mindset and temperament to do the right thing on a daily basis, and will not shy away from trouble that they may discovered.