File Formats and Management

stack-of-papers

Digital Image File Formats.

Much of the information in this section is based upon an article by Ken W. Watson. (). Photo File Types. Available: http://www.rideau-info.com. Last accessed 28th Jan 2015.

There are literally dozens of digital image formats that have emerged for specific graphics / animation / digital manipulation purposes, but the three most common that are likely to be encountered by a photographer are : Firstly JPEG a lossy format, TIF, a lossless format and RAW an in-camera lossless format. Lossy means that image data is lost when the image is compressed (saved) while a lossless format retains all the original data, even when compressed.

Each format has pros and cons and is suitable for particular tasks.

JPEG (JPG, JPE) stands for Joint Photographic Experts Group and is a standard developed in the 1980s to handle colour digital images. It works best with photographic images (as opposed images of text) because it relies on the blending of colour.

It is the most common file format and is used and supported by every major software provider of photo edtiting tools.

It is a “lossy” format, it reduces the file size of an image by blending “redundant” image pixels. As the image is compressed blurriness can appears around edges of objects in the photo. In photo editing computer programs JPEG compression is usually expressed as a percentage where 100% is no compression and 0% is maximum compression (think 100% quality vs. 0% quality). Usually Adobe Photoshop uses a sliding scale from 0 to 12 (really 0% to 100%). Once compressed in JPEG format an image cannot be uncompressed (you cannot regain the original quality).

It is for this reason that professional quality images are saved in a lossless format and then a final images – for printing or publication – is saved in .jpg format.

Pros : Relatively small image size, very good photographic reproduction, best format for emailing or posting to the web, compatible with virtually every image editor and viewer.
Cons: “lossy” format, it compresses by removing information which can never be recovered, photo degradation after editing on save (even at highest quality setting).

TIF/TIFF – This is a lossless image format that is, no pixels are modified in the image. TIFF stands for Tag Image File Format. This generally results in very large image sizes (in terms of computer file size). TIF has the option of being compressed, using either LZW or ZIP, both lossless compression methods.

Using LZW or ZIP will shrink the TIF image with no loss of data. Note that some programs also provide an option for JPEG compression – if used it will result in data loss as per the jpg format Even compressed, TIF files are very large, much larger in computer file size than their JPEG equivalents.

TIF is a favourite of graphic designers since it was an early standard on the mac.
Pros: “lossless” format – all image information is retained.

Cons: Huge file size even when compressed, has multiple “standards” so not all programs can read all TIF files. Not web browser compatible.

RAW – This is a lossless image format offered by some digital cameras and especially DSLR format cameras. The JPEG image produced by a camera and typically displayed on its screen is an image processed by the camera’s software in which variables such as sharpness, contrast, saturation and white balance are applied to the digital image based on the camera’s settings.

RAW on the other hand is the direct unprocessed image as seen by the camera’s sensor. It allows the post-image processing using any of the camera parameters (i.e. sharpness, contrast, etc.). This format is generally preferred by professional photographers for their image “negatives” (originals) since it allows the greatest post-processing flexibility.

A downside of RAW is that the format is currently proprietary to the camera manufacturer and therefore it is not a good long term archival standard. There is work being done to standardize the RAW format, but a standard is unlikely to be set since new features (e.g. white balance, focus and HDR bracketing) being added into digital cameras end up being part of their RAW data and this keeps changing as new innovations are introduced by individual manufacturers.

Pros: “lossless” format – allows full post processing of all in-camera variables (white balance, saturation, sharpness, etc.).
Cons: Proprietary camera manufacturer format (multiple standards), not all software can view RAW files, large file size. Not web browser compatible.

Storage and Management of Digital Formats.

This is a discussion on good practice to manage photo files and design ideas to build storage systems.

The speed reliability and capacity of personal computers has improved measurably since they became widely available and affordable in the 1980’s none the less all computer media is subject to failure, this is often catastrophic i.e. total loss of the contents and is usually unexpected and can be triggered as simply as improperly removing a card from a mac or pc computer. The risk varies with the type of media. For example portable formats – memory cards, keys, CD’s and so forth are by their nature much more prone to physical loss or damage than data stored on internal hard drives on a PC kept in an appropriate environment.

There are a number of strategies that can be adopted to minimise such a loss but basically all involve having multiple copies of the data as you are clearly most at risk when only a single copy of the data exists. This is most true when the image is freshly taken because at that point only one copy of the image exists i.e. on the camera. To mitigate this some modern cameras aimed at the professional e.g. Canon 5D Mk III have the facility to write to 2 memory cards at the time of image capture thus mitigating the risk of corruption but you are still vulnerable to physical loss of the card(s). Ultimately managing this data in a risk adverse manner is a combination of good practice and resilient hardware.

First priority should be to create a copy of the image ASAP, for most people this will be by uploading to a PC however for some,  generally professionals, e,g, News and Sports photographers this may be done by wireless upload to a central server. For those working “off-line” e.g. in a remote location there are a variety of portable devices that will enable a copy to be made until you are able to return to base however the risk of physical loss remains during this time. Hence photographers will rarely allow this data out of their possession whilst travelling e.g. will retain as hand luggage rather than checked baggage on an airline. Good storage management is basically about adopting a “risk adverse state of mind” and so for example despite the large memory card capacities of modern cameras wedding / event photographers will often prefer to use multiple smaller cards to minimise the risk of losing large numbers of images in the event of a card becoming corrupted.

Once the image is copied the next step should be to ensure there is a backup copy created – typically using utility software that automates the process on a regular basis and preferably to a device separate from your main computer e.g. a simple external hard drive or a more sophisticated cluster of external drives – a so called NAS device (Network Attached Storage)  which typically employ built in resilience technologies such as RAID (Redundant Array of Inexpensive Disks) typically this means the data is physically spread over several disks and the hardware can automatically recover all data in the event of a single disc failure and replacement. Mirroring is also a resilience technology and as the name implies generates a mirror copy of the disk, this is relatively fast compared to RAID but at the expense of halving the available disc space.

Back up versus Archive. In the context of this article Backup typically means a working copy of images and is designed to cope with issues on the local PC e.g file corruption/ disc failure. Archiving is meant as long term, off site storage designed to mitigate a major catastrophe such as total loss of personal PC e.g by fire or flood. Data stored in an archive should be tested regularly to ensure it can be retrieved if and when required – no computer format has so far lasted more than 10 years or so before being replaced in common use e.g. an archive on floppy disc;s isn’t going to be much use if your current PC no longer has a floppy disc drive.

A good archiving strategy can be achieved at reasonable cost by using 2 external HD’s as follows. One of the HDs stays with the main machine, the second (archival copy) is stored at an off-site location far enough away to be unaffected by the same disaster e.g. if you live on a flood plain your best friend and neighbours house is probably not the the best choice but otherwise a friend’s house, a relatives house, bank safety deposit and so forth). Every month swap these, i.e. make a current archival set to the local HD and then take it to the off site location and swap it with the remote HD. If your house gets flooded out, blown away or burned down, the archive copy of your photos will still be available. With the monthly swap of the HDs, the drives are automatically checked and the data on them refreshed and any issue can be investigated and resolved.

If you are in an area with fast broadband then it may also be possible to do this with on-line remote storage but the volumes of disk space required maybe cost prohibitive and depending on your level of paranoia (or risk aversion) the idea of handing control of your data to a third party maybe unattractive.

Rule of 1/2/3 is a summary of a good storage strategy courtesy of : Ken W. Watson. (). Storage and Archiving of Digital Photo Files. Available: http://www.rideau-info.com. Last accessed 28th Jan 2015

RULE OF ONE – this rule states that there should be one set of untouched (unedited) photos. These are the original photos from your camera, in film terms, these are your “negatives”. Never overwrite these – if you’re editing, always edit a copy, never the originals.

RULE OF TWO – at any given point in time, ensure that no matter what you are doing, there are at least two separate sets of your photos. Initially, when you copy photos you’ve just taken from your camera to your computer, the two sets are the photos on the camera’s memory card and the copied set on your computer. Before you erase the photos on your memory card, ensure that that the Rule of Two is maintained i.e. make a backup of the set on your computer (to another HD, memory stick, offsite storage, a DVD, etc.). So, when you erase your camera’s memory card, there are still two separate sets of your digital photos. 

RULE OF THREE – The Rule of Two is for immediately working with your photos, the Rule of Three states that at some point, preferably sooner rather than later as disasters arrive without warning, you must have a further copy of your photos located off-site (away from a catastrophic disaster such as fire or flood – which would wipe out all the backups in your house). So, the Rule of Three adds a third, archival set, stored off-site.

 

 

 

 

 

This entry was posted in Digital Imaging.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*