Geoff Coupe's Blog

Reflections on life at “De Witte Wand”…

Category: Photography

  • Geotagging and Metadata in Picasa 3.8

    Last month I wrote about the geotagging disaster that the current beta of Windows Live Photo Gallery is causing. At the moment, I daren’t have it running on my PC because it wantonly writes garbage GPS coordinates into my photos.

    While I’m waiting to see what Microsoft will do in the next beta of WLPG, I thought that I’d take a look at its closest rival, Google’s Picasa, to see how that’s shaping up.

    While I found on past experience that there’s lots to like about Picasa, I’d ruled it out up until now because it did not support XMP-based metadata. That meant as I use metadata following the IPTC Core standard, which itself uses XMP, then Picasa just didn’t cut it.

    However, things change, and the current version of Picasa, version 3.8 released last month, is being trumpeted by Google as now supporting XMP.

    So I downloaded and installed this new version of Picasa. And while it certainly seems to display XMP-based metadata (see below), it doesn’t seem to support writing out all of this metadata into image files. I also came across a major bug in how Picasa handles Geotags.

    It won’t display the correct GPS coordinates of many of my files on its map. Here’s an example, the contents of a folder containing images shot in the local area here in The Netherlands (click on the image to see it full-size in a new window).

    Picasa Geotag 2

    Here’s a close-up of the map. As you can see, Picasa claims that many of the images have GPS coordinates corresponding to places outside of The Netherlands, in fact many of the images are literally out of this world, according to Picasa.

    Picasa Geotag 1

    These files have all had GPS coordinates added to them using IDimager. (Note: IDimager is no longer available. Its successor is Photo Supreme, which I am now using) All these files will display correctly in IDimager itself, and also in the map interfaces of Microsoft’s Pro Photo tools and Geosetter. Here are the files being displayed correctly in the map interface of Geosetter:

    Geosetter 1

    Clearly, this is a bug in the current version (3.8) of Picasa. Fortunately, Google have acknowledged that there is a problem, and it should get fixed at some point in the future.

    As to the XMP metadata support, it looks as though the following IPTC Core elements are at least read by Picasa for JPEG files:

    Description
    Description Writer
    Headline
    Keywords
    Title/ObjectName
    Job ID
    Instructions
    City
    Location
    State
    Country
    Creator
    Creator’s Job Title
    Provider
    Source
    Copyright Notice

    In this version of Picasa, there doesn’t seem to be a way of writing data into all of these fields, but only a subset, so Picasa isn’t yet suitable for maintaining IPTC Core metadata. Picasa also doesn’t read and display these metadata fields at all from RAW files (at least for my Canon CR2 format). It only appears to display the Exif metadata from these RAW files. So, once again Picasa is getting closer, but it’s not good enough for what I’m looking for.

  • Geotagging in Windows Live Photo Gallery–Part 2

    Last month, I wrote about my findings on the experience in using the Beta of Windows Live Photo Gallery to geotag my photos. I wasn’t too impressed with the experience.

    This week, a second beta of WLPG was released by Microsoft so I’ve been revisiting the experience of geotagging.

    And today, it’s been borne in upon me just what a disaster geotagging in WLPG has proved to be.

    You see, I made an assumption. That was that WLPG would not alter any metadata in a photo without my explicit permission and knowledge. Wrong…

    I was naive enough to think that WLPG would only write out GPS coordinates to the Exif metadata in a photo when I explicitly added a Geotag using WLPG. Wrong…

    I realised that WLPG was reading in IPTC Core Location metadata from my photos and using that to create a geotag in WLPG’s internal database. I also realised that it was copying the metadata into the “Location Created” section of the newly specified IPTC Extension metadata. Since these are a set of text fields that reflect what already existed in the original IPTC Core Location fields, that didn’t bother me unduly.

    What I hadn’t also appreciated is that WLPG not only constructs a geotag in its internal database and creates Location Created metadata in the IPTC Extension section, but that it then proceeds to write out a set of GPS coordinates into the photo’s Exif metadata of where it thinks that the photo was taken.

    This is an unmitigated disaster!

    As I said last month, WLPG makes false assumptions about what the GPS coordinates are. If it doesn’t recognise the contents of the Sublocation field, it uses a GPS position derived from the contents of the City field. If it doesn’t recognise the contents of the City field, it uses a GPS position derived from the contents of the State field, and if it doesn’t recognise the contents of the State field, it uses a GPS position derived from the Country field.

    I have a collection of over 40,000 photos. The majority of these have IPTC Core Location metadata that I have catalogued over time. Only a very small percentage of these had GPS coordinates that I had carefully added myself.

    Now, WLPG has gone through my collection reading the Location metadata and has written out GPS coordinates to all of the photos containing Location metadata. And, of course, in a lot of cases, it doesn’t recognise the terms I’ve used for a particular location field, so it’s plucked a GPS value out that bears no relation to where the photo was taken.

    Worse still, I now have lost the needles of those photos which have accurate GPS positions in a haystack of huge proportions, which consists of photos with false GPS positions.

    Aargghh!

    Update 23 August 2010

    1. I’ve now looked at a backup of my photo collection taken on the 1st June 2010 (i.e. before the WLPG beta was installed). No photos had false GPS values inserted in the Exif at that time. Therefore I conclude that WLPG is the culprit.
    2. Contrary to what I first thought, not all photos with IPTC Location metadata get GPS values inserted into them by WLPG – there may be a pattern, but it’s not obvious to me why some files are hit, but not others.
    3. However, even though all files are not affected, I have still found over 7,000 photos with GPS values, and of these, only 2,359 of these photos have genuine GPS values that I have explicitly inserted. The rest have GPS values inserted by WLPG itself and which are also inaccurate (false).
    4. Some false values come from WLPG ignoring a sublocation, and inserting a central GPS value for the City.
    5. Some false values come from WLPG misinterpreting a sublocation and inserting a completely wrong GPS value for another location altogether; e.g. Sublocation: Voortman Bos, City: Heelweg gets interpreted as Voortmanweg in Deventer, 37 kilometres away…
    6. Some false values make no sense whatever; e.g. some photos I have of St. Pancras station in London have a GPS value assigned to them of Cuxham, a very small village in Oxfordshire…

    All together now: Aaaarrrggghhh!!!

    Update 8 September 2010

    I posted about this issue on the WLPG Help Forum. Now, Analy Otero, who works in the WLPG team, has posted a response  to confirm that WLPG does indeed write out what it thinks are correct GPS coordinates to image files based on the content of the IPTC Location fields. As she says:

    “The behavior you’re experiencing is the design of the feature and we’re working to improve both reliability of the process (to ensure all photos get proper geotags) and to improve the accuracy of the places.”

    Unfortunately, I don’t believe that Microsoft can ever sufficiently improve both the reliability of the process and improve the accuracy of the places to the extent where I can trust that accurate GPS information will be included in my images. My experience thus far has been an eye-opener of just how bad it currently is. I see that in the current release notes, Microsoft themselves say:

    “Landmarks (such as the Eiffel Tower) are not supported in the current implementation of geotagging”.

    The problem is that the textual IPTC sub-location field, in particular, will always be down to what the user decides, e.g. “the stern of HMS Ark Royal in dry dock”. I’d like to see Microsoft be able to give an accurate GPS for that. Of course, they can’t, and so the chances that rubbish GPS coordinates will be introduced by WLPG into an image remain very high.

    A further twist is that, apparently, once WLPG has introduced a GPS value (false or accurate) into a file, it can’t ever be subsequently changed by WLPG. Elsewhere in the release notes it states:

    If a photo or video contains no GPS data, coordinates will be added when the item is geotagged. However, updating or deleting a geotag string won’t modify the GPS coordinates. Any additional updates to the geotag field don’t change the original coordinates written to the file. (my emphasis)

    It seems to me that the way to cut this Gordian knot is for Microsoft to give us a proper mapping interface in WLPG itself (such as Picasa, IDimager, or Geosetter do) so that we can check locations prior to allowing GPS coordinates being written, and to use the map to modify or delete GPS coordinates. That is, writing of GPS coordinates is under the explicit control of the user, instead of something that WLPG does by itself in the background as a write-only operation.

    The current implementation of WLPG writing out what it thinks the GPS coordinates should be is dreadful and appalling. I simply cannot afford to have WLPG installed on my PCs as it is. It has already introduced garbage information into thousands of my images.

    Update 30 September 2010: Well, the final version of WLPG 2011 is now released, and as far as I can see it is still screwing up my GPS metadata.

    I’ve just found some photos taken this month in the Netherlands which now have GPS info for Wimereux in France inserted into them by the final release of WLPG.

    I am definitely not impressed.

    Update 2 December 2010

    There’s an update to WLPG 2011 that addresses the geotagging issue. See here for more information.

  • Slide Show Quality in Windows Live Photo Gallery

    Microsoft’s Windows Live Photo Gallery (WLPG) can display selected photos as a slide show. In the new beta of the next version of Windows Live Photo Gallery, the slide show capability is still present, but Microsoft have changed the way in which it is done. Instead of having this capability within WLPG itself, it uses the new version of Windows Live Movie Maker to make and display the slide show.

    The problem is, the quality of the slide shows produced by Windows Live Movie Maker is terrible. Photos displayed as slides are blurry and noticeably degraded in quality.  I would be ashamed to show slides to family and friends using it.

    I raised this in the Windows Live help forum for Photo Gallery. The first response back from Microsoft was to deny that anything had changed between WLPG version 3 and the beta of version 4. However, once I sent them proof, then they admitted that things had changed and:

    “it appears that photo quality in slide shows in Windows Live Photo Gallery Beta is indeed a bit degraded when compared to the original file source”.

    I love that “a bit degraded”. No, Microsoft, it is noticeably degraded to the extent that it is unacceptable. The quality of the slide shows produced by Windows Live Movie Maker is simply not good enough. So now I will have to find an alternative to WLPG in order to show slides to friends and family.

  • The Courtyard of the British Museum

    Here’s a Photosynth of the courtyard in the British Museum that I did last year.

    On this blog, hosted on WordPress.com, I can at least embed a link to the Photosynth, even if I don’t get a thumbnail (as I can on the same entry in my blog on Blogger). On my old blog on Microsoft’s Windows Live Spaces, I was unable to embed Photosynth pictures or links, because Spaces strips out the embed code. This while Photosynth is yet another Microsoft product.

    Yet another case of Microsoft’s left hand not knowing (or caring) what the right hand is doing…

     http://photosynth.net/embed.aspx?cid=7303a8f0-f8af-435c-8685-afc774c898c6&delayLoad=true&slideShowPlaying=false

    (tip: when viewing the Photosynth, switch to Grid View to select another set of photos shot from a different point in the Courtyard)

    Addendum: And of course Microsoft has now scrapped the Photosynth product and technology, so none of these links work anymore. It’s dead, Jim.

  • On the Internet…

    …nobody knows you’re a dog. That’s the caption of an iconic New Yorker cartoon by Peter Steiner.

    The Dutch newspaper, De Volkskrant, has a page in its weekly magazine devoted to photographs sent in by readers on a theme that is given in the previous week’s issue. Last week, the theme was “the view from your webcam”.

    I thought I’d do an update on Steiner’s joke. So here is “On the Internet, nobody knows you’re a dog – unless you use a webcam”

    GeoffCoupeHeelweg

    It didn’t get published… Oh well, there’s always another day…

  • Photos of the Year

    Here are my photos from 2009 that I am most pleased with. They’re no great shakes from a professional point of view, but I am happy with them…

  • Pretentious? – Moi?

    Olympus UK have launched their new PEN camera with a series of video adverts. You can see them here – click on the video tab to choose between them (once you’ve pressed play to activate it).
     
    I have to say that I’m in the camp that finds the Kevin Spacey adverts pretentious twaddle; more evidence that Marketing people should be first up against the wall, come the revolution. However, the ‘Stop Motion’ advert is a little gem. It also tells you absolutely nothing about the camera, but as a work of art, it shines.

  • All You Wanted To Know About Digital Photography

    There’s a new web site that just been launched, which is devoted to the topic of Digital Photography. It looks very comprehensive, and covers topics such as best practice in Digital Photography and Workflows.
    The site has been set up by the American Society of Media Photographers with funding from the US Library of Congress. The project team includes Peter Krogh, who has written a well-respected book on Digital Asset Management and Digital Workflows.
    If you’re interested in Digital Photography, this definitely looks like a site worth checking out.

  • De Witte Wand

    I made a new photosynth of the front of the farmhouse and garden yesterday. You can see it here.

    Addendum: And of course Microsoft has now scrapped the Photosynth product and technology, so none of these links work anymore. It’s dead, Jim.

  • Tagging Digital Photos – Part III

    As a result of my last post on tagging, I’ve had a couple of questions, so I thought I’d answer them in this post. The questions were:

    1. Say I went on vacation and came back with a whole bunch of pictures. How do you relate those pictures to each other? I.e. is there a way to do some kind of “Description tree structure” effectively?
    2. I’ve seen that many users of JRiver Media Center (JRMC) use the hierarchy structure in keywords that you describe above (they usually call it nesting). In JRMC this is displayed nicely in a tree structure. How portable is this hierarchy structure? How does it display in IdImager, WPLG, or in any of the Adobe applications?

    Question 1 is a bit like a combination of “how long is a piece of string” and “what’s the best way to skin a cat?”. I can only answer it by describing what works for me, but I would not claim that it’s the most effective for everyone.

    So, here’s what I do when I get back from a vacation… First, ingest all the photos from my camera(s) into the tool that happens to be my choice for the heart of my digital workflow: IDimager. (Note: IDimager is no longer available. Its successor is Photo Supreme, which I am now using) Other folks will have their own choices such as Adobe Lightroom, or even free tools such as Picasa or Windows Live Photo Gallery (WLPG). I’ve ended up with IDimager simply because, for me, the combination of its flexibility and power combined with a reasonable cost gives me the biggest bang for the buck of all the tools I’ve looked at.

    During the ingestion process, IDimager renames all the photos from the camera’s IMGnnnn format to my preferred date/timestamp format of yyyymmdd-hhmmss-nn, and also places the photos into a folder structure that is ordered by date to three levels: year, month and day. If a particular folder does not already exist, IDimager will create it on the fly. At the same time, the first pass of assigning metadata to every image is done by IDimager. This is to add the Contact Info, Copyright Notice, Usage Rights, and Title fields to the images.

    The next step is to cull the obvious failures from the set of images using the Light Table view of IDimager. Then I’m ready to do the first pass of assigning the detailed metadata to the working set of images. Since these are vacation photos, they will all have the Events/holidays/vacation Keyword assigned. I will also add location information, simultaneously to both the IPTC Core Location metadata fields, and as a structured Keyword; e.g. Places/Europe/Spain/Catalunya/Barcelona/Las Ramblas.

    Then I’ll do a few more passes, adding further Keyword metadata as appropriate, until I feel that sufficient metadata has been added. The last step at this stage will be to add geotag metadata, if I’ve had my GPS logger with me.

    The “Description tree structure” (a.k.a. the “Keyword” or “Tag” tree structure) that I have is as I described in the last post. It’s effective enough for me, but I won’t claim that it will work for everyone. Still, if you build up your structure following David Rieck’s suggestions, then I think you’ll have a good start. I’ve also just found a short article that gives some guidelines about how you might go about designing your own controlled vocabulary and taxonomy for digital images.

    Now I have the starter set of vacation photos. Further steps in the workflow will be to choose the best, with further processing and editing as necessary, by assigning ratings, using the “power of 10” or “pyramid” rule. This is described in Peter Krogh’s “The DAM Book” . The rule is that each additional “star” rating will have an order of magnitude fewer images in it. For example, if you have say 100,000 images in your total collection, then 10,000 might be rated with 1 star, 1,000 rated with 2 stars, 100 rated with 3 stars, and so on. Of course, at this rate, I’ll be lucky to have even a single image that is worthy of 5 stars, but that’s the idea – be aggressive to drive up your idea of quality.

    Because this starter set of vacation photos has sufficient metadata (for me), I’m able to select them quickly out of my total collection using either IDimager or WLPG. For simple queries, either will do; e.g. select vacation photos taken in 2006 in Barcelona. But for more complex queries, then I have to use IDimager; e.g. select vacation photos that show ball games but not tennis…

    There are other things that IDimager can do that less capable tools (e.g. Picasa and WLPG) simply can’t. Synonyms and relationships, for instance. Synonyms mean that if I search my image collection for cars, for example, IDimager will also suggest synonyms for the search term, e.g. automobiles. Relationships means that, for example, People/Family/John is related to People/Family/Anita as “husband to wife”.

    By creating relationships between your keywords, you are able to switch instantly to related keywords. If the keyword John is selected, you get instant access to Anita because Anita will be included in the sub list of John.

    Question 2 is somewhat difficult for me to answer, because I don’t use JRiver Media Center (JRMC). So I can’t say for certain how JRMC will expose the keyword structure to other tools. But any application has in theory two possible approaches to this. The first is a static approach, whereby you can export the structure from tool A into an intermediate file that can then be imported into tool B. The intermediate file usually takes the form of being in CSV or XML format. These two formats are pretty widely understood by a variety of Digital Asset Management (DAM) tools. Being static, it’s generally best used to set up a new tool, or to take a snapshot of your current structure, either for archiving purposes, or for sharing with other people.

    The second approach is a dynamic interchange, whereby all the DAM tools being used will track changes to the keyword structure preferably themselves, or with minimum user intervention. Because we are dealing with photos here, we do in fact have the possibility of achieving that dynamic interchange by using the photo metadata in the files. So, for example, if I make a change to the keyword structure in IDimager, I will then synchronise my image collection with the new structure. IDimager then writes out changes as necessary to the metadata of files that are affected by the change.

    WLPG monitors for changes in the metadata of images and reflects those changes in the metadata and keyword structure displayed in WLPG and held in its database. This means that both tools should show the same keyword structure. Here’s an example looking at part of the Places hierarchy with both IDimager and WLPG (click on the pictures to see the full-sized images):

    Church1 Church2

    You’ll notice that although the subtrees for the Glenfaba and Michael Sheadings (the Manx equivalent of State) are shown expanded in the IDimager screenshot, and not in the WLPG screenshot, both do have the same structure. And on the right, you can see that both have the same keywords. Also note that in IDimager, the full delimited keywords are shown (e.g. Places/Europe/Isle of Man/Middle Sheading/Baldwin, whereas in WLPG only that leaf of the full delimited keyword is shown, e.g. Baldwin.

    WLPG is reading the whole delimited keyword, and using it to recreate the keyword structure shown on the left (in this case under the Descriptive Tags heading). So in this case, IDimager and WLPG are able to share the keyword structure dynamically. I’ve been able to set it up this way, because I’m able to tell IDimager to use delimited keywords, and that the delimiter character to use is “/” (which is required by WLPG). If JRMC can write out delimited keywords into the IPTC Core metadata of your image collection using “/” as the delimiter, then WLPG should be able to reconstruct and track the keyword hierarchy on the fly.

    Be aware that not all tools use the same delimiter. For example, Lightroom always uses the pipe character ( | ) as a delimiter for its hierarchical keywords. In addition, Lightroom uses a separate XMP property to store these hierarchical keywords separated from the normal keywords. That’s why IDimager also supports the hierarchical keywords method as used by Lightroom as well as delimited keywords written to IPTC Core specs. So portability between tools is by no means guaranteed, but often a way can be found if at least one of the tools is flexible enough. Hope this helps.

  • Tagging Digital Photos – Part II

    I’ve been asked by someone for some detail on how I tag my digital photos. I thought that it might possibly be of some interest to others, so here’s the answer as a blog entry.

    The starting point is that I tag my photos using some (but by no means all) of the data elements specified by the IPTC Core standard. The following table lists the elements that I use; the descriptions that I have provided are drawn from the excellent Field Guide provided by the PhotoMetadata.org web site.

    Element Description
    Title This field is a shorthand reference for the image or “photograph” – primarily for identification. The title of an image should be a short, human-readable name – text and/or numeric reference – and may take several forms.In my case, I now use a date/timestamp to uniquely identify all my photos, i.e. a title has the form: yyyymmdd-hhss-nn.
    Description The Description field, often referred to by applications as “Caption,” should report the who, what and why of what the photograph depicts.I don’t always follow the guideline given by the Field Guide’s explanation, I usually just rely on the fact that this metadata is covered by the Keyword and Location metadata content for my image:”If there is a person or people in the image, this caption might include their names, and/or their roles in any action taking place. If the image depicts a location, then it should describe the location. Don’t forget to also include this same “geographical” information in the appropriate fields (location, city, state/province, country) of the IPTC Core. The amount of detail to include depends on the image and whether it is documentary or conceptual”.
    Keyword Terms or phrases to describe the subject of content in the photograph. I have cobbled together a controlled vocabulary (see later) for my own use here.
    Copyright Notice The Copyright Notice should include any legal language required to claim intellectual property. It should identify the photograph’s current holder(s).
    Rights Usage Terms The Rights Usage Terms field should include free-text instructions on how the photograph can be legally used. The majority of my photos are licensed under a Creative Commons license.
    Contact Info The Contact Info fields provide a generic structure for storing basic information that should make it easy to reach the person or organization that created this image.Note: “City,” “State” and “Country” used in Contact Info should not be confused with fields bearing the same names that refer to the picture’s origin.

    Address (Contact info)
    Address is a multi-line field.

    City (Contact info)
    Enter the name of the city where your business is located.

    Country (Contact info)
    Enter the name of the country where your business is located.

    Email(s) (Contact info)

    Enter your business or work email address, such as name@domain.com. Separate multiple email addresses with commas.

    Phone(s)  (Contact info)
    Enter your business or work telephone number here. Separate multiple numbers with commas. Be sure to include the complete international format of a phone number including: +{countrycode} ({regional code}) {phone number} – {extension, if required}

    Postal Code (Contact info)
    Enter the local postal code (such as ZIP code) where your business is located.

    State/Province (Contact info)
    Enter the name of the State or Province where your business is located. Since the abbreviation for a state or province may be unknown to those viewing your metadata internationally, consider spelling out the name.

    Website(s) (Contact info)
    Enter the URL or web address for your business. Separate multiple addresses with commas.

    Since I’m not a commercial photographer, I use the bare minimum here: just city, country, email and web site.
    Country
    (legacy)    		 The full name of the country pictured in the photograph. This field is at the first level of a top-down geographical hierarchy. Use a verbal name and not a code.
    State/Province
    (legacy)    		 The name of the subregion of a country – usually referred to as either a State or Province – pictured in the image. Since the abbreviation for a State or Province may be unknown to those viewing your metadata internationally, consider using the full spelling of the name. Province/State is at the second level of a top-down geographical hierarchy.
    City
    (legacy)    		 The name of the city where the image was captured. If there is no city, I use the Sublocation field alone to specify where the photograph was made. City is at the third level of a top-down geographical hierarchy.
    Sublocation
    (legacy)    		 The name of the sublocation shown in the image. This might be the name of a specific area within a city (Manhattan), or the name of a well-known place (Pyramids of Giza), monument or natural feature outside a city (Grand Canyon). Location is the most specific term, at the fourth level of a top-down geographical hierarchy.

    You’ll notice that in the above table, the location fields of ITPC Core (country, state/province, city, sublocation) are marked as legacy. That’s because there’s a newer IPTC specification (IPTC Extension) that seeks to remove the ambiguity implicit in the term “location”. After all, does it refer to the location of the camera, or the location of the subject in the picture? So the Extension defines two classes of location: the location where the image was created, and the location shown in the image itself.

    For the moment, I’m continuing to use these legacy terms from IPTC Core (and for the most part, the location of the camera and the subject are the same for all practical purposes as far as I’m concerned). At some point in the future though, I’ll probably move over to using these newer metadata items from the IPTC Extension.

    One other thing, I “echo” the content of the location metadata back to a structured Keyword entry. For example, a photo taken in the Dam Square in Amsterdam will have the following items of metadata in it:

    Sublocation: Dam Square
    City: Amsterdam
    State: Noord-Holland
    Country: Netherlands
    Keyword: Netherlands/Noord-Holland/Amsterdam/Dam Square

    This is handled automatically by the application I use in my digital workflow: IDimager. (Note: IDimager is no longer available. Its successor is Photo Supreme, which I am now using) When I tag an image with a known sublocation, then the other corresponding location metadata, as well as the structured Keyword entry are automatically written into the image. Having a structured keyword echoing the location metadata in this way helps when the images are viewed as a collection using an application that does not support IPTC location metadata, but does support IPTC keywords. Windows Live Photo Gallery, for example (but see Note 1 below).

    Keywords and Controlled Vocabularies

    Choosing the right set and structure of your keywords that are used in your image library is a big topic, so I can only scratch the surface here. I’ve borrowed ideas from a number of different sources to come up with mine.

    First of all, I followed the excellent set of guidelines that David Riecks put together on his Controlled Vocabulary web site. Then, having a flat list of thousands of keywords is not very useful, it’s much better to arrange them into a hierarchy that connects related terms together; a so-called Controlled Vocabulary. For example, my keyword cows is actually part of a hierarchy that starts Nature/Animals/livestock/cattle/dairy cattle/cows. That way, when I search for photos (using either Windows Live Photo Gallery or IDimager) with the keyword cows, it will just show me those with cows in them. But if I search for photos with the keyword livestock, it will show me photos of cows, horses, pigs, sheep, and so on. When I use IDimager to tag a image of a cow with the cows keyword, it will automatically add all the parent keywords in the structure: dairy cattle, cattle, livestock, Animals, and Nature.

    My hierarchy has been built up from a number of sources:

    I’ve ended up with a structure that has the following items at the root:

    • Activities
    • Events
    • Nature
    • Objects
    • People
    • Places
    • Science
    • Styles

    Each of these splits down into further categories as necessary as you go down the levels. For example, Activities splits into

    • Disciplines
    • Hobbies
    • Physical and mental activities
    • Processes and techniques

    So then a photo of a tennis match would have the structured keyword of Activities/physical and mental activities/games/sports/ball game/tennis assigned to it.

    While setting up a controlled vocabulary takes a bit of work, and it will require ongoing “gardening”, there’s no question but that it’s useful when you have a lot of photos. I’ve currently got in excess of 37,000 photos in my collection and having them all with metadata makes it much easier to manage them.

    Note 1: Since I wrote this post, there has been a new version of Windows Live Photo Gallery released that does actually use the IPTC “Location created” metadata. It uses those fields to hold what it calls “geotags”, but which are in fact textual descriptions of the geographic location: street address, city, state/province and country.

  • All You Wanted To Know About Photo Metadata

    I’ve mentioned photo metadata lots of times in the life of this blog. Now I’ve just come across a really excellent web site that tells you everything you ever wanted to know about photo metadata. It’s at http://www.photometadata.org/
    Worth checking out.

  • Photos of the Year

    Here are the photos that I am most pleased with of all those that I took in 2008. I know that they’re not very good from a professional’s point of view, but I like them…

  • Windows Live Wave 3

    I see that the latest versions of the standalone applications (including Windows Live Mail, Windows Live Photo Gallery and Windows Live Writer) have now been released. This download page is still currently describing the applications as “beta”, but the applications themselves seem to have dropped that moniker from their titles.

    I’m pleased to see that at least one bug in Windows Live Photo Gallery that I reported to Microsoft over a year ago has finally been corrected.

    I’m using Windows Live Writer to create this post, and one thing that I want to check is how it handles image metadata. While it’s very easy to use WLW to insert images into your blog, the previous version seemed to be stripping out image metadata, and therefore creating orphan works, which I think is a very bad idea. So, here’s a test image, which in the original has my copyright information and IPTC Coreinformation as metadata embedded in the file.

    20080915-1152-07(1)

    Once an image is published in my blog, it can be downloaded from there as well. Let’s see what has happened to the metadata…

    Yep, all the metadata has been stripped out – copyright, creator, keywords – everything. That’s not good, in my opinion.

  • Postcards Home

    One of the careers that my father had was as a ship’s engineer. He began with the Isle of Man Steam Packet Company on the ships that crisscrossed the Irish Sea. The Island at that time (the 1920s) was a popular holiday destination, which meant that during the summer months, far more ships would be sailing than in the winter. At the end of the season, the junior engineers would work on the overhaul of the laid-up vessels. When the overhaul on a ship was completed, the men were paid off, and as my father wrote:

    We walked round the town until the next vessel had her overhaul. This happened every year, and meant that over 100 men could be out of work for between 12 and 16 weeks. This did not appeal to me – I had seen too much of it, and I applied for a seagoing job with the Ellerman Line. I received a letter offering me a post as 4th Engineer on the City of Wellington from the Ellerman Line and this is what I really wanted because I would then begin to get my 18 months sailing time in before I could sit for my 2nd Class Marine Engineer’s Certificate.

    I left Douglas on the 11th November 1925 and joined the City of Wellington on her maiden voyage round the world. Our first port of call was St. Johns, Nova Scotia, where during the war a munitions ship had blown up and destroyed the town.

    From there, the ship (and dad) visited Boston, New York, Newport, Panama, Honolulu, Yokohama, Kobe, Shanghai, Hong Kong, Suez, Gibraltar and Rotterdam. Dad bought postcards when he had the chance. Some he would send home – usually to his younger brother, Doug – but others he kept for himself, to remind him of where he had been on this, and subsequent voyages. After his voyaging days were over, he put them in an album where they’ve been ever since. They are a wonderful record of places and peoples that in many cases have changed beyond recognition or even vanished completely.

    Scan10016 Scan10021

    Scan10120

    Scan10035

    Dad wrote of Yokohama:

    The massive destruction of the town by the earthquake in 1923 was there to be seen, and I will always remember the forts at the entrance to the harbour and the large blocks of concrete tossed higgledy-piggledy about.

    Scan10030

    Scan10032

    Scan10029

    Scan10050

    Scan10077

    Scan10063

    Scan10135

    I love the fact that the publisher of this postcard has pasted in, not very convincingly, some ships in the foreground…

    Scan10151

    Scan10166

    Scan10167

    This is just a small selection of about 250 postcards. I think I’ll post a few more illustrating the places he visited in other voyages another time.

  • Windows Live Photo Gallery Revisited

    Last month I mentioned that I was trying out the latest release of Windows Live Photo Gallery, and that I’d run into a couple of bugs. Subsequent to that, I’ve been in communication with the WLPG team trying to sort out the bug whereby WLPG doesn’t seem to be tracking changes to IPTC/XMP keyword metadata.

    I had documented what I’d seen and sent it off to Michael Palermeti, a Program Manager on the WLPG team for investigation. He reported back that the developers could not find anything untoward with the test files and data that I’d provided. They were unable to reproduce the failure to track metadata changes. This struck me as being very odd, since I was clearly seeing the behaviour on two systems at home (my desktop and laptop PCs). So I went back to do more testing. And I think I’ve found out what’s going on.

    It is, as I suspected, associated with the fact that I’m using hierarchical keywords. A hierarchical keyword means that, for example, my keyword trees is actually the leaf node of a hierarchy:

    Objects/built environment/settlements and landscapes/landscapes/natural landscapes/vegetation/trees

    What’s going on is that when I assign a keyword to an image, I also explicitly assign the parent keywords to the image. So when trees is added as a keyword to an image, I’m also explicitly adding the additional strings:

    Objects/built environment/settlements and landscapes/landscapes/natural landscapes/vegetation
    Objects/built environment/settlements and landscapes/landscapes/natural landscapes
    Objects/built environment/settlements and landscapes/landscapes
    Objects/built environment/settlements and landscapes
    Objects/built environment/

    And there’s the problem: WLPG doesn’t seem to register these six instances of hierarchical keywords (trees, vegetation, natural landscapes, landscapes, settlements and landscapes, built environment) properly when they are assigned at the same time as a group. Worse, WLPG will not respond to subsequent deletions or additions to any keywords on the image. It’s almost as though a repetition of a part of a keyword hierarchy has the effect of locking the file as far as WLPG is concerned.

    When I change the method of assigning keywords to simply assigning only the full string once, then for the most part, WLPG appears to be happy. It will correctly register the keyword as a tag, and also track subsequent changes to the keywords.

    However, there is one crucial set of circumstances where WLPG still does not work correctly. That is where I assign keywords that share part of the same hierarchy to an image.

    Let’s take an example. I have a number of images with the keyword natural landscapes assigned (I.e. I have the hierarchical string: Objects/built environment/settlements and landscapes/landscapes/natural landscapes assigned to the images). Subsequently, I want to refine the metadata of some of these images by also assigning the keyword trees to them. So I go ahead and assign trees – in other words I’m adding an additional keyword string:

    Objects/built environment/settlements and landscapes/landscapes/natural landscapes/vegetation/trees to the selected images.

    The problem is that WLPG does not register this change – it continues to display only the natural landscapes tag associated with these images. Because the trees keyword is actually part of the same hierarchy as natural landscapes, WLPG fails to work properly and add this additional keyword string as an additional tag.

    What I actually have to do is explicitly delete the natural landscapes keyword from these images before I add the trees keyword to the images. This also has the effect of never being able (in WLPG) of having both the natural landscapes and trees tags associated with the same image because they are part of the same hierarchy. This strikes me as being somewhat of a limitation… In fact, I would say that it’s a bug, since now I cannot have certain combination of keywords associated with my files and have them tagged correctly in WLPG. Take house, castle, and dining room. They all share parent keywords in common in my hierarchy, so within WLPG I cannot tag images in such a way that will distinguish between a dining room in a castle and a dining room in a house. I suspect that this all goes back to the design decision that appears to have been made for WLPG that selecting multiple tags is an OR function and not an AND function. In my opinion, that was a very bad decision, but that’s another story…

    Anyway, to sum up, I’ve now found out that WLPG cannot cope with multiple instances of hierarchical keywords that share the same parents. I hope that it can be fixed in future versions.

  • Managing Metadata

    Metadata is “data about data”. While the term is used to cover a wide variety of fields, the one that most interests me is metadata used in digital images.

    The most prevalent standard for digital image metadata is Exif, implemented in every digital camera. It’s metadata focused on technical information about the picture itself: camera shutter speed, exposure, etc. The standards of IPTC-IIM and XMP are focused on metadata about the subject of the photograph: topic, location, photographer, etc. IPTC-IIM is an earlier standard that has been in use by the press agencies since the 1980s – so there’s a large installed base. However, it’s not easily extensible. Enter XMP, which is XML-based, first created by Adobe Systems Inc. in 2001. In 2004, Adobe got together with the IPTC organisation and others to create the “IPTC Core” standard, an XML-based successor to IPTC-IIM.

    Of course, as the old joke has it, “the great thing about standards is that there are so many to choose from”. What is worse, is that different manufacturers (of cameras, application software, operating system software) interpret the standards in subtly different ways so that the resulting implementations are often incompatible with each other.

    This got to such a stage that in 2006, Microsoft proposed the setting up of a cross-industry working group to deal with the matter. As a result, the Metadata Working Group was created in 2007 by five founding members: Apple, Adobe, Canon, Microsoft and Nokia. In 2008, Sony joined the party. The goals are:

    • Preservation and seamless interoperability of digital image metadata.
    • Interoperability and availability to all applications, devices, and services.

    Not before time, I hear you cry. Certainly I get very frustrated by the current lack of interoperability that exists.

    Yesterday, at the annual Photokina exhibition, the Metadata Working Group released version 1.0 of their Guidelines for Handling Image Metadata. In the Group’s own words, “this document describes how best to use existing standards such as Exif, IPTC, and XMP to address the key organizational metadata questions that most consumers have”.

    I’ve only had time to quickly glance through the document, but it does seem to be a good effort to attempt to herd the bunch of cats that are software developers – many of them in their own companies.

    It was with some irony that I noted this paragraph (page 32):

    Hierarchical keywords are not covered. However it’s well understood that this is an important use case even in the context of the consumer and will be added to future versions of this document. There are existing solutions available e.g. Adobe Bridge, Adobe Lightroom as well as Microsoft Expression Media and Windows Live Photo Gallery that have introduced hierarchical keyword workflows specific to their needs.

    Irony, because it’s the way that Microsoft’s Windows Live Photo Gallery has currently implemented hierarchical keywords that is causing me some interoperability problems and frustration at the moment.

    Oh well, back to herding cats…

  • Managing Photo Libraries – Part 6

    Time for another episode in my saga of trying to manage my library of photos. This has been prompted by the release this week of the beta of what will be the next major release of Microsoft’s Windows Live Photo Gallery.

    While there’s a lot to like about Photo Gallery, it continues to have shortcomings that stop me from making much use of it. Some of them are down to bugs that Microsoft really should address, while others are limitations in its capabilities. But even here, at least one of the limitations could be removed by Microsoft as I’ll illustrate.

    First, the bugs. One that I find particularly irritating, because it’s been known about by Microsoft for nearly a year, but still has not been fixed in this new beta, is the “Publish to Flickr” bug. It’s probably even a very simple fix – a change to a single line of code would probably do it – so I fail to understand why this has not been done. Update: this bug was finally fixed by a release of WLPG on 16 December 2008.

    Another long-standing bug, but one that may be more tricky to fix, is that Windows Live Photo Gallery (WLPG) is aware of IPTC/XMP metadata tags that images may have, but seems to deal with them on a hit or miss basis. Sometimes WLPG will correctly read in the IPTC/XMP tags from an image file and add them to its own tag list, and sometimes it won’t. Here’s an illustration of this. The following image is a WLPG screenshot of twelve photos that have been tagged with IPTC/XMP metadata. I happen to have used IDimager, which is my current tool of choice at the heart of my digital workflow, but I could also have used Microsoft’s own Pro Photo Tools 2 to add IPTC/XMP metadata. (Note: IDimager is no longer available. Its successor is Photo Supreme, which I am now using)

    In this screenshot of WLPG, I have selected the first of the twelve images, and in the information panel on the right are the description tags associated with the image. I’ve highlighted the tags with a red box for clarity. Now, I did not use WLPG to add the tags to the image, they were automatically read in from the image by WLPG itself and added to WLPG’s list of tags.

    WLPG Wave 3 Beta 1 test 1

    That’s how it should work – WLPG should check images for IPTC/XMP metadata and use this to maintain its own tag hierarchy. But now look at this next screenshot. Here, I’ve selected the second image in the sequence of twelve. Remember that all twelve images have had IPTC/XMP metadata added to them, in fact they all have exactly the same metadata. But here, in this screenshot, WLPG is showing that there are no tags associated with this image, so it seems to have failed to read in the metadata from the image.

    WLPG Wave 3 Beta 1 test 2

    In fact, in this set of twelve images, all of which have the same metadata, WLPG failed on nine images, and only correctly read in the metadata on three (numbers 1, 10 and 12). While WLPG has a “Refresh” command, this doesn’t seem to have any effect on reading in metadata. It still stubbornly claims that nine of the images have no descriptive tags associated with them.

    This is a showstopper of a bug as far as I’m concerned. For me, the “truth is in the file” – in other words, the metadata describing an image file must be preserved in the file itself. That means that the IPTC/XMP and EXIF metadata is central for management of my photo library. Having metadata held outside the image files (as WLPG is doing in its own database) may be necessary for performance reasons, but the content must always reflect the metadata in the files themselves.

    A rather good analogy that I came across is this… Imagine that you have an album of family photos. It’s full of photos of members of your family stretching back several generations. Underneath each photo is a handwritten description of who is in the photo – that is the metadata for the photo. It’s a marvellous resource for you and your family – a record of your family history. But over time, the glue degrades, and the photos become loose. Worse, many of them become unstuck. What do you have then? A pile of loose photos, and an album with blanks where the photos should be – and no way of knowing which photo should go where. The metadata has become separated from the photos.

    What’s the solution? Well, what should have been done in the first place is to write on the back of every photo who is in the photo before sticking the photos in the album. In other words, the metadata should be directly associated with the photos themselves. Then it doesn’t matter if the photos fall out, the album can always be reconstructed. Indeed, a new album can be made when the old one falls apart.

    Anyway, back to WLPG. I mentioned limitations. One is in the photo import process (when you transfer off a batch of photos from a camera or memory card into the PC). While WLPG gives some options for creating folders and renaming the files as part of the process, they are nowhere near flexible enough to meet my requirements. I spelt out what I was looking for in part 5 of this series of posts. I also mentioned that Microsoft had claimed that the underlying import engine was flexible enough to do what I wanted. It’s a pity then that in this latest beta, the claimed flexibility has not been exposed. We still have the same old limited options that we had a year ago.

    There are a couple of new features in this beta of WLPG. The first is one that has long been asked for: the ability to select photos based on rating. I’m a little surprised that it’s taken so long to be included, given that it has always been in Vista’s Photo Gallery. Still, I suppose I shouldn’t be churlish.

    The second is a totally new feature: face recognition. You can tag faces in your photos with people tags. Interestingly, if you are using Windows Live Contacts, then your people tags are automatically populated with your contact list to start with, and changes to this list are reflected in WLPG. I still need to investigate this feature a bit more carefully. Unless these tags are written back to the individual image files as XMP metadata, so that they can be used in other applications, then I would not want to make much use of this feature. It would be too much like simply writing names underneath photos in an album. No guaranteed longevity there, then…

    So, all in all, one cheer for WLPG. But until the metadata bug in particular is addressed, I’m not going to be making much use of this application.

    Update 24 September 2008: I’ve done some more testing on the metadata bug, and I think what is happening is that WLPG is confused by hierarchical keyword metadata. In WLPG, you can have a long list of keywords/tags, or you can start to group them in a hierarchy. So, for example, my keyword cows is actually part of a hierarchy that starts Nature/Animals/livestock/cattle/dairy cattle/cows. That way, when I search for photos with the keyword cows, it will just show me those with cows in them. But if I search for photos with the keyword livestock, it will show me photos of cows, horses, pigs, sheep, and so on.

    It looks as though WLPG will recognise a hierarchy of IPTC/XMP keywords in new files, and uses it to add to its own hierarchy. However, thereafter it refuses to recognise any changes to the metadata of image files containing hierarchical metadata, and so ignores them. Not very useful, and what I consider to be a showstopper bug.

    Update 4 December 2008: here’s the blog entry where I track down this issue of hierarchical keywords causing a problem in WLPG…

  • Photosynth

    I mentioned a couple of days ago that Microsoft has released Photosynth. The site seems to be a bit overwhelmed by the demand, as my first experiments in creating Synths failed on the uploading stage. However, they seem to be there now. Here’s an example. 
     
    Note: if you want to explore it, the Photosynth applet will first have to be downloaded and installed into your browser.

    Addendum: And of course Microsoft has now scrapped the Photosynth product and technology, so none of these links work anymore. It’s dead, Jim.

  • Photosynth Released

    I see that Microsoft has now released the first version of Photosynth that allows you to use your own photos to create 3D spaces. I can’t wait to try it out…

    Addendum: And of course Microsoft has now scrapped the Photosynth product and technology, so none of these links work anymore. It’s dead, Jim.