Your important metadata lives in two, maybe three, places in your file; which one are you seeing?
Which instance of the IPTC metadata does your favorite application prefer? Inquiring minds want to know.
Let’s step back for a moment for some background. Because all things that should be dead simple usually aren’t, the IPTC metadata – important information like the caption, your byline, and copyright notice – is stored in multiple places in your file.
First, there’s the old-school IPTC/IIM version of the basic info.
(Some people mistakenly think this is all there is to IPTC metadata. Actually, the IPTC specifies what kind of information is stored in its “properties”. Physically storing the data in the file is a separate pile of acronyms.)
Then there are the extended IPTC fields, which live elsewhere in the file, in the newer Adobe XMP format. Copies of the basic fields live in the XMP, too. One day, the IIM format will be phased out and everything will be more or less under one roof in XMP.
It’s up to your metadata authoring application to keep the two copies of IPTC data in sync between the two formats.
Watch the video version of this post
Then there’s something in Exif…
On top of that, there are a few “IPTC-ish” fields that can be found in the Exif data, as well. The Exif format stores mostly logging information from your camera, like the camera’s serial number, settings used to make a photo, and GPS coordinates. Modern, upscale, cameras can write your byline and copyright information to fields in Exif.
Generally, while anybody can write Exif information to a file, it’s cameras that do so.
The Exif fields that applications might write to are for copyright notices, the image creator’s name, and the “Image Description” – a caption, in other words. (And time corrections. We’ll ignore those.) Some applications can read your copyright information, for example, and copy it to the relevant IPTC field. Some can copy your IPTC Caption to the Image Description field in Exif. “Can” being the operative word here. Applications can keep all this data in sync if they feel like it. Or not.
For the most part, this synchronization “just works” and you see the metadata that you should see.
On the other hand, sometimes it doesn’t. Denise May Levenick talks about adding metadata to heirloom family photos in this post on her Family Curator blog. “Metadata is a ‘funny’ thing.” she writes, “It doesn’t always ‘stick’ so that it’s visible in other applications.”, Levenick is obviously a conscientious person. But she and her readers were tripped up by some less-than-competent applications that don’t sync their metadata properly. (Some of which are examined later in this post.) So beware! Hazards lie in wait for real-world users.
Synchronization is largely a one-way affair. The application prefers one or another location and reads the data from there. When it comes time to write data, the application simply writes what it has at hand to all the places it can. We should remember that, in most combinations, these programs all “work”, particularly if you stick to the ones I recommend and respect their limitations.
We should remember that, in most combinations, these programs all “work”, particularly if you stick to the ones I recommend and respect their limitations.
Some applications give you some say in the matter. Photo Mechanic, for example, lets you choose which data to prefer, XMP or IIM, and which platforms to write to. Many applications allow you to decide how metadata is to be handled when there is a choice between embedding the data in the file or writing it to an XMP sidecar file. But, generally, applications simply write new data over any out-of-sync data that may exist. I don’t know of any program that alerts you to mismatched data. (But you can see if you look, in XnView or ExifTool.)
You’ve probably not spent a lot of time thinking about this…
Because of all that, and perhaps because you have a life, you’ve probably never stopped to think about what’s up with the multiple copies of your data living in your files.
And it’s worth noting that despite all the inconsistencies, most of the programs you might use for metadata purposes manage somehow to “just work”.
But you’re you and I’m…
When I was writing about how WordPress treats captions, I did stop and wonder.
So, I fired up ExifTool, the wonderful command line program by Phil Harvey that allows you to do pretty much anything you want to a photo’s metadata. I made a file with different captions in each of the possible locations. Then, I made files that each had one or two of the possibilities empty. I could then read and attempt to edit these files in different programs to see which metadata format each preferred and which way they would fall back if their favorite didn’t exist.
Here’s what I found:
WordPress only looks for a caption and a headline. It prefers the IIM version of the IPTC Caption. If that’s absent, it falls back to the Exif Image Description. WordPress doesn’t understand XMP at all. WordPress doesn’t write to embedded metadata, so that doesn’t apply.
More on WordPress and embedded metadata here, and here.
A little uncomfortable
Frankly, I’m not at all sure I’m comfortable with this business of relying on Exif data for any long-term purposes, especially since there’s no assurance that every application that has touched the photo makes a copy of its caption in that Exif field. It flies in the face of the concept of “single source of truth”, which is a bulwark of data integrity.
It’s bad enough that there are two copies of the actual IPTC data. But adding yet another player in the mix makes me kind of nervous. Yes, the apps (apart from a few outliers) I’ve tested don’t look to the Exif unless the data doesn’t exist in the normal places, so it’s only a fallback. But in twenty years of working with embedded metadata, I’ve never seen a single instance of an Exif Image Description coming to the rescue.
In the Metadata Working Group’s Guidelines there is considerable emphasis on the notion that some products may only have access to Exif data and it should be accommodated, and even preferred, if at all possible. What products they really had in mind, I Have no idea. Whatever product it was, I’ll bet one of the companies in the Working Group made it. (I wrote this paragraph before I had run the caption test on a second batch of applications. Now, I have a pretty good idea of who it was.)
Next, I looked at what happens when you choose ‘Get Info’ in the Mac’s file manager.
The Mac prefers IIM and falls back to Exif, just like WordPress. (I used OS X Yosemite. If this behavior has changed since, please let me know.) The Mac was not able to read XMP data.
Shockingly, in Photoshop, Adobe’s own metadata format is preferred last. Where there’s a choice, it will only look for XMP if neither IIM nor Exif is available. Go figure.
Photoshop: IIM > Exif > XMP. That finding was unsettling enough to me that I went back and checked. Three times. Yup, IIM > Exif > XMP. Photoshop wrote the test caption to all three platforms.
Reading the same XMP-vs-Exif file as Photoshop, Lightroom chose to read the XMP. So, Lightroom prefers IIM >XMP >Exif. Lightroom wrote to all three platforms.
See the Lightroom How-To here.
Behaved as Lightroom did: IIM > XMP > Exif. Bridge wrote to all three places.
The Adobe products were all able to read and write XMP data. I would hope so. Adobe developed the standard. And they display fields that only exist in XMP just fine. I’m still scratching my head about Photoshop’s behavior, though.
A break for some more background
I guess at this point, I guess I should mention that the Metadata Working Group Standard does actually advise developers to prefer Exif data where it and XMP both exist. There is a rationale offered for this behavior. However, I stand by my advice to webmasters that by the time an asset is ready to publish, the Exif data block is perfectly expendable and can be removed.
Another tidbit from the Working Group standard is that there is a checksum within the metadata that allows an application that feels like it (or conforms to the standard, to put it a bit more formally) to know whether the IPTC/IIM version of the basic IPTC data matches what’s stored in the XMP block. If it doesn’t match, the application is supposed to read the IIM data. If it does match, the application is supposed to read both the IIM and XMP and prefer the XMP version. (If the checksum doesn’t exist at all, the application is supposed to prefer the XMP. This bit seems inconsistent as all get out to me. (If anybody can explain it, please rescue me in the comments.)
The test files I used for this post obviously have mismatched IIM and XMP blocks. That means that the applications that prefer the IIM in my test may well present the XMP version of the data in the more normal case where the data in the two blocks do match. For our purposes, though, it doesn’t matter. If the information is the same, which version we see is irrelevant. (And do remember my guidance that publishers in today’s environment should leave both IIM and XMP data untouched.)
The standards – IPTC, Exif, XMP, and Metadata Working Group are all available here.
Back to the plot…
In its Info Panel, XnView reads and displays simultaneously all three versions of the caption on my test file.
In its metadata editor, it reads the only the XMP caption or the IIM one, whichever you have set the dialog to prefer. Go to the ‘Options” tab in the editor and select “XMP, update or create IPTC-IIM”, or “IPTC-IIM, update or create XMP”. DO NOT click the ‘Write’ button. Just click ‘Cancel’. When you reopen the editor (CMD/CTL+I) it will display whichever version of metadata you selected in the pulldown. In this dialog, XnView doesn’t read the Exif Image Description at all.
It writes to XMP and IIM (assuming you specified that it write to both in the Options tab), but remember it can’t write extended IPTC fields (which only live in XMP) at all.
That’s a bit confusing. But OK.
On its tooltips and slide mount labels, XnView will display either IIM or Exif, according to which variable you assign in Preferences >Browser, or Preferences>Thumbnail>Labels. XMP is not available as a variable.
That means that, if it matters to you, you can set a tooltip or slide mount label to display the IIM caption and the Exif one, and set the Edit IPTC-IIN/XMP dialog to prefer XMP. Then you’ll be able to quickly see if you have caption metadata out of sync. I don’t expect to use it much, but I saved a tooltip for this purpose.
See the XnView How-To here.
The superpower of metadata authoring programs allows considerable customization in its preferences. You can choose whether it prefers XMP over IIM, or vice versa. You have multiple options for RAW and other file types where sidecar files are applicable.
It always writes to XMP (whenever there’s an appropriate XMP field) and it allows you to choose whether or not to also write to IIM. And you should choose to do so!
Photo Mechanic does read from the Exif Copyright and Artist fields, but it doesn’t write to any Exif fields (at this time). Information in the semantically equivalent Exif fields will be written to the corresponding IPTC fields if the information is not changed in the PM interface. But information entered in Photo Mechanic won’t be written back to Exif fields. So, Photo Mechanic effects a one-way sync from Exif to IPTC.
The Photo Mechanic How-To is here.
Apple preview follows the lead of its operating system. It reads all of the basic IPTC fields. It prefers the IIM version of the caption and falls back to the Exif if the former isn’t present. Interestingly, it reads the Rights Usage terms Field, which lives in XMP, but it won’t read an XMP caption, even if the IIM and Exif are not present. (Cue the spooky music.) Preview doesn’t write metadata at all.
In Mac OS Sierra (10.12), Preview has gained the ability to read a useful subset of XMP fields. It still prefers IIM.
The world’s most annoying photo management program prefers IIM and falls back to Exif (like the Mac operating system’s Get Info function). In the caption test, it couldn’t read XMP at all. But it writes to all three formats. (By the way, if anyone knows how to delete or disable this program, please let me know. My efforts have all failed. It keeps coming back like a bad penny.)
By Mac OS Sierra, it appears that Photos has lost all metadata functionality except a limited grasp of Keywords. I can’t say I’m all broken up about that personally since I despise the program. But it really doesn’t seem like a good thing.
The Properties > Details tab in Windows 7 reads the Exif version of the caption and a small sampling of IPTC fields. It doesn’t read either IIM or XMP captions. I guess we found the reason for the Metadata Working Group’s fixation with Exif.
It turns out that Windows’ metadata behavior is more complicated – and flat-out bizarre – than I thought when I wrote this post.
Here’s the scoop: Windows 7 Properties dialog will display the IIM Caption in its “Title” field IF the IIM Object Name field is blank. If the IIM Object Name field has a value in it, then Windows displays the Object Name field’s contents in its “Title” field and will display the Exif caption (if it exists) in Windows’ “Subject” field. But if there’s no Exif caption, then “Subject” is left blank. If there’s an Exif caption, but no value in the IIM Object Name field, then Windows displays the Exif caption in BOTH its “Title” and “Subject” fields.
IIM Caption MIGHT appear in the “Title” field. Exif caption MIGHT appear in the “Subject” field and/or the “Title” field. And IIM Object Name MIGHT appear in either “Title” or “Subject”. Wow.
So, if metadata is written by an application (Photo Mechanic or XnView, for example) that doesn’t write to the three Exif fields that correspond to IPTC ones, Then Windows users will or won’t see the caption in their Properties dialog, according to whether there is or is not a value in the IIM Object Name field.
For the Creator and Copyright fields, the Windows Properties dialog prefers Exif and displays IIM if those fields have no values in the Exif.
If anyone knows a reason (or excuse) for this behavior, please speak up in the comments. Otherwise, I can only advise you not to trust metadata displayed in Windows. (Applications that run on Windows, like Photo Mechanic, or XnView, or members of the Adobe suite, can work just fine, of course.)
Windows appears to ignore XMP data altogether.
Worked the same way as Windows 7. The Properties dialog read the Exif Image Description, which it called “Subject”, but neither IIM nor XMP versions. The contents of the Object Name field appeared to be mapped to a “Title” field in the dialog.
Windows 10 Photos application
This worked the same way as the Properties dialog. Only the Exif version of the caption was revealed.
This means that IF you author your metadata in a product that writes the caption to the Exif Image Description field, it will be visible in the Windows 7 or 10 Properties dialog and Photos app. If you use something else – likely including many of the off-brand programs that are available only for Windows – you’re probably screwed.
This program features separate dialogs for Exif and IPTC/IIM metadata. IrfanView can read the IPTC/IIM caption in its pane and the Exif one in its. It can write to IIM only. It can neither read nor write to XMP.
If you use IrfanView, which is only available for Windows, this means that your captions won’t show up in the Windows ‘Properties’ dialog. And if you edit in IrfanView a caption that was originally written in a program that does copy the caption to the Exif Image Description (like the Adobe products, for example), you’ll end up with different captions in the Exif and IIM data. That’s not good. (XnView doesn’t sync Exif with IIM or XMP either, by the way.)
(On KDE, running on Fedora 25) – gThumb can only read metadata, and only a subset of the IPTC fields. In the caption test, it prefers IIM > XMP > Exif.
(5.4.0 on Fedora 25) – DigiKam is the open source world’s photo asset management application of choice. Think of it as a Lightroom without nondestructive editing. Its metadata capability is good enough to make it a practicable alternative. I use it to keep track of my own family pictures. DigiKam preferred XMP > Exif > IIM in the caption test. It writes to all three.
Linux users note that a Linux version of XnView is available, and is fully functional. But it’s not open source, if that’s a deal breaker for you.
What a bunch of chaos! We should remember that, in most combinations, these programs all “work”, particularly if you stick to the ones I recommend and respect their limitations. Looking under the hood may have been an exercise in frightening ourselves unnecessarily, like learning about an untreatable, but nonfatal, disease. On the other hand, maybe it’s better to know.
And I’ll have many blog post topics to write to try to pressure all those software developers to get their ducks in one row.
If you have an application that I didn’t try in my caption tests and you’d like to check it out, please drop me a line and I’ll send you some test files. As always, please let us know what you think in the comments.
Nice article, thank you. I have a question though about WordPress and metadata. When WordPress makes versions of your images at smaller sizes, does the metadata, like copyright and author, stay with the new images? Often it’s a smaller version used in a page, so it wouldn’t be good if those data were missing.
>so it wouldn’t be good if those data were missing.
Whether WordPress honors metadata on the various versions of the file create when you upload to Media Library depends on imaging library your server is using. By default, WordPress will use ImageMagick, if it is installed and activated. If that’s the case, you’re good to go. All your versions will have proper metadata. There’s a bunch of information on this in this post: https://www.carlseibert.com/wordpress-honors-metadata-sort-of/
If ImageMagick isn’t on your server (Like, say, here. We haven’t got it set up on the server that serves this site yet, sadly.) WordPress will use the GD library (that’s the name, not what I think about it) that ships with PHP. GD destroys metadata. Doesn’t do as well with images themselves, either. If you’re stuck with GD, there is a workaround, which is described in this post: https://www.carlseibert.com/wordpress-metadata-workaround/
Upcoming, I plan on doing a chart of hosting providers that support ImageMagick.
Interesting post. Thanks
Good post! I did a similar analysis for Windows Photo Gallery a while back – https://jmoliver.wordpress.com/2017/02/12/accessing-windows-photo-gallery-metadata-using-exiftool/
Good work! I took the liberty of saving a copy of your chart for future reference.
It’s a pity Microsoft appears to be making progress in the wrong direction.
When I was working on my Creative Commons posts, I came across several documents from about the time Adobe released XMP as an open standard that were all full of optimism about how everybody was going to migrate to XMP and it would all make sense – very soon – from that point in 2009 or so. Microsoft was mentioned. Of course, it hasn’t exactly worked out that way 🙂
Sure, it is a Google Docs document, I am still finding out some behaviors and revise it. Most recently I added some old Digital Image Library keyword tags which WPG also reads, and then, interestingly, moves them to xmp when edited.
For quite some time I have been digitizing all of my family’s photos, painstakingly cataloging and adding metadata as a hobby. I started using Microsoft Digital Image Suite’s Library app which later evolved into Windows Photo Gallery, which I still use today even if it is now unsupported and adding “People Tags”. I also use GeoSetter, which I highly recommend for geotagging and also adding other metadata. Among the things which I like about GeoSetter is that it uses exiftool and allows you to execute post editing exiftool commands which I use to ensure certain tags are in sync.
I post from time to time about my metadata struggles on my blog – https://jmoliver.wordpress.com
Now that I want to start a database for my pictures I came across your fine article. I just started to struggle in comparing all kind of programs. I need two things. A good metadata system and a good program to treat my ARW and CR2 RAW-files with the possibility of local editing. I prefer to combine those two wishes. For metadata I followed your advise to use DigiKam or XnView.
I found out that I can use Darktable for RAW editing. The problem is that it is very hard to let this programs interchange data with DigiKam. Darktable reads all the keywords, titles etc from Digikam, but after changing/removing a keyword in Darktable, Digikam doesn’t read that back.
In your comparison you didn’t mention Darktable which is an interesting program.
What do you think of this program and do you know a solution for this problem It can be one program that does both or a good combination of two programs. What about Capture One for example. Quite expencive for private (but serious) use, but cheaper then Lightroom
I’ve only briefly played with Darktable, so I’m of no help there.
Since you mention Capture One, I’m going to guess that you’re on Windows or Mac. In that case, I would commend ON1 Photo RAW to your consideration. It’s an all-in-one, like Lightroom or Capture One, but it has some advantages that I take seriously over either. By default, it does not depend on a central database for its non-destructive editing information. It handles that in sidecar files. That means that you can work on photos and archive them any way you want without having to worry about that database. ON1 is completely standards-compliant with metadata, and its metadata functionality is efficient to use. On the archiving side, you simply set watch folders and it catalogs anything that’s put in them (like the way DigiKam works). I like its RAW editing, too. It’s the program I actually use. I don’t usually use it alone. I do a lot of stuff in Photo Mechanic, which works perfectly with ON1. ON1 is inexpensive. Right now, it’s only $100 USD, and frequently they have it on sale for even less.
Thanks for your advise. Yes I am on Windows. I will try ON1. BTW I found out that Darktable also makes sidecar files like DigiKam. Although the XMP files have a different structure, there is an interchange of tags/labels between the XMP files of Darktable and DigiKam. You can find the tags made in DigiKam in the XMP files of Darktable and vice versa. But Darkroom itself doesn’t show up with the tags and titles made in DigiKam.
Both DigiKam and Darktable have problems with connecting my Sony a7III.
Darktable and camera work reasonable with tethering but not with bringing files direct from camera to the program/computer.
I followed your suggestion and tried ON1. It was worthwile a try. ON1 does a perfect job considering metadata and has some cool editing features. The problem with ON 1 is that it is to slow in editing and uses to much computer capacity. The CPU is used often for about 95 % while the GPU is used max 25 %. At this moment I prefer other programs for editing in combination with Digikam or XnView for searching. I hope ON1 will develop soon as a perfect solution. It is promising but not yet there.
Indeed, ON1 is not the fastest bunny in the race. I find it exports significantly slower than Lightroom and (along with every other program, WAY slower than Capture One.) Although, frankly, the slow export hasn’t really bothered me. I don’t see it hogging machine resources on my laptop. Right after starting the program, while it is updating my cataloged folders (of which I have a fair number), it will make the fans spin fast. But it that scan is a low priority process and it doesn’t seem to affect anything, including ON1 itself. Good news is that they make it faster with each release. So, maybe someday….
Thanks for the informative post!
I am writing a command line program to quickly view and delete images
within a terminal window. I wanted to put in the ability to edit
comments, but had no idea what a can of worms I was opening. After
reading the IPTC recommendations I was even more confounded —
“headlines” sound more like what I call “titles” and “titles” are what
I’d call “filenames” — but this blog post did help, somewhat.
I do have a couple questions (for you or your knowledgable readers):
1) What is your suggested order for reading metadata? I was going to
go with XMP > IIM/IPTC > EXIF, but it seems like I’d be in the
minority. And, since some programs you mentioned cannot actually
write XMP, it seems like the EXIF or IPTC data has a better chance
of being current.
2) Which tags should I make it easy for people to edit and what should
I call them? “Caption” (AKA “Description”) seems important, but
what about “Comment”, “Title”, “Headline”, “Subject”, “Label”, etc?
Which ones do people actually use? And how do the tag names people
are familiar with map to the underlying XMP/IPTC attributes?
Thanks for reading!
For a reading order, I usually recommend a simple XMP > IIM > Exif. The machinations in the Metadata Workling Group guidelines of 2010, in addition to being complicated, were predicated on assumptions that don’t make sense anymore. (To me, anyway.) Back then, it was assumed that the world would move quickly to XMP and IIM would soon be deprecated. Hasn’t happened. In that world, it was assumed that rogue, out of sync, data would generally be written by an archaic IIM-only-writing program. In today’s world, out-of-sync data is just as likely to be written in the XMP. Exif has always been a political football.
My logic is that XMP doesn’t pose tag length restrictions and is thus more likely to be authoritative than the possibly truncated IIM. If any descriptive data exists in the Exif, it was probably put there by the camera, would most likely have been superseded by, or transferred to, the IPTC fields. If it’s out of sync, it’s pretty likely NOT to be as authoritative as the more recently written IPTC. Few programs can read it anyway. It’s just not where descriptive data is SUPPOSED to be.
I have a post here describing all the fields and, IIRC, their many and various names through the years. I have to confess I don’t do much to help the confusion. I often refer to fields by their traditional names, rather than their proper ones.
(It also doesn’t help that ExifTool uses it’s own naming convention, with IIM fields usually called by their old school names and XMP ones by the new names. The fields, of course, are the very same fields. The field is one entity in the schema; it’s just written in duplicate.
Fields that I think matter: Caption/Description is obvious. As are the Creator and Copyright fields.
(Google Images supports Creator, Copyright, and Credit, BTW. Credit is poorly understood and rarely used outside of the publishing and stock photo industries. That Google chose to include it is a testimony to the fact that they were, shall we say “urged” to do the right thing by a pending lawsuit by a company entrenched in both of those industries.)
Keywords is important, too. I would avoid dallying with the special Lightroom hierarchical keywords field. Just the normal ones will do fine. It’s very important to always put keywords in the standard IPTC place. There are unfortunately programs out there that write them to idiosyncratic places, causing tragic data losses when people migrate away from them.
Title/Object Name and Headline are interesting in that they are used differently by different organizations. Headline is supposed to be just exactly that. Object Name is supposed to be a “slug”, a human-readable not necessarily unique identifier for the image. But usage in the real world is all over the place. I doubt most individual photographers use either one of them.
“Subject” is the XMP place for keywords. It is synchronized with the IIM “Keywords” field. The pair are always labeled “Keywords”.
Rating and Label are the fields that carry the star rating and color label fields that most GUI programs use. Both live in the XMP. Image Rating is an official IPTC field and its use is standardized. Label is not part of the standard and its use is all over the place. I recommend using the Adobe Lightroom color set as it’s something of a lowest common denominator. (I covered labels in, I think, the Photo Mechanic 6 post or video.)
Comments is actually not an IPTC field. It’s in the Exif and is not terribly widely supported. It makes no sense, frankly. The Exif is generally written to by the camera and how is a camera going to comment on anything? The closest thing to a real comments field is Special Instructions. (Which, refreshingly, is never called anything else.)
Now that I’ve brought it up, there are three Exif fields that are semantically equivalent to IPTC ones, “Artist”, which is Creator by yet another name, Copyright, and Description. Caption/description is useless because cameras can’t write anything useful there. Cameras can do just fine with Copyright and Creator because those are standing values, the same for every picture. But information in those fields doesn’t do much good unless it is later transferred to the proper IPTC fields.
The IPTC standard has the current official names for all of the fields and you can glean from it which fields are synchronized between the IIM and XMP. (There’s actually a chart in the standard. But it omits a couple of obscure fields that are actually synchronized, so it isn’t as authoritative as poring over the field descriptions themselves.)
There is a link here to download the current IPTC standard. Just use the search function.
And I would be happy to check over your work if you’d like.
I’m embarking on a project to scan our family’s old and sizable slide collection, and your remark about “tag length restrictions” made my blood run cold. My plan had been to write metadata about scanned slides to the three description fields (Exif Image-Description, IIM Caption-Abstract, XMP dc:Description) in a bid to (1) ensure consistency between these fields and (2) make the information visible via as many programs as possible (hello, Windows’ File Explorer). I don’t anticipate writing giant amounts of information per slide, but the thought of length restrictions is troubling, and the remark at https://support.captureone.com/hc/en-us/articles/360003412157-XMP-and-IPTC that IPTC fields are limited to 32 characters is positively horrifying. Your mentioning of IPTC field truncation makes it worse. I’m now concerned that if I write a description longer than 32 characters, it will be truncated in the IPTC Caption-Abstract, but not in the Exif or XMP description fields, thus giving rise to the inconsistency writing to all three fields is intended to prevent.
I’d be grateful for any information you can provide regarding tag length restrictions on these description fields, and I’d very much welcome your advice regarding the best way to represent general free-form descriptive metadata for scanned images such that the information is likely to be viewable (in its entirety) by as many people using as many programs on as many platforms as possible. The thing about scanning family slides is that the resulting images are likely to be distributed to lots of different people with various levels of technical sophistication who will view the images in a variety of different computing environments. I want to maximize the likelihood that what I do will work for everybody, both now and in the future.
Unrelatedly, thank you *very* much for publishing the original article. It represents a huge amount of work (as does the writeup itself). It’s enormously helpful to those of us grappling to get a handle on the practical realities of working with this kind of metadata.
Actually, the character limit for the caption/description in the IIM is 2,000 characters. So, in any but the most extreme cases, you should be OK. Different IIM fields have different character limits. Some are indeed 32 characters.
Your concern is well-placed though. Character limits aren’t the only thing that can bite when reading back text fields from the IIM or Exif. XMP effectively has no character limits and has good character set support, which is particularly important in captions, since many names have diacritics and ASCII doesn’t do diacritics. Exif, I should point out, doesn’t have set character limits – although authoring software might – and some Exif fields can support more than ASCII, but this one doesn’t (Pretty much everything about Exif sets my teeth on edge.)
All of this is why I advise any software developer who asks to prefer – for fidelity’s sake – to read XMP and then go to IIM and Exif, in that order, if the XMP doesn’t exist.
In order to populate all three fields, you’ll need metadata authoring software that will write to all three fields. Of those that I cover, ON1 Photo RAW, Lightroom Classic, and the Adobe products that use Photoshop’s metadata functionality (Photoshop, Bridge, Illustrator, and some others) will all write your caption to all three places.
Photo Mechanic will (generally) write to the Exif field that’s semantically equivalent to an IPTC one only if Photo Mechanic finds that field already populated. Why, you might ask? Well, it’s a bit complicated. There’s a general reluctance to write stuff into logging data, which Exif is. That’s one thing. But there’s data in the Exif that might be incorrect and might need to be fixed. Think the Create Date, or Date/Time Original as it’s known in Exif. Or the Creator and Copywrite fields. All of which can be written by the camera and can be wrong, meaning that a program like Photo Mechanic is obliged to overwrite those fields to keep the file correct and in sync. Every developer resolves the “Exif conflict” differently. Photo Mechanic tries not to fix it if it ain’t broke.
This matters for you because the Exif Image Description field is a super-prime candidate for getting out of sync down the road. With some software reading it or writing to it and some software not, it’s easy to see what might well happen if your captions are edited in the future.
Thank you for your well-founded comment and especially for caring about your photos’ legacy. It’s people like you who motivate me to do this.
Doesn’t look like he ever replied to, or possibly even read your thorough and authoritative response – but I did, and wanted to thank you for putting out all this very useful information, that I’m sure took many years to collect and analyze.
You make it look obvious, simple even, in a way that only a true subject expert can master. Thanks for sharing this knowledge and your expert advice!
I just copied the Photos library to my external Archive drive, then I deleted the default macOS Photos library, and made the new one my default in the Photos app. So, now only the pictures I take with my iPhone are transferred to the hard drive + backed up.
I found this to be the easiest/best way, and not take up any space on my machine (because I don’t use Photos on my Mac at all), but at the same time, I get my pictures from the phone archived and a good backup.
Regarding your comments about Apple Preview, I just did some tests using the most recent version of MacOS, and at least some XMP fields are now read if Exif and IIM information is missing. So Apple Preview now seems to be IIM > Exif > XMP. Shockingly, my tests indicate that this is regardless of the tab you’re looking at in the Preview Inspector. So if all of Exif, IIM, and XMP metadata are in the file, you’ll see IIM metadata on the Exif and TIFF tabs, not just on the IPTC tab. If IPTC and Exif metadata is missing, but XMP is present, you’ll see XMP field values on the Exif and IPTC tabs (as well as the TIFF tab). I don’t know what the TIFF tab is supposed to show, anyway, but since all metadata in all tabs seems to follow IIM > Exif > XMP, the different tab labels seem to be purely cosmetic.
If you have time to run some tests of your own, I’d be interested to know if you can reproduce the above. I was really shaken to see IPTC information on the Exif tab.
Very cool. Thank you for doing that. This is great news (I think) as far as some support for XMP data is concerned. The weirdness of things popping up here, there, and everywhere is a bit troubling. But a small step forward is still a step forward.
I have been massively busy lately and I haven’t devoted as much attention to the blog as I should. I need to go back and refresh this whole post, looking for developments like the one you have so kindly documented.
It’s interesting that you focus on the XMP support, because I can’t get past a tab labeled “Exif” that shows IPTC metadata if the file has IPTC metadata in it. If everything is in sync, this is harmless, but if you’re looking at the metadata to check to see *whether* the IPTC and Exif info are consistent, you’ll come away thinking they are, even if they aren’t. Ditto if the file has only XMP data, because you’ll then be presented with tabs for Exif and IPTC claiming to show metadata that is not actually in the file.
I can’t forgive an app that lies, and Apple’s Preview does when metadata in Exif, IPTC, and XMP are not consistent. That XMP can now be part of the lie is hardly consolation.
“Unforgivable” is indeed a good word for it. And, honestly, this stuff isn’t that hard. If teeny little companies can get it right, the world’s most “valuable” corporation should be able to get it right.
You can check to see if a file is in sync with the IPTC’s online tool at getpmd.iptc.org (There’s a link below in the footer.) Out-of-sync files are a real problem. I don’t know of any programs that people would be expected to use to work with their files that check to see if the file is in sync. Yes, a good program like Photo Mechanic will write your edits in sync. But if the file was out of whack, you’ll never know what you just wrote over.