PGTS PGTS Pty. Ltd.   ACN: 007 008 568               Mobile Version Coming Soon

point Site Navigation







Valid HTML 4.01!






   Stop Spam! Stop Viruses!
   Secure And Reliable Ubuntu Desktop!

   Ubuntu

   If you own a netbook/laptop~
   Download Ubuntu Netbook!





ADJT - A Perl Utility Script To Adjust Timestamps In EXIF Tags

By Gerry Patterson

My collection of digital photos is growing.

Finally, threatened with an avalanche of digital images, I have turned to an off-the-shelf photo management software. Only to discover that most of the common packages expect accurate information from the timestamps in the EXIF META tags inside JPEG images. With almost fifteen hundred photographs, I discovered that there was a problem with the date information in my photos.

It might be possible to fix them but the process that did it would have to be a batch process.

So I wrote this script.

My First Digital Camera.

Initially, the only digital photos I had were the ones that came back with prints when we had family photos processed. I would store them on CDs. If the company that developed them supplied labels, I'd write on them and put them in a plastic case. As is usually the case, this was started with the best of intentions. Well labeled and ordered chronologically. But as different disks started to come from different sources, and as various things mounted up on my desk, this catalogue became increasingly untidy.

Eventually we stopped ordering prints altogether. When we sent the film off to be developed we just asked for digital images. After about ten years there were about sixty CDs with digitised photos on them.

Then I purchased a digital camera. It was a Pentax Optio 60. After this, the number of digital photos began to grow more rapidly. I would just put them in folders on my hard drive. Each folder would be named after the date I moved the pictures from the camera. I didn't use any software to manage the photos, except gimp.

As the collections became larger, I started to experiment with F-spot. There were number of features that I liked, but I noticed that F-spot organised my photos in a strange way. It seemed to be sorting them by date. But it wasn't the date that was on the file, it was some internal date in the photo. I didn't bother looking into it much. I continued to file the photos in the same way. If I needed to re-touch a photo (e.g. remove "red-eye"), I would use gimp. I did notice a red-eye reduction icon in F-spot, but I could not figure out how to use it.

Then I started to use a Mac and I encountered iPhoto.

I discovered iPhoto had some features that I really liked. It could do most of the things that I used to do when I massaged photos with gimp, but it was simpler and quicker to use. There are many one-click controls, that can adjust exposure, brightness, etc. There are some very good effects, and best of all a drop-dead simple, one-click red-eye removal, that is almost as good as the gimp red-eye removal. Occasionally the iPhoto red-eye removal goes feral. But most of the time it just works, and when it does, it does it so much quicker than gimp that I find it hard to go back to the old method. With gimp, each red-eye adjustment can take between 5 and 20 seconds. A really difficult one can take as long as a minute. But in iPhoto this can be accomplished with a single mouse-click.

My First Second Camera.

Then we got a new camera. It was a Nikon D40X. My photographic skills are minimal. But I only have to hold this camera and I feel like a pro. The SLR view finder is what I am accustomed to seeing when I use a camera. The camera does not have the usual "delay" that I have experienced with other digital cameras. It is very responsive and the auto mode is so good that even a photographic dunce like myself can just point and shoot, and the results look good.

The little Pentax is still in use, but mainly by our kids. There were a large number of photos from the old Pentax, that I had not sorted through. A lot of these might not be worth saving. A lot of them could be touched up with iPhoto's great tools (especially one-click red-eye removal). When I tried bringing them into iPhoto, I ran into the same problem I had encountered earlier with F-spot. The photos were all over the place and no longer in sequence.

Now I was getting annoyed.

I had almost 15 hundred photos taken with the first digital camera, that could not be imported in sequence into F-spot or iPhoto. And if I could get them in the correct sequence, they would show the wrong date.

The Future Looks Digital.

As it turns out, the Pentax Optio that I purchased seems lose its' clock settings quite easily. Some digital equipment (like the Nikon DX40) have a reserve power supply, so that when you disconnect the main battery, they can still remember who, when and where they are. The Pentax however, loses its' tiny mind in a matter of seconds. At first I didn't realise this was a problem. After I discovered this I would try to remember to the clock whenever I changed the battery. But sometimes I would forget to. Especially if the battery failed while I was trying to take a photograph. And on a number of occasions, in my haste, I set the clock incorrectly. This was why the internal dates on the photos, were incorrect. In any case, at the time, I did not realise that the internal dates were important. I thought I could just use the modification times on the files ... I had some vague idea about using "touch" to set the file modification times. It was only after I had a large number of photos that I discovered that the internal dates were important.

It seems that after closing the base of the camera, the date will usually stay in memory for a month or so. At least until the battery got low again. Also the clock did not go all the way to a base value, sometimes it would just go back two years or so. So there might be as many as hundreds photos with a sequential timestamp. If I could adjust each batch with an offset, then the whole collection would be consistent. For example I might have a hundred photos, starting on 2005-07-01 00:00:00, and in the middle of this group would be one with an internal timestamp of 2005-07-19 21:40:06, and someone is holding up something with a date on it, or it was an event that I know occurred on known date. Let's suppose that I know for a fact that it is the date of 2007-09-24 at about 19:30. All I have to do is add the offset for that timestamp to the entire group of a hundred photos and the timestamp would be correct.

If I could adjust those internal dates, I would be able to manage them with iPhoto (or F-spot for that matter).

Of course, now that I know how important it is. I will be diligent about checking the date and time on any digital camera that I use.

RenRot A Batch Utility That Renames And Rotates JPEG Files.

I started looking for a program that could adjust the dates. It would have to be a batch process. I had no intention of going through almost fifteen hundred photos and adjusting the date on each one. I found a program called renrot (see the bibliography for details of where to obtain it). For those of you who have Ubuntu systems, you can install renrot with this command 

	sudo apt-get install renrot

It turns out that renrot is a perl script. This handy little script will rename and rotate image files. There are a number of options. It uses a config file to store default options and it is built for the command line. It does have an annoying default behaviour of coloured screen output. The colour can be turned off with the "no-color" option. The "no-color" option can be specified in the file ~/.renrotrc with the following command: 

	use color=0

However the "no-color" setting is bypassed by the help screen, or any process that seems come before the image processing section of the script. So one of the first things I did was modify the renrot script. At the top of the script the defaults for the hash %cfgOpts are set. I changed this line: 

	'use color' => 1,
to this: 
	'use color' => 0,
On the version, of renrot that I had installed (version 0.25), this was line 76.

While I was examining the code to work out how to alter it, I found a very interesting module, Image::ExifTool, which manipulates the EXIF tags inside image files, and is the key to how renrot works. The list of manufacturers that ExifTool supports is impressive indeed. Both my cameras were there.

Surely, I reasoned, it would not be difficult to write one myself?

ExifTool - A Perl Module That Reads And Writes EXIF Tags.

A little bit of experimenting with the powerful little module, ExifTool, and the utility exiftool, revealed that the ExifTool module returns date stamps from image meta data in the POSIX format "%Y:%m:%d %H:%M:%S". I found references to two tags, DateTimeOriginal and CreateDate. As far as I could tell these dates seemed equivalent. Any photo I looked at in my digital collection would have both these dates set to the same value. In iPhoto you can view these values in the Info panel by choosing Photos->Get Info. In Ubuntu you can view these values by looking at the File Properties in the Nautilus file viewer. You will have to scroll down to get them however. Nautilus shows the File Modification dates first.

The EXIF tags of interest are:

Also, when I read the ExifTool man page I discovered that there is a Shift method for the procedure that sets the value of tags containing date information. You can find this by entering the command "man Image::ExifTool" or "perldoc Image::ExifTool". I read the code in Shift.pl and came to the conclusion that it would be best to use an offset in days with the fractional part expressed as HH:MM:SS.

One way to alter the datestamps would be with this command: 

	exiftool -CreateDate+="d hh:mm:ss" -DateTimeOriginal+="d hh:mm:ss" *.jpg

This would adjust the dates by the amount specified (as days, hours, minutes and seconds). There were several drawbacks to using this command on a large number of files:

So I decided that the first thing I would need would be a script that worked out the difference between two dates. Let's return to the example above. I have two dates.

  1. 2005:07:19 21:40:06, The date stamp in the DateTimeOriginal and CreateDate tags.
  2. 2007:09:24 19:32:06, The time stamp that should be in the tags (I don't really know about the minutes and seconds, but I am certain that it that it was around 19:30).

It's a bit clunky, but this script should work out the difference in days, hours, minutes and seconds:
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;

die "Wrong number of arguments" unless (@ARGV == 2);
my ($d1,$d2) = (@ARGV);
($d2,$d1) = (@ARGV) if ($d1 gt $d2);
$d1 =~ s/ /:/;
my @d1 = split(/:/,$d1);
$d2 =~ s/ /:/;
my @d2 = split(/:/,$d2);
# adjust for month (Jan == 0)
$d1[1] -= 1;
$d2[1] -= 1;
die "Bad date: $ARGV[0]" unless (@d1 == 6);
die "Bad date: $ARGV[1]" unless (@d2 == 6);
my $diff = timelocal(reverse @d2) - timelocal(reverse @d1);
my $d = int($diff/86400);
my $h = int ( ($diff - ($d * 86400) ) / 3600 );
my $m = int ( ($diff - ($d * 86400) - ($h * 3600) ) / 60);
my $s = $diff - ($d * 86400) - ($h * 3600) - ($m * 60);
printf "%d %02d:%02d:%02d\n",$d,$h,$m,$s;

There is probably a much more elegant way of doing this. But this will do for now. If we call this script "diff_date" and put it in our BIN folder, it can be invoked with the following command: 
cmd> diff_date "2005:07:19 21:40:06" "2007:09:24 19:32:06"
796 21:52:00

Developing A Perl Script That Uses ExifTool.

The next thing I needed was the script that would actually do the adjustment. Apart from doing the adjustment, it will also need to give information on the files. The Nikon stores files in the following format: 

	DSC_nnnn.JPG
	where: nnnn is a number e.g. DSC_0102.JPG

Each number is incremented sequentially. When this photo is moved into the iPhoto library it would be renamed to DSC_0102.jpg.

The Pentax stores its' images in a similar format, i.e.: 

	IMGPnnnn.JPG
	where: nnnn is a number e.g. IMGP0102.JPG

So, if I took some raw camera files from the Pentax, I would like to run these through the script to get a sequential list of the dates. The first cut of this script looked like this:
#!/usr/bin/perl
use strict;
use warnings;
use Image::ExifTool;
use Getopt::Std;
use POSIX;

getopt ('ad');
our ($opt_a, $opt_d, $opt_i);
my $mod_time = POSIX::strftime("%Y:%m:%d %H:%M:%S",localtime);

my %extr_tag = (
        DateTimeOriginal => 1,
        CreateDate       => 2,
        ModifyDate       => 3,
        FileModifyDate   => 4 );

foreach my $file (@ARGV) {
        next unless (-f $file);
        my $exifTool = new Image::ExifTool;
        my %DateInfo;
        foreach (keys %extr_tag) {$DateInfo{$_} = ""}
        $exifTool->Options(Unknown => 1);
        my $info = $exifTool->ImageInfo($file);
        my $group = '';
        my $tag;
        foreach $tag (keys %extr_tag) {
                $DateInfo{$tag} = $info->{$tag} if ($info->{$tag});
        }
        if ( $opt_i) {
                printf $file . "\t";
                foreach (sort {$extr_tag{$a} <=> $extr_tag{$b} }
                        (keys %extr_tag) ) { printf "%-20.20s",$DateInfo{$_} }
                print "\n";
                next;
        }
        die "must specify -a and -d option" unless ( $opt_a && $opt_d);
        die "-d must be '-1' or '+1'" unless ($opt_d eq "-1" || $opt_d eq "+1");
        die "Missing tag: DateTimeOriginal in $file" unless (defined $info->{DateTimeOriginal});
        $exifTool->SetNewValue(DateTimeOriginal => $opt_a,Shift => $opt_d);
        die "Missing tag: CreateDate in $file" unless (defined $info->{CreateDate});
        $exifTool->SetNewValue(ModifyDate => $mod_time) if (defined $info->{ModifyDate});
        die "Cannot write file" unless $exifTool->WriteInfo($file,"$file.new");
        rename ("$file.new",$file);
}

The script above has two modes;

  1. Query mode. This is invoked with the -i switch. So if you enter: 
    	adjt -i *.JPG
    You will see output like this: 
    IMGP1119.JPG    2005:07:01 00:00:54 2005:07:01 00:00:54 2007:11:08 03:11:46 2007:11:08 03:11:46
IMGP1120.JPG    2005:07:01 00:01:36 2005:07:01 00:01:36 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1121.JPG    2005:07:01 00:02:15 2005:07:01 00:02:15 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1122.JPG    2005:07:01 00:02:59 2005:07:01 00:02:59 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1123.JPG    2005:07:01 00:03:19 2005:07:01 00:03:19 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1124.JPG    2005:07:01 00:03:41 2005:07:01 00:03:41 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1125.JPG    2005:07:01 00:03:51 2005:07:01 00:03:51 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1126.JPG    2005:07:02 02:05:33 2005:07:02 02:05:33 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1127.JPG    2005:07:02 02:05:44 2005:07:02 02:05:44 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1128.JPG    2005:07:02 02:05:56 2005:07:02 02:05:56 2007:11:08 03:11:47 2007:11:08 03:11:47
IMGP1129.JPG    2005:07:02 02:06:12 2005:07:02 02:06:12 2007:11:08 03:11:48 2007:11:08 03:11:48
IMGP1130.JPG    2005:07:02 07:17:08 2005:07:02 07:17:08 2007:11:08 03:11:48 2007:11:08 03:11:48
IMGP1131.JPG    2005:07:02 07:32:19 2005:07:02 07:32:19 2007:11:08 03:11:48 2007:11:08 03:11:48
IMGP1132.JPG    2005:07:02 23:50:55 2005:07:02 23:50:55 2007:11:08 03:11:48 2007:11:08 03:11:48
IMGP1133.JPG    2005:07:02 23:51:05 2005:07:02 23:51:05 2007:11:08 03:11:49 2007:11:08 03:11:49
  2. Adjust mode. In this mode both the -a switch and the -d switch must be specified. The -a switch gives the adjustment as: "d HH:MM:SS" and the -d switch must be "-1" or "+1". So in order to adjust the clock for a group of files this would be the command: 
    	adjt -a "796 21:52:00" -d "+1" IMGP118[6-9]* IMGP119* IMGP12* IMGP13[0-5]* IMGP136[01]*
    This would add the adjustment to the DateTimeOriginal and CreateDate tags. It would operate on only the files in the specified range. On the other hand if you wanted to subtract one year from a group of files (because you pressed the down arrow too many times whilst setting the clock), this would do it: 
    	adjt -a "1:00:00 00:00:00" -d "-1" IMGP11[4-9]*.JPG

I had only used this script a few times when I realised that there were several improvements I could make. For instance it would be more useful useful if it included an upper and lower limit for the operations, since I was usually operating on a range of sequential files. For example if all my files were in a particular folder, and I wanted to adjust the range from IMGP1142.JPG to IMGP1197.JPG, it would be easiest to include an -l switch (for lower bound and -u switch for upper bound). So then I could use something like: 

	adjt -l IMGP1142.JPG -u IMGP1197.JPG -a "1:00:00 00:00:00" -d "-1" *.JPG
And this would adjust all the files from IMGP1142.JPG to IMGP1197.JPG (inclusive).

Since most cameras store these files sequentially and one is usually trying to adjust a range, this saves all that time trying to figure out regular expressions. Another handy feature might be the addition of a -m switch, which would then use the file modification time as the time to write into the Original Date and Creation Date tags. And also it might be handy to add a -c switch to use a constant value (specified with the -c switch).

And while I was about it if I added the code from diff_date into the adjt script, the one script would be able to carry out all of all these functions. I could call it the -C switch which would be used to calculate the difference between two dates. When the -C switch was used it would expect two arguments in the POSIX format "%Y:%m:%d %H:%M:%S". Since it contains a space, the arguments must be enclosed in quotes. Perhaps I could even put these into an environment variable for future operations?

By this stage my little script had grown to be a small utility. So I decided to formalise the process.

ADJT - A Utility Script That Adjusts The Timestamps In JPEG Files.

Note: For instructions on how to use adjt see the ADJT manpage. Or you can view the ADJT perl code.

Although the script above will do the job. It has a number of limitations, that became apparent when I started to process large numbers of files. The final version is a much better package. If you would like to try it out, you can download a tarball from this site (see below).

As of writing there are no conditions on copying this script. However, I may add a GPL to it at a later date, if it proves to be popular. For the time being however adjt is, like all the scripts on this site, available for copying to do whatever you wish, provided that your enterprises do not include spamming or other illegal activities.

I added a usage() subroutine to the script, which prints out a help screen and I added some documentation. At first, I tried to use the utility help2man. The help2man program looks as though it could be useful. However, I had not included support for long options in the adjt script. Although the help2man documentation claims that it can still process the output from executables that do not support long versions, I couldn't get it to work. This may be because help2man works best with binary executables. I didn't pursue it further, although I found a few links that might be useful (see the bibliography).

I decided to create the documentation by copying an existing man page. the latest Version of the man page for adjt can be viewed here.

If you want to view the final version of the adjt script you can see it here.

I created a tarball of the package here. It uses a very simplistic install method that is unorthodox, but should work in most cases. If it doesn't work the script will have to be installed manually. It's just a perl script. So you can install it to your local library.

ADJT - INSTALLATION:

Here are the instructions for installing the adjt script, on platforms that I have tested:

ADJT Version 1.0

Adjt can be useful for adjusting dates inside image files, especially if your camera had the wrong date settings. Already I can see there are several enhancements that will probably be included in a later version. For example:

ADJT Version 2.0

Since writing the above, I have returned to examining F-spot, and have concluded that it can perform most of the tasks that iPhoto does. For instance it allows you to easily sort by folder and name and to adjust tags. It is also possible to adjust the original date and the digitised date. There is also a very handy red-eye reduction. In order to get to work you must click on the edit image, and then select a small rectangle around the eyes and then click the red-eye reduction tool. Sometimes the F-spotred-eye reduction algorithm works for cases where the Apple software fails.

Most of the adjustments that iPhoto has are also available in F-spot. They just weren't as obvious and easy to use. But although they don't look as "slick", they work just as well. In some case they even work better.

Probably if I'd sorted the photos by folder initially in F-spot, I may have been able to adjust the timestamps using the tag adjustment tool. This would have been a huge task however, because there were many hundreds of images. Once F-spot had imported the files with wrong date in the wrong folders, it was difficult to "unbake the cake". Although it is may not be as easy to use as iPhoto, there are many features about F-spot which make it a serious competitor with the Apple product. Not least of all the fact that it is Open Source. As all the various packages that ship with Ubuntu become more integrated, F-spot will only improve.

I tried using the adjt perl script to adjust the dates on analog photos, and soon came to the conclusion that I should add the capability to run interactively. Although we tend to use the digital cameras most often, there are a large number of digitised analog photos.

So, before I could finish version 1.1, it was already looking more like 2.0.

Version 2.0 of the script will run interactively and step though a roll of digitised film. As mentioned above, the Cannon EOS 500 winds the film on entirely and so when a lab processes the film they come back in reverse order. The Cannon still takes good photographs and since we would not get a very good price for it, if we sold it, we have kept it. It still gets used sometimes. And at the risk of sounding a bit too nostalgic I have to admit that, with some good film, the EOS 500 can take a photograph that has that old-school analog warmth.

I have also discovered that the Pentax date problem may have something to do with the Pentax firmware.

I will be describing version 2.0 next month.

BIBLIOGRAPHY:

The following are some links which might prove useful.

Linux renrot

The renrot man page (for Linux). RenRot is a perl script which you might find handy if you need to process a batch of files. It can also serve as an introduction to the Image::ExifTool module. RenRot will rename your files to have a name formed from the timestamp. It can also rotate the images. It's only one of the many examples of how Open Source programming is changing the face of global computing.

Ubuntu renrot

The renrot package (for Ubuntu). This is the Ubuntu page with details of the Ubuntu package for RenRot. An easy way to install it is to use aptitude. I did not see a package for the Mac. However, I was able to get it to work on the Mac, by installing the ExifTool module and extracting a copy of the renrot perl script from the Ubuntu package. You can use the man page from the link above. Also renrot does have a help screen.

Joe Brockmeier

Creating your own man pages. A simple summary of how to create your own man pages. Or you may prefer the more comprehensive Linux Man Page HowTo

help2man

GNU help2man. Command line utility for converting help text to man pages. This looks useful also. I could not get it to work with adjt. It seems to work best with long options.

ADJT tarball

PGTS ADJT download. This is the latest tarball which should install adjt on most Unix systems. Let me know if it gives you any trouble. For the time being version 1.0 can be found here.

ADJT script

ADJT perl script. This is the source code (Version 2.0) which goes with this article.

ADJT manpage

ADJT documentation. This is the manpage which accompanies the script. It has been created with the Ubuntu man2html utility.

PGTSJ March 2008

ADJT - Example Session. This is the most recent web page about ADJT. This includes more information about the interactive mode and a sample session using the script in interactive mode.