Images on Gmail

In honor of WordPress.com’s addition of custom header images as a test to one Theme (finally!), the blog looks a little different today. [but today is no longer today, so we’re back to normal] I’ll put it back the way it was soon, and hopefully not too long after that, custom header images will roll out to a Theme I like. Isn’t that a great shot T took of #2 blowing a dandelion? Of course, it’s heavily cropped to fit into the requisite shape for a header image, but even still, it looks great; it really captures the gentleness of #2’s personality.

Anyways, it is fitting also to have an image up there, because the geeky topic of this post is that I got a Perl script working to backup digital photos to Gmail. I am motivated to do this because of a hard drive failure, which lost approx 2 years of family photos. So in addition to being more faithful with my CDROM backups (and verifications of those backups), I will also be trusting in Google (who are not too evil) to not lose my data.

For posterity, and so any mildly competent PC user can follow in my footsteps, here’s how it works:

  • Get a Gmail account.
    • If you already have one, invite yourself to another one. Each allows close to 3Gig of storage, so probably only a few accounts will meet your needs. If you don’t already have one, you can sign up for one via mobile phone. If you don’t have a mobile phone, then I love you, and I will gladly send you one of my 90+ remaining Gmail invites. (There’s actually no reason you have to use a gmail account. Yahoo email currently offers 1G of free storage, and there’s probably no practical limit to the number of yahoo accounts you can create for yourself either).
  • Install ActiveState Perl
    • (the latest is 5.8.something), which is a free download from ActiveState.com (the first link should directly pull a 12.8Meg Microsoft Installer file; if that link should become broken, go to the second link and tool around). Accept all the defaults so that your PATH is updated and .pl files are recognized as perl scripts.
  • Install the Image::MetaData::JPEG Perl Module
    • (this is so that my script can extract text tags (such as shooting date) that your camera stores in each image, and build a searchable email body text). The automatic CPAN install worked well for me: in a cmd window, just type

perl -MCPAN -e “install Image::MetaData::JPEG”

    • (This should activate CPAN download capabilities that are installed with ActiveState Perl, and automagically install everything needed for the Module to work. If you are missing some utilities required to perform the install (i.e. nmake), this command should even download and install those utilities for you. If it says there are some package dependencies, would you like for them to be prepended to the current queue of modules to install, say yes. Note that I only moved to blat (below) because auto’magic’ CPAN install failed for Email::Stuff, so it’s not perfect. And if it doesn’t work for you, then you’re on your own!)
  • Install blat,
    • which is a Win32 Command-line mailer. And anything you can do on the command line (by typing), Perl can do faster (and with fewer typos)! It’s very simple to install; after you extract the .zip, copy the .exe, .dll, and .lib files from subdirectory full/ into some directory in your path (for win2k, C:\WINNT\SYSTEM32\). Or don’t, and invoke blat all the time (and edit the perl script to invoke blat) using it’s full path.
  • Configure blat
    • to be ready to send emails. Undoubtedly your ISP has documentation on how to setup your email client; somewhere in there they specify a SMTP server for outgoing mail. Maybe something like smtp.yourisp.net. Well, whatever that is, choose an email address (preferrably an email address that actually belongs to you; you should be able to use the address of your new gmail account), say youraddress@somedomain.com, and in a cmd window type

blat -install smtp.yourisp.net youraddress@somedomain.com

    • If blat is installed correctly, you should be able to send (and on the other end, receive!) test emails, like this:

blat -to newaddress@gmail.com -s “This is a subject” -body “This is a test” -attach “animage.jpg”

  • Paste the Perl script (editing to use the email address of your choosing)
    • Fire up notepad, copy & paste the Perl script below. Right up front, change “rightHereDumDum\@gmail.com” (yes, that’s a backslash, and yes, it needs to be there) to the address for the gmail account you want to use, and save as pixbak.pl (or whatever you want to name it) into a directory that is in your PATH (or a directory of your choosing, if you want to invoke it by its full path each time).
  • Run
    • In your cmd window, cd to a directory that contains images you want to backup. Then invoke the Perl script, simply by typing

pixbak.pl

When the script runs, it allows for a 10-second delay between images; partly out of general kindness to the internet, and also so you won’t look so much like a spammer and maybe get your outgoing emails blocked. After you get pixbak.pl working nicely, you might want to set up some filters in your gmail account, so that incoming emails will automatically get labels like ‘2000’ or ‘Oct’, etc.

Well, that’s it! Happy using; I’d love to hear from you if you tried and either succeeded or failed to use this utility. Just drop me a comment below! And without further ado, here is the script pixbak.pl (OK, maybe a little ado: I apologize for the lack of indenting, but I think WordPress is preventing me from using HTML to format well — be careful that all of the line wrapping and special characters (quotes, backslashes, etc.) copy&paste correctly!):

#! /bin/perl

use Image::MetaData::JPEG;

# make sure @ is \@
$toaddr = "rightHereDumDum\@gmail.com";

@longdir = `dir`;
foreach (@longdir) {
if (/Directory of (.*)$/) {
$cwd = $1;
last;
}
}
@dirs = split /\\/, $cwd;
foreach (@dirs) { $dirtag .= "DIR_TAG=$_\n"; }

foreach $jpg (glob "*.jpg") {
$img = new Image::MetaData::JPEG($jpg);
$dat = $img->get_Exif_data('IMAGE_DATA', 'TEXTUAL');
$date = $dat->{DateTimeOriginal}->[0];
($yyyy,$mo,$dd,$hh,$mn,$ss) = split /\D+/, $date;
$body = "DATE=$date\n";
$body .= "YEAR=$yyyy\n";
$body .= "MONTH=$mo\n";
$body .= "DAY=$dd\n";
$body .= "HOUR=$hh\n";
$body .= "SEC=$ss\n";
$body .= $dirtag;
foreach $k (sort keys %$dat) {
$body .= "$k=$dat->{$k}->[0]\n";
}
$body =~ s/[^\w\s:=]//go;
open BODY, ">body.txt";
print BODY $body;
close BODY;

$cmd = qq(blat);
$cmd .= qq( body.txt);
$cmd .= qq( -to "$toaddr");
$cmd .= qq( -s "$jpg");
$cmd .= qq( -attach "$jpg");
print "$cmd\n";
system($cmd);
sleep 10;
}

Advertisements

19 Responses

  1. I skipped the entire babbledy-blah about Perl, but just have to comment on the header image. Hooray! This is a day long coming. I can’t wait to try it out myself!

  2. But, but, I've automated backing up thousands of photos on gmail!

    You Philistine! Just wait until I get around toBlog'ing 'Code is Poetry'. Then you'll see.

  3. Aww, your blog is back to its normal self. I liked the other look!

  4. My head hurts.

  5. Tonight, I filled up my first gmail account; I maxed out at almost 3200 images (each attached to a single email), totalling 2.7 Gig. And that’s without the two years of family photos that are still trapped on our dead hard drive, waiting for me to get off my butt and lay out the long green required to retrieve them.

  6. So, if your harddrive crashes, do you have an automated way to extract each photo from each email – sounds like a pain to get them back!

  7. To start with, gmail offers the advantage of POP (unlike most other free email services), so I could set up a Eudora or Outlook Express and pull all the emails back down to my desktop. From there, it would still be a problem of wading through all the emails, but at least they would be in one mbox file, and I could probably knock together another perl script to extract the images. Let’s hope it never comes to it!

    Worst case: manually fetching 3000 images is possibly better than paying >$1000 to have the disk recovered!

  8. Why not spend a little bit of money on a couple USB hard drives for backup?

    or…

    Purchase a few flash drives and use them for backups. I think as long as you have two backups plus your original, you’re pretty safe.

    Seems like a lot less of a restoration incovenience.

  9. Just for the record, don’t trust flash drives. When you frequently save to them, they can become corrupted quite easily. But definitely a few USB hard drives is the way to go plus CDs – but if you find a free option in Gmail, seems like a great last case scenario.

  10. Plus, Images on Gmail has the bonus that I get to write some Perl!

  11. I’m not sure that for backup purposes I would agree that the flash drive is less reliable than a USB drive. The reason being that there are no moving parts. A usb hard drive has the same moving parts that caused RubeRAD’s HD to crash in the first place. Plus, if flash drives corrupted so easily by writing to it all the time (which backup purposes wouldn’t really require anyway), then why are there so many successful flash drive mp3 players such as the iPod Nano? If it were that unreliable, I don’t think Apple would have used it.

    Regarding PERL, there would be nothing stopping you from writing a PERL script to do the backup. You could even challenge yourself to make it a fancy backup script that didn’t delete the old and only copied the new, etc.

    (BTW, Who’s Laura? I’m a friend of Rube’s from church)

    :)

  12. Follow her link, and I’m sure you can figure out who Laura is…
    I got a call from the disk recovery place, and they actually said it’s about as bad as it gets, with the arm damaged, and nicks and scratches on the platters. $1800 (out of a maximum possible $1900). And the platter damage will cause holes in some files, so there’s no guarantee of being able to recover all the data.

  13. Bummer, Rube! :(

    Do you need a loan ;).

    Sorry to hear it. I pray that the LORD will provide.

  14. Thanks to Jehovah Jireh, I can cover it. But maybe #3 will have to drop out of college 3 credits shy of a bachelors…

  15. I am very pleased to report that ADS has cut their quote to me almost in half! They rock — everybody go use them if you have data recovery needs! (And Jeff, I put your name down as referrer, so hopefully you get some bonus points with them as well).

  16. Why was their quote double to begin with?

  17. I didn’t want to ask too many questions, at least not until I got the actual quote sent and signed and faxed back… But they were originally saying that I had damaged heads as well as nicks and scratches on the platters. Partly the discount could be explained by accidentally verbally giving me the even lower quote for another customer. Also partly it could be explained by the very detailed instructions (file types, names, directories, dates) I was able to give for the relatively small portion of data I needed recovered.

  18. The restore is complete, and out of the two years worth of pictures, only about half a dozen were corrupted as far as I can tell. They are all backed up to Gmail, so my backup backup account now has 1137 images, and as a coincidence, “You are currently using 1133 MB (41%) of your 2746 MB.”

    That’s almost exactly 1MB per image, which is odd, since each individual image is only in the 600-700KB range, and the image metadata that forms the body of each email is about 1K. Gmail must be counting all the indexing necessary for Google-searching my gmail in my size total. 300-400KB of indexing per 1KB of searchable text!?!? That’s a lot of overhead!

  19. That is so awesome. I’m so happy for you that you got all of your photos back.

    Congrats.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: