Monday, February 1, 2021

Creating a time-accurate mp4 video out of a collection of photos

Way back when I got married, I asked the attendees to send me their digital photos so I could get various angles of the event.  I got back photos from about 10 different cameras, plus two digital videos.

Using Picasa, it was easy to view all this content in one album, but the first problem was time stamps (these photos were all digital camera photos, with manually-set clocks).  In Picasa, using one reference camera (my own), I then shifted each camera's photos times until it matched mine.  I could now view the photos on my computer in sequence, as long as it was in Picasa.

My photos sat this in state for over 10 years.  Recently, I got the urge to work on my wedding video, but the two non-HD video streams look quite dated. Wouldn't it be nice to insert higher resolution photos into the wedding video at key moments.  But with over 500 photos, it would not be practical to manually place them at the right timestamp my video editing software.

My solution was to turn these photos into a video, with each photo displayed at the right timestamp relative to the very first photo in the stream. From searching around, I couldn't find anyone who had actually done this, hence this article.

I used the exiftool/ffmpeg tools, free to use on multiple OS platforms, but the syntax below is for Windows.

What you need:

Download exiftool and ffmpeg (I won't post a link - they're both popular tools easily findable by the great Google)

Make sure you can download this Excel spreadsheet. You'll need to edit it on your own Excel-capable machine (or upload it to GoogleDocs to perform further editing)

https://1drv.ms/x/s!AgFCNBReexfMnnn_cN0OlyR1yYaK?e=1JvgYJ

Instructions:

From a command prompt, navigate to where your jpg files are stored. Then run:

"<path to>\exiftool.exe" *.jpg -s -DateTimeOriginal -csv > photos.csv

(note - It's possible that the Exif field your camera/photo program uses might populate/prefer a different field from DateTimeOriginal, so change that as appropriate)
Open photos.csv in excel (or whatever program you have for .csv), then copy/paste the two columns into the first two columns of the ToFFMPEG.xlsx file I shared above. All the other columns in ToFFMPEG.xlsx past Column C are calculations - don't modify those. If the number of files you're pasting in exceeds the number of pre-populated formulas in Column C and beyond, just Fill Down those formula rows as well.

Important - sort by Column B, so that the durations will calculate correctly.

Note - the last row in the file will not be able to calculate a duration, so it will just take the duration from cell K2.
The calculations are in Columns D, E, F, and the final results are put in Column C. Copy that column C into your favourite txt editor. You'll see double quotation marks everywhere that weren't in Excel, but are an artifact of copy/paste when there are multiple lines per cell. Delete all of these double quotation marks with a search/replace. Save it as ToFFMpeg.txt . This is the input into FFMPEG.

Then, run the following ffmpeg command:

"<path to>\ffmpeg" -f concat -safe 0 -i ToFFMpeg.txt -vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,setsar=1" -pix_fmt yuv420p -c:v libx264 output.mp4

(the above command should be run on a single line, despite how this forum is splitting it across multiple lines)
This will create your mp4 file. Change the 1920:1080 resolution to whatever you need to optimize with your output video.

Limitations: exif data has only per-second granularity. If two photos are taken at the same second, one will have a duration of 0 seconds, and will probably be hidden by the second photo in your output video.

Apparently, ffmpeg has the ability to read and output exif data itself, but as I was already familiar with exiftool, that's what I used.

Here's a screenshot of the photo mp4 stream embedded with two video camera shots in my Movie Studio project: