Disclaimer: I work for Google but not on any of the mentioned products, I’m just one of Google One’s paying customers. Opinions are my own, not necessarily those of my employer. And no, there’s no privileged or confidential information.

Party’s over, unlimited Google Drive storage for Education is ending. I suppose we all knew this won’t last forever, but I what didn’t foresaw was that my school immediately decided to jump to OneDrive.

It's nice that they even drew a nice visualization of your files flying to OneDrive.

I guess it’s finally time to start paying for things. After all, eat your own dogfood, right? So I pulled out my credit card, and within minutes, “Welcome to Google One, Chris!”:

It’s nice that they mention “expert support”, Google is finally listening to its users.

"You can't go a day on HN and many other places without people bitching about Google customer support."

Table of Contents

  1. Attempt 1: rclone copy
  2. Attempt 2: Partner Sharing
  3. Attempt 3: Google Takeout
  4. Attempt 4: Album Sharing
  5. Final Thoughts

Attempt 1: rclone copy

Now I just need to migrate things to my fresh new storage, what could go wrong? Isn’t that just as ez as some:

rclone copy --progress school:\* gmail:

Nope. Remember when the two-way sync between Google Photos and Drive was discontinued in 2019? rclone copy gets you a copy of all your photos up until 2019 as files, but nothing after is present in that directory. So that doesn’t work.

Attempt 2: Partner Sharing

Google Photos has this convenient feature called Partner Sharing, where you can share all your photos along with all their metadata to a partner account, and the partner account can be configured to save all shared photos automatically. Sounds like the perfect way to move photos, right? Nope, I ran into this issue where Partner Sharing doesn’t include location metadata, which is unacceptable for me. I tried to ██████████, but ██████████, and ██████████. Then again, ██████████. So I gave up🤷‍♂️.

Attempt 3: Google Takeout

At the end of the day, Google Takeout is always there for you. I went ahead and requested a full copy of my Google Photos data. After about a day, I got 5 50GB .zip archives. Glad I still have unlimited Drive storage so I can just rclone copy them from my Drive’s Takeout folder, cause good luck if you try to download them from web.

I put them on an external drive, unzipped the archives on top of each other, and let the Google Drive app start uploading them to Google Photos. EZ.

Aaaaand, nope. After a few hours, I went to photos.google.com and took a peak at the uploaded photos, and saw a ton of photos appears as they were taken a few days ago. Then I realized it was because they don’t have the “Date Taken” EXIF information (that metadata is in the sidecar .json file), hence Google Photos has fall backed to the file creation date, which is the date that I requested Takeout. Damn it.

One example:

➜  Photos from 2014 exiftool -DateTimeOriginal IMG_0609.JPG
➜  Photos from 2014 cat IMG_0609.JPG.json
{
  "title": "IMG_0609.JPG",
  "description": "",
  "imageViews": "23",
  "creationTime": {
    "timestamp": "1454503582",
    "formatted": "Feb 3, 2016, 12:46:22 PM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1413867781",
    "formatted": "Oct 21, 2014, 5:03:01 AM UTC"
  },
  "geoData": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "url": "https://lh3.googleusercontent.com/█████████████████████████████████████████████████████████████████████████████████████████████████████████",
  "googlePhotosOrigin": {
    "mobileUpload": {
      "deviceType": "IOS_TABLET"
    }
  },
  "photoLastModifiedTime": {
    "timestamp": "1622985542",
    "formatted": "Jun 6, 2021, 1:19:02 PM UTC"
  }
}

There’s no DateTimeOriginal in EXIF, and that information is only available in the photoTakenTime field of the sidecar JSON file.

I wrote a little script to assess how bad the situation is:

#!/bin/bash

# Print file name if the photo does not have DateTimeOriginal EXIF.
if [[ ! $(exiftool -DateTimeOriginal -extractEmbedded "$@") ]]; then
    echo $@
fi

Then I ran this command (“execute the script above for all the non-JSON files under the current directory”):

find . -type f \( -not -iname "*.json" \) -exec print_if_no_datetimeoriginal {} \; | tee ~/no-date.txt

… which revealed O(1,000)s of photos without DateTimeOriginal EXIF, damn it.

By a quick glance, most of these photos falls into one of these categories:

  • Majority of those are the .mov files of Live Photos / motion photos (only the .heic file sets DateTimeOriginal EXIF, the .mov sets CreateDate/ModifyDate instead)

  • Photos saved from chat, and some chat apps strips EXIF when sending

  • Screenshots, like VLC screenshots

  • Edited photos where the photo editor doesn’t set EXIF correctly

  • Old photos that are just really really old, old enough that its EXIF was already lost in time

Can’t really blame Google Takeout for not including EXIF for those photos as they are not there in the first place.

Speaking of Live Photos, they contain one .heic file and one .mov file, and Google Takeout presents them to you as exactly that. I didn’t test it but I suppose that this means that if you attempt to re-upload the photos back into Google Photos, Live Photos would become one still photo and one very quick video, even if you go all the way to fix DateTimeOriginal EXIF issues with mattwilson1024/google-photos-exif or Metadata Fixer.

I don’t like Live Photos either, but I don’t want them to be broken.

Attempt 4: Album Sharing

Takeout didn’t work, so I have to find a way to move photos within Google Photos. If partner sharing doesn’t work, at least we could do album sharing.

After hours of painful trial-and-error, this is the workflow that moved my photos.

  1. (On web) Create an album
  2. (On web) Share it with the target account
    • The app doesn’t allow you to share empty albums
  3. (On web) Enable “Share photo location” in album options
    • That option won’t be available until you have shared the album
  4. (In iOS app) Make sure the “Share photo location” is enabled
    • Because it often show up as disabled in app after enabling it on web
  5. (In iOS app) Add no more than 5,000 photos to the album
    • Do it in the app because you select all photos of a month, much better than keep on scrolling and getting RSI
    • Not exceeding 5,000 is very important, if you add more than 5,000 photos, the iOS app won’t actually add those photos to album, and it will keep showing as 0 photos on web. The official limit is 20,000 photos, this is probably some internal API limit issue.
  6. Wait for an iOS notification that your sending account added photos to a shared album
    • Or you can keep refreshing on web if that’s your thing
  7. (Either app or web) In the receiving account, open the shared album and “Save All”
    • You may need to repeat this step several times

Repeat until you have moved all your photos. I made “month” the atomic unit here, photos of a specific month will always be in the same album batch, this makes it easy to: 1. Make sure you have included all photos, 2. verify you have got everything, and 3. fix a specific batch if needed.

Then, to verify everything is there, for each batch album:

  1. (In iOS app) Select all photos in the time range of that batch, take note of the number
  2. (On web) See the number of photos in that shared album, see if they match
  3. Randomly sample a few photos and see if they have all the metadata, mostly location
    • Note that some photos won’t have location after moving because the old location was an “estimated location”, based on your location history and other things, so it’s not in EXIF

Final Thoughts

Why go through so much trouble just so you can continue to use Google Photos? Why not PhotoPrism or any other photo service? They are self-hostable and they preserve your file structure so you can do easy rclone copy for backups and migrations in the future?

Because:

  1. Feature-wise, Google Photos has no competition. It’s not just the easy syncing and the web interface, it auto-suggests rotations, auto-stitches panoramas with quality far better than the panoramas that I shot on my phone
  2. This may sound like Kool-Aid, but one thing about working at Google is that: When it comes to handling my personal data, I seriously trust Google more than self-hosting an alternative, especially for photos because 1. I need access from the Internet 2. photos are sensitive. After going through these internal processes that ensures security and privacy of the data, and the ease-of-mind knowing there will be enough replications.
  3. Again, eat your own dog food. Even if things go horribly wrong, I can still easily reach out to the engineers and hopefully avoids affecting other normal users.
One of those photos that Google Photos automatically stitched, from photos shot on my crappy iPhone in Dali, Yunnan, China.

But!

  1. This experience SUCKS. Glad I don’t need to do it very often, hopefully not until Google Photos ends up here🪦. I’m glad for you that I have tried all the paths that doesn’t work so you don’t have to. I’ll admit that part of the motivation for this post is to just rant how bad this is. I know Google Takeout is kind of a compliance thing but you are just doing the bare minimum, how am I supposed to do this? Where is Google Takein? Why can’t we keep the nice things like Drive-Photos sync?
  2. ████, ████ everywhere.