Making Add Me a reality required collaboration between the Pixel Camera, Creative Camera and Google XR teams. (The Google XR team works on Android XR and ARCore, platforms for building augmented and virtual reality experiences.) To blend an image, Add Me uses augmented reality (AR) to show the second photographer an overlay of the first image so they can accurately frame the new photo to match the composition of the first one. Since aligning the two images is crucial to get the best result, the team explored a few methods to get the perfect shot. They eventually realized augmented reality was the most useful way to visually guide the user, so they turned to the Google XR team.
“We on the XR team have always wanted to do more and enable helpful features in the Camera app,” says XR software engineer Ryan DuToit. The team saw the vision for Add Me and recognized how useful it would be for personal photography.
Still, developing an interface where the AR feature was self-explanatory — so even those unfamiliar with the technology could use it — wasn’t easy, and took ample experimentation. Plus, the XR team had a busy 2024, Ryan explains, pointing to the recent Android XR launch. “Collaborating with another team on another project was challenging — but we did it, and I’m very proud of it,” he says.
Add Me was a fairly complicated technical feat: It uses various machine learning models, which are powered by Pixel’s TPU and the Tensor G4 chip to run on device. “If it weren’t for the TPU, I don’t think we would have been able to converge to a reasonable latency,” Adi says. If they’d had to run these ML models on a GPU or CPU, the feature wouldn’t have been able to display the images from the first shot and then later the blended shot quickly enough.