Backstage Story: How Portrait Lighting Mode was born on iPhones 8 Plus and X

Think fast: What is the main feature of an iPhone today? The answer you most hear from camera. And no wonder. Smartphones today are the cameras of the modern world and, like the iPhone, one of the best selling smartphones in the world, its most widely used camera around the globe.

Apple knows this and, with each new generation, tries to improve what is already spectacular today. This obsession with photography on a device that fits in the palm of our hands is paying off: as we saw here, iPhones 8/8 Plus cameras are currently the best on the market, according to tests by DxO Labs and iPhone X has it all. to overcome them, since it has optical stabilization in the two rear cameras and a telephoto lens with aperture /2.4 (against /2.8 of the 8 Plus).

<img data-attachment-id = "611677" data-permalink = " picture-lighting-mode-present-us-iphones-8-plus-ex / 26-iphone-x / "data-orig-file =" https://.uol/wp-content/ uploads / 2017/09/26-iphone-x.jpg "data-orig-size =" 2162,1050 "data-comments-opened =" 1 "data-image-meta =" {"aperture": "0", "credit": "", "camera": "", "caption": "", "created_timestamp": "0", "copyright": "", "focal_length": "0", "iso": "0 "," shutter_speed ":" 0 "," title ":" "," orientation ":" 0 "}" data-image-title = "iPhone X" data-image-description = "

Official image:

"data-medium-file =" https://.uol/wp-content/uploads/2017/09/26-iphone-x-600×291.jpg "data-large-file =" https: / /.uol/wp-content/uploads/2017/09/26-iphone-x-1260×612.jpg "src =" https://.uol/wp-content/uploads/ 2017/09/26-iphone-x.jpg "alt =" iPhone X "width =" 2162 "height =" 1050 "class =" alignnone size-full "wp-image-611677" srcset = "https: // . 2162w, -x-300×146.jpg 300w, https://.uol/wp-content/uploads/2017/09/26-iphone-x-600×291.jpg 600w, https://.uol / wp-content / uploads / 2017/09/26-iphone-x-1260×612.jpg 1260w "sizes =" (max-width: 2162px) 100vw, 2162px "/>

Leaving aside the hardware improvements, this year the big news presented by Apple was the Portrait Lighting Mode. It's a feature that combines depth sensor and face mapping to enable the creation of studio-quality lighting effects, according to Apple. That is, not just a simple filter that you apply on top of a picture you have taken or will take, Apple's dual camera system working alongside strong machine learning to recognize a scene, mapping its depth to, then change the lighting contours of the photographed subject. It's all done in real time, and you can even view the result thanks to the A11 Bionic chip.

Taking this to an Apple scale, I could no longer agree with this statement by John Paczkowski of BuzzFeed News, who talked to some people inside Apple about it:

The result, when applied to the Apple scale, has the transformative power for modern photography, with millions of suddenly professionalized amateur clicks. In many ways, the most complete accomplishment of the democratization of high quality images that the company has been working on since the iPhone 4.

I'm not saying that the photographer profession is over and that the iPhone alone solves anyone's photographic problems. Of course, it is not. Put an iPhone 8 Plus in the hands of a regular user and another in the hands of a professional photographer that you will quickly understand what I mean. It is undeniable, however, that we no longer depend on equipment that was specifically designed for professional photographers now, so that we can take amazing pictures, with quality never before imagined for a smartphone-sized device.

How did Apple achieve this? Much study is a process of deconstructing the artistic form that the company wishes to emulate. In the case of the new feature of iPhones 8 Plus and X, this meant studying how others (Richard Avedon, Annie Leibovitz, and Vermeer) used lighting throughout the story.

<img data-attachment-id = "611693" data-orig-file = " png "data-orig-size =" 800,1601 "data-comments-opened =" 1 "data-image-meta =" {"aperture": "0", "credit": "", "camera": " "," caption ":" "," created_timestamp ":" 0 "," copyright ":" "," focal_length ":" 0 "," iso ":" 0 "," shutter_speed ":" 0 "," title ":" "," orientation ":" 0 "}" data-image-title = "Portrait Lighting Mode" data-image-description = "

Official image:

"data-medium-file =" https://.uol/wp-content/uploads/2017/09/26-mode-iluminacao-de-retrato-300×600.png "data-large-file = "×1260.png" src = "https://i1.wp/ .uol/wp-content/uploads/2017/09/26-mode-iluminacao-de-retrato.png?w=401&h=803&ssl=1 "width =" 401 "height =" 803 "data- original-width = "401" data-original-height = "803" itemprop = "" title = "Portrait Lighting Mode" alt = "Portrait Lighting Mode" style = "width: 401px; height: 803px; "/> <img data-attachment-id =" 611694 "data-orig-file =" https://.uol/wp-content/uploads/2017/09/26-modo -light-of-portrait-2.png "data-orig-size =" 800,1601 "data-comments-opened =" 1 "data-image-meta =" {"aperture": "0", "credit" : "", "camera": "", "caption": "", "created_timestamp": "0", "copyright": "", "focal_length": "0", "iso": "0", " shutter_speed ":" 0 "," title ":" "," orientation ":" 0 "}" data-image-title = "Portrait Lighting Mode" data-image-description = "

Official image:

"data-medium-file =" https://.uol/wp-content/uploads/2017/09/26-mode-iluminacao-de-retrato-2-300×600.png "data-large- file = "https://.uol/wp-content/uploads/2017/09/26-mode-iluminacao-de-retrato-2-630×1260.png" src = "https: // i1. wp/ "width =" 401 "height = "803" date-original-width = "401" date-original-height = "803" itemprop = "" title = "Portrait Lighting Mode" alt = "Portrait Lighting Mode "style =" width: 401px; height: 803px; "/>

If you look at the Dutch masters and compare them to the paintings that were being made in Asia, in a unique way, they are different. So we ask why they are different? And what elements of these styles can we recreate with software?

We spent a lot of time lighting and moving people, a lot of time. We had some engineers trying to understand the contours of a face and how we could apply lighting through the software, and we had other silence engineers working to make the process super fast. We really did a lot.

Johnnie Manzari, designer of Apple's human interface team.

Phil Schiller, Apple's world marketing head and enthusiastic photographer, explained Apple's creative process and the strong collaboration between various teams:

There's the augmented reality team saying, “Hey, we need to get more out of the camera because we want to make augmented reality a better experience and the camera plays a part in that.” And the team that is creating Face ID, they need the technology of camera, hardware, software, sensors and lenses to support biometric identification on the device. And therefore, there are many roles that the camera plays, as a primary thing taking a photo or as a support, to help unlock your phone or enable an RA experience. And so there is a great job between all the teams and all these elements.

This is a time when the biggest advances in camera technology are happening in both software and hardware. And that obviously favors Apple's virtues over traditional camera companies.

When asked about the evolution of the iPhone camera, Schiller acknowledged that the company has been working deliberately and incrementally toward a professional-caliber camera. Still, he said the idea is not just to create a better camera, but how Apple can contribute to photography.

Paczkowski asked something interesting about the feature: does something get in the way when we use software to simplify and automate a historically artistic process? After all, there is a bit of dystonic feeling when pressing a button and essentially narrowing the huge gap that exists between professionals and amateurs.

For Manzari, it is not about simplifying or reducing the intellectual content, but accessibility, helping people to take advantage of their own creativity. Manzari's point is that there are many good photographers who are not professionally trained and who do not need or want to deal with a range of lenses and tools to calibrate focus and depth of field when taking pictures. And in fact, they should not. So why not take all these things and give them something that can help you take great pictures?

For Schiller, the feature is not an attempt to mimic any particular style, but to try to brush the various ranges of styles out there so that there is something for everyone. The idea is not to make “Stage Light” the same as Vermeer, but to have enough scope for everyone to have different choices for many situations that cover major use cases. To do this, Apple had to learn how others used lighting throughout history and around the world.

<img data-attachment-id = "611696" data-permalink = " picture-lighting-mode-present-us-iphones-8-plus-ex / 26-iphone-xa-11-bionic / "data-orig-file =" https://.uol/ wp-content / uploads / 2017/09/26-iphone-xa-11-bionic.jpg "data-orig-size =" 1957,1714 "data-comments-opened =" 1 "data-image-meta =" { "aperture": "0", "credit": "", "camera": "", "caption": "", "created_timestamp": "0", "copyright": "", "focal_length": "0 "," iso ":" 0 "," shutter_speed ":" 0 "," title ":" "," orientation ":" 0 "}" data-image-title = "iPhone X with A11 Bionic chip" data -image-description = "

Official image:

"data-medium-file =" https://.uol/wp-content/uploads/2017/09/26-iphone-xa-11-bionic-600×525.jpg "data-large-file = "×1104.jpg" src = "https://.uol. br / wp-content / uploads / 2017/09/26-iphone-xa-11-bionic.jpg "alt =" iPhone X with the A11 Bionic chip "width =" 1957 "height =" 1714 "class =" alignnone size -full wp-image-611696 "style =" border-bottom: 1px solid # e9e9e9; padding: 0 50px; "srcset =" https://.uol/wp-content/uploads/2017/09/26-iphone-xa-11-bionic.jpg 1957w, https: // .×263.jpg 300w, /09/26-iphone-xa-11-bionic-600×525.jpg 600w, 1260×1104.jpg 1260w "sizes =" (max-width: 1957px) 100vw, 1957px "/>

Improvements in the cameras of the new handsets don't stop, obviously. Apple has been working on enhancements that are much less flashy than Portrait Mode and Portrait Lighting. Cameras on the 8 Plus and X, for example, detect snow as a type of situation and automatically make adjustments to the white balance, exposure and other information so that the user has nothing to worry about. “Everything is perfect, the camera just does what it needs,” said Schiller. “The software knows how to take care of the photo for you. There are no settings. ”

We think it's the best way to build a camera by asking simple and fundamental questions about photography. What does it mean to be a photographer? What does it mean to capture a memory? If you start this way and not with a long list of possible features to incorporate you often end up with something better. When you take away the complexity of how the camera works, the technology just disappears. Then people can apply all their creativity the moment they are capturing. And you end up with some amazing photographs.

So good.

via Daring Fireball