Artificial intelligence used to restore the film of the arrival of man on the moon in a way that has never been possible before
On July 20, 1969, Apollo 11 astronauts entered history by becoming the first humans to set foot on the Moon. man on the moon, which was broadcast live on TV to the entire world, can now be seen with the same quality as a recording made in 2020.
With the help of artificial intelligence algorithms, the owner of the YouTube channel DutchSteamMachine – that he is an expert in restoring old photos and films – he managed to make magic with the original films recorded by the Apollo team. The big difference in the work of the youtuber with that of other improved versions of Apolo's recordings that the final result does not have any type of granulation, and they appear to have been recorded with current HD cameras – and not with the type of equipment existing in the 1960s.
The restoration was done using the DAIN (acronym for Depth-Aware video frame INterpolation, or interpolation of video frames with depth recognition in free translation), an open source AI algorithm that allows you to process a video in order to create new frames between the existing ones, allowing you to it can make an old video more fluid and help to eliminate image grain that is a feature of older cameras.
The advantage of this AI is that, as it is open source software, programmers are always launching improvements to the program, which allows the results obtained to be increasingly impressive. This is the same AI that, in April this year, was used to restore the first film in the history of cinema to 4K quality, and is a tool widely used by restorers of old films.
How man's arrival on the Moon was restored
The youtuber explained how he did it to restore the video of the arrival of the man on the moon in a quality never seen before. The first step was to look for the best existing video about the event – since, as it is the filming of one of the most important moments in human history, there are several improved versions of the original footage to choose from.
Thus, the version chosen to start the work was a 720p version with a high bit rate. But there was still a second obstacle to be resolved: the frame rate per second. One of the biggest challenges in restoring old videos is that many times even the restored sequences have an appearance of many cuts due to the way the original video recorded, which can vary between 12, 6 or even 1 frame per second. And, as this value often fluctuates within the same video, it was necessary to compare the astronauts' speech with the video images in order to discover the frame rate used in each video segment.
From there, each frame of the movie was separated into a PNG image and inserted into the AI with the original recording rate of that frame (1, 6, 12 frames per second) and the number of times that the algorithm needs to multiply the playback speed to reach 24 fps (frames per second, or frames per second in Portuguese), which is considered as the minimum value for the human eye to be able to see a video without noticing the cuts between frames.
THE DAIN then it checks the inserted frames and, based on them, creates new frames so that the video reaches 60 fps, generating the necessary fluidity for the final result.
For example, between each frame of a segment recorded at 12 fps, the AI will insert 2 new frames (because it is necessary to increase the playback speed by 2x to reach 24 fps) created by her own algorithm, using the images from the recording as a basis original to infer which images would exist between these two frames if they had been originally recorded in this configuration. This means that, from 12 real frames, the AI will create 24 fake frames in order to make the transition between frames much more smooth for those watching.
After that, you should then apply color correction to the film (since these old films often have a kind of orange or blue filter in the filmed images) and synchronize the result obtained with the audio and, if possible, with other films recorded in the same period (if there is more than one camera recording at the same time). The surprising result you can see in the video below, which shows the arrival of the man on the moon in a way never seen before:
Source: Science Alert