Image Manipulation By Using Generative Hallicunation Of Faces and Pixels

Let's say you've found a printed photo from your childhood at home. You decided to do a throwback Thursday on Instagram. But you're worried that its quality may not be well if you take a photo of it with your camera or scan it. What is the first thing you try? You start to search for an image enhancer. Well, I know, you wouldn't. However, I advise you that it's the right thing to do these days. Artificial intelligence became superior at doing these tasks. This article gives a first-hand insight between AIs that try to fix pixels vs AI's trying to recreate pixels in photos.

Albumera Book a Free Photoshoot

Expertise and Background

At Albumera, we solved the long-waiting photo delivery period after your photoshoot by training an Image Retouch AI. Generally, after any photoshoot, you need to wait a minimum of 2 days to get your photos from a photographer. Photographers are busy people. This boredom affects both parties. So we decided to train an AI to delegate the photographer's photoshoot work, and we developed it to cut off photo delivery times after any photoshoot with a professional photographer. This article tells you why recreating pixels give better results for people than fixing pixels. So read on.

Backstory

Photos that are taken outdoors mostly do not require retouch. However that's not the case if you are taking a photo of a person. People would like to see their beautiful faces more beautiful therefore a photographer do a lightly retouch to the photos. However this requirement makes the photography service a longer process. To solve this we decided to do an AI retoucher that retouch the face slightly to make it more beautiful. So we conducted a research, here is the summary:

Research Results

• Photo retouch, enhancement, super-resolution, editing, color correcting solutions don't to different things to human faces.
• People care about their face when compared to other parts of the photo.
• People require extra retouches to their face if you let them do revisions.
• There is no ugly and beautiful faces dataset so we can't teach AI to learn how to retouch faces specifically.
• Facial image quality and facial operations on images are mainly researched in China and Asia.
• Photographers do detailed operations on face when compared to other parts of the photo while retouching it.
• Examined many auto photo editing apps and realized the most successful and expensive one is only doing exceptional things on faces.
• Our github research led us to Face Hallicunation and we're SHOCKED.

Reasons that shocked us

• It's not editing or retouching actually the most successful apps are recreating you. Please welcome Syntectic Media. Syntectic you.
• Recreational AIs are working quite different from AIs that are fixing pixels applying and learning function(x)=y where x is the ugly pixel and y is the beautiful pixel.
• Recreational AIs are placing glasses if they feel that your face needs one!
• So face reconstruction process demands so many trained faces and model's manipulation success is mostly based on how beautiful the dataset faces are.

A typical Fixing Type AI mostly follows this pattern:

1. Find a dataset where photos and their human retouched ones exist. ( Image Pixel Fixing type AIs fail at this level because there only 1-2 database about this with very limited number of photos available. Reason: human labor is so expensive.)
2. Train a conv neural net to teach a retouch function. ( Image Pixel Fixing type AIs fail at this level because you need so many photos-retouched photos duo to converge the success rate of the learned function.)
3. Apply this retouch function to new photos.

A typical Generative Type AI mostly follows this pattern:

1. Find a dataset where photos exist. (Wins at this level because of plenty of datasets)
2. Apply GAN to teach system to be better at generating HQ photos from downsized, blurred, downsampled photos . ( From the nature of GAN you can nearly run endless iterations over this level. Downsize a photo using T function teach an f function to create a very appealing candidate photo, calculate the score, repeat)
3. Apply this generation function to new photos. Use the photos as seed input.

See an example here: A generative AI

A very common problem trying to be solved with two different approaches. The second one gets it's power from plenty of computer generated data where you can imagine adding random blur, downsample function etc.

Current situation in Image Editing Industry:

It was so interesting that only one commercial project is using face restoration and generative AI. Others are only using fixing type AIs and they don't even touch faces. It's also interesting to see that the commercial project that use generative type AI is 17x expensive than the other ones. Editing 1 photo with API is $1. Well that's why we made this research because we store and deliver 1M photos until today, which approximately so expensive for us to handle for now. After we polish our own network I will be happily sharing the results with you by comparing it to others. However it's not so ok to compare projects based on cherry picking perfect examples. Although we will be using a very specific AI enhancement network to enhance photos that are taken by DSLR photo cameras. The lens on the DSLR creates a very different photo than iPhones therefore editing them is different too. We get so many requests from photographers about this need. We decided to take a step on it and solve it. Feel free to join our newsletter so I can share the result with you when we finish polishing our version.

References

Face Hallucination and Face Restoration Research - Using both a fixing type AI and Generative type AI together brings in amazing results.

Face Generating GANS comparison - Image comparisons between generative research projects

A good face hallucination survey - Survey of face hallucinations.

A fixing type AI - An example fixing type AI project.