These researchers create mouth-watering (but fake) pizza images. Why?
Fake face generators come with the risk of being used for misinformation or harassment. This web application sounds way more fun and less dangerous: scientists found a way to teach computers to draw fake pizza images from scratch.
Researchers from Rutgers University and Samsung AI Center developed a Multi-ingredient Pizza Generator (MPG). They based it on a state-of-the-art GAN structure called StyleGAN2, in which they developed a new conditioning technique.
In human terms, these researchers developed a new model that teaches computers how to draw a realistic pizza from scratch. Even though others had tried to do it before, fake pizza images were blurry and lacked flair.
The New MPG model can effectively learn to draw photo-realistic pizza images from combinations of specified ingredients. The research is academic, but this model can also be put into practice, for example, to experiment with menus or teach children about different ingredients.
CyberNews spoke to Fangda Han, one of the researchers, to find out more.
“Teaching a computer to draw an image from nothing is harder than teaching it to distinguish different ingredients. It’s the same for human beings. One can easily distinguish between a dog and a cat, but it takes years to learn how to draw a dog or a cat,” Han told CyberNews.
He and his colleagues have been working on food-related image generation problems for more than two years now. For example, one of his colleagues is working on food ingredient segmentation, while another tackles food ingredient identification, trying to teach computers to identify what ingredients are included in the meal and in what quantities.
Meanwhile, Han focuses on the food image generation problem. Han has been making rapid progress because of a new technique called GAN.
“That’s a start. My initial idea in the group was to generate food images directly from a text description,” Han told CyberNews.
There’ve been attempts to generate food images, he added. He and his colleagues tried generating pictures of cookies or salad, but they weren’t good because of the complexity of the meal.
Therefore, researchers changed the initial idea and started experimenting with pizza images. They succeeded in creating a model that draws photo-realistic pizza images from a given list of ingredients.
The MPG model draws a pizza from scratch. You specify the ingredients that should be in the picture (salami, broccoli, etc.), and they come up with your desired pizza. For computers to understand what broccoli is, you have to teach them beforehand by collecting many photos of the ingredients and labeling them.
You can play with it here.
Some of those really look like pizzas straight from the oven, right?
The StyleGAN model that researchers used in creating the new approach can already generate photo-realistic human face images.
“We tried to keep most of this model's ability and add our modules which, we think, will give us the power to control which ingredients appear in the image. The other reason why this pizza image is real-like is that compared with our previous work, this time we use a relatively more constrained dataset,” Han said.
Previously, researchers tried to generate salad pictures but were not satisfied with the results.
“Pizza is more consistent in shape, and ingredients that people usually add to pizzas are relatively simple. Now the model doesn’t need to learn all those fancy shapes, the combination of colors, but only concentrate on the combination of ingredients on the pizza image, which is naturally easier,” he said.
At the moment, MPG draws a picture from 10 different ingredients. It is limited because of the data set.
“If we collect more pizza images from the internet, and label them, and add them to the current dataset, and retrain the model, it would be able to add more ingredients,” Han explained.
Is it useful?
How can all of this be used in everyday life? Although it was designed more for academic purposes, it can also be applicable in schools or restaurants.
“It can help children develop intelligence, to build the relationship between the text and images. When we teach children different words like broccoli or black olives, we can show them how they look. In this way, we are helping them to build the connection between their visual system and their language system,” Han explained.
In the future, It can also help learn about healthy food. “We can not only work on pizza but also on a salad to create new fun looking but healthy recipes. In this way, children can learn about healthy and nutritional food,” Han added.
MPG could be useful in creating photo-realistic icons of dishes and help customers visualize what kind of food they are ordering.
“It can help in designing new dishes. People can have a visual understanding of what a dish looks like before really making it, which could lower the cost for designing new dishes,” he said.
Researchers continue working on food image generation problems. They are keen on controlling it more. They want to be able to specify whether pizza should be fully cooked or not, or if a user wants all of it or just half. Also, researchers should be able to control the angle of the pizza picture, i.e. whether we should look at it from the top or any other specific angle.
“We also want to be able to cut ingredients into pieces. We want to let the model know what a cut means. It’s really important. If we can do it, it’s going to be really interesting,” Han explained.