By Anna Wexler
Photos by Avi Levine

At the airport, Michael Bronstein is often stopped by a suspicious security guard, who questions him about the yellow rubber glove in his backpack. Then Michael must explain that he’s on his way to a lecture, where he will use the rubber glove as a prop in helping the audience understand his research. What the guard doesn’t know is that Michael Bronstein’s work may end up radically transforming the entire method of airport security.

In 2002, while studying as undergraduates at the Technion Institute of Technology in Haifa, Michael and his identical twin brother, Alex, developed a revolutionary system for three-dimensional facial recognition. In a story that has already become a legend, the brothers were challenged by their computer science professor, Ron Kimmel, to create a system that would successfully distinguish between their faces.

The twins, 29, look exactly alike: both have shaggy brown hair, wire-frame glasses, and slim builds. At a recent conversation at their Technion office, they often chimed in to finish each other’s sentences. The brothers’ strikingly similar appearance makes their work on facial recognition technology all the more remarkable.

While the human brain has an astounding capability for facial recognition—you can meet someone briefly, and years later, still recognize his face—for computers the problem of facial recognition is much more difficult. Two-dimensional facial recognition, such as comparing two photographs, is hindered by factors such as head angle and illumination conditions. In 3D facial recognition, the main limiting factor becomes facial expression: whenever you smile, frown, or crinkle your nose, the 3D shape of your face changes.

The challenge for the Bronstein twins was to somehow model the underlying 3D shape of the face and then develop a set of algorithms that could successfully compare the models.

Working with lab engineers Eyal Gordon and Yaron Honen, the brothers built a system—consisting of a camera and a projector that flashed a light pattern onto a face—that efficiently captured the image of a face in 3D. By measuring the way the pattern deformed, the system was able to reconstruct the 3D model of a face within a single second.

Then, building on algorithms developed by Kimmel and his student Asi Elad, the brothers constructed a unique system for comparing faces. The algorithms took the complicated 3D information and converted it into simpler two-dimensional geometry, creating an entirely new representation of the face, one that undid all facial expressions. “If you smile, this will ‘unsmile’ you,” says Michael.

The twins, at the age of 22, had made a major leap in facial recognition technology. Their work resulted in numerous awards, celebrity meetings with everyone from Bill Clinton to George Lucas, and a flurry of media activity.

As the media attention died down, Michael and Alex continued working: they weren’t completely satisfied with their system. “You cannot fully undo the smile,” Michael says, because if you undo the smiles of two different people, their faces might look too similar in the representation. “The representation introduces distortion,” says Alex, explaining that it is not possible to reliably represent 3D geometry in two-dimensional space.

By analogy, imagine converting two high-quality digital photographs into lower resolution ones; comparing these low-resolutions images would yield accurate but not entirely exact results. The brothers wanted to find a way to directly compare the 3D images—the equivalent of directly comparing the high-resolution images.

To accomplish this they turned to the work of Russian mathematician Mikhail Gromov (winner of 2009’s Abel prize, mathematics’ equivalent of a Nobel) and began applying his theories of computational geometry to computer vision and pattern recognition problems. Their work in this area (in which they developed an efficient numerical procedure called generalized multidimensional scaling) became the subject of both of their Ph.D. theses, completed under the supervision of Prof. Kimmel. Their research also resulted in them co-authoring a book, Numerical Geometry of Nonrigid Shapes.

By applying their newly developed procedures to the facial recognition system, they achieved their goal: a direct comparison of the 3D images. To illustrate how the new system works, Michael takes the yellow glove and demonstrates that he can easily slip his fingers into it without stretching the rubber. “But imagine that I have a very small glove and a very fat hand,” he says, tugging at the glove, “I will need to stretch the rubber significantly.” Michael explains that the new system tries to “fit” one 3D face into another, like a hand fitting into a glove. If the system finds that a large degree of stretching is required for the two faces to fit, it will deem the two faces to be different.

In early 2010, the brothers sold the facial recognition technology to Invision, an Israeli company, and are working with them to develop the technology into a marketable product. The current system is able to acquire 3D images in just a fraction of a section.

In today’s post-9/11 world, where security is of the utmost importance, governments around the world have been heavily investing in biometric research in an effort to discover a fast, cheap and accurate method of identification based on an individual’s inherent physical qualities. To-date, fingerprints and iris scans have been the leading biometrics, as a reliable facial recognition system has yet to appear on the market. That could all change when the Bronsteins and Invision release their product.

Biometric identification is not without its critics. A recent bill that passed in Israel’s Knesset, which allowed for a two-year trial period of a biometric database of Israeli citizens, met with outrage. And the US’s decision to issue biometric passports—which it began doing in 2007—was highly controversial. But there will always be a debate over the tradeoff between security and privacy, and the brothers seem more focused on the scientific end of the technology.

Michael and Alex immigrated to Israel from Russia in 1991, and have always been interested in science. As adolescents, they built homemade rockets and bombs, a hobby that ended at the age of 13 when their parents bought them a fish tank in exchange for their promise to stop all explosions. Later, the brothers excelled at math in high school, and studied both electrical engineering (as undergraduates) and computer science (Ph.D.) at the Technion.

The brothers’ work on facial recognition and shape analysis led them to explore related fields. Their latest idea, the one that took them to California after their Ph.D.s, is the one they seem most excited about. Michael points to the giant computer server that’s been humming in the background next to us: “What was done for the human genome, we are doing for video,” he says.

Video Genome, as the project is called, was inspired by the computer science problem of comparing complicated sets of data, such as photographs. Currently, the web’s most powerful image search engine, Google Images, is only partially successfully. For example, a search for “Eiffel Tower” will yield many hits (images of the Eiffel Tower), but also many misses (a person on a Paris street, or Michelle Obama), because the search engine compares the text associated with the images (Michelle Obama recently toured the Eiffel Tower) and not the content of the images themselves.

The Bronstein brothers started to think about ways to analyze an even more complex kind of data—video—and struck upon an analogy between video analysis and the field of bioinformatics: what if you could assign a genome (a simple code) to each video based on a set of algorithms? Videos could be analyzed based on content information such as edges and shapes, and their “genomes” could be decoded and placed in a database. Then, the system would be able to analyze any new chunk of video, find its genome, and by comparing it to an existing database of known genomes, identify the video itself.

One advantage of the system would be in its ability to track videos, both popular movies and those uploaded on sites like YouTube. For example, if a video of Avatar was recorded via a handheld camera in a movie theater and then uploaded to the web, the system would be able to decode the genome, compare it to an existing database, and correctly identify the video as Avatar.

The brothers moved to California to work on their Video Genome ideas, where they concurrently held visiting appointments at Stanford. Now, two years later, they are back at the Technion, where they continue to work on the project.

“Imagine the media center of the future,” says Alex, describing how the system would transform video into a more interactive medium. For an individual user, the system would allow video to be enriched with subtitles and other meta-data. It could also be an engine for advertisement, allowing for the placement of ads at targeted points.

The brothers see the Video Genome Project as part of the natural continuation of their work, the common underlying theme being theoretical and numerical methods in metric geometry. “We understand what theoretical mathematicians do, and we also understand the problems of computational science,” says Michael.

Together the twins have co-authored over 50 scientific articles. Outside of the scientific world they are also similar: they shared a Technion Humanities prize in 2001 for translating Shakespeare into Italian (preserving the rhyme), and separately, they both married Italian women. They are fluent in multiple languages, and both want to travel around the world.

Michael says that he can walk into a room, and if Alex is discussing something with someone, he can pick up mid-sentence and continue the conversation. “We’re on the same frequency,” he says.

Now, as Michael and Alex turn an eye towards future academic positions, they are realizing that they need to prove that each twin, as Alex puts it, “can work independently without the need of the other.” But at the same time they are so productive together that it doesn’t make sense for them to work separately. Together, they are like one big brain, using theoretical math to tackle some of the most daunting problems that face computer science and engineering today.