Updated: Jul 24, 2020
A groundbreaking project showcases the potential for virtual remembrance.
Did you catch the recent 60 Minutes special, hosted by correspondent Lesley Stahl, about the project to create conversational avatars of Holocaust survivors so that their stories could be shared long after they pass away? It’s an incredible, groundbreaking demonstration of the potential for the technology of digital immortality, an effort that HereAfter cofounder James Vlahos wrote about the project in his book, Talk to Me.
Here’s an excerpt for those looking to learn more:
Private companies aren’t the only ones creating clones, as a ground- breaking project called New Dimensions in Testimony illustrates. A joint effort by the Institute for Creative Technologies and the USC Shoah Foundation, the project aims to memorialize Holocaust survivors.
In 1943 the Nazis captured a ten-year-old boy named Pinchas Gutter and his family, and imprisoned them in concentration camps. Gutter’s sister and parents were killed in the gas chambers before he could even say good-bye. He would go on to be beaten, shuttled between different labor camps, and put on a death march before finally being freed by the Red Army in 1945.
Gutter, now in his eighties, has devoted himself to sharing these horrors, giving talks and answering questions. But like all of the remaining Holocaust survivors, Gutter will not be around much longer to do so. Testimonies like his have, of course, been captured in print, audio, and film. But telling a story in person has unique power. As Rabbi Marvin Hier, the dean and founder of the Simon Wiesenthal Center, once explained, “There’s nothing like the human witness who can look you in the eye and say, ‘Look, this is what happened to my husband. This is what happened to my children. This is what happened to my grandparents.’”
Targeting the immediacy of in-person storytelling, the New Dimensions team interviewed Gutter and more than a dozen other Holocaust survivors, asking them hundreds of questions apiece in multiday sessions. I interviewed my dad in his bedroom; these interviews took place on a soundstage inside of what looked like a giant geodesic dome. Mounted on the inside of the dome were thou- sands of tiny LED lights and thirty cameras recording the survivors from every angle.
Using visual-effects technologies that were originally developed for military training simulators and movies such as Avatar, the project’s scientists transformed these recordings into movie clips, which, when projected onto a special screen, appear to be three- dimensional. As holographic display techniques continue to improve, the scientists will be able to create even more lifelike holograms. They could project Gutter or another survivor into any room, illuminate him as if by the ambient lighting of that space, and allow people to walk around him. Paul Debevec, a professor of computer science at USC, says that the goal is to make it “seem like they are sitting in the same room as the audience.”
ICT’s conversational-AI experts, in turn, created a natural- language system to interpret what people were asking about and retrieve an appropriate answer in the digital version of the survivor’s brain. Those answers might be short but often are many minutes long. There’s less back and forth than with the Dadbot. But this system has matching video of someone actually speaking. The system is currently being displayed in museums around the United States.
David Traum, an ICT computer scientist who worked on the conversational part of the system, says he believes that interactive preservations of the dead will become widespread in the future. If the price of the technology comes down enough, ordinary people may keep versions of their late relatives around. And conversational avatars will almost certainly become a standard part of education. Traum says students will be able to talk to Plato, Einstein, and the “top people in the world, the people who have invented things, made historical decisions, and lived those experiences.”
Fritzie Fritzshall, another Holocaust survivor who participated in the New Dimensions project, is a believer in the technology. Most of Fritzshall’s family perished in concentration camps, for- ever silencing their voices. Fritzshall, too, will die before long, and she says she is glad that her digital double will continue to share her narrative. “I have passed it on to my twin, so to speak,” Fritzshall said. “When I’m no longer here, she can answer for what’s asked of me. She will carry on the story forever.”