History brought back to life with AI

You must have heard the question before, “If you could have a dinner party with any three historical figures, who would you invite?” Would it be Albert Einstein? Maybe you prefer the company of Leonardo Da Vinci or Nicola Tesla? Frida Kahlo or Florence Nightingale? Would you request a tyrant, an artist, a humanitarian or a war hero? 

This thought experiment recently became not so far-fetched of an idea. This is what we can soon expect; applications and exhibits that give these famous characters from history a second life. They will talk to us on our devices, welcoming us in the halls of museums and art galleries. We will hear their voices read passages from their written works. They will tell us stories from their lives and the times they lived. We are in the beginning of a new age—the age of the digital resurrection. This might sound like the plot of a Black Mirror episode. But its true. Soon these AI-driven replicas will guide many experiences for us, for better or worse. They will live among us. 

For more than a year, our team at MoMoLab has researched the possibilities. We considered ways to combine history, folklore and art with modern technology, storytelling, and game design. We planned new experiences incorporating digital avatars into our projects. In this process, we developed some exciting new concepts for interactive experiences allowing visitors the chance to interact with characters from the past. Imagine having full dialogues with people from the past. It will mean history brought back to life. We studied how historical figures could be given 3D faces, bodies and clothing, and more importantly, personalities, replete with voices, backstories, and gestures. We tested each step in making this type of experience as robust and enjoyable as possible, and pitched the idea to interested clients. 

It turns out we were on to something. In the past six months this concept, which at one time seemed prohibitively expensive, complicated, and time-consuming to develop, is now possible. In fact, it is relatively simple to implement (albeit still time-consuming) with the use of readily available tools and software. 

The rise of digital avatars 

The biggest steps forward in this new medium have been in online marketing and customer relations. Since we started our research, an entirely new industry has popped up offering real-time avatars. These services offer avatars to improve online visitor engagement and to drive sales. The common use case is for these avatars is as the face on company websites, used as approachable sales assistants. These avatars are the latest iteration of interactive chat bots, answering queries and assisting with online navigation. Other services, such as HeyGen, are promoting digital avatars as video presenters, giving marketing teams the chance to type copy text and render out videos with surprisingly convincing voices and gestures—all while saving a small fortune on video production costs. 

Other software-as-a-service companies specializing in real time avatars, like Uneeq, claim these avatars improve the user experience. Uneeq claims their clients recorded a 350% increase in online visitor engagement due to these live avatars. They also claim they double the rate of conversation between visitors and chatbot. These numbers are incredibly attractive to many businesses. While it seems that much of the heightened engagement stems from the peoples’ interest in a new shiny toy, it is apparent that many find this technology compelling. Having tested it out myself, I can attest to how persuasive it is. It is far more interesting to speak directly with these avatar chatbots and get realistic answers in return rather than typing text bubbles.

All signs point to this trend continuing. This is especially true as the technology used to generate these digital avatars, voices and gestures become ever more convincing. These types of generative characters will soon be pervasive in our world. They will be business ambassadors to improve visitor engagement. Game studios will develop them while striving for the most unique and immersive games. And even individuals will spend their time with them longing for synthetic companionship. The eerie and slightly dystopian future depicted in films like Her is now a reality, with all the messiness you might come to expect. 

AI models in history museum exhibits 

Shortly after we mapped out the necessary steps to develop our concept for historical characters brought to life, a startup in France called Jumbo Mana, presented a version of this same idea. They launched an exhibit at Viva Technology 2023 along with art historian, Dr Wouter van der Veen, called “Hello Vincent.” This project allowed visitors to have (almost) real-time conversations with a digital representation of Vincent van Gogh.

Tech news circles celebrated Hello Vincent as a great leap forward in generative AI. This project then moved to Musée d’Orsay in Paris. There the digital facsimile of Vincent van Gogh gives museum visitors an entirely new way to interact with the art and history of this artist. This digital twin was built on an AI model composed of details from his life, a database of his real letters, and his catalogue of artwork. The results, while not perfect, show the power and scope of this concept. It also poses some new concerns that deserve discussion. 

How real can these digital twins be? 

Now that digital avatars from history are in museums near you, there are some ethical considerations that need addressing. Such avatars, though constructed with meticulous detail, are inherently incomplete representations. They rely on fragments of a full and complicated life story. These avatars built on language models which, despite their sophistication, inevitably inject interpretations, assumptions, and even errors into the portrayals. In truth, this is no different than the issues surrounding biopics depicting historical figures. Just as Daniel Day-Lewis’s portrayal of Abraham Lincoln in a biopic is a skilled interpretation rather than an exact replication, so too are digital avatars mere interpretative representations. They embody the fusion of historical research and creative interpretation. They exist primarily for entertainment rather than as true reflections of reality. This is how they should be evaluated, as interesting portrayals of historical figures and new mediums for storytelling, not 1:1 recreations. 

The interactive experiences centred on digital avatars may be avoided and even chastised by many museums that pride themselves on accuracy of the historical record. This accuracy is often more important than the entertainment and interactions of guests. In many academic circles and archaeological institutions, the emphasis is on looking at the hard facts. This type of experience takes too many liberties for conservative organisations. But for other museums or visitor centres, these interactions might be the perfect vehicles to convey crucial information and inspire visitor engagement.  

Still despite all the progress, many of these avatars are still creepy. The Uncanny Valley factor is still a major obstacle of the wide-scale adoption of these types of interactive experiences. Right now, even the best facsimiles still have a certain stiffness and repetitiveness to their movements. Their facial expressions makes them feel fake and lifeless. They often fall short and become poor interpretations of what it is to be human. Currently it remains a challenge to find that right space between realism and stylistic interpretation. Often the best results for these experiences are from art directions that do not strive for realism. Aiming for stylized representations often works best. 

The uncanny valley effect is a hypothesized psychological and aesthetic relation between an object’s degree of resemblance to a human being and the emotional response to the object. Examples of the phenomenon exist among robotics, 3D computer animations and lifelike dolls.

Wikipedia 

On top of the visual obstacles, there are still issues with making real-time reactivity between the users and the digital avatars. The amount of computing power required to make these experiences happen with as little latency as possible cannot be ignored. The faster the applications, the more expensive and resource-heavy it will be. This does not consider the energy needed to power all the data centres humming away while processing these language models. There is a cost to generative AI that is currently eclipsed by its techno-optimistic promises. 

Another major obstacle to the true-to-life quality of these exhibits is the data wrangling needed to make each digital twin. An incredible amount of data is required for training models to form the backbone of the realistic personalities. A historical figure’s body of work needs to be collected; a large part of what they wrote; what was written about them; the historical context from the time they lived. It will take a discerning eye, a new form of historiography, to make these characters feel true to the real people. Without this attention to detail, the ChatGPT and other LLM will take many liberties. In the end all the characters generated will sound and think alike. There is also a chance that the less flattering parts of these historical figures’ lives may be scrubbed from the model’s database to hide the warts of the past. 

Coming to an exhibit near you 

Despite all the complications and potential pitfalls, we at MoMoLab remain fascinated by this concept. Our AI creations will someday soon take centre stage of an exhibit near you. This idea is still central to many of our plans. Our team is looking for the best approaches forward, ironing out the details to bring such a character to life. We are finding the best ways to depict history in an educational, informative, and interesting way. All the while, we will preface the inherently incomplete nature of our recreations. Now is the time to see how this new form of storytelling can add new context to historical works. 

For more information on the subject, reach out to our team. Or if you wish to consider other ways digital twins can improve the experience of your history exhibition, send us an email or call.