"The world is falling into decay – that's obvious" – overall, with this phrase, you can comment on many things. However, Olena Malkova, a producer at FILM.UA, used it in a very specific context during our conversation, commenting on the potential changes that the rapid development and widespread availability of neural networks, including ChatGPT, have caused in the established order of things in the audiovisual industry. Olena knows what she's talking about because, for her and the younger producer from her team, Juliia Mordas, the use of ChatGPT in their professional activities has been not just a theory but an everyday practice for several months now.
Based on this small but already diverse experience, Olena draws the following conclusion: "Yes, some professions in the industry will unfortunately disappear. However, new ones will emerge. So, in the end, everything must somehow balance out." According to Malkova, the industry currently lacks a clear understanding of how to utilize neural networks in practical work and why it is necessary. This applies not only to the Ukrainian industry. "During meetings with foreign partners, when we discuss FILM.UA's experience in working with neural networks, we observe that people mostly have no idea about the possibilities of these technologies in the context of film and television production. There is no logical chain in their minds on how it can be applied. But we already have it because we constantly implement it in our own projects and on demand."
This practice became possible thanks to FILM.UA Design, an innovative design studio whose team combines unique knowledge in the field of artificial intelligence technologies and expertise in the audiovisual content industry. FILM.UA Design specializes in creating titles, posters, teasers, trailers, motion design, face reconstructions, deepfakes, and more, using technologies such as AI, VR/AR, virtual production, game engines, NeRF, 3D scanning, and others.
Yevhen Sannikov, who is also the Head of the Research and Innovation Department – a subdivision of FILM.UA Group, has become the Art Director of FILM.UA Design. He has been conducting practical experiments with neural networks for years, back when ChatGPT was still unheard of among the general public.
Yevhen, who has been working in design for 20 years, is passionate about innovation and has dozens of creatively realized ideas in his portfolio. He is a true advocate for the integration of science and creativity. "When I conducted lectures for FILM.UA producers on the application of artificial intelligence in their work, I emphasized from the beginning that things will no longer be the same. In just a few years, films will not be produced in the same way as they are now (unless it's done out of inertia for some time). Therefore, knowing the possibilities of technology and being able to apply them is a competitive advantage for producers," says Sannikov.
Currently, FILM.UA Design offers its services to the market, and the market actively utilizes them. We spoke with Yevhen Sannikov, Olena Malkova, and Juliia Mordas about how AI can be applied in film and television production and thoroughly explored these possibilities – both potential and those based on practical cases.
Scripts: Research, Writing, and Proofreading
Since ChatGPT became freely accessible to users from Ukraine, many have had the opportunity to engage in conversations on various topics with it and independently assess the extent to which the neural network is capable of generating human-like text. Moreover, if you have tested ChatGPT not just once but consistently over at least a month, you may have noticed how quickly AI improves its Ukrainian language skills while interacting with Ukrainians.
Yevhen Sannikov started his text experiments with neural networks not just a few months ago, but several years ago. He experimented with the GPT-3 system, which was made available by OpenAI, using the Russian version since the Ukrainian version was unavailable at that time. During those times, neural models did not have access to the internet and could only learn from information specifically provided by developers. And "language lessons" for the neural network were an expensive endeavor. According to Yevhen, training GPT-3 in a specific language costs nearly $400,000, which requires funding from the government or a large business. Even with an extensive vocabulary, there were evident issues with the logic in the neural network. To conduct experiments, they loaded synopses of some FILM.UA projects into the network and asked it to continue writing, but the resulting texts were impossible to read without bursting into laughter. However, much has changed since then.
"Yevhen introduced us to the GPT network somewhere in December 2020, and that's when our wonderful journey with it began," recalls producer Juliia Mordas with a smile. Together with her colleagues, she shares various practical milestones that have already been achieved on this journey. Indeed, this new tool allows fruitful work with ideas. By properly providing input, one can obtain numerous variations of logical combinations of multiple plotlines or fully functional directions for the development of a complete project, starting from a two-sentence concept. "Many authors are already using AI to break free from creative deadlocks. They present ideas, based on which they further work," explains Juliia. The chat can be useful when it comes to developing characters and their motivations. Additionally, the neural network can be used as a kind of focus group. "There were cases when we used ChatGPT to test which ending to choose for a series in development. We fed the story into the neural network and asked it to come up with an ending. And when the chat suggested an ending almost identical to ours, we thought, 'Yes, this could work,'" adds Mordas. The producer also tested the neural network in the context of naming, and in this case, just like with the endings, ChatGPT "thought" almost like humans do.
Another potential application area for the neural network is dialogues. Of course, we are not talking about the semantic content and style yet, but ChatGPT is quite capable of creating the framework of a dialogue based on a given situation, which can then be expanded in "human" language. "I was impressed that the neural network writes with script formatting upon request: it uses capitalization where necessary, opens a scene, closes a scene, and properly labels dialogues," comments Sannikov on the technical aspect of the matter.
Writing a screenplay typically requires a lot of research work, and here ChatGPT can also come in handy, but it's not advisable to rely on it completely. Olena Malkova explains why: "Let's take medical dramas as an example – they require medical cases. Marina Zhuravskaya, the editor, and I have been working on such projects for years, and Marina has become an expert in surgeries, but this research – diagnoses, what to cut with a scalpel, and so on – takes an immense amount of energy and time. And recently, we realized that the neural network can solve this perennial problem for us – writing medical cases! Moreover, it does it very quickly. However, there's a catch: the network can lie, so everything needs to be verified. Nevertheless, even with verification, it significantly speeds up and facilitates our work by providing us with a basis for screenplay writing."
In addition, the neural network can create lists of experts on a specific topic and justify their relevance. "For the Risen From the Ashes project, I needed a list of ten experts from the World War II destroyed cities. I gave this task to the chat and asked for arguments to support the inclusion of those individuals in the list. It compiled the list, providing justification for the expertise of the included people. Then I asked for the books written by these experts and received a detailed bibliography. Later, we discussed this with the team and concluded that in just a few minutes of research, the neural network accomplished what an editor would have worked on for at least a day. And it did a good job. Such functionality is very useful for documentary projects," says Sannikov.
Another useful functionality of the neural network is its ability to outline loglines for synopses, and in the future, it can greatly facilitate the work of selecting scripts and script submissions.
Since ChatGPT is a "polyglot," it is capable of assisting in researching the specifics of countries and markets whose language you do not speak. Of course, you can independently search and use Google Translate for translation, but the chatbot can do it faster and with better quality. "Recently, we had a request to adapt several series to a new market and a different cultural environment. And thanks to the chatbot, we gathered a lot of information," says Mordas. "What's important is that before that, I did my own research on English and Ukrainian-language resources. However, I was able to find much less useful information than what the chatbot discovered while directly working with the language of the country we were researching." Sannikov notes that the additional "knowledge" of the neural network can also be attributed to its work with digitized books, especially those of a specialized nature, which are not available through simple searches.
Deepfake Technology for Face Replacement of Actors
The use of Deepfake AI has sparked a heated discussion in recent weeks. As a reminder, just before the premiere, STB channel released a trailer for the series The Last Letter of the Beloved, which was filmed before February 24, 2022, where the Russian actor Prokhor Dubravin was replaced by Ukrainian actor Dmytro Saransky using deepfake technology.
After a heated discussion online, the broadcaster postponed the release of the eight-episode melodrama "for further industrial consultations to discuss and ratify the use of digital technologies in content, taking into account the best international experience."
There is indeed something to consider, as discussions on the ethical issues of using deepfake technologies in films and series have been going on for at least seven years. However, while in Ukraine these discussions were sparked by the replacement of one actor with another, in the global context, the primary cause was the replacement of actors with themselves, but either younger or at least... alive. It can be said that ethics in this context were first loudly discussed when Rogue One: A Star Wars Story was released in 2016, in which actor Peter Cushing "returned" to the role of Grand Moff Tarkin, despite having passed away almost a quarter of a century earlier. It was then that, for the sake of chronological consistency in the Star Wars universe, Carrie Fisher (Princess Leia) was "de-aged" in the film.
Peter Cushing was "brought back to life" by digitally overlaying the late actor's face onto the face of Guy Henry, who worked on the film's production.
But let's return to our context. Although The Last Letter of the Beloved is the first example of such ethical controversies in Ukraine, it is by no means the first instance of practical use of deepfake technologies in Ukrainian series. This work began long before the full-scale invasion due to the so-called blacklists of Russian actors who supported the aggression of the Russian Federation since 2014 and illegally traveled to occupied Crimea, among other reasons. The first case was the series Inseparable (2013), which is perhaps the only Ukrainian artistic product about the tragedy of Chornobyl, the release of which was impossible for a long time due to the aforementioned difficulties: one of the supporting actors ended up on the "blacklist." In order to release the project on the anniversary of the Chornobyl tragedy on April 26, 2021, Yevhen Sannikov's team found the following technical solution: they invited a man whose face resembled the anthropometric characteristics. He spent half an hour in front of the camera, speaking a non-scripted text and demonstrating various facial expressions. After that, the neural network was "fed" with both old and new materials, and it gradually began to "apply" the new face to the old one, simultaneously testing the level of realism. Meanwhile, another neural module worked on the natural aging of the face, adding wrinkles, folds, and so on. Finally, the AI work was slightly adjusted in a "human" manual mode, and as a result, a quarter of an hour of the series (equivalent to at least several hundred thousand hryvnias) became suitable for screening again.
Another case from several years ago is the "barber services" provided by Yevhen's team for the series Love in Chains. The situation was as follows: the filming was completed, the set was dismantled, and one of the actors, who was clean-shaven throughout the entire series, grew a beard and mustache for his next project. And then it turned out that a particular scene from Love in Chains needed to be reshot. This meant additional time, resources, and the actor who couldn't shave due to other commitments. So the AI "shaved" him and "placed" him back into the already filmed set.
Since then, the realism of AI-generated images has only increased, and Yevhen's team has gained experience. The offering of services by FILM.UA was timely, as the market's demand for such services became more prominent. "Before the full-scale war, many projects were filmed that are now impossible to show due to legal and, primarily, ethical reasons – Ukrainian audiences will not watch Russian actors," comments Malkova. On the other hand, channel libraries require salvation, and viewers demand a constant supply of new Ukrainian content, the production volumes of which have significantly decreased due to the war.
Therefore, deepfake technology in this context is a practical solution that, however, requires timely and well-grounded public communication, as evident from the case of The Last Letter of the Beloved. But even after Ukraine's victory, the need for such services will remain relevant. "Just imagine: you're filming a series, have already made 17 episodes, and in the middle of the shooting process, the actor who played the main character leaves, and you can't 'kill' them without ruining the whole story. Or there's a need to age or rejuvenate an actor (by the way, we're currently working on such a case for our colleagues): changing age through traditional post-production methods takes a long time, so AI can provide a solution much faster," the producer gives an example of this everlasting relevance. And AI listens and learns...
Posters, Trailers, Animation
"I must confess that neural network is involved in almost every poster we work on," says Yevhen Sannikov. "Avoiding neural network generated fragments simply doesn't make sense, as it greatly speeds up the work process. Previously, to create a poster, I would have to search for additional images in the desired perspective from stock photos for a long time. Now I simply give the network tasks, such as a road leading into the distance, crashed cars on the roadside, fire, and a spring-winter landscape. And I receive a result that I quickly 'fine-tune' to the desired outcome. Without the neural network, it would take me days or even longer to create and edit such a collage."
Olena Malkova adds an example: "Have you seen the poster for Maxym Osa on Netflix? That's also the magic of human-machine collaboration: Yevhen Sannikov created it using a neural network. By the way, we showcased this poster at meetings in Los Angeles, mentioning that a neural network was used, and the Hollywood audience was amazed: even in the film industry, they haven't widely embraced this yet."
The poster for the war drama Myrnyi-21 by Akhtem Seitablaiev was also created using a neural network.
Neural networks are already capable of independently editing trailers, albeit not brilliant ones, but rather decent. "We tested the editing on the film Koza Nostra," says Sannikov. "You ‘feed’ the entire film, the full hour and twenty minutes, to the neural network and it produces an edited trailer for you. I can't say it was a masterpiece, but it was logical and dynamic: the dialogue was well selected, and the tempo followed the pattern of 'action – calm – action'. The trailer that resulted from this experiment probably wouldn't be suitable for cinemas, but even without any additional adjustments, it is perfectly suitable for TV promotion."
In the context of television, neural networks can also be used in the editing of sports event broadcasts. In particular, Sannikov explains that this has already been implemented by an Estonian company that collaborates with FILM.UA Design. "Their neural network has been trained specifically on American football, hockey, and other sports. They provided it with a dataset of edited highlights from various matches, which it learned from, and now it can edit seamlessly on its own. For example, if they need to show a goal, it selects shots with the overall context of the playing field, shows the player running and passing, provides a close-up of the feet during ball impact and the ball's flight, and inserts the crowd's reaction in the stands. Following this principle, the neural network is capable of autonomously editing highlights for different sports."
And of course, neural networks open up extraordinary new possibilities for animation – so wide-ranging that they probably require a separate material to highlight their technical details. However, let's outline just the key ones: neural networks are already encroaching on the once-groundbreaking technology of motion capture. "Mo-cap involves bulky, expensive studios, sensors, and so on. Whereas with neural networks, you can simply provide a video captured on a phone, and it will read the structure, the specifics, and the interaction of movements. Based on this video, it creates a model onto which you can then 'dress' 3D animation," explains Sannikov.
Neural networks can create an animation based on videos or graphic references much faster than animators could. Of course, experienced individuals are still necessary in the process of refining this animation to achieve the final result, but not everyone understands this and, in turn, close themselves off from professional opportunities. "We recently undertook a design project for a European television channel, which involved a scene featuring dancing individuals. Instead of relying on earlier AI versions like Midjourney and Stable Diffusion, we employed a novel technology that learns from our video animation experiments using neural networks. Upon reviewing the results, the project producer enthusiastically suggested incorporating this new approach. However, the designer expressed reluctance, citing the complexity involved as a deterrent," notes Sannikov.
Communications and Marketing
Neural networks can greatly accelerate and facilitate business correspondence, effectively performing secretarial functions. They can also create annotations, assist with formulations for presentations, find relevant references, verify press releases and other texts for conformity to how native English speakers would write them, as well as assist in market research for marketing purposes.
Let’s not forget about social media: neural networks can generate ideas for publications, select visuals, and write posts. But we'll repeat once again, they need to be verified. The importance of this is vividly demonstrated by a recent case where Novyi Kanal had to apologize for posting on their Instagram fabricated facts about Oles Honchar's life. These 'facts' were actually created by an overly creative ChatGPT, and people believed them without verifying. So, this story shows that artificial intelligence cannot replace humans or eliminate their work, but it can help make that work more efficient. “ChatGPT is a new tool that allows us to free up time from technical work and focus more on developing creative thinking. It's great that we now have this toolkit. However, I emphasize, it's just a tool. And how to use it correctly depends on people,” summarizes Juliia Mordas.
Imaginary Worlds vs Real Responsibility
The logical next step in the application of artificial intelligence for the needs of the audiovisual industry would be content created entirely in the concept of AI + Human. And work on such a project is already underway: the documentary film The Imaginary Worlds of Stanislaw Lem, conceived by Yevhen Sannikov last year, will essentially become a compilation of all the AI solutions mentioned above. "One of the projects currently in development at FILM.UA is a documentary film about the genius science fiction writer and futurist Stanislaw Lem, who was born and spent a significant part of his life in Lviv. During World War I, he moved to Krakow, where he became famous," reminds Olena Malkova of the overall concept of the project, which already has a teaser and is ready to seek international partners.
In the past, creating such a teaser would have taken a considerable amount of time: writing the text, translating, finding an actor and recording the narration, editing, making corrections... But with artificial intelligence, everything was done in a week. And it only becomes faster. However, this doesn't mean a reduction in cost: while using similar solutions provides brand-new creative possibilities, and significantly speeds up and streamlines production, it doesn't make it much cheaper, although it does change the cost proportions considerably.
"Tablets, nanorobots, virtual reality, AI – all of this was predicted by Lem long before it became part of our lives," the producer continues. "Artificial intelligence held a special place in the works of the writer: he was skeptical of the idea that AI could replace humans and their creativity, but he recognized its ability to surpass human abilities in certain fields. Therefore, Lem believed that humans and AI should 'collaborate' to change the world we live in for the better. That's exactly what we, as a team, are doing – integrating cutting-edge technologies into the creative process and striving to 'collaborate'. We thought it would be cool if a film about Lem was created together with AI. We made a teaser in which all the images were generated by AI under human supervision, the text was written together with GPT, and the music and narration were generated by AI directed and edited by humans. And such a result could only be achieved through synergy."
However, despite the genuine excitement about the possibilities that the use of cutting-edge technologies opens up for creative industries, it is important not to forget about caution when working with them. "It is crucial to understand that currently, nobody knows exactly how the algorithms of a chatbot, especially GPT-4, work or how they gather and process information," summarizes Olena Malkova, drawing on her own experience with AI. "We need to learn how to navigate this new reality, as a significant number of ethical questions, discussions, and attempts to limit the use of neural networks and their inevitable further development await us in the near future. Everyone who utilizes them becomes both a tester and a teacher of these programs. Therefore, our team takes all possible risks and changes brought into our lives by neural networks very seriously."
