The new reality, in which we've been living for the past four months, requires from the industry non-standard solutions, completely new projects, and unexpected partnerships. Recently, we've seen many examples of such synergy – from the United News telethon to the Ukrainian Content. Global Cooperation initiative. Today we will talk about another example of a successful joining of efforts, but now within one company. Until February 24, the head of FILM.UA Group's Research and Innovation Department Eugene Sannikov, and producer Olena Malkova have been working on different types of projects: Eugene was focused on technologies – from content creation using game engines to experiments with artificial intelligence and NFT, while Olena was mainly shooting game series (her works include digital cycle Back to Basics and the documentary film Back to Basics. Wedding). But with the beginning of the full-scale war, their teams combined efforts and competencies, launching more than ten projects with a powerful social mission – from helping the state in reconstruction to restoring historical justice.
– I think this synergy was born out of friendship, – says Eugene. - At FILM.UA, it usually happens like this: collaborations are born from personal sympathies and the comfort of working together. Olena and I have a good experience with joint projects: I worked on the first and second Back to Basics, and I made credits for several TV series. Moreover, our department was created to help producers – to make their projects easier, faster, and more financially efficient. And I advise all producers who contact me, who are interested in technologies, ready to consider and understand them.
– On February 23, all the executive producers, authors, and editors with whom I worked were busy with startups, production, and writing new scripts, – Olena recalls. – And on February 24, everything stopped. But we didn't stand still for a single day, because we quickly regrouped and ran in another direction. Maryna Zhuravska, Vira Loy, Yulia Mordas, Tanya Nosenko – literally on the very first day [of the full-scale war], we started working on the Back to Basics project together. We added English subtitles that explain that this is the same Ukrainian culture that russia is trying to destroy and cancel. Various departments of the company helped us with this, as well as Viktoria Yarmoshchuk, and Zoya Soshenko – everyone got involved. After releasing the updated Back to Basics, which our distribution helped to spread in many territories, we immediately started making relevant short videos, the production of which involved Eugene.
One of these videos was a clip about Ukrainian women participating in military operations
– And then we realized: if we can't film now, then it's time for documentary projects. And then it turned out that Eugene, who once created the cycles Great Dreamers and History of Criminalistics, is almost the only person in the company who year after year methodically accumulated ideas, developed documentary projects, and waited. There were so many treasures!
– Why were these ideas not implemented?
О. М.: It's no secret that documentaries (an amazing genre) were not in demand on Ukrainian television. So new ideas were accumulating in Eugene's head. His documentary work has always been at the interface with technologies: AI, AR, and VR. In general, we adopted Eugene's ideas and now we have 11 beautiful projects on the shelf, each of which can be done right away whenever we find a partner. Together with the management, we have chosen four main projects. To start developing scenarios for them we are looking for new authors or contacting those with whom Eugene started the projects.
– The headliner on this shelf is the documentary series Rising from the Ashes, which has also been in the works for quite some time.
E. S.: Back in 2012 or 2013.
О. М.: This project is about destroyed and restored cities. There was a great teaser shot ten years ago, a great presentation, and a lot of work, but no one thought Ukraine would be a part of this project when the idea came up. It was planned to include 8-10 cities – Warsaw, Dresden, Rotterdam, etc. – and simply tell how they were rebuilt after the Second World War. Today we have Chernihiv, Irpin, and Kharkiv – a large number of points on the map of the country have turned into ruined cities. Of course, we immediately grabbed this idea, but now we are looking at it from a completely different angle.
E. S.: Because the project was conceived as a Discovery-style story. Here is a city in ruins – tentatively, Dresden. Here are fascinating numbers and facts: how many days it took to clear the ruins, how many bricks were used for construction, how pre-war building plans were found, and residents were given the option of rebuilding in a new way or trying to reconstruct the old one, how the renovations took place... And when the war in Ukraine began, we realized that we could no longer tell the story in the Discovery style – we were the story. And many human destinies and tragedies are intertwined here. Therefore, we are now looking for the tonality of the project.
– How do you intend to unite European and Ukrainian cities?
О. М.: We want to use their experience of rebuilding cities, to show what that situation taught them, because, through the destruction, a new story emerges – for the country in general and us as a nation. There is a certain metaphor and symbolism in this destruction. From the point of view of Ukrainian cities, our project is not about what was destroyed, but about how we will restore it. Sofiia Chemerys – a very kind, experienced director – joined the development, and we already know for sure that we want to make eight episodes: we are currently looking for foreign partners for each. To be honest, the concept is not completely thought out at the moment, but this is both a weak and a strong side of the project at the same time.
– That is, the European cities destroyed during the Second World War will not be considered independent heroes, but an experience that can help Ukrainian cities, is that right?
О. М.: We are using eight European cities, and each of them will have a Ukrainian counterpart – certain connections, and similar historical experiences. In the eight-episode series, we will talk about European cities in detail, but Ukraine will be a horizontal connecting line. And in the future, we want to make a full meter, in which more attention will be paid to Ukraine, and the rest of the cities will act as support. There are so many layers to this story, and we find them now in endless arguments and correspondence. First of all, we are looking for a balance between the human stories, the technological things that will help visualize all this, and what our director wants.
The project is growing rapidly – we have a great architectural consultant Oleksandra Gerasimova, who helps with both architectural and historical issues; we will involve military consultants, authors, and historians. Eugene, with our executive producer Tatyana Nosenko and her team, filmed flyovers of Irpin with drones and made 3D scans. Using the scans the Department of Innovation, including creative technologist Valentyn Kravets, can make models of the destroyed objects. By the way, some of the footage was included in the Discover Ukraine video about Ukrainian locations, updated for the Cannes Film Festival. We also managed to capture the destroyed Mriya plane.
We are already thinking about implementing the presenter in the destroyed objects to create the feeling that he is walking through them. To achieve the necessary volume for this, we need 3D scanning. We are preparing to develop the first teaser and a concept to show [otential partners – the studio will not produce Rising from the Ashes on its own. In addition, we want it to be not just a television project but perhaps a common reference point to help reconstruct the country.
– Tell us more about 3D scanning.
E. S.: Scanning using the photogrammetry method: the drone methodically, square by square, flies around the selected object, taking a thorough picture. It can fly over one location for an hour. As a result, we get a very detailed 3D model of the object, accurate to the centimeter, which can be used not only in the content. For example, architectural bureaus, and Western architectural enthusiasts, can create ideas for reconstruction based on this model. In specialized programs, it is much faster to work with a 3D model.
This is what a scanned area of 10 hectares looks like after photogrammetry
Moreover, all images have a very high level of detail
– What objects have already been scanned?
E. S.: The 3D scanning process is quite expensive, so we cannot cover many objects. In Irpin we covered only a few significant ones – the central House of Culture, built in the 1950s, a factory, the owner of which is one of our potential heroes, and the residential complex "Irpinsky Lypky", where guests from other countries and journalists are coming now. For example, when we came there to scan, we were not allowed in at the appointed time, because the ministers of foreign affairs of Lithuania and Latvia were there.
Of course, we do not have the resources to fly around and photograph the entire Kharkiv, for example. But we can point-scan some objects. This way, we would not only be filming the project but also conducting scouting for Western specialists in architecture. Given these scans, they will be able to participate in the future reconstruction of the country. In addition, 3D models can be used for a virtual studio.
– Could you go into more details?
E. S.: The Mandalorian is the most famous example of content created in virtual production. The virtual studio is a room consisting of high-resolution LED panels, even the floor is made of LED panels with reinforced glass (the only studio that exists in Kyiv can accommodate a car if necessary). The screens display a scene – an image built on the game engine. There are sensors on its perimeter and cameras. When the cameras move, the system understands their coordinates, adjusting the virtual scenery to the location and perspective strictly in the visual field of the cameras.
LED lighting provides realistic colors and light reflections on actors and props that cannot be achieved with chromakey
– Does this mean that in Rising from the Ashes the presenter will not work in an actual location but in a virtual one?
E. S.: Yes. If the presenter is a celebrity (I hope so), we can shoot them in any European city: we send the prepared scene and they just go into the setting. Moreover, you can add infographics, and various chips and the host will interact with this virtual scenery. I have always liked such elegant solutions when a lot comes out of one thing. In this case, we fly out on a drone once and receive a model for architectural bureaus that want to participate in the renovation of Ukraine; virtual scenery, thanks to which the presenter could be just inside, for example, the destroyed House of Culture; as well as a model for visualization, infographics, a documentary film that can be rotated, showing everything from different angles.
– By what criteria are the objects selected?
E. S.: In Irpin, by the way, all these criteria were met on this shooting. The first category is important historical objects. To be honest, I don't know if the House of Culture is an artistic heritage, but it is significant for the residents of Irpin, especially the elderly, it was one of the oldest buildings in the city, as I understand it.
The second category is the most affected buildings, such as the "Irpinsky Lypky" housing complex. This is the place where they bring most foreign delegations.
О. М.: As our director says, Irpin and Bucha are about young families, cozy new buildings where everyone wanted to live. And "Irpinsky Lypky" is the very heart, a fashionable and beautiful house. Everything is new – lanes, parks. And now there is nothing. Just a house to be demolished...
E. S.: The third category is objects related to human stories. The factory we scanned was inherited by one of our potential heroes from his father. It exported its products to a large number of European countries, but now it is destroyed. Two of his sons helped the Armed Forces during the occupation of Irpin, and both were wounded: one was seriously wounded in the stomach, the other in the back. In addition, their house was hit by a bomb. But that's not all – the debris broke the cross on his parents' grave at the Irpin cemetery. So his factory is gone, his house is destroyed (although there are whole houses nearby), both his sons are wounded, and his parents' grave is also shot... Therefore, we also scanned the factory, and in this way, you can make virtual bridges between European cities and local stories: for example, the presenter approaches the factory from one side, being in a virtual studio, let's say, in Berlin, and Vova appears from the other in Irpin.
Also, we are currently trying to apply one AI algorithm to 3D scanning. That is, it will no longer be photogrammetry but the transformation of 2D images into a 3D scene. If drones make a snapshot from above, then with the algorithm, it will also be possible to make digital snapshots from the height of human growth. This is NVIDIA's NeRF technology – we have already received this algorithm and are starting to test it.
Посмотреть эту публикацию в Instagram
This is the way to an interactive project where it will be possible to "walk" through all the destroyed cities, and rotate them in a video from all sides.
– How expensive and resource-intensive is it?
О. М.: It's hard to say at the moment – this is a pilot thing. We have a certain fixed figure allowing us to produce pilot materials, a teaser, a scene plan, and minimal development to show potential partners. In addition, an important factor is finding accessible locations free of mines, and many similar, purely technical issues. It is now much easier to get to Irpin than when we were filming – only getting there through all the roadblocks and inspections took half a day. We have to give credit to Tanya Nosenko, who organized all this.
– Let's talk about other projects. How did Born in Ukraine appear?
О. М.: During the first months of the full-scale war, while we could not film, the FILM.UA Group was doing something all the time, constantly releasing new videos. Producers were inventing new things because the content is our weapon, a contribution to the common cause. Subsequently, we began to think about what other relevant content can be produced inexpensively, roughly speaking, with improvised means. Maria Kvitka, with whom we once made Back to Basics and were also in touch all this time, came up with the idea of the Born in Ukraine project. It is about famous Ukrainians who were "appropriated" by russia. Together with Maria, we have chosen the characters for eight episodes of the first season (and we already decided on the characters for the second season). Our permanent author is currently writing the scripts, and the universal template developed by Eugene is the visual basis. The theme music is an ambient track once written by the great guys for the Back to Basics movie.
– Judging by the length of the episodes, is this digital content?
О. М.: Yes, we will post it on YouTube and other FILM.UA digital platforms. But not only. We are already translating the project into English (here is a special thanks to the MRM company, which also participates in the production of the project, helping with the translation). Together with distribution, we will think about how to sell it.
In parallel, we are doing the second cycle – Lost Names. Different graphics, different ideas, but the same team and the same approach. In general, this is an endless drive: russians steal paintings in the occupied territories, and our screenwriter has come up with an interesting way to look at it – starting with the Germans, who during the Second World War, stole paintings from everywhere. In Kharkiv, the russians are bombing the genetic bank of plants, and we are thinking about how to tell about it now. We are developing a project about recipes for Ukrainian cuisine and household items of Ukrainians – we are looking for solutions at the intersection of animation and technology. We have a lot of ideas for short, relevant content, and all this needs to be done.
In addition, Eugene has an amazing project called Kyivan Rus. Imagine finding yourself in those times, in the central square, and not knowing what to do. Here is a guide for you: where to go, what to eat, and where to wash.
E. S.: The idea was born four years ago, and Kyiv Rus was envisioned in a hipster way: where is the best smoothie in ancient Kyiv, and where are the shopping rows (laughs).
О. М.: Yes, in a modern language with modern features. And the most important thing is that it is a format. You can do the same thing for 13th century Istanbul, for example. This is a mixture of documentary, animation, and, again, technological gadgets.
– What animation are you talking about? The one made on the game engine?
О. М.: No, we want something simpler – perhaps something along the lines of Netflix's How to Become a Tyrant project. It turned out that our editor-in-chief, Maryna Zhuravska, was a multiplicator in her past and perfectly understands how it works. So we will try.
E. S.: I want to combine animation with video in augmented reality: for example, shoot the Golden Gate, and add AR to it, so that all this is complemented by a walk through the city.
– But behind such simplicity and lightness, there must be a serious historical basis.
О. М.: Of course, we are going to invite consultants. But we will get in trouble anyway because every historian has their own opinion. The name itself Kyivan Rus causes a million disputes (smiles).
– Then let's get back to technology. Last year, you replaced a russian actor in the series Inseparable using artificial intelligence. I think this will be a very common request now.
E. S.: We are calculating and testing this service for one TV channel now. In a year, technologies have advanced, new graphics processors have been released, and while in Inseparable we couldn't take close-ups, now we can. Then we will be able to remove russians from TV series even more easily (laughs).
– Considering the number of projects in the works, are you recruiting people for the Department?
E. S.: Yes, we are always open to meeting market professionals, there is a lot of work.
О. М.: Of course, in an ideal world, all this relevant content should be created by other people so that Eugene can generate new ideas. Here is an example of a project he came up with after February 24. The famous futurologist and science fiction writer Stanislav Lem, as it turns out, spent half his life in Lviv. When the Germans arrived, he was there – this is our shared history with the Poles. And Eugene, while he was in Lviv, thought that it would be good to make a film about Stanislav Lem.
– Feature Film?
О. М.: No, a documentary for 60 or 90 minutes. This man predicted a lot of what is happening now. And what influenced him to become the way he was? First, the film involves the reconstruction of his life, and second, the search for answers to these questions. He thought a lot about the essence of artificial intelligence, and this is also one of the storylines. Eugene worked with an artistic neural network, setting it up in a certain way, and it made incredible posters. We would like to make a part of the project's visuals (all concepts for sure) with the help of a neural network.
It would be cool to tell the story of a science fiction writer who has reflected a lot on the subject of artificial intelligence, using artificial intelligence. Here, of course, many new and unusual legal questions arise: for example, who owns the rights to what a neural net generates? But I hope we will solve the questions. We already have an author who is developing an idea, we are writing an application and preparing a presentation for Polish partners because this is a co-production project. And when we make one about Lem, maybe it will become a whole cycle of science fiction.
– What other services does the Department provide?
О. М.: Eugene makes excellent titles for TV series: Hide and Seek, There Will Be People, Coffee with Cardamom... This can be a separate direction – we are currently preparing a website and a showreel for it.
E. S.: And among technologies, primarily, Deepfake, since many russian actors have starred in Ukrainian TV series.
О. М.: It turns out that today the channels are left without a large part of their libraries. But whether they will now have the money for such services is a big question.
– On the other hand, replacing an actor in an 11-minute episode, as was the case with Inseparable, is cheaper and easier than making a new series.
E. S.: We are not limited to this technology and together with the technical director of FILM.UA, Philip Petrenko, we are exploring interesting algorithms that are currently unavailable, are in closed beta, or are simply very expensive. He is contacting development companies and hopefully, they will help us with these technologies.
– For example?
E. S.: Automatic re-lipsync technology can be applied not only to video but also to animation. Let's say there is an English-language version of Mavka, and for the French market, the algorithm will change the characters' lipsync to the French track. Or Chinese. The neuronet does this automatically, and it makes the movie much more pleasant to watch.
– Doesn’t the picture lose quality in this case?
E. S.: Not at all. We tried to do it ourselves, and something even worked out, but not in production quality. Therefore, we contacted the company that owns this technology, and we are waiting for a response. The same story with Instant NeRF technology, which allows you to capture any scene in 3D, and make it as detailed as possible. If we can't handle it on our own, we'll ask NVIDIA for help. Then we will be able to use it not only for movies but also for some architectural measurements or for helping the police in recording the destruction. So, perhaps, the difficult situation in which we all find ourselves, paradoxically, will help us to unlock new technologies that were previously unavailable on the Ukrainian market.
