Franz Rosati: The Collapse of Truth

Pau Waelder

Musician and digital artist Franz Rosati recently presented on Niio a series of three videos titled DATALAKE: GROUNDTRUTH (2024) in which he worked with AI models to generate mesmerizingly fluid landscapes that evoke chaos and disaster, but also regeneration and impermanence. Widely recognized on the international digital art and electronic music scene, Rosati creates a unique combination of visual landscapes, soundscapes, and cinematic narratives that embody the ideal of the total work of art (Gesamtkunstwerk) for a fully immersed audience.

In a previous interview for Niio Editorial, on the occasion of the launch of Rosati’s series LATENTSCAPE (2021), the artist offered us a general approach to his work. In this article, we focus on his most recent series, diving into his use of machine learning programs, his reflections about our current visual culture, and the elusive concept of truth.

Franz Rosati. DATALAKE:GROUNDTRUTH N1, 2024

This series brings to mind our fascination with disasters, which has been enhanced by digital technologies allowing the creation of ever more spectacular images of cataclysms in films and TV shows, and the growing tendency in mainstream media to feed us with disturbing images in the midst of the raging attention economy. Would it be correct to interpret the artworks as a commentary on our current media landscape? 

I don’t know if I want to see the work as a commentary on the media landscape, this is not the very first target of the project. 

For sure the theme of disaster, the cataclysm, and the overwhelming flux of visual information, has extended not only to the natural and ecological context, but also to the perception of a cognitive vortex. So the disaster as something disruptive, unexpected and out of control, is exactly what emerged during the production and the selection of the final materials.

I do not start any of my projects with a specific political or critical theme, it’s a pretty more subconscious process, I just start doing things and stop when something resonates with some parts of me, then I try to figure out why it resonates with me so much. This is why I’m very interested in understanding how my work might adhere to a vision of reality.

I think every artwork is political in some ways but I also think that every artwork must speak for itself, it’s not about releasing a statement through images and sounds, but instead leaving it there and observing how it resonates with the surrounding environment.

In this case, after months of generating materials and testing sounds, I decided that I wanted to catch the vibe of a visual multiverse inflated by images, that through an inescapable hyper-aesthetization process, disastrously decorate and beautify its own impermanent deflagration and collapse.

“After months of generating materials and testing sounds, I decided that I wanted to catch the vibe of a visual multiverse inflated by images that disastrously decorate and beautify its own impermanent deflagration and collapse.”

The use of split screens in some scenes reinforces the connection with newsreels and mass media. What do they bring to this project, in terms of visual composition and subtext?

The splitscreen aesthetics idea came out in an advanced phase of the work and was absolutely the most difficult part to do from a technical aspect.

I didn’t want to do it by using masks to tell AI where to put different things, which is a pretty common and straightforward approach. Instead I wanted to let AI interpret several words in the prompt, going back and forth to see how a comma or the weight of a specific word can alter the results and recall some features in the model. This was a pretty long process.

The screen is a very iconic and persistent element of our visual culture since Nam June Paik (and before him Guy Debord, Isidore Isou or Abel Gance) which was probably the first in visual arts to understand how powerful could it be, but today we have incredible artists such as Baron Lanteigne who is able deal with a simulacra of the screen in extremely powerful, subtle and unexpected ways, as in the content so in the setup of his works.

The splitscreen in this series is my “tribute” to the device, in its embodiment as a magic portal, a simulacra, a magic surface opening up to other worlds. A multidimensional device shaping our reality more and more, first in our homes, then covering walls and buildings of our cities, then fitting in our hands and pockets to slowly landing in the art galleries and exhibitions.

In my case I wanted multiple screens representations to be a strong part of the artwork: The television layout which is the one telling what’s happening in our world shaping our perception of reality as well as the LED wall or the stage design which is instead the most diffused example of mass worship during concerts and large events. 

“The splitscreen in this series is my «tribute» to the device, in its embodiment as a magic portal, a simulacra, a magic surface opening up to other worlds.”

I want these elements to be almost like a living organism that is symbiotic with the natural elements, absorbs nature and is absorbed by nature. Virtuality and biology melted together.

So it is not just a way to visualize different contents and context. It is more about the splitscreen, the “news layout”, the LED wall, as a part of a new and complex hybrid ecosystem.

Franz Rosati. Still from the series DATALAKE: GROUNDTRUTH, 2024

An interesting detail in these scenes of chaos and destruction is the presence of cars. Artist Iñigo Bilbao once stated: “Confronted with images of a disaster, in order for them to cause us a real (and perhaps pleasant) impact, we need to identify something among the chaos, a reference that informs us of the scale of the tragedy […] Apart from the most macabre evidence, cars offer an easily recognizable shape, allowing us to glimpse the seriousness of the accident, the power of the bomb or the height of the floods.” Is the presence of cars in these artworks connected to this sense of scale mentioned by Bilbao? Or maybe it is something embedded in the imagery of disasters that AI models replicate?

Cars and humans have a very accidental yet central role in this work, retrospectively. That wasn’t planned at all, my idea at the beginning was only focused on depicting this process of nature, architecture and technology struggling to coexist and find their place, fighting, morphing and twisting.

I tried everything to totally get rid of both humans, cars and vehicles, and keep only images of technology. For some reasons the CLIP (Contrastive Language-Image Pretraining) part of the code responsible for learning and connecting visual concepts from natural language supervision, was strongly trained with tons of images labeled as cars, vehicles and people or something strictly related. So the semantic domain of technology is not separable from cars and vehicles.

The observation made by Iñigo Bilbao is revealed to be dramatically true and fitting in this case. In the end I decided to keep cars and vehicles and play with them, giving them a role, as a reminder of the many biases we can discover in AI, and how that speaks so much about our world and how some peculiar aspects of our society will always percolate into the tools we use and back in a recursive process, shaping their functions and role in our evolution or progress. 

This could lead to a very simple deduction: The tool is the message or at least the part of the algorithm generates part of the message.

“I wanted to let AI interpret several words in the prompt, going back and forth to see how a comma or the weight of a specific word can alter the results. This was a pretty long process.”

The soundtrack is a crucial element of the artworks, as it brings in a strong emotional element to the images. As a musician, how would you describe the role of sound in this series?

Sound is very central for me, for many years I’ve worked with materials gathered from streaming platforms, and social networks as well as field recordings or unusual sounds, trying to build some kind of accidental sound design and music from the social bubble as Burial first taught us.

I always liked the realistic imprint coming from a mediated support, the new sonic landmarks and the emblematic sounds of audio compression and the limits of some new media devices; it’s like collecting field recordings in a digital world but recorded by non-professional devices such as smartphones, dash-cams, live recorded concerts, cracks, crashes, impacts, then reinforced with classic sound design process. All of this is mediated by audio compression and so on, ending up building a totally different soundscape. The question in this case is: we’re immersed in a virtual multiverse since the ‘90s, so what has become of our listening and hearing process from a cognitive and perceptual point of view?

In the first iteration there was a very strong musical presence, since it’s more than a couple of years that I’m working on the integration of real and AI generated strings, especially cellos, and my first thought was to go in that direction.

What you hear is basically the sound of disaster, something close to white noise on a liminal level but full of patterns your brain can hook up to.

In this case, I decided to use AI systems to produce a sort of massive but realistic sonic avalanche instead of composing music.

By analyzing some frames of the videos using Llava, I was able to generate a prompt to feed a Latent Diffusion Model for audio generation and enrich my AI-Foley Sound Library which was built around 2021 with the Latentscape project using SampleRNN at the time.

“What you hear is basically the sound of disaster, something close to white noise on a liminal level but full of patterns your brain can hook up.”

I came to this conclusion after seeing the workshop Martina Carbone and Daniele Imani Nobar, which also works with me as assistant, held for Re:Humanism. They focused on using AI for Foley in a classical sense, generating and designing sounds for silent movies.

So after browsing my Sound Library and after listening to it repeatedly I then decided to only focus on these massive realistic sounds, following the morphings and the behavior of the images, while pushing the musical layer in the background, beneath all the other sounds.

At the same time the sound design workflow for an artwork series is very different from the one I use for my music or audiovisual performances. 

For this reason, Mattia Magionami‘s support has become essential in the last year. I usually come up with a general workflow, a well defined sound aesthetics and an overall idea for some details, I make some starter templates and collect or generate materials. Mattia then takes charge of all the creative editing, synching, mixing and mastering of the final piece.

Franz Rosati. DATALAKE:GROUNDTRUTH N2, 2024

Another mesmerizing aspect of these animations is the fact that they are constantly morphing, without end, every disastrous event merging into another. The scenes therefore are captivating in presenting a thrilling spectacle of chaos, but at the same time, since the disaster is not permanent, but part of a flow, it does not seem like a tragedy. What have you found interesting in the technique of morphing and constant transformation?

I always worked with chaos intended as a high number of complex events happening together. This definitely depends on my fondness for musicians like Iannis Xenakis or Zbigniew Karkowski, Masami Akita or Paul Dolden, just to cite a few who had the first relevant impact with contemporary culture in my teenage years.

Chaos is potentially where you can create the most interesting connections, a pool of everything happening simultaneously, to be explored and where to find unexpected patterns and relationships.

When we talk about AI, learning is one of the key terms, but remembering is never mentioned. I feel that learning and remembering relate to a different sensibility rather than applying the same exact mechanism. The ease with which mass media and people tend to forget things is a crucial issue in our reaction to information overflow.

The constant morphing flow of images and events, forces you to check for patterns, to hang on landmarks in the screen space and try to learn what you’re seeing and remember something to not get lost in the flux and start to decode what is happening.

“Chaos is potentially where you can create the most interesting connections, a pool of everything happening simultaneously, to be explored and where to find unexpected patterns and relationships.”

Going back to the theme of disaster, this continuous morphing reminds me of that specific instantaneous feeling you have while the disaster is unfolding: a fall from the bicycle, or something more tragic, for instance, not necessarily physical.

In that moment your brain freezes the time: the unexpected event leads to a condition where it’s impossible to find patterns, and time stretches to infinity until this perception ends and you go back to a new, a new different reality to be re-semantized.

Franz Rosati. DATALAKE:GROUNDTRUTH N3, 2024

Both the images and the soundtrack have been generated using AI models. Can you explain the process of creation, and the differences between generating images and generating sound with AI? Given that you already worked with generative algorithms, what do these artificial intelligence programs contribute to your creative process?

I don’t want to go deep into technical stuff about workflows and code. I’ll focus on the creative experience with Generative AI compared to “old school” Generative Algorithms.

There are very huge differences in some aspects. We’re confronting with the same kind of approach at the foundation, based on giving rules and observing narrow variations as results, but we’re dealing with different kinds of complexity.

Dealing with Generative AI means we have to handle visual data condensed in a Latent space and then used to represent words we can prompt. It’s like a magic spell, and numbers are hidden behind this, since the final users have very few numerical parameters.

In “old school” generative process, we’re dealing with numbers and parameters in a very Pythagorean way, using equations, working with euclidean spaces, coordinates or controlling emerging features and systems behaviors.

In a creative approach, this leads the artist and the creative, in a very peculiar position. At the same time this technology is evolving so rapidly that is very much subject to paradigm shifts.

“When we talk about AI, learning is one of the key terms, but remembering is never mentioned. The ease with which mass media and people tend to forget things is a crucial issue in our reaction to information overflow.”

In this series you introduce the concept of “ground truth”, which is used in statistical models, cartography, meteorology, and the military to refer to information that is known to be true. Evidently, the fictions created by 3D simulations and AI models are putting into question what we can call “ground truth.” What has led you to work with this concept in this latest series?

Ground Truth is a technical term used in AI as well as Geographical Information Systems or Remote Sensing (which I used a lot in projects such as Distantia from 2023) and Statistics in general, used to prove if something is real based on empirical evidence.

It’s basically the opposite concept of Inference, which is a more common and popular term in the field of AI.

At the same time, if you ignore the meaning of the term, it could sounds a bit ambiguous: the word “truth” is very central since the advent of the internet and the alleged democratization of the media, this word is being used to represent and declare the state of declaring a single truth itself especially by conspiracy theory influencers, Q-Anon militants and alt-right propaganda at the point that we are now used to identify the word “truth” as a sort of red-flag…for fake, dangerous bad stuff and in general to deny the existence of many truths. So “Truth” is probably the word that best represents the gray areas of our time.

“«Truth» is probably the word that best represents the gray areas of our time”

This exact dichotomy and hard contrast led me to go for his title, to exactly evoke that kind of broken and twisted information system, depicting the distorted infosphere, the flow we are exposed to every day, in contrast with the quantum conception of reality, which gives us the tools to see the many truths reality is made of, that is instead becoming more and more strongly established.

Lauren Moffatt on the intimacy of VR

Fabien Siouffi

On the occasion of our collaboration with Fabbula on the artcast Worlding with the Trouble, which features the work of Lauren Moffatt, Serafín Álvarez, and Xenoangel, we are featuring in this post an interview by guest author Fabien Siouffi with artist Lauren Moffatt.

Following the selection of Lauren Moffatt as the first recipient of Fabbula’s Worlding with the Trouble programme, Fabien Siouffi discusses with the artist her trajectory towards the VR medium. 

Worlding with the Trouble is a commission and production programme designed to support artists, hackers and thinkers in the creation of disconcerting, heady virtual worlds, translating radical thoughts into multi-sensory experiences. 

I’d love for you to trace back your trajectory as an artist. When did you start with VR, and what have you done with it so far?

I’m a graduate of painting and drawing but actually all through my studies there was an almost even balance between time-based media and painting, and even while I was studying I was integrating animation and different types of experimental image making into my painting. This culminated in a painted animation from my graduate work, and around that time I started to get really interested in embodied experiences and how to visualize what someone sees through their eyes. I started creating self portraits and then I moved onto trying to show other people’s views.

Considering this form of representation also led me to think about how our visual system works, the fact that we’re only focused on one thing that our eyes are scanning all the time and that we’re seeing parts of our faces as we look at the world around us. To me, this was a very intimate way to try and represent a world from inside someone’s body. What I found missing was questioning the visual system when certain aspects of it don’t belong in this objective table system in which everything is delineated with a horizon line. Everything starts with this fictional line. This is quite different to the way that we subjectively see things and the way that we also trace the narrative and our surroundings, as we go about our day to day. And so, it became clear at some point that painting wasn’t the right medium for these experiments that I was doing because it was taking too long. It was too complicated to bring these images that I wanted to make to the surface of a canvas.

I started working with digital images and animating them using video editing software and then I started making videos and editing them. This progressively led to a series of works in which I was building multi camera rigs, performing in public spaces while wearing these camera costumes and filming with them. I made this footage into a massive collage by manually knitting together 360 degree perspectives to create immersive videos. And by chance, it was around this time that 3D was becoming big and I received some funding and support to train in stereoscopic filmmaking. Actually, Céline Tricart was one of my trainers in Prague. She taught me how to make stereoscopic images interesting in video. 

Lauren Moffatt, On Hybrids and Strings. Image courtesy of the artist and Fabbula.

From there I went on to do a fellowship at Le Fresnoy, and it was there that I started working with VR. I wanted to do something really different and so I created the first documentary piece that I’d ever made, which was also the first VR piece that I’ve ever made. This was a piece called The Oculist Reason. I was really interested in history and the way that virtual documentation could possibly change the way that history is written. And so I used as a case study a dome-shaped painting in Liverpool and looked at it from different points of view, creating a virtual reproduction of the painting and telling its story and that of the events it describes. The next project I did was made in collaboration with a Korean filmmaker. It was an adaptation of a sequence from one of his films to VR, and it led us to think about the way that this translation changes the rules for cinema. And also about how there isn’t this cutoff between cinema and life anymore, everything is cinema and cinema is life. 

I started working with digital images because I realized that it was too complicated to bring these images that I wanted to make to the surface of a canvas.

From there I went on to make a piece called Image Technology Echoes, which has been in production for the last two years and in development for another year before that. It deals with the separation between the body and the mind, and the idea that there could be an homunculus that lives inside your mind and that is controlling everything and perceiving everything from this more interiorized point of view than the one that you are  aware of in your everyday dealings. And so in this case you can step inside each of the characters: there are two characters in an art gallery, watching an exhibition and having a conversation. As you approach them, you become transported into their mind space, a room of their own where there are some clues about who these people are. So as a viewer, you jump between these different realities and if you choose to pay close attention, then you might find out why these two people are together and what’s going on inside their minds and what’s going on between them in this conversation. However, a lot of people just like to move between all of these different spaces and look at things from different points of view, so there is no right or wrong way to to experience it.

Lauren Moffatt, Image Technology Echoes. Image courtesy of the artist

About your relationship with the medium VR in general, I’m interested in knowing why it has caught your attention? What do you see in this medium that feels special for your work?

What I find really special is that I can build a subjective space that brings together many things that I’ve been working on for many years before all of this technology became available to me. And I find it also quite powerful in the sense that you can build an entire architecture that encloses the person and, if the viewing conditions are right, they can feel safe inside it and completely suspend their disbelief in this thing that you’ve built. And this is even more powerful than a physical installation because it becomes so intimate. The intimate relationship that is created between the viewer and the piece is something that is quite appealing to me because I’m often working with intimate concepts that I’m trying to transmit to the people I’m showing my work to. There is an intensified relationship between the viewer, and you as an artist, expressing something about the medium itself, and producing subjective realities. 

Which subjective realities are you interested in conveying or which ones do you think come out with this medium that could not come out with others?

In VR you can create quiet meditative spaces where you have time to engage with ideas that play a little bit foreign or a little bit difficult to take seriously unless you really pay attention. When you get the viewer’s undivided attention, you can build empathy, and that can be really powerful when you tell human stories with this medium. The attention that the audience gives to the objects or the surfaces or whatever it is that you’re constructing in these environments is much more focused because of how they are delivered to them.

What I find really special about VR is that I can build a subjective space that encloses the person and make them feel safe inside it and completely suspend their disbelief in this thing that you’ve built. 

You once stated that VR can represent realities that we hold inside our minds.

Yes, I find it interesting to think about the way that reality is for us intimately, how we build our perception of reality and how often our ideas and our imagination are suppressed by our need to adapt to our environment. So it’s interesting to create spaces where it is possible to explore the interior life of a person and that this is not something that’s scary, or formless, or unhealthy. 

For instance, I was quite inspired by Notes on Blindness (2016), a VR experience based on a film which I found very interesting. There have been a number of works that I thought were really interesting, because they were not just constructing visible realities but also constructing points of view and  allowing you to realize how much of the lives of all of the people around me are invisible to me.

Lauren Moffatt, The Unbinding. Image courtesy of the artist

Do you feel that there is a particular area or subject matter that comes out in your work that only comes out with this medium?

Yes, there are some formulas, some narrative resources and themes that tend to surface, but it’s difficult to point them out because I’ve noticed that the audience who visit my work, had a really different experience of it to what I saw when I made it. So there’s an openness to interpretation, while it is also true that frequently strong feelings such as anxiety or melancholy emerge from the VR experiences. However, the artworks are more of an invitation to explore other realms of realities all in their complex layers rather than simply an exposition of a theme. It is often rather cryptic, so there are a lot of different interpretations that could come from it.

Samsung & Niio Art Launch Prestigious Global Digital Art Competition Celebrating Visual Arts on ‘The Wall’

Open to artists worldwide, competition will culminate in a first-of-its-kind global exhibition in May

Originally Published by Samsung Newsroom

Samsung Electronics is today announcing the launch of an open call competition in collaboration with Niio, a premium platform for new media art. Now open for entries from nascent, emerging and established artists, the Samsung The Wall x Niio Art Awards aims to discover the most inspiring digital art from around the world. Entrants will compete for a chance to be exhibited internationally on Samsung’s The Wall, the highest quality, large form factor 4K Micro LED digital art canvas, and qualify for up to $15,000 in cash prizes. The competition will culminate in summer exhibitions at ten showcase locations including Korea, Singapore, the United Kingdom and the United States. 

The first-prize winner of the competition will be invited to attend the unveiling event of their winning entry at Samsung’s The Wall showroom in May. All winners will be selected by a panel of renowned experts in visual art. When assessing each submission, the judges will be looking for artworks that display artistic and conceptual excellence, including creativity, originality and detail. 

“Samsung is always exploring new ways to make premium content as impactful as possible and take users’ experience to the next level,” said Hyesung Ha, Senior Vice President of Visual Display Business at Samsung Electronics. “This open call, in partnership with Niio Art and its global community of leading artists, celebrates the latest and greatest talent in digital art and provides the art community a meaningful platform on which its artistic creations can be showcased. The competition will also play a major role in heralding the next generation of display experiences – all brought to life on the awe-inspiring The Wall.” 

“In today’s connected world, artists rely on digital tools to tell their stories. We are thrilled to partner with Samsung to offer artists everywhere a chance to share their creations on the most iconic and exquisite large-scale canvas there is,” said Rob Anders, CEO of Niio. “Moreover, our plans for the synchronized, global unveiling of the winning artworks in May will change the way digital art is experienced in the world.”

Niio inspires people everywhere by providing seamless access to premium video and new media art through its state-of-the-art technology and curated distribution platform. Niio’s platform now empowers a growing network of over 4,000 artists and art partners from over 70 countries to self-publish media art; the published artwork can then be borrowed or acquired to be viewed on high quality connected display devices anywhere in the world.

Now available globally, The Wall represents the latest application of Samsung’s modular MicroLED technology which can be custom-tailored to any size and aspect ratio, immediately enhancing the aesthetic of any space.

To learn more about the open call, artists can visit https://www.niio.com/get/the-wall-open-call/ and enter their artwork, as well as find submission details, specification requirements, rules, important dates and prizes. And to discover The Wall, please visit: https://displaysolutions.samsung.com/led-signage/the-wall.

About Niio

Niio is reimagining the way humans interact with art in their everyday lives. Underpinned by a robust technology platform that powers the ‘digital art’ ecosystem, Niio has amassed a global community of leading artists, galleries and institutions who store and publish the largest catalogue of the finest moving image art in one place. By enabling seamless access to premium digital art on any screen, anywhere, Niio is unlocking an entirely new form of media consumption: digital art, on demand.

Niio @ Unpainted Art Fair ’16 (Munich, Germany)

Earlier this year, Niio Co-Founder Oren Moshe had an opportunity to visit and participate in UNPAINTED lab 3.0 in Munich, Germany, a unique art fair featuring 40 international digital artist organized by artistic director Annette Doms in collaboration with New York curator Nate Hitchcock,  Co-Founder of East Hampton Shed and former Co-Curator of Rhizome (NY).

Oren had the opportunity to join colleagues Chris Fitzpatrick (Director Kunstverein Munich), Nate Hitchcock (Co-Curator, UNPAINTED, New York), Ioannis Christoforakos (Collector, Athens) & VT ArtSalon (Taiwan) on the stage for a panel discussion about “The Impact of Digital Media on Our Real World.” 

Fair participants included international curators and thought leaders, art collectors, gallerists, and artists, as well as an interdisciplinary team, who understand the challenges of these new art forms. It’s fair to say that all were “united by a love of art, innovation and the changing times”. 

Some personal snaps from the weekend:

Unpainted 5
Susanne Rottenbacher

unpainted laptopsUnpainted 4Unpainted 3Unpainted Museum