Cosette Reyes: narrating stories to AI

Pau Waelder

This interview is part of a series dedicated to the artists whose works have been selected at the SMTH + Niio Open Call for Art Students. The jury been selected at the SMTH + Niio Open Call for Art Students. The jury members Valentina Peri, curator, Wolf Lieser, founder of DAM Projects/ DAM Museum, and Solimán López, new media artist, chose 5 artworks that are being displayed on more than 60 screens in public spaces, courtesy of Led&Go

Cosette Reyes is a Mexican designer, artist, anthropologist, and biochemical engineer. Over the last few years, she has participated in international research projects in the fields of mental health, human evolution, and cognition. This extensive research has inspired a deep exploration into the phenomenon of the mind and its corporeal expressions through design and art. Currently residing in Valencia, Spain, Reyes is in the third year of a Degree in Graphic Design and Digital Media at LABA Valencia, School of Art, Design & New Media. In addition to academic pursuits, she leads several creative and community-building projects.

Since 2022, Reyes has collaborated with House of Chappaz, a prominent contemporary art gallery in Spain, contributing her expertise in motion graphics, 3D, and video art. The choice of the nickname “Ammoniite” reflects a fascination with its connection to science, art, and mystery. The fossilized figure, with its ideal aesthetic proportions, aligns with her main interests. The added “i” in the name symbolizes a commitment to interdiscipline and innovation in professional practice.

Cosette Reyes. Instante, 2024

Your work focuses on the exploration of the mind, with a direct application to human-machine interaction research in interface design. Can you tell me about these two facets of your work, how they relate to each other?

I seek to articulate the whole corporeal part of the human mind, with its behavior and the relationship it has with the new media. I find it especially interesting to explore how the human being leaves its mark on new technologies and this feeds all the systems that will later interact with the humans of the future.

“We have, as designers and artists, the great responsibility to show the options that are available, instead of imposing a unique vision or use.”

As a designer, I think we are at a key point where we are not using all the knowledge we have about human behavior. We have gone from using color psychology and marketing techniques to having a wealth of information about users’ habits, which we can transfer to the users themselves. This opens up a wide range of possibilities, paths, ways of interacting with technology and allows us to go back to introspection and get to know ourselves through our own body and the environment, whether digital or tangible. We have, as designers and artists, the great responsibility to show the options that are available, instead of imposing a unique vision or use. Something that characterizes us as humans is to be curious and to have the possibility to choose. Through design we provide solutions, challenges that we solve with our creativity and the answer to these challenges are creative solutions that provide many options for our user or viewer: it is not about guiding them, or giving them a guideline towards one choice or another, but letting them know that they have all these options, always within an ethical framework, within the legal framework and the context in which we find ourselves and above all with the exercise of our own rights and respect for the rights of others.

Cosette Reyes. Instante. Displayed at CC Plenilunio (Madrid) as part of the SMTH + Niio Open Call. Photo: SMTH

You are studying in the Graphic Design Degree at LABA Valencia. How has your experience in this degree been, what does it contribute to your professional and artistic development?

In LABA Valencia I have developed a lot of technical skills, I have acquired a lot of basic knowledge and I have learned a lot of software, especially new technologies. But what I have valued most about LABA in terms of knowledge in the field of design and the creative sector is that we have contact with working professionals. These professionals make us very aware of what the sector is like, what the market is like, and how they have had this approach with a client, whether it is the Generalitat Valenciana, or the education sector, or even in associations where the clients are themselves. In LABA Valencia, the human aspect stands out. The teachers are very up to date with their syllabus, with all the educational proposals, but they always put a lot of emphasis on the human side, on the tangible and physical side, and on working with real cases, with a global perspective but also, so to say, with the feet on the ground. In the Design Thinking process they put a lot of emphasis on empathy. Empathy has greatly enriched all the knowledge I had and I have been able to articulate it now with design, which has led me to observe human behavior and people in a more global way. But always with that sense that connects and that is incipient to us: being creative, being curious, and above all something very important that is collaborative. That is what LABA Valencia has given me the most: the experience of collaborating.

It has also helped me a lot to meet people who are starting out in the sector, and to share the day-to-day with my colleagues and peers. That is very enriching. Valencia is a city that inspires, a quiet, green city, which is closely linked to design: it has quite an important history in terms of design. All the designers who are now active have had this contact, not only with the community but also at a national level. In the field of design in Valencia I have seen that there is always a discourse and a social motive. They look not only at vulnerable sectors but at things that matter to the community. They are always at the forefront, not only in terms of graphics but also in social movements. It is one of the cities with the highest quality of life.

“What I have valued most about LABA in terms of knowledge in the field of design and the creative sector is that we have contact with working professionals.”

You collaborate since 2022 with House of Chappaz, from this perspective, how do you see the art market today in relation to digital media? What possibilities do you see for artists?

I have had the opportunity to collaborate for about a year with Ismael Chappaz’s gallery, which is a reference gallery in Spain. The gallery is very supportive of design, but above all conscious design, design with a reason. As for the position of digital artists in the market, I think it has evolved a little slower than all the new media and everything digital. Acceptance has always been complicated, because especially after the pandemic we have gone more towards contact, towards everything tangible. But currently and in the last few months I have seen that this impulse to go towards digital and for artists to express themselves in these non-tangible media has been much more supported. I have also noticed that when projects flow better is when there is another physical, tangible and “classic” part, so to speak. In interactive exhibitions, where there is participation of the body, is where I see more possibilities for digital artists.

Cosette Reyes. Instante. Displayed at CC Max Center (Bilbao) as part of the SMTH + Niio Open Call. Photo: SMTH

Your work was selected in the previous edition of the SMTH university call. What was your experience then? How do you see the current collaboration with Niio and the options it brings to artists?

I found it very interesting to bring this art to all kinds of audiences, and not to limit it to museums or spaces where a public that is already interested will experience it. It is important that this happens after the pandemic, because we were quite disconnected in the sense of the body and the tangible, but very connected in a more ethereal sense. I don’t consider that to have been a bad thing, but rather that we’ve been given the opportunity to see how we can be in both environments and have this more hybrid essence. That you go to a physical space of leisure, recreation, being more connected to yourself and seeing the works that speak to your own body, I think that was a very important point to reconnect with all of that. It can be for all audiences a point of inspiration, and I have also seen it in the second call: the artists have brought a much more introspective and more conscious discourse in terms of new media, and also in terms of the body. It’s revealing that most of us have touched the collective: that’s a very interesting process in which you discover yourself as a person and then you see what’s around you and how you can collaborate.  

“I found it very interesting to bring this art to all kinds of audiences, and not to limit it to museums or spaces where a public that is already interested will experience it.”

Tell us about Instante: you have worked with Artificial Intelligence programs to generate the dreamlike images that populate this video. How does this piece fit in with your work and your fascination with surrealism? What has it been like to work with AI?

In this work I had thought of doing something else more inclined towards video art, much more about the body but with touches of reverie, capturing physical spaces in which we find ourselves safe, but that we can no longer find. Regarding working with AI, although it seems that these systems are automated and that we only have to give them a few instructions so that everything builds itself, it turns out that there is a kind of dialogue: when I started, I had something very concrete in mind, but when after interacting with these models of artificial intelligence I realized that it wasn’t two or three clicks. So I decided to take up the idea of how our mind doesn’t always reflect what’s going on through our own corporeality. So I gave myself the task of looking for many more references, to give a twist to the idea I had, because it was going the other way and I found it an interesting challenge to say to myself, “I don’t think I should take what this one artificial intelligence engine is giving me, but I could combine it with others and make them collaborate and feed back to each other.” 

Cosette Reyes. Instante. Displayed at CC Zaragoza as part of the SMTH + Niio Open Call. Photo: SMTH

I used three models and thanks to that combination I was able to give my original idea a life and essence of its own. As designers we always have the challenge of having an idea, and in the course of being able to materialize it we have a lot of possibilities of tools and new media to be able to transmit it. I wanted to make a tribute to everything we have and everything we enjoy, always being aware that we don’t know when is the last moment we will be able to enjoy it. And I’m not just talking about the natural and tangible environments, but also that our own tools as designers and artists are changing. The Photoshop we used to know, where we could spend hours removing a background, now artificial intelligence is included and in two clicks it removes the background and then you have to adapt to these new times, to these new speeds, and to the new results that technologies are giving you. All this can help you and give you many more possibilities to develop your creativity.  

“Now most artificial intelligence engines ask you not only for a prompt but also for a context, a story, an aesthetic.”

Part of the process of working with AI is the use of language, through the prompts with which the images are generated. For me this has been very interesting, because I like writing very much, I have always enjoyed writing and it is one of my best ways of expression. I have always considered it necessary to accompany the visual pieces with text to communicate what I wanted. Now most artificial intelligence engines ask you not only for a prompt but also for a context, a story, an aesthetic, with a description as extensive and precise as possible of what you want to create. The program adapts more and more to the subjective, to the associations of ideas, and in this way gives you results that are less and less strange or sinister and more and more familiar, with which it is easier to connect. You have to narrate a story to it, and then tell it “from all this that I’ve told you, create an image of what happens when this or that happens.” 

Cosette Reyes. Instante. Displayed at CC FAN Mallorca as part of the SMTH + Niio Open Call. Photo: SMTH

At the end of the day, that’s what we designers and artists do, we create a story and share it through our own experience but always looking to connect with those who will experience it. Then you have to keep in mind that you have to work with different AI models, for example one that can enhance your prompts to be better understood by another AI engine, or one to work on color or lighting. It’s a new process that you need to adapt to.   

Disordinary Beauty, a work in progress (part 4)

Domenico Barra and Pau Waelder

DISØRDINARY BƏAUTY is an ongoing art project by Domenico Barra that explores ugliness through glitch art. The project has been developed as a series of NFTs, with a new phase taking place on Niio as a work in progress, in which the artist will periodically upload new artworks and accompanying documentation. Here in the Editorial section, we are publishing email exchanges bringing light into Domenico’s creative process and the ideas and influences behind this project.

Follow Domenico Barra’s work in progress on your screen in DISØRDINARY BƏAUTY: art canon

Fourth ɛʍǟɨʟ exchange

from: Pau Waelder
to: Domenico Barra
date: Feb 8, 2024, 8:48 PM
subject: Re: Disordinary Beauty #04-05 on Niio

Hi, Domenico!

It’s been a year since we started this collaboration and I am deeply thankful for your unwavering commitment to creating new artworks and your generosity in sharing them in this series on Niio.

So, let me get to the questions! (in attachment) I explored your blog and found a treasure trove of content there.

Please let me know if anything is not clear, I know the questions are a bit long, so please bear with me 🙂

Best wishes,

Pau


from: Domenico Barra 
to: Pau Waelder 
date: Feb 12, 2024, 11:14 AM
subject: Re: Disordinary Beauty #04-05 on Niio

Hola Pau, qué tal?

Thank you so much for finding the time to read my blog and prepare these questions.

Taking part in the Art cast series for Niio has been essential, it is allowing me to expand my DB🥀🪞 project and explore more the aesthetic of portraits and the [thorn] sense of beauty, learning about artists of the past, new stories, and also test new video codecs. I have a long list to try, my FFmpeg is on fire. I must say that I hope one day to see some of these works exhibited in a gallery, maybe, who knows. I am also looking forward to adding audio, the next video will have audio.

I hope you will find my answers to your questions interesting, and thank you so much for this opportunity. It’s an honour and pleasure to share my thoughts and stories with you. Great respect for that. 

Sincerely,

d0/\/\!

Domenico Barra. DB a̶r̶t̶ ̶c̶a̶n̶o̶n̶ | a̶f̶t̶e̶r̶_̶g̶e̶m̶i̶t̶o̶, 2023

Let’s start with the portrait of Vincenzo Gemito. I didn’t know about this artist and became fascinated after reading your blog post. This is the second self-portrait in the series, after the one by Carl Joseph Begas, and it is a very different kind of work: a very intimate and mesmerizing depiction of the “scultore pazzo” which makes me think about the internal struggles that all artists go through and the possible connection between madness and glitch, as both diverge from the expected, the system that works and is stable. I wonder why you chose this portrait, did you want to explore these connections? Maybe some self-introspection of your artist persona?

My journey into the realm of art began at a pivotal moment in my life. Back in the day, I had recently embarked on a new chapter by moving to England, mostly motivated by the necessity to face my inner demons, seeking to discover my true self. At the cusp of turning 20 years old, it was my first experience living independently as a foreigner abroad. I needed to overcome my insecurity, anxiousness, and restlessness, I needed to find some peace of mind. Amidst this transition, art emerged as my guiding light, offering a profound avenue for exploring my fragilities and grappling with the myriad challenges I encountered. 

It was during those years that I found out and finally started to face my OCD and experimenting ways to hack it and keep those intrusive thoughts under control, something that finally I managed to do brilliantly. Reverse engineering my brain was one hell of a trip. 

But it all started well before that. The human mind glitches I encountered along the way are quite few. Living alongside my two autistic cousins from a young age exposed me to the diversity of human experience and instilled in me a fondness for the atypical. This early exposure to altered states of the human consciousness also fostered a curiosity within me, driving me to delve deeper into the complexities of human nature. 

“The disability is in the society as a whole, and not the individuals, everyone is a single story.”

Through art, I found solace in embracing the vulnerabilities that define us, recognizing that true understanding stems from empathizing with the struggles of others who are conditioned by a society that is not able to allow everyone to be who they are based on their particular condition. The disability is in the society as a whole, and not the individuals, everyone is a single story. By this I mean that if a society doesn’t manage to be inclusive, enabling people to have a peaceful life, adapting to peculiar necessities, then it is the society having issues and not the people in need of certain conditions to have a decent life.   

My interest in the human psyche was further fueled by encounters with institutions like the old Psychiatric Hospital Leonardo Bianchi in Napoli during the projection of a documentary that was telling the stories happening behind those walls. I walked by this place everyday and that always tickled my curiosity and imagination. Witnessing the resilience and humanity amidst conditions often stigmatized by society reshaped my perception of madness and reinforced my commitment to amplifying the voices of the vulnerable whenever I had the chance to tell their stories. 

The attention towards the reality of a psychiatric hospital became even stronger once I had the chance to visit the dismissed forensic psychiatric hospital S. Eframo in Napoli. It had been redeveloped by a group of activists that now operate part of this building to support minorities and marginalized people offering various social services such as medical assistance, language schools for asylum seekers, entertainment and politics. I remember walking through the corridors and reaching the rooms where the psychiatric inmates were kept and assisted, probably one of the strongest experiences ever in my life. I spent hours reading the messages they left on the walls, it gave me goosebumps. 

“Witnessing the resilience and humanity amidst conditions often stigmatized by society reshaped my perception of madness and reinforced my commitment to amplifying the voices of the vulnerable.”

In the pursuit of authentic artistic practices and driven by inclusivity, I drew inspiration from The Surrealist Manifesto by André Breton, a book I treasure with admiration. It influenced me to the point that I felt the urge to celebrate the wild and untamed manifestations of creativity, probably somehow with the intention to give space to my disorder and exploit its tendency for the absurd. Living in a shared house with friends akin to the surrealists’ ethos, where freedom of expression reigned supreme, opened my eyes to the transformative power of art in fostering connections and dismantling conventions. At the time we were also hugely influenced by a successful series titled The Mighty Boosh and its unusual storytelling. This was an incredible experience because I learnt the empowering potential of art for self expression and self awareness. 

When selecting Vincenzo Gemito for my series on Niio, I was drawn to his tumultuous past and relentless pursuit of perfection—a stark contrast to my own embrace of imperfection, but also being aware of Gemito’s background and surroundings as I walked these same streets of Napoli in my own boots too, I was also shaped by its conflicts and culture. We are both artists, neapolitans. I have known his work for a long time, since I’ve seen it many times at the Museo di Capodimonte, a place we had previously had a chance to talk about, and it felt natural to relate to him and his condition as artist and human.  

Through glitch art, I find a means of decoding the complexities of life, using glitches as metaphors for the fragility and disordinary beauty inherent in the human experience. More personal experiences, such as witnessing the ravages of Alzheimer’s and dementia in people very close to me, have profoundly influenced my artistic journey, imbuing my creations with a sense of empathy and introspection. In essence, my artistic odyssey is a testament to the transformative power of creativity in navigating life’s complexities and embracing the inherent imperfections that define us. 

“Through glitch art, I find a means of decoding the complexities of life, using glitches as metaphors for the fragility and disordinary beauty inherent in the human experience.”

Through glitch art, I seek to illuminate the beauty found within life’s and society’s glitches and celebrate the diversity of human existence. In the first series that gave birth to the DIS/verse projects, DISØRDINARY BƏAUTY | ⚡ BÆUTY IZ CH∆ØZ ⚡ , those hauntingly beautiful generative glitch art portraits are unknown lost souls in the short circuits of society, of life. A reminder that many people stay invisible, unrecognizable, and unknown because society isn’t inclusive at all.

The DIS/verse projects include: Noise Source, Ecstasy of Creation, Witness of Madness, The Beautiful Minds and D.B. | Beauty is Chaos

Gemito’s stare in this self-portrait is so powerful and it resists all distorsions and remains as a haunting presence. How was it “breaking” this image? Did it resist the glitches? Would you say you succeeded?

It was definitely a bit of a challenge to choose the right portrait from Gemito’s extensive body of self-portrait works. Ultimately, I settled on this particular one that I felt captured the essence of what I wanted to convey through my glitch art version. The decision to focus on this specific portrait was influenced by several factors. Firstly, its composition lent itself well to the format I needed considering the constraints of adapting it to fit a screen, allowing me to center it effectively within the frame was an important feature. Gemito’s posture, with his bending and leaning forward, added a dynamic element that I found intriguing and visually engaging. Moreover, the intensity of Gemito’s stare in this portrait struck me as particularly powerful. I wanted to capture and amplify this sense of tension and vulnerability in the final glitched rendition. To achieve this, I deliberately chose codecs known for their ability to heavily distort and break images, pushing the boundaries of visual disruption to create a thunderous effect. In addition to utilizing these codecs, I also employed a generative glitch script to further enhance the explosiveness of the sentiment and of the final composition. 

Despite the chaotic nature of the glitches, I wanted Gemito’s presence to remain palpable and unwavering amidst the pixelated turmoil. His greatness, I believe, transcends the distortions and disruptions, serving as a testament to the enduring legacy. The resulting artwork stands as a visual manifestation of his unstable journey, a poignant reminder of the enduring power of artistic expression and vulnerability of the human mind, so great and so fragile. 

Domenico Barra. DB a̶r̶t̶ ̶c̶a̶n̶o̶n̶ | a̶f̶t̶e̶r̶_̶v̶i̶g̶é̶e̶_̶l̶e̶_b̶r̶u̶n, 2023

The fifth artwork in this series is dedicated to a portrait by the celebrated painter Elisabeth Louise Vigée Le Brun, who is also the first female artist in the Disordinary Beauty project. The portrait you chose is quite special, because it is a commission that Vigée Le Brun had trouble finishing due to the “extreme ugliness” of the sitter, Princess Maria Luisa di Borbone. It is a captivating story in the context of a work exploring beauty and ugliness through glitch art. What drew your attention to this portrait? Since the canons of beauty have particularly been imposed on women, what do you make of this depiction of a woman despised for her apparent lack of beauty made by a woman artist who put her talent into applying a “beauty filter” to her portrait?

Initially, I must admit, I found this story rather amusing. The idea that even a figure as powerful as a queen could be subject to mockery and ridicule due to her physical appearance struck me as both intriguing and somewhat ironic. It highlighted the harsh reality that ugliness is not forgiven – regardless of one’s status or privilege, it can still be used as a tool for discrimination and derision, especially if you are a royal and your status is close to the divine, and beauty is strongly related to the divine. 

I couldn’t help but wonder if Maria Luisa’s royal status played a role in how her appearance was perceived and depicted. Historically, powerful figures often sought to be portrayed not as they truly were, but rather in a way that projected an idealized image of themselves. This was particularly prevalent among royals, where marriages within the same bloodline often led to genetic conditions and physical imperfections, I believe artists were often invited to leave these possible particulars out.

“The idea that even a figure as powerful as a queen could be subject to mockery and ridicule due to her physical appearance struck me as both intriguing and somewhat ironic.”

I presume that for both the subject and the artist, there was a vested interest in creating an image that aligned with societal standards of beauty, and if not perfection at least tradition. This becomes especially significant when considering the vulnerability of women in a male-dominated society, where power and talent are often associated with the male sex, a portrait was also a symbol of power and wealth, it had to be good for the royal and the artist, of course, wanted her work to be at least in her standard, I guess.

While ugliness has long been a theme in art, typically associated with depicting societal outcasts and villains, this artwork challenges conventional beauty standards in a contemporary context. It addresses at some level the use of synthetic solutions, like the ongoing issue of ugliness being stigmatized and filtered out of society even if often resulting in an exaggerated, grotesque version of beauty that becomes rather ugly, I am thinking at some selfie beauty filters, and plastic surgery that most of the time are big time failures.

Of all the pieces I’ve created for DISØRDINARY BƏAUTY | a̶r̶t̶ ̶c̶a̶n̶o̶n̶s̶, I believe this one stands out as the most thought-provoking. It serves as a reflection of society’s obsession with masking imperfections and the consequences of striving for a manufactured version of beauty, and I am not talking just about beauty in the context of beauty, imperfections are filtered out from all sort of images that need to convey an aura of superiority, because that ugly detail can easily be seen as a weakness and the appreciation of artists often relies in their ability to make someone “ugly” look more majestic, beautiful, divine. While I am here writing, I just received an email from a photo editor offering me his service to correct the imperfections present on the headshots published on my Instagram because he could with his editing magic make those portrait photos look more beautiful. That’s a very ironic coincidence. 

Finally, it feels necessary to mention a parallel research that you are carrying out using AI tools,  “THƏ L∆B ØF ∆NØM∆LIƏS.” Here you produce much more unsettling images, which your followers can find on Instagram. Which parallels and differences would you draw between “traditional” glitch and this “AI-powered glitch” of beauty in the creation of machine-assisted ugliness?

The DIS/verse project has been an ongoing journey, constantly expanding as I delve deeper into the ever-growing sphere of aesthetics, particularly as new media technologies become increasingly integrated into our daily lives. My initial foray into AI experimentation began several years ago with StyleGAN, where I was intrigued by the potential of AI to infuse new layers of meaning into artistic works and the actual practice of art and what is means to be an artist in the age of “intelligent” machines. Can artistic intelligence and artificial intelligence create something that is more puzzling and arousing than what we have seen so far in art?

However, as I delved further into AI-influenced artistry, I couldn’t help but notice the inherent limitations that often arise when we become entangled in the very media we seek to explore. It’s a delicate balance between harnessing the potential of AI as a creative tool and avoiding the trap of allowing the medium to dictate the outcomes.

“By deliberately misusing AI tools, I aimed to create images that defy conventional expectations and provoke a deeper engagement with the uncanny.”

My motivation has always been to push the boundaries of what is possible with AI, to transcend the standard modes of use and challenge both myself and the audience to the unexpected. This parallels my approach to glitch art, where the exploration of the unexpected is paramount. I want to put the machine in the condition to generate something where I can lose, or have little, control on it. In most cases of databending I select the glitches I believe are the most interesting, the same happens with the present text-to-image/video apps. I select the images that I believe have some intriguing features. Sometimes it feels like pulling the bar at a slot machine hoping to strike a winning jackpot.

This series titled DISØRDINARY BƏAUTY |THƏ L∆B ØF ∆NØM∆LIƏS”, specifically designed for Instagram, was born out of curiosity and a desire to explore how AI could reinterpret my glitch art works part of DISØRDINARY BƏAUTY | ⚡ BÆUTY IZ CH∆ØZ ⚡ . It all began with a simple question: What would my glitch art look like if created by AI? This curiosity was sparked when I stumbled upon one of my works in an AI dataset used for training models, prompting me to explore further.

Using an AI app called Starry.AI powered by Stable Diffusion, I experimented with crafting prompts to generate images inspired by my disordinary glitch art. With each attempt, I sought to evoke novelty, complexity, and ambiguity – qualities that I believe are essential to true aesthetic arousal, as described by psychologist D. E. Berlyne, an approach that has also inspired Dr. Ahmed Elgammal for his strategy to develop artistic AI models at Rutgers University, a methodology described by Marcus du Sautoy in his book The Creativity Code.

I gradually and intentionally crafted prompts that veered away from the mainstream pursuit of beauty, as most of the AI images online aim at creating images that are luminous, soft, polished, shining, unless candy-coated weirdness, instead I preferred embracing imperfection and ugliness as a counter-narrative to the prevailing aesthetic norms spreading online. I wanted my AI portraits to be raw, rough, cruel, and oddly more realistic somehow than extreme standardized beauty. This exploration also highlighted the biases inherent within AI systems, particularly regarding representations of gender, race, and disability. Tendentially text-to-image apps works well with standard words that mostly recall beauty, harmony, peace, but do start to struggle with words that are a little more offgrid like “disfigured”, “deformed”, “disable”, “arab”, “asian”, “albino”, “autistic”, and so on.  

Domenico Barra. Artwork from the series THƏ L∆B ØF ∆NØM∆LIƏS, 2023

By challenging these biases and deliberately misusing AI tools, I aimed to create images that defy conventional expectations and provoke a deeper engagement with the uncanny. It’s about exploring the limit within the AI system itself while misusing the standard of prompts, pushing its boundaries to generate something truly unexpected and thought-provoking. I also didn’t want to produce AI images that looked glitchy. In a conversation with other glitch artists we all agreed that probably a glitch in these AI apps won’t look like the glitches we are used to. But probably it had to do with something unexpected. This is why I I like to go against the tide. This is why I focused on the prompts, crafting either over complicated prompts, using words that often even get blocked by the AI systems, or using L33T SP34K. Doing the wrong thing the right way. Approaching AI with a glitch mentality. Thinking of prompting as a sort of databending.   

“I didn’t want to produce AI images that looked glitchy. This is why I focused on the prompts, using words that often even get blocked by the AI systems, or using L33T SP34K. Approaching AI with a glitch mentality. Thinking of prompting as a sort of databending.” 

In essence, my goal is to embrace the imperfections and anomalies within both glitch art and AI, blurring the lines between human creativity and machine capacity to create a truly unique and unusual, and eventually disturbing and unsettling, maybe annoying, artistic experience online in the context of a social media, Instagram, that is the temple of staged perfection. Imagine scrolling Instagram feed, or even the page for the hashtag “beauty”, or “pretty” and you come across one of these images. I think this is also a glitch practice, alter the experience of a steady numbing scroll infecting the flow with something that is completely the opposite of what people are comfortably ready to expect. The image itself becomes a glitch on the system of the photo grid and the system of the content experience on Instagram.  

Ronen Tanchum: reality interpreted

Pau Waelder

A contemporary artist, developer and an interaction designer, Ronen Tanchum has developed a body of work that explores the representation of natural phenomena and our perception of reality as it is mediated by the entertainment industry and digital media. At a time in which the attention economy fosters a visual culture based on spectacularity and evasion to fantasy worlds, his work draws attention to how digital technologies, from 3D modeling to machine learning, reshape our perception of the world around us.

In his long-time collaboration with Niio, Tanchum has presented numerous artworks that we are now gradually collecting in a series of solo artcasts, offering a glimpse into the many facets of his artistic practice. In this interview we dive a little deeper into the main subjects of his work.

Ronen Tanchum. Particle Forest, 2022

Your work is characterized by an interest in nature and natural phenomena, particularly the behavior of fluids. This is obviously related to your work in the film industry, but if you look at it from the perspective of your artistic research, what does nature as a subject and fluid mechanics as a tool bring to your art practice?

Yes, this is the DNA of my artworks and what they convey. Ever since I learned computer graphics for the first time and had access to 3D software, some 20 years ago –when I was 16– I was trying to learn the software and to make the computer create something that is believable. This notion always brought me back to study the real world. So, I had to carefully observe the world around me, from the little imperfections of a corner of wall that needs to be reproduced synthetically, to complex natural behaviors that need to be recreated digitally in order to create realistic content. This required a lot of work, but additionally it was not only about making the recreation realistic, but rather a hyperreal, exaggerated reality that made the content visually attractive and engaging. 

“Instead of starting with nothing (a blank canvas) and adding on to it, I start with a lot of chaotic data and I shape it little by little, tweaking the algorithms, refining, and testing again and again until I reach a result that I’m satisfied with.”

During my whole career as a specialist in 3D technologies and simulations I had to recreate a lot of natural effects synthetically, so that they are used in key moments of Hollywood films, where reality is presented as a spectacle. For instance, an effect of clouds covering the sky and then dissipating, that has a narrative role in the film, so it has to be created in a way that looks as realistic as possible while also supporting the narrative. I worked with many natural phenomena, like waterfalls and tornadoes to rain, snowfall, and fire, and I found that the possibility of reproducing these phenomena synthetically within the machine was fascinating. So I continued to explore these technologies while also playing with the boundaries of what is real and what is not, and the way that natural forces and elements behave. Exploring these techniques led me to a deep understanding of the human role in the synthetic reproduction of nature, and how we do not simply reproduce what we observe, but we interpret it. We play with it, we make it more expressive, we manipulate the behavior of the elements, time, and natural forces to give a dramatic quality and visual appeal to something as mundane as a splash of water from a bucket on the floor. 

So my artistic practice has focused on exploring the creative possibilities of reproducing natural elements and landscapes, flora and vegetation synthetically through different technologies,  programming languages, and mediums. Using computer algorithms to create these simulations of nature is quite a challenge in itself, because instead of starting with nothing (a blank canvas) and adding on to it, I start with a lot of chaotic data and I shape it little by little, tweaking the algorithms, refining, and testing again and again until I reach a result that I’m satisfied with. I find this practice very challenging and encapsulating in ways that I could never do with a pen, paper, and ink, or with a canvas, a brush, and paint. I design systems that have a life of their own once the program starts running, so there is also a sense of creating a situation with a certain degree of control, and also letting go.

Ronen Tanchum. FEELS I, 2021

You have mentioned how the depiction of reality in films leads to spectacularity, and that is also something we frequently find nowadays in digital art, with large installations and projections in public spaces, that lead to equating digital art with a visual spectacle. As an artist, how do you see this expectation of digital art being eye-catching?

That’s an interesting question. Certainly, spectacularity is a tool to tell your story and convey or emote feelings. I do believe that art needs to be felt more than understood, and I also see that the spectacular aspect of digital art is there by choice. As a medium that is relatively new and exciting to a large audience, digital art is often perceived in this way, as something that catches your attention, and for artists that is a powerful tool to have in their hands. So, I understand the pull, both for artists and the audience, to expect spectacularity from digital art, but I also don’t feel that this is a necessity. Digital art doesn’t always have to cause a strong visual impact or be displayed in large LED screens. Of course, screens are its habitat, it is where digital art is meant to be experienced. We’re moving into a new age where art is no longer only on canvases, or sculptures, but on different mediums, and also everywhere. The screen is often understood as a digital canvas, but that is only the beginning, there will be many more ways to experience art digitally. 

In my practice, I would say that it is not so much about making art that draws attention, but using the medium in interesting ways. Exploring the possibilities of software, of generative algorithms, 3D modeling, artificial neural networks and so on, to question our reality and our experience of nature is what feels interesting to me.

“Certainly, spectacularity is a tool to tell your story and convey or emote feelings. But digital art doesn’t always have to cause a strong visual impact or be displayed in large LED screens.”

Another aspect that you’ve mentioned is the idea of control. You sometimes work with software that lets you control every element, every detail and behavior. But you also work with generative algorithms and machine learning programs, with which there is more of a “dialogue.” How do you balance your creative authorship with the outputs of these autonomous systems?

A lot of my practices are procedural and generative in nature. So even when I want to create a specific thing and aim for a certain output, I test a lot of methods to get there, naturally. I’ve been building systems and algorithms before releasing them as long format and as something with the aspect of randomness in them before, and I often work with JavaScript, and GLSL, to create long format, generative art, which is not AI. It is a way to release control and let go, so it’s interesting, because at first, I start building towards something and then I find myself thinking about variations of that original intention. To give you an example: a random function gives you a different number every time and then you can use that number to perform visual modifications on the artwork. So, for instance, every time some element appears, it can have a different color or a different size or a different shape. And then I use these somewhat random functions in order to create the output. But this output that you’re looking at lives in a spectrum of outputs: every time that you iterate on the algorithm, there will be a different output. How different that new output can be, of course, depends on the degree of so-called “randomness” you give to the system. So, if I want to get a certain degree of control over this spectrum of outputs, I must limit the amount of unexpected results that might come out of it.

“Generative art on the blockchain is a match made in heaven because here the algorithm is not only producing an endless amount of random outputs, it is creating a series of artworks that people can own and say «okay, this one belongs to me.»” 

I particularly like this method of working, to experience and be surprised by the interaction with the machine. Working with algorithms gives me an opportunity to do something that is not necessarily static. It could be dynamic, or it could be influenced by something and become interactive, or it could be a data sculpture, using real time data, or a data set that you train, and then play with. This is a really powerful tool: generative art and algorithmic art on the blockchain is a match made in heaven because here the algorithm is not only producing an endless amount of random outputs, it is creating a series of artworks that people can own and say “okay, this one belongs to me.” And that  is really interesting because the outputs become unique, but also part of a series, and the owners of these artworks become part of a community. This generates some very interesting dynamics between the pieces of a collection and the owners of those pieces.

Ronen Tanchum and Ori Ben-Shabat / Phenomena Labs. Rococo, 2023

Continuing with the subject of generative art on blockchain, can you tell us about your experience with the series Rococo? How was the response to these artworks?

Rococo is a project Ori Ben-Shabat and I developed together. It is an exploration of how we can reproduce synthetically digital paintings that represent flowers. Flowers, as you know, can come in many shapes and colors, for instance with six or fifteen petals, and that gives us a lot of possibilities, in the form of functions and numbers for the algorithm. Working with the algorithm we created a type of flower that we liked, and then duplicated it a number of times, introducing variations in the number of flowers, petals, and colors. The code itself describes a bunch of spheres that move in space, and while doing so they draw and create the final painting that you see. It is a similar approach to that of a painter who would choose a brush, and a bit of paint, and then perform a series of movements spreading the paint on a canvas with the brush in order to create the image, the gestures of his hand determining the particular shape of the flowers and a certain style of depiction. 

The response was very good. As you know, when you present generative art on an NFT marketplace, you put the code of the system that creates the artwork on the blockchain, then people can explore what the algorithm does prior to minting. Usually, they can explore and see the spectrum of outputs that the algorithm creates, and then they decide if they want to buy it or not. But they actually don’t know exactly which composition they will obtain, which is in a way the opposite of buying a painting. This process becomes very engaging and very surprising and personal, both to the artist and to the collector. It introduces the element of luck and chance into collecting artwork, which is an interesting way to release art. And it also creates a dynamic within the collection: some will be worth more than others, just because more people like them. This is really interesting, and it could be explored endlessly. So for instance, you can have an algorithm that creates an infinite number of outputs, but then only X amount of them are locked to the blockchain, and only those are what collectors can own. 

Your work easily transitions between photorealistic 3D animations, abstract compositions, and what could be described as digital painting: artworks that explore painting as a compositional and stylistic reference using digital tools. Which of these approaches is more interesting? Which is more challenging?

What interests me is to work with the edges, to play with all of them and transition between them. I am very influenced by both traditional art and contemporary art. So in projects such as Rococo, a major goal was to find a way to use code while simulating something as materially specific and expressive as a brushstroke. This could have very well become a generator of perfectly identifiable, realistic, 3D looking flowers, but with Ori we decided that it was much more interesting to explore what the act of painting looks like and find out how to evoke the level of expression and abstraction that a painter achieves applying painting on a canvas, but using computer software.

Ronen Tanchum. The Expressionists ~ Couple #2, 2020

You have mentioned your collaboration with Ori Ben-Shabat, with whom you work at Phenomena Labs, a studio that creates immersive art experiences. How does the work at Phenomena Labs differ from your individual work as an artist?

I founded Phenomena Labs almost 10 years ago with a mindset of collaborating: on the one hand, to develop a collaborative approach to creating with my friends and on the other hand, to collaborate with clients and art collectors in commissioned work. Basically, anything that I do collaboratively takes place in the context of the studio and is presented under Phenomena Labs as a brand and identity. Ori and I frequently work with other artists, designers, and architects to create immersive installations and generative art. This work is generally addressed at public spaces and large audiences.

Phenomena Labs. Moments in Time, 2023. Jönköping (Sweden)

Moments in Time is a fascinating project from Phenomena Labs that connects an architectural space with its environment through real time data animations, in which we see several recurring elements in your work. Can you tell us more about this project and the possibilities of creating art with real time environmental data?

This is a unique project we’ve worked on throughout 2023. The objective was to create a mirror for the vibrant community that is about to inhabit a building in Jönköping (Sweden). We were approached by our client and the architects and we thought about a piece that is alive, and is inspiring the startup community allocated in that building. On a large screen in the lobby, the artwork displays a series of chapters, different compositions that use data in real time. We chose to use a few different metrics and data points for different visual chapters of the piece. Each data point refers to an aspect of the building and its surroundings, as well as the people inside, in order to visualize how the environment and the human activity in the building can change and evolve over time. We used motion sensing to create visual trails from the movement of people in the lobby, and turned it into a paint brush effect where people apply brush strokes on a digital canvas by walking through the lobby, thus creating a visual composition in real time. Then we used weather information to apply wind turbulence on a set of particles displayed on the screen. And we also introduced real time energy data from the building to create a virtual waterfall that becomes a sort of data visualization of all the energy that is being consumed in the building every day. It was really interesting to see that, for instance, the waterfall flows faster and has a higher volume of water when there’s people in the building, and when they go home, it settles and slows down.

Phenomena Labs. Still from a chapter of Moments in Time, 2023. Jönköping (Sweden)

You state that your work is about trying to connect humans and machines, and reflecting on our dependence on technology. Recently, the launch of Apple’s Vision Pro was greeted by enthusiastic customers who gave the world a glimpse of what is to come: more dependency on our devices, that increasingly shape how we perceive reality. As an artist and professional creator of fantastic digital realities, how do you see this relationship evolving in the future?

The launch of products like Apple’s Vision Pro remind me that in our relationship with technology, there is a constant tension between what we are familiar with and what level of innovation we are ready to adopt. This tension oscillates in cycles, so that when something pushes too much into the unknown or becomes uncertain, such as this possibility of really isolating oneself from the world, then there is a backlash. At this point, people long to go back to a simpler relationship with the environment, and instead of adding more layers of digital content to their surroundings, reconnect with nature, or at least with a calming and comforting view of nature. Finding a balance between the two and making the digital environment more familiar is a challenge that may take more than a generation. 

“For me, the question is how to embrace the better aspects of digital technologies without letting them alienate us from the real world or shape our perception of the environment.”

For me, the question is how to embrace the better aspects of digital technologies without letting them –or those who market them– alienate us from the real world or shape our perception of the environment. In this sense, I intend to explore real time data in my work to let people understand and appreciate the world around them, and at the same time visualize the systems and networks that provide that data. It is important to understand that we live surrounded by systems (natural, legal, informational) that we have to think in terms of the environment and our interactions with others and with these systems. Often disruptive technologies are created thinking only in short-term solutions and specific goals that do not consider the world they will have an impact on. But there will always be a reaction from the world, society, systems, etc. Within this constant tension, and back-and-forth reactions in where gradual change, maybe progress, happens. 

Chun Hua Catherine Dong: “My body is a material for my art”

Pau Waelder

Chun Hua Catherine Dong, Meet Me Halfway – part 1, 2021

A performance and conceptual artist whose work spans different media, Chun Hua Catherine Dong successfully navigates the space between an artistic practice characterized by the physical, bodily presence of the artist in the same space and time as her audience, and another one based on the mediation of digital technologies and a distributed and almost immaterial existence. Dong has taken her performance artworks worldwide, combining action with documentation in the form of photographs and videos that often become artworks on their own. She is also exploring the creative possibilities of VR, AR, and Artificial Intelligence in a series of artworks that are still deeply rooted in her research on gender, memory, identity, body, and presence.

Dong has exhibited their works at The International Digital Art Biennial Montreal (BIAN),  The International Biennial of Digital Arts of the Île-de-France (Némo), MOMENTA | Biennale de l’image, Kaunas Biennial, The Musée d’Art Contemporain du Val-de-Marne in France, Quebec City Biennial, Foundation PHI for Contemporary Art, Canadian Cultural Centre Paris, Museo de la Cancillería in Mexico City, The Rooms Museum, Canadian Museum of Immigration at Pier 21, DongGong Museum of Photograph in South Korea, He Xiangning Art Museum in Shenzhen, Hubei Museum of Fine Art in Wuhan, The Aine Art Museum in Tornio, Bury Art Museum in Manchester, Art Museum at University of Toronto, Varley Art Gallery of Markham, Art Gallery of Hamilton, among others. She is represented by  Galerie Charlot in Paris.

The artist recently presented the artcast Meet Me Halfway, which collects four videos from her multi-channel VR video installation that explores the perception of time and space in virtual reality and the inability to return to the present from searching the inner world.

Experience Chun Hua Catherine Dong’s immersive VR spaces in Meet Me Halfway

Chun Hua Catherine Dong, The Lost Twelve Years (2015)

As a Chinese-born, Montreal-based artist, the issues of identity, culture, belonging, and distance are present in your life and your work as well. In our globalized world, these issues can sometimes be overlooked, or else exoticized and clichéd, even demanding of an artist with a mixed cultural background to address them. Would you say that there is still a dominant Western perspective on multiculturalism, and if so, how do you address it in your work? 

This is a very interesting question. I can’t speak for others, but it’s natural for me to explore these topics. Living in a different cultural context often prompts questions about one’s identity.  If I lived in China, I would probably never feel the need to deal with these difficult issues. But I immigrated to Canada a long time ago. I need to reconnect with my roots because I feel that something that nurtured me has faded and been forgotten. It is important for me to renew it from time to time. I addressed this issue in my earlier performances. For example, in my performance The Lost Twelve Years (2015) I use a Chinese teapot to pour ink over my head and a squirt gun to shoot ink to my heart and head, which are actions that force me to remember who I am.  

“After living as a «living sculpture» for a long time, I came to the conclusion that it is wise to keep life and art separate.  Now, I state that «I use my body as my material in my artwork» rather than «my body is my artwork.»”

Your body is a key element in your work, both as “the body of the artist”, representing you as an individual and your personal experiences, and as “a female body,” addressing issues of the representation of women in a patriarchal society. When you conceive your performances, how do you weigh these two possibilities?

As a performance artist, my “body as an Asian woman” and my “body as an artwork” frequently change. When I first started doing performance, I considered performance as an attitude, and that “life is a performance, performance is life.” The two were inseparable; thus, my life was always in a performance/artwork mode, or “living sculpture” mode. But I realized that I was quite weary of being my own artwork. It is also harmful to one’s mental health and sanity because the concept “life is art and art is life” could mess up your life. After living as a “living sculpture” for a long time, I came to the conclusion that “Life can be a performance, but performance is not life—at least, not my entire life.”  It is wise to keep the two separate.  Later, I use the statement that “I use my body as my material in my artwork” rather than “my body is my artwork.”

Chun Hua Catherine Dong, Skin Deep (2014-2020). Photographs with Augmented Reality

In your work, we can find on the one hand a direct approach to the body, naked, as a canvas or an object, and on the other hand the body veiled by masks and disguises. What do you find more interesting about playing with the different levels of displaying and hiding the body, maybe also seducing or unsettling the viewer’s gaze?

This is a very interesting question. Yes, there were naked bodies in my early performance work. For me, the body is a blank canvas, and any type of clothing or even makeup can give “identity” to it. Perhaps viewers perceive me as vulnerable when they see me naked, but I don’t feel that way. Being naked doesn’t challenge me but rather challenges the viewers. The power of the naked body in performance art lies in its rawness, it’s a pure form of art. Anyway, who isn’t born naked?

“For me, the body is a blank canvas: any type of clothing or even makeup can give “identity” to it. Being naked doesn’t challenge me but rather challenges the viewers.”

In the digital world, physical distance, the presence of the human body, and even identity tend to be blurred or seemingly erased. For instance, your work Meet Me Halfway is strikingly different from your performance work in both aesthetics and the presence of the body, yet you have incorporated your body in the form of camera movements. How do you navigate the differences between an immaterial digital environment and the materiality of your performances?

Meet Me Halfway (2021) was created during the pandemic. According to reports, many Asian people were attacked in public places during the pandemic. I was afraid of going out. If I had to go out, I wore a big hat and mask to cover myself because I didn’t want to be recognized. This situation subconsciously influenced my work Meet Me Halfway, which is why my body is absent in this work but just camera movements.  I became interested in VR during the pandemic as well because I discovered that VR can help me to escape from reality. VR space is less political, at least, you won’t get physically attacked. You can build your own virtual world in VR and visit it from time to time whenever you want. It is interesting that you mentioned immateriality in the digital environment. Actually, performance art is often regarded as an immaterial practice as well. Because of its immaterial nature, it is very easy for me to shift my practice from performance art to digital art.

Chun Hua Catherine Dong, Mulan (2022)

Following with the previous question, Mulan addresses gender identity through a folk heroine placed in an underwater landscape. What seems at first a scene of pure fantasy contains numerous symbolisms. How would say that a viewer immersed in this VR space can connect with the message you want to convey? 

Gender is an important component of my work. Mulan (2022) was inspired by Beijing Opera. You are right. “Mulan” depicts a pure fantasy scene because Beijing Opera is my fantasy. I used to dream of wearing the Beijing Opera costume and performing on stage when I was little. But Beijing Opera is a form of high art, not many people have a chance to access it. For me, art provides a space for asking questions and discovering; I’d be very happy to see that people have questions when they experience Mulan, such as, “Why Mulan? Why are there two Mulan? What outfit does Mulan wear? What are the names of the sea creatures surrounding Mulan?” If people ask questions, they will find answers.  Sometimes I realize that I am more interested in how viewers feel and think about my work rather than telling them what my work is about. Viewers’ different interpretations enrich and expand the artwork itself.

“I am more interested in how viewers feel and think about my work rather than telling them what my work is about. Viewers’ different interpretations enrich and expand the artwork itself.”

The mise en scène is an important element in a performance, which in your work translates to carefully set up photographs, installations, and VR environments. What is the role of space in your work across the many different media you use?

Mise en scene is a stage. Most of my works are staged. In performance, “mise en scene” can be in any place, including public, private, virtual, or imaginary spaces. Camera frame is a type of stage too because activities must occur within the frame in order for the camera to capture them. If we apply this concept to traditional art, a plinth is a stage for sculptures, and a wall serves as a stage for two-dimensional artworks.

Chun Hua Catherine Dong, Meet Me Halfway (2021). Four-channel VR video installation. Exhibition view at Foundation Phi.

You have stated that you initially wanted to become a painter, but found that performance was more expressive. Yet there is a painterly quality to much of your work, particularly in photography and digital art, besides the use of paint in some of your performances. Which would you say is your approach to painting nowadays? 

Yes, I wanted to be a painter before. But painting has its own limitations because you work in a two-dimensional space, and you must sometimes wait for it to dry before applying another layer. Performance is an expressive medium, I never wanted to go back to painting after I fell in love with performance. My work does have painterly quality, I guess it is because of my painting background. Regarding how I approach painting nowadays, I think it is VR drawing/ painting. It doesn’t limit you in a 2D space like traditional painting, but rather you work in a 3D space. When you draw a line in VR, it is a 3D line, and you can zoom in and out to see your drawing/painting in 3D perspective, which fascinates me.

“I approach painting through VR. It doesn’t limit you in a 2D space like traditional painting, but rather you work in a 3D space. When you draw a line in VR, it is a 3D line, and you can zoom in and out to see your drawing/painting in 3D perspective, which fascinates me.”

In your recent work Out of the Blue, you address your childhood and feature a teddy bear character that has been present in your work over the last three years. Can you tell us more about this character? You frequently use 3D printing techniques to create sculptures, why have you chosen this technique over more traditional forms of modeling and sculpting?

The teddy bear is a symbol of childhood.  With its eyes closed, the bear refuses to look at the world, rather prefers to dream. In my digital art practice, I began with AR and VR, and then 3D printing. It is very natural for me to use 3D printing to make sculptures because 3D printing is a type of digital fabrication. 3D printing is also a practical choice. Traditional sculpture requires a large studio space and special tools, which I don’t have. On the other hand, 3D printing doesn’t require much space; simply having a table or a desk at home is sufficient. Traditionally, 3D printing has been used to make molds or prototypes for further work. However, I embrace its rawness. I use 3D printing as the raw material for my finished artwork, with no additional touches such as sanding or painting. The unpolished raw nature of 3D printing fascinates me because it captures the essence of the technological and digital process, demystifying how artwork is made.

Chun Hua Catherine Don. Solo Exhibition: At the Edge of Two Worlds. TRUCK Contemporary Art, 2022

You have recently started experimenting with AI, first in the photographic series For You I Will Be an Island, and lately creating animations of what appear to be underwater creatures. Can you tell me about your experience with this technology? Which are your objectives when using AI programs? How does working with these programs differ from your VR and 3D animations?

I like AI. For me, AI is more than simply a tool; it’s like having an assistant. I understand that people have concerns about AI. I completely respect that. However, as an artist with limited resources and financial assistance, AI helps me save time and money when creating artwork.  For example, in For You I Will Be an Island (2023) I printed 23 pieces of 2.5 m x 2.5 m AI generated graphics; I can’t imagine how I would do this without AI. I could paint 23 pieces of 2.5 m × 2.5 m paintings, but how long would it take? Or I could use photographs, but where would I find such locations to photograph? I probably can find them if I have the financial freedom to travel around the world to look for them, but how long would it take?  Now AI is able to create animation and 3D objects, although it is not there yet, it is still very exciting. Animation and 3D modeling are often very time consuming and costly. If I have a budget, of course, I prefer to work with creative people, but if I don’t, AI is a good way to go.

Chun Hua Catherine Dong, For You I Will Be An Island (2023)

As we are starting the year (in the Gregorian calendar, and soon the Chinese New Year), it begs the question: what are you currently working on, and which projects do you have in store for the coming months?

Thanks! I am very excited that the Chinese New Year is coming soon. This is the year to celebrate the dragon. I am currently working on a public art project with 35 video displays at Place des Arts in Montreal. I am also working on an upcoming solo exhibition at Galerie Charlot in Paris in April. And I will participate in Montreal’s International Digital Art Biennial (BIAN) in May.

“If I have a budget, of course, I prefer to work with creative people, but if I don’t, AI is a good way to go.”

Mario Klingemann: into the wilderness of AI

Pau Waelder

Mario Klingemann, born in Laatzen, Germany in 1970, is an artist who integrates algorithms and artificial intelligence into his creative process. His artistic exploration delves into visual expressions, linguistics, and our intricate interactions with technology. Klingemann’s creations, which primarily utilize generative adversarial networks (GANs), manifest as screen-based artworks or interactive installations. These installations captivate audiences with a never-ending array of visual experiences, where the AI models crafted by Klingemann generate portraits, abstracted figures, or textual content in real-time.

Recognized as a pioneering figure in the realm of AI-art, Klingemann’s contributions include a tenure as Artist in Residence at Google Arts and Culture from 2015 to 2018. His collaborations extend to notable institutions like the British Library and the New York Public Library. In 2018, he was honored with the prestigious Lumen Prize Gold Award. Klingemann’s work has been showcased at eminent art venues globally, including the Centre Pompidou in Paris, The Barbican Centre in London, and the Hermitage Museum in St. Petersburg.

In this article, based on the text written for the exhibition Latent Spaces that I curated for La Bibi Gallery (Mallorca, Spain) featuring the work of Mario Klingemann and the Finnish duo Grönlund-Nisunen, I offer a brief overview of the main subjects in Klingemann’s work. On the occasion of this exhibition, Niio is presenting a selection of artworks by Mario Klingemann, courtesy of Onkaos.

Explore the wilderness of AI in Mario Klingemann’s artworks

Mario Klingemann. Sirius A, 2019. Courtesy of Onkaos.

Latent space is the position relation of information

Borrowing László Moholy-Nagy’s reflections about architecture in his book The New Vision (1927), Klingemann describes the term “latent space,” commonly used in machine learning processes, as “the position relation of information.” Latent spaces are not physical spaces, but rather a way to describe how an artificial intelligence system processes the information it takes from a data set and creates clusters of items that resemble each other, according to a set of variables. Klingemann explores latent spaces as realms of endless possibilities, looking for the unexpected, the rare and weird, that which pushes his creativity further. For him, AI is not a technology that replaces the artist, but one that provides creators with new ways to develop their talent. In his work, we find endless processes fueled by “untamed” artificial neural networks that generate uncanny images and responses. Faced with this creative otherness, one must find one’s position as a viewer whose aesthetic and narrative expectations are challenged.

Nowadays, the most popular AI applications excel at creating realistic depictions of things we have seen thousands of times, or unconventional combinations of familiar images (such as a dog dressed as an astronaut riding a horse on the Moon) with impressive accuracy. However, Klingemann finds this approach predictable and boring, and aims to drive the system into the unpredictable: “[the popularization of] one-click AI art tools… forces me to look for areas out there that I still consider “wilderness” and to learn more about what it is that we humans find truly interesting and captivating.” The Hyperdimensional Attractions series addresses the unexpected by applying a three-body problem to a latent space, resulting in a triptych showing images that change according to how the feature vectors they represent “orbit” around each other. What the viewer sees is a triptych of familiar images always mutating into weird shapes and amalgamations: it is precisely these unsettling figures that interest Klingemann in his exploration of the fringes of representation. 

Mario Klingemann. Three Latent Body Problem, 2023. Courtesy of Onkaos.

Datasets and appropriation

Generative neural networks are trained using datasets, and therefore the content of these datasets is crucial. Artists such as Anna Ridler create their own datasets by making hundreds of drawings or taking thousands of photographs, while others incorporate their own writings into a machine learning model in order to create an alter ego of sorts, as do Mark Amerika and Sasha Stiles. Mario Klingemann carries out a particular form of appropriation by using datasets of thousands of images or, more particularly, taking Hyeronimus Bosch’s The Garden of Earthly Delights (1490-1500) as the base material for The Garden of Ephemeral Details. In this artwork, an autonomous AI machine made of several generative adversarial networks (GANs) constantly reinterprets the famous triptych by adding new forms to its unsettlingly surreal iconography. Klingemann uses Bosch’s masterpiece as a field of algorithmic interpretation, forcing the machine to explore the “wilderness” of artistic creation by providing a peculiar data set in the form of the Dutch artist’s singular combination of religious imagery and unbridled fantasy. The resulting artwork thus becomes an experiment in machine hallucination and artificial imagination.

Mario Klingemann. The Garden of Ephemeral Details Reserve #2, 2020. Courtesy of Onkaos.

Motion and process

Moholy-Nagy was one of the pioneering artists exploring kinetic sculpture and the use of light and industrial materials in assemblages and installations. Kinetic art also influenced early algorithmic art, with pioneers such as Manfred Mohr seeking to portray motion in abstract generative artworks and finding in personal computers a tool to create visual compositions in perpetual transformation. Mario Klingemann follows this tradition by using generative adversarial networks to create images that are not static but constantly and seamlessly morph into new shapes, accentuating the fact that the process takes precedence over the finished product. The Hyperdimensional Attractions series and The Garden of Ephemeral Details clearly show this particular decision. The former explore the position relations of feature vectors in the latent space, the resulting images becoming a way to visualize these relations. The latter deconstructs Bosch’s triptych in order to portray the effort of the machine as it tries to stretch its imagination.

Mario Klingemann. Imposture Series – The Butcher’s Son, 2017. Courtesy of Onkaos.

Latency

Motion and process imply change taking place in a certain time and space (real or virtual). Change, in turn, implies that something is about to happen. There is expectation, as one waits for the next step in the process. Paradoxically, latency is found in Klingemann’s prints, static images that nevertheless potently evoke transformation. The Imposture series explores the representation of the human body through AI models, resulting in six compositions that the artist selected from 50,000 images generated by the program. Some of the images vaguely remind of paintings by Francis Bacon but have a decidedly non-human quality to them: they lack the natural notions of the shape of the human body and the empathy that a human artist would feel. In an even more disturbing turn, the Neural Decay series evoke early photographic portraiture, again in a form that distills otherness and the uncanny. These artworks seem in the process of dissolving, their shapes blending into each other, as if they were about to turn into an amorphous substance or simply vanish. As we observe and wait for this to happen, a space of latency opens before our eyes.

References

Jochen Gutsch (2021). “Words behave like pixels and sentences like pictures”: An interview with Mario Klingemann. Goethe Institut.

Mario Klingemann (2023). Latent Talent. A*Desk. Critical Thinking.

Antoine Schmitt: coding movement

Pau Waelder

Paris-based artist Antoine Schmitt describes himself as a “heir of kinetic art and cybernetic art,” aptly indicating the two main aspects of his work: the interest in all processes of movement, and the use of computers to create generative and interactive artworks. With a background as a programming engineer in human computer relations and artificial intelligence, his career spans almost three decades and is characterized by a combination of interactive installations, process-based abstract pieces, and performances. He has collaborated with a wide range of professionals from the fields of music, dance, architecture, literature, and cinema. He also performs in live concerts and writes about programmed art.

Schmitt’s award-winning artworks have been exhibited internationally, in prestigious venues such as the Centre Georges Pompidou and Musée des Arts Décoratifs in Paris, and world-renown festivals Sonar (Barcelona), and Ars Electronica (Linz). A selection of video recordings from his generative works have been featured in our curated art program, including the artcasts Unvirtual Art Fair (Paris) and Possibles, which was exhibited at the ISEA2022 Barcelona Symposium. The artist kindly answered a series of questions about the concepts and processes behind his work.

Antoine Schmitt and Franck Vigroux. ATOTAL. Audiovisual concert, 2021

From your early works to the latest installations, there is a constant interest in the relationship between the artwork and the viewer, and more generally between a human and a machine, that often become intimate, connected to emotions and to physical proximity. What do you find interesting about this strange relationship between an individual and a machine, or an apparently sentient entity?

Programming has always been for me a means to approach reality, by recreating it. I consider programming as a radically new material, in art and in general, because of its active nature: programs are processes embedded in reality and can react to it and act upon it. This specificity allows me to recreate programmatically aspects of nature that interest me. One of the most complex entities in reality (known so far) is the human being. Many of my artworks stage a programmed artificial entity that embodies a deep aspect of human nature. These artworks act for me as mirrors for the viewer, a way to question deep human mechanisms or ways of being, like desire, curiosity, language, conflict, gravity, etc… not forgetting that humans are also animals, and are also bodies in space. 

This approach also allows me to reflect on the way we humans are programmed, by laws, evolution, society, etc… My artworks are, like deep science fiction, very much fueled by philosophy, physics, metaphysics, sociology, psychology, anthropology, etc… Using programming to create artificial entities, more or less intelligent, more or less sentient, but all embodying dynamic aspects of human life, allows me to focus each artwork on a specific concept or aspect of human nature. They are forms of living caricatures that are all the more effective.

“I consider programming as a radically new material because of its active nature: programs are processes embedded in reality and can react to it and act upon it.”

Your work is characterized both by its interactivity and the generative processes that bring it to life. What do you find most interesting about these two types of processes, the one carried out by an autopoietic generative artwork and the one carried out by an interactive installation?

All my artworks are active and exist in real time, i.e. the same time as the spectator. Some artworks are not sensitive to the real world, they are not interactive, they live their life in their own universe, and we watch them like we would watch a strange animal in an aquarium. With these artworks, the main link between the audience and the artwork is through empathy. By projecting oneself in the existential universe of the artwork, the spectator recognizes and feels the situation. It is the same process as with movies and books, with the additional dimension of the real time: with realtime artworks the spectator knows, or feels, that what happens happens here and now. It is not a recording. This gives a different dimension to the empathy, like when watching a live performance which also happens here and now.

Antoine Schmitt. Systemic. Interactive installation, 2010

With interactive artworks, I usually want to question the behaviors and inner mechanisms of the audience themselves. It is the actions of the viewer which are the artwork, I create the dynamic situation in which the viewer is immersed and I orient it so as to highlight and question certain deep ways of being. For example, the Systemic (2010), Lignes-mobiles (1999) and La chance (2017) installations draw dynamic arrows on the floor in front of passers-by to question their intention. In Psychic (2007), a text on the wall describes the movements and intentions of the spectators in the exhibition space (“Somebody is coming”).

I tend to adopt a minimalist approach: I don’t use an artistic dimension (color, figure, interactivity) unless it is mandatory for the artwork. So I don’t use interactivity unless the artwork’s subject is the spectator themselves.

“In my interactive installations it is the actions of the viewer which are the artwork, I create the dynamic situation in which the viewer is immersed and I orient it so as to highlight and question certain deep ways of being.”

Since the beginning of your career, you have collaborated with performing artists, among which composers such as Vincent Epplay, Franck Vigroux, and Jean-Jacques Birgé, performers such as Hortense Gauthier, and choreographers such as Jean-Marc Matos and Anne Holst. How did these collaborations take place? What have they brought to your own work and your creative process?

I have two different approaches to performance, whether I’m on stage or not. When I work with professional performers who use their body and actions as their main material, we craft situations where the human entity is confronted to an artificial one. This allows us to precisely stage the encounter and focus precisely on certain aspects, which become the subject of the performance. The situation usually centers on the concept of an encounter with an “other” and on the modalities of dialog. In Myselves with Jean-Marc Matos, it is about exploring various modes of dialog like imitation, fight or fusion. In CliMax with Hortense Gauthier, it is about finding mutual pleasure. In these setups, the mirror effect happens between the performer and the artificial entity rather than with the audience. The audience is watching the encounter. The artificial creature becomes an actor of the performance, in the spirit of performance: taking risks in a staged delicate situation. 

Antoine Schmitt and Hortense Gauthier. CliMax (Préliminaires), 2018

When I am on stage, I usually play live images, using a videogame-like visual instrument that I program myself and that recreates a specific abstract though consistent live universe, while the other performer plays live music. We are in a situation of semi-improvisation and we create an audio-visual temporal exploratory journey around a specific theme (the birth of shapes in Tempest, the cohabitations of multiple timelines in Chronostasis, totalities in ATOTAL, flows in Cascades, etc…). As a performer, I appreciate sharing the energy of the present moment with the audience, especially while being delved into an artificial universe and struggling with it, which the audience can feel.

Antoine Schmitt. Generative Quantum Ballet 21 Video Recording, 2022

Besides the performing arts, another strong reference in your work is scientific research: you often mention theories from mathematics or physics as the conceptual ground for your pieces. What does science bring to your work? How do you build a bridge between the scientific method and your creative process?

I am very sensitive to the deep and strong laws of the universe that math and physic theories can give us, as they allow me to both approach our reality and imagine other possible realities. What is interesting with these laws is that they are programmable so I can recreate them using programs, thus focusing on deep mechanisms, to stage them or alter them. For example, in the Tempest show, I created a universe containing many of the forces of our universe but also invented forces, thus opening the doors to parallel universes.

I often say that science and art are interested in the same subject : the crack that exists between reality and our abstraction of it. This crack is our curse as human beings. Animals do not feel this pain but as soon as one has the gift of abstraction, the distance between what we abstract and what is, is the source of all mental suffering. Science tries to close that crack by explaining as much as possible through theories and language, more and more precisely, even though it is an impossible task (as was demonstrated in the 20th century by the scientists Heisenberg and Gödel). On the contrary, Art delves in the depths of the crack, exploring all its modalities, playing with all the emotions that stem from it. And the narrower the crack, the deeper it is.

“I often say that science and art are interested in the same subject: the crack that exists between reality and our abstraction of it.”

The aspects of your work that we have previously addressed all point to a main subject which are the processes of movement, as clearly highlighted in your artist’s statement. These processes are explored in a wide range of contexts, from the quantum realm to urban societies, and among different actors, be it people, bodies, or particles. Why are these processes so important to your work, and which of these contexts is more rich, engaging or interesting to you?

I think that I’ve always had this abstract approach to reality which can be synthesized in the question “why does it move like this?”. I started with a rather scientific approach through my studies as an engineer, and when I decided to become an artist, I continued to explore this question in a different way. It is an analytical approach, a way of looking at the world, and a way to question it. I frankly appreciate all the dimensions of it and will continue to explore them, but I think that the strongest and the ones that give me the biggest satisfaction are the most abstract approaches, the ones that are the most remote from reality and still apply to many aspects of reality, existing or perceived. Black Square (2016), where a flock of white pixels try to enter an invisible square and bounce on it thus revealing it, can lead to multiple interpretations. It is a fundamental delicate situation. 

Antoine Schmitt. Black Square Video Recording, 2016

The signature element in your work, the pixel, is introduced in Le Pixel Blanc (1996). There, you describe it as “a minimal artificial presence… something that almost did not appear, but that still would be «there».” Over time, the pixel has gained more presence and become as much an object, a presence, and an absence, as part of a flow or the representation of an individual. How would you describe the evolution of your conception of this basic element and its influence on your work?

The pixel and the square are omnipresent in my work. I like my artworks to be minimal, like mathematical theorems. This naturally led to the pixel, the minimal visual element in the universe of the computer. A pixel is a small square, and by enlarging it, you get a large square. And like Malevich, I consider the square like the symptom of the human being’s power and curse: the ability of abstraction These two elements are the basis of most of my artworks. What I work on is their movement, relatively to the space around them, or relatively to the other elements. They are minimal but open to all the possibles, through their movements and the infinitely rich possibilities of programming.

“The pixel and the square are minimal but open to all the possibles, through their movements and the infinitely rich possibilities of programming.”

Your career spans almost three decades, in which you have explored many different formats of creation and distribution, from multimedia projects on CD-ROM, to Internet-based artworks, interactive installations, video mapping, screen-based pieces, software art, live performances, generative cinema, NFTs, and much more. What is your opinion on the way technology has evolved over these decades and how it has influenced art making? How have you experienced this period of constant innovation and obsolescence?

These have been very exciting years, for one because computers are more and more pervasive (we all now have a powerful computer in our pocket) and also because art made with computers is now widely accepted. It is therefore easier to create programmed artworks and to show them. The technology is more easily available, the distribution channels — in the wide sense — are numerous and the audience is listening.

On the other hand, technology is nowadays mainly used for advertising, surveillance, entertainment and manipulation of opinions, which is a social problem and has an effect on art made with technology. Many approaches build upon or react to these social dimensions, which are all needed and interesting but leave little room for the more conceptual and radical approaches. This may be true for all forms of art, but it is stronger with technological art as technology so much shapes our society these days.

Antoine Schmitt. FaçadeLifeGrandPalais. Generative mapping at the Grand Palais in Paris, 2016

What is interesting also is that I think that no new concept was really born in the field since Alan Turing invented the computer, the “universal machine”. All computer-based technologies are avatars of this unique concept. This can probably account for the fact that my artworks have not radically changed since I started. My work does not reflect on the social impacts of technology on society, nor are impacted by the various technological “innovations” and obsolescence. It is minimal so does not make use of the innovations toward more “power”, and it is rather rooted deeply in the concepts of the universal machine which have not changed : with a universal machine, all thinkable processes are programmable.

“Art made with technology often builds upon its social dimensions, which are all needed and interesting but leave little room for the more conceptual and radical approaches.”

You were already working with generative text twenty years ago, in The Automatic Critic (1999). What is your opinion about the current trend among artists to use machine learning models such as ChatGPT?

Although I am quite impressed by the quality of the interactions of users with ChatGPT (I thought that this level of quality would take more years to happen), the generative approach on these systems are in the normal continuation of the original concept of the computer. We are at the stage of imitation: these algorithms generate media that look like media created by humans, as the central mechanism of neural networks is pattern recognition and pattern generation, whether it is text, images, music, reasoning, etc… This is quite fascinating for users and it is similar to the caricatural mirror effect that I was referring to at the beginning. The art, or more generally the forms of expression, created by these algorithms in imitation of ours are a mirror to our forms of expression and thus question them.

But art is intention and responsibility. These two notions are still unique to humans. But maybe one day, we will be able to create an algorithm able to feel pain, express it with intention towards its fellow humans and take responsibility for it. There is no theoretical impossibility for this in the theory of the universal machine and I look forward to it.

In the meantime, as an artist, the most interesting aspect of AI systems remains for me the creation of biased algorithms which focus on some dimension of human nature, like Deep Love (2017) which answers all questions with “I don’t know, but I love you.”

Antoine Schmitt and Franck Vigroux. Tempest. Audiovisual concert, 2013

You entered the NFT scene in 2021 with Buy Me! a particularly conceptual, and generative piece. What has the NFT market brought to your practice? Has it influenced your production? Have you found new forms of creation or sources of inspiration, beyond its commercial dimension?

It took me some time to understand that the main new concept behind the NFT market boom was the perspective of financial profit, for collectors and for artists. This is the reason I created the satirical piece Buy Me! (2021), which embodies an algorithm desperately trying to convince its viewers to buy it, using language techniques inspired by advertising and psychological manipulation. It is a piece on the processes of marketing.

Apart from greed, the NFT market has opened the field of computer art to a new audience, which was really interesting, but I am eager to see the fusion of the traditional art market with NFT seen as a new way to buy and collect artworks.

Antoine Schmitt. The Fall of Leviathan. Interactive installation, 2021. Photo: Quentin Chevrier

You recently quoted the mathematical theory of catastrophes to describe the year that has begun and may bring sudden change, positive or negative. How does this year look for you? Which upcoming projects can you share with us?

I am very excited to start a collaboration with the DAM Projects gallery in Berlin. Its owner, Wolf Lieser, has been involved in computer art for a few decades and I look forward to working with him and his team. We will start with a solo show next autumn, with a selection of historical works and new artworks.

I am also very excited by two new live audiovisual performances, Videoscope and Nacht, with Franck Vigroux, which are in the making, and that will tour the world along with the existing performances (Melbourne, Gijón, San Francisco, etc..).