Perception might be reality

In today’s context, related to health emergency, some aspects of technological design assume a peculiar role. The first is the theme of distance; the second is the theme of the extension of sensory capacity through digital technologies; and the third is the recognition of the state of the subjects. These three sets of factors, the need to manage distance, the need to extend technological sensoriality into the distance, and the need to recognise in the distance not only the sensoriality but also the state of the user—whether a patient, driver or viewer—thus constitute a new axis of technological design.

Picture left: Interfaccia suono, Copyright: RE Lab

On the one hand, therefore, it is an expression of the Reggio Emilia Approach, a socio-constructivist educational philosophy based on research and the values of lifelong learning that strongly believes in the dialogue between theory and practice, and sees the child in his relationships with others, as an active part of the learning process.  On the other hand, the interest of the multidisciplinary research group in the rendering of sound spatialisation has also approached the concepts of Acoustic Ecology described by the teacher/composer Murray Schafer in his great work The tuning of the world (R.Murray Schaefer, 1985): The soundscape is our acoustic environment, the ever-present noises we all live with. The author suggests that today we suffer from acoustic overload and are less able to hear the nuances and subtleties of sound. Our task, he argues, is to listen, analyse, and make distinctions despite the noise pollution. 

The ‘Station of Being’ is a fully functional, experienceable prototype of a Smart Bus Station in the Northern Swedish city of Umeå that was opened in 2019. The project was initiated by the City of Umeå and partly funded by the H2020 Smart City Lighthouse project RUGGEDISED.

The aim of this design is to make public transport more attractive, to promote an increase in use and, in turn, to lower the carbon emissions of the city. The Station achieves this by affording a positive waiting experience, aiming to turn wai(s)ting time into time to feel, reflect, pause and move: in a nutshell, to be.

Two main components of the design contribute to the passenger’s experience.

–   A dynamic light and soundscape informs passengers in or near the station which bus is about to arrive. A subtle play of lights refers to the arriving bus line through its colours, while a soundscape plays to tell stories about the character and history of the line’s final destination. This combination offers ambient information which can be perceived on one’s sensorial periphery, removing the need for passengers to pay constant attention to the coming and going of busses. 

–   Wooden ‘pods’ hanging from the ceiling provide a physically comfortable place for waiting passengers and contribute to a feeling of safety. The pods can be turned 360 degrees, providing a way to lean and stay out of the wind or just mindlessly sway. The pods allow one to seclude oneself, or to create a social space.

Picture above: Station of Being, Copyright by Samuel Pettersson

Transformation is a substantial change, often associated with innovative and radical change. Transformation occurs when one configuration is converted or changed into another, whereby the change is major or complete (Mirriam Webster Dictionary). We use the term ‘transformation’ when designing for transforming practices (TP) to denote substantial enduring change of values, ethics, and related behaviours of a person, a community or society, triggered by the need of creating alternative ways to engage with the world, addressing specific societal challenges. In this context, people and communities become (and are) transformation itself. This demands a leap, a paradigm shift, even for what seems to be smaller personal transformations (Hummels et al., 2019; Hummels, 2021, Trotto et al. 2021).

Picture right: Copyright by Ambra Trotto, Caroline Hummels, Jeroen Peeters, Daisy Yoo and Pierre Lèvy

The original brief included a number of practical and functional requirements, and stated amongst other things that the bus station should: (a) be highly innovative, smart and sustainable; (b) promote a feeling of safety for passengers; and (c) involve citizens in its development. Most importantly, however, special emphasis was placed on the requirement to prevent heat loss from buses when opening their doors to (un)boarding passengers. This is a typical example of a narrowly-defined, reductionist approach to requirements, which can lead to inadequate, mediocre solutions. It was clear to the design researchers that a reframing of the project and requirements was necessary in order to approach the challenge as a ‚wicked problem‘ and come up with innovative interventions.

The first step in this reframing process was taken through a course at the Umeå Institute of Design. About 20 design students carried out design interventions in the city,

In which metaverse do we want to live?

The Metaverse—A ubiquitous, synchronised universe where presence and sense of presence become indistinguishable.

For those of us who were born in the 20th century, the word „metaverse“ came to be associated with science fiction books and films of the 1990s which depict a dystopian future dominated by technology. In most of them, humanity is dominated by mega corporations, totalitarian governments or dehumanised scientists who use simulated worlds and robotic environments to subject people to situations of oppression, domination and injustice.

But for those who grew up with the online world as an integrated space within the physical world, the metaverse is simply a collection of all the virtual universes that coexist in the cloud and are necessarily shared, interactive, immersive and collaborative.

One is inherited from the current model of society, led by closed platforms and Big Tech that are not necessarily the bad boys of so many dystopian novels. But there are indeed few and dominant players of the digital economy of the 21st century. They claim the central role such huge machines should be allowed to have, to make this Metaverse happen in an ordered and controlled way, far from actual wilderness of the internet sphere that includes the worst we can see from ourselves in the deep and dark web.

Another vision, in stark contrast, talks about an Open Metaverse created with open technology and open protocols that allow increasingly decentralised and accessible environments for almost anyone. This is the vision of those creators of—and believers in—Web 3.0, which has been evolving for at least 10 years. They advocate for an Open Metaverse based on principles of digital sovereignty and easy access to data and protocols for people to achieve a new world economic order, an order  perhaps better, perhaps fairer, perhaps more promising for those who now lie at the bottom of the actual world’s economy.

A third vision, probably the most plausible one, is a coexistence of both centralised and decentralised visions, convergent and divergent environments in which any kind of ordering or regulation will be complex but might prove necessary.

multiple identities for each person—physical and digital—and a possible total sovereignty for each opens up an unexplored universe.

The sovereignty of identity in the physical world, in the virtual world and in what emerges from the combination of both universes for future generations may completely change social relations in the decades to come.

Somehow, once again, as in Europe’s First Renaissance, the New Renaissance will be all about the convergence of Culture, Economy and Technology.

Towards digital humanism

The European Renaissance began first in Italy after a period of exhaustion by plagues, misery and wars. In the mid-14th century, the plague had plunged Europe into one of the worst catastrophes of mankind, an incurable disease that brought great pain to those afflicted and usually a quick death. Even before the plague, a long-lasting famine had taken hold, for which climate change in the form of significantly falling temperatures probably played a decisive role. The structures of social order eroded and everyday life became brutalised. The population declined markedly, with the paradoxical effect of a valorisation of human labour and an increase in productivity brought about by new technologies. The European Renaissance was preceded by a creeping decline in the authority of clerical and princely authorities, and was characterised by a return to ancient thought, especially that of the Greek Classical period and the Roman Empire. Aristotle was disposed of—prematurely—because the Thomasian worldview, unlike patristics, was based on his writings and had made them the authoritative source alongside the Holy Scriptures.

The renaissance of digital transformation is based on the empowerment of human authorship, on the development of human creativity, on the targeted, intelligent and measured use of new technologies for human purposes. It aims to humanise the world of work through sustainable production and the establishment of fair practices in the global economy, while the current development path has established monopoly structures in the form of large tech giants, shifted greater parts of economic value creation to platforms and made a successful business model out of the skimming of user data from digital service offerings for marketing purposes. This humanistic form of digital transformation will not be feasible without a state framework in the form of digital infrastructures, without an independent European path of „human centred AI“ and transparent uses while safeguarding informational self-determination rights, without a European legal framework of digital dynamics. But the chances are good that the humanistic form will ultimately prevail over the commercial model of Silicon Valley and the state control model of China and other autocratic and totalitarian states.

The show must go on

Cultural institutions the world over collectively seek out new ways to connect meaningfully to local, global and diverse audiences—To co-produce and programme new kinds of experiences created with new kinds of creative teams and new kinds of partners offers exciting opportunities. Audiences are eager to connect with both big ideas and one another and have challenging and new experiences ahead. Audience Labs is an artist-led initiative looking at the intersection of Performance, Innovation and Social Change and was born at the Royal Opera House in London’s Covent Garden, a location with a centuries-old tradition of performance. Introducing the creative practices of opera and ballet to immersive technologies provided a unique opportunity to look at bringing human emotion and collective experiences together. We want to look at what is possible but always continue to ask what is desirable and in what way technology enriches our life in the wider societal shifts and upheavals of the digital and green transformations taking place before our eyes.

One of the projects Audience Labs presented in 2021 was Current, Rising. Described as the world’s first ‘hyperreal’ opera experience, it invites audiences to step into an atmospheric virtual world and take centre stage in the performance.  

When I joined the Royal Opera House I was inspired by the ‘epicness’ of opera, the way the storytelling was driven by music first, narrative is secondary, the events are there to support the ideas and emotions. So many of them are about heightened emotions, to let you feel all the feels, so to speak, everything is bold and big. In opera the word Gesamtkunstwerk is used, or „all-embracing art form“.  

The show that emerged from this process was Current, Rising. Inspired by the liberation of Ariel at the end of Shakespeare’s Tempest, Current, Rising takes four people at a time on a journey through the six phases of the night. Participants are empowered to collectively explore a series of imaginary dreamscapes, travelling from twilight to dawn as they encounter ideas of isolation, connection and re-imagination. It explores freedom as a process within our control, rather than as a state in which we exist, and asks how we might harness personal responsibility to join with others to re-think the future.  

The audience enter a real life theatre set wearing VR headsets and go on a dreamlike journey carried musically by a poem layered in song. It takes the epic, music-driven and poetic qualities of opera and combines it with the possibility of VR to create any possible landscape, defying all natural laws, pushing beyond what is possible on the stage. In Current, Rising music, the visual world and the physical experience are completely enmeshed, changing the relationships between the creators, the usual sequence of creation, and the relationship of the audience to the work. Here the audience members are the protagonists: they are inside the work, and their physical experience is a part of the work itself.

Art can make a world worth living in. After shelter, equality, equity and a healthy planet, there is art. Audience Labs explores possible futures through art—we aim to construct shared experiences and transformative moments that enrich people’s lives.At a time when we’re all struggling with what the future holds, we wanted to bring makers and audiences together to imagine a shared future, reflecting collectively on: Where shall we go? How do we want to feel? What do we want the world to look like? We are interested in expanding where culture takes place, and broadening who it is for, who gets to make it and how it interacts with other agents of connection and change in the world in the Next Renaissance

As Audience Labs is starting its time at King’s College London, we will be working with a network of industry experts, collaborators and experts to focus on critical questions surrounding the future of performance. 

Picture left: Current, Rising CGI Production Shot House of Subconscious, Copyright: Joanna Scotcher & Figment Productions

Why Cultural Led Urban Developments are necessary for Resilient Cities

Placemaking is a global civic trend in urbanism. However, a novel combination of arts, technology development and processes of urban governance empowers placemaking to be responsive and forms an essential contribution to a next renaissance of global cities. Not necessarily by making them less global as the following case studies show, but by bringing in a perspective of interconnected local publics and public values to urban policy, from the design of public spaces to the organisation of urban resource management and carbon-neutral cities.

Picture above: Digital Calligraffiti 2017-21 commissioned by Public Art Lab in cooperation with the artists Michael Ang, Don Karl and Hamza Abu Ayyash, Copyright by Michael Ang and Public Art Lab

Responsive placemaking can take various forms, and we’d like to highlight three of these here as examples of projects in which we have been involved ourselves.First, through projects of representation, responsive placemaking highlights the connections between people and places, and contributes to the construction of local publics. They can aid bringing out a ‘sense of place’ as well as a layered, variegated ‘sense of us’. 

Second, as collaborative management and governance, responsive placemaking aims at providing these local publics with tools to organise themselves as collectives, addressing civic engagement and social inclusion. It seeks to set up alliances and coalitions of citizens, government representatives and professionals to organise around issues of communal concern, and the collaborative and inclusive management of local resources, from reusable energy communities to commons-based housing cooperatives. 

And third, illustrating practices of translocal exchange and mutual learning, responsive placemaking does not approach local initiatives as isolated communities, set in their own time and place. Through the various scapes they are often part of various translocal movements and communities that enable mutual learning.

The European Lighthouse project CONNECTING CITIES⁸ explores translocal exchange through artistic practices, the networked infrastructure of urban screens, media facades and projection walls worldwide. Its goal is to engage citizens from distant places in new creative forms of togetherness and exchange. Under the guiding principle ‚Urban Screens are our Walls’, the CONNECTING CITIES NETWORK transforms these digital infrastructures into platforms and visualisation zones which provide new opportunities for communication, community building and translocal dialogue and telepresent scenarios in the public space by overlapping layers of virtual reality, augmented reality and networked society. For example, DIGITAL CALLIGRAFFITI⁹ allows all social groups to freely share their messages, wishes and visions on burning issues of our society. It combines the traditional cultural technique of calligraphy and graffiti with the possibilities of urban media art and new technologies. DIGITAL CALLIGRAFFITI transforms walls and digital screens into a canvas of civic expression to build bridges of intercultural understanding and promoting diversities as the medium to bring us closer.

Case study: The Pignatelli Chapel, Naples: How bringing past Renaissance alive with digital tech opens new civic and economic spaces

The chapel or small family church of Santa Maria dei Pignatelli, located in the heart of Naples, in Piazzetta Nilo, has 14th-century origins and was first renovated from as early as the last decades of the 15th century at the behest of Ettore Pignatelli, Duke of Monteleone and Borrello and future viceroy of Sicily. The works lasted until 1515 and included the construction of the two important funerary complexes that make the chapel one of the jewels of Neapolitan art of the mature Renaissance: the tomb of Carlo Pignatelli, on the left wall, the work of Tommaso Malvito’s workshop around 1506-07; and the small chapel of Caterina Pignatelli, the work of the great Spanish sculptor Diego de Silóe around 1513-14, rich in decorations taken from Antiquity and close to the culture of the Papal Rome of Raphael, Michelangelo and Sansovino. From 1736 the chapel was restored and assumed baroque forms. It was provided with a new altar in polychrome marble designed by Gaetano Buonocore and realised by Gennaro Di Martino, with rich marble decorations on the walls; a floor in inlaid marble was also added, realised by the same Di Martino and by Antonio Di Lucca on a drawing by Ferdinando Fuga (1761). A bowl-shaped dome was frescoed in 1772—as well as the drum, pendentives and sails of the adjacent vault—by Fedele Fischetti.

Our video recounts not only the recovery of an asset of such extraordinary value, returned to the city, but also how this reunification with the city context is enhanced by the crucial role of digital technologies. It is therefore an exemplary journey: from the Renaissance as a material and cultural heritage to be preserved, to the New Renaissance with the potentialities opened up by new technologies for cultural and creative industries.

Sources pictures: All pictures are screenshots from the Video below, Curtesy of Università Degli Studi Suor Orsola Benincasa.

Mission accomplished?

The world is about to enter the Metaverse, says Mark Zuckerberg. The Metaverse, in his vision—or in his eagerness for more power and wealth disguised as a noble contribution to humanity—is a virtual place where we work, have fun, meet friends, watch movies, play games, and, most of all, consume. The Metaverse will be the internet, email, new reality, and everything. The Metaverse is where we will live. Monika Bickert, Metaverse’s vice-president in charge of policy content, wrote an article for Le Monde on November 10, 2021, in which she stressed that the Metaverse is a new kind of life.Facebook, Mark Zuckerberg’s previous company name that he is now trying to sweep under the rug, claims to have 2.9 billion users in 2021. It may be so. If it is, the foundations for Zuckerberg’s Metaverse have already been laid—in a very short time, considering Facebook’s founding in 2004. The Metaverse is not here yet, but Facebook is—and Facebook is the dress rehearsal for the Metaverse.

Picture above:  High-Performance Computing Center HLRS University of Stuttgart, Copyright: Boris Lehner for HLRS

Picture left:

Several human hopes and desires are driving this prescription. Mathematical models avoid ambiguity by helping us to identify specific solutions in an ambiguous world. We can further claim that our solutions are not distorted by human intervention, with mathematical algorithms executed by a flawless machine—only experts understand that computers are neither flawless nor do they operate without human intervention. The idea of excluding human error allows us to pretend that our digitally created realities based on mathematical models are just—devoid of any prejudice and human flaws. In addition to this idea of an artificially created intelligence, the vision of creating life without resorting to nature is tempting, especially if we believe that this is a species created only from human minds and exempt from all the flaws that come with the dirty and mortal nature that surrounds us.

Our enthusiasm for our creations leads us to ignore the warning issued by John Culkin in 1967: „We become what we behold. We shape our tools, and then our tools shape us.“ Taking the models we create for reality has always been the danger of our model-based approach

MSC has therefore structured its Reach Project as the Research and Creation Center for eCulture and the Humanities. The Center intends to develop a program building a bridge over the ever-growing gap between the Humanities and science and technology. This distancing between the two fields has become widely addressed since the beginning of the 20th century—the conflict between the „two cultures” C.P. Snow described in his 1959 eponymous book.⁴

Picture left: Logo Media Solution Center Baden-Württemberg

Recognised as a major influence on economists, Leonard Reed’s 1958 essay ‚I, Pencil’ demonstrated that the invention of even the simplest of objects is the result of mass collaboration. A pencil is made by loggers, graphite miners and factory workers. Its form is predicated on the invention and widespread adoption of writing and drawing, and it requires an ecosystem of other technologies from pads of paper to marketing and distribution networks. The workers involved need to eat food and travel to work, and this all takes place within an elaborate economic and social web.

As interwoven, interdependent, cross-disciplinary and intricate as those systems were, Reed could not have foreseen the world to come. The extent to which the processes, raw materials and interconnected networks would become exponentially and globally more complex sixty or so years later would have been absolutely unthinkable. What had been the paradigm-shifting impact of Reed’s observation now needs to happen again at the next order of magnitude. We live in a context where the affordances of digital technologies, the entirely inescapable global supply chain networks and the ecosystemic challenges of the 21st century implicate and involve every single individual. We can no longer extricate ourselves from those global systems. Collaboration and the intersection of diverse ideas and specialisms are not merely beneficial for innovation in this context but fundamental for survival.

Patenting is still by far the most commonly used system of IP valorisation, even though it still carries the signs of the era that launched it, when married women were not allowed to own any kind of property:

Women do not get a fair share when it comes to patenting and are far less likely to own patents. This disparity is in part because of not only the inherent biases in science and technology and in the patent system itself, but also because of the high costs of even applying for patents. […] Patents are a glaring exception to the unregistered protections provided in other areas of intellectual property, which are more egalitarian in design.¹²

Currently, European patent granting procedures are expensive, slow (three to five years, on average), burdened with obscure legal terminology and fraught with problems relating to the definition of ideas and similarity to other patents. IP Screener¹³ makes a powerful case for making all patents findable with AI-assisted technologies, so independent inventors can affordably source and track existing inventions using plain language, build their knowledge from existing inventions and avoid infringing on protected IP.

Beauty, epiphany, neurons

Six hundred years ago a small group of innovators changed history. They lived in Italy, and the name of their era, the Renaissance, is delightfully compact. It means rebirth, renewal and the emergence of new character. Europe has experienced more than one renaissance, but the Quattrocento occupies a special place in the imagination. The reason is clear. Unlike comparable eras, the Italian Renaissance produced colourful characters who embody wide-ranging achievement: Renaissance people who excelled in the arts, science and engineering. The influence of Filippo Brunelleschi, Leon Battista Alberti and Leonardo da Vinci spread throughout Europe, and they continue to inspire the world. In most eras we discourage polymaths, but we make an exception for Quattrocento genius.

Header photo: View from Erte Ale volcano in Ethiopia
This was a study for the performance installation ›Epiphany: The Cycle of Life.‹ (c) 2015 Ali Hossaini

Photo below: Epiphany: The Cycle of Life
Singer Netsayi and the Young People’s Chorus of New York City perform ›Epiphany: The Cycle of Life‹ at BAM in New York City. (c) 2015 Ali Hossaini

It was not a given that 15th-century Italy would invent perspectival drawing. Euclid codified optics over two thousand years ago, and some scholars argue that the ancient Greeks applied perspective to painting, architecture and theatre. As the name implies, the Renaissance was a rebirth of classical learning. But the Italians of the Quattrocento went further than the ancients, and perspective spread across Europe from Italy, and from Europe to the world. We can describe perspective in one stroke. By applying science to art, and shaping science with art, Renaissance polymaths created the flat panel display. 

The beauty of perspectival art reveals the structure of space. Our first guide to perspective is Alberti’s 1435 treatise On Painting, a slender volume which generated incalculable economic value when its techniques were applied to new disciplines. Within a century of Alberti’s treatise, inventors used the camera obscura to automate perspectival drawing, and, within decades, Louis Daguerre and William Henry Fox Talbot industrialised it by inventing photography: drawing with light. To this day, cameras are a locus of innovation, notably in mobile handsets, that charge economic growth. At the same time, the camera is a useful metaphor in digital media. Gaming and animation developers use a ‚camera tool‘ to define the user’s point of view. Today’s 3D design software is one of Alberti’s many heirs.

We can speculate on why a spatial multiverse eludes us. Is it because polymaths invented the flat panel display? Renaissance engineers were accomplished artists and scientists; indeed, it is impossible to characterise them with a single word. Their aesthetics embodied intellectual and technical rigour, and they combined innovation with humane values. Consider Brunelleschi’s architectural masterpiece, Il Duomo. Accidents commonly mar construction, but, during Brunelleschi’s management, Florence’s cathedral was erected without a death. Renaissance polymaths studied ancient texts, but they probed nature with tools of their own invention. They envisioned modern science by creating humane environments filled with beautiful objects.

Contrast the Quattrocento with today. Contemporary professionals specialise while the paragons of the Italian Renaissance embodied wide-ranging talent. Perhaps this explains their lasting influence. Drawing in perspective was not instantly popular. Now we take the medium for granted, but 15th-century audiences treated perspective as advanced technology: difficult to learn, marginalised and controversial. Perspective’s utility did not immediately show, and its power to create illusions seemed a gift from Satan. But Renaissance paintings incorporate fine aesthetics. Their masters created paradigms for the media they invented.

Let me ask again: What lessons can we take from the Renaissance? Aesthetics and contemporary ethics should drive engineering. Technology should express its time; it should express the possibilities and necessities of the moment. Consider another great polymath. In the early 19th century, the Prussian explorer Alexander von Humboldt travelled the world in a quest for grounding principles. Arguably the founder of ecology, von Humboldt published illustrated books that demonstrated interconnections among the physical world, the biological environment and human interventions. Though celebrated while alive, von Humboldt’s star faded and his holistic, interdisciplinary theories were replaced by specialisation and linear causality.

Humanity stands at a precipice, and creative industry is both problem and solution. Our ability to create artificial environments has masked disastrous mismanagement of nature. At the same time, cultural conflicts have paralysed our capacity to act. Why can’t we save ourselves and the innocent creatures who share our planet? The answer turns on the exclusionary cultures we have created—cultures which oppose the ecologically transcendent humanism of Alexander von Humboldt and other ecologists.

Let us return to spatial media. How should the multiverse evolve? Should it sever humanity from nature, or can it bind society to ecology? Should it enable monopoly, even tyranny, or provide opportunities for all? Markets will determine the multiverse’s form but so will designers. Rather than inventing for its own sake, the multiverse’s engineers need to judge outcomes. Good intentions alone are a recipe for disaster. In the 1990s, I deployed social media into television networks. My goal was to create media for a democratic society. Massive channels, controlled by a few, should be fragmented into capillaries run by individuals. Social media would—I thought—encourage critical thought, dialogue and solidarity.

Here’s another lesson from the Quattrocento. Its polymaths did not separate thinking, making and doing. Nor did they separate ethics, aesthetics and technics. Today holism is deeply unpopular, but I think it is vital for innovators to adopt it. And I argue that holism is vital for producing commercially viable forms of spatial media. The multiverse is not an end in itself: it is the foundation for future generations to innovate. My work in developing standards for ethical design of AI and brain-computer interface has convinced me that media will converge with architecture, urbanism and biology. The next-generation Internet will transcend our current conceptions. Did Alberti imagine streaming media when he wrote On Painting? Imagine the value unleashed by 21st-century infrastructure which supports decades, even centuries, of progress.

The New European Renaissance could mesh creative industry, science and engineering with an ethos of global awareness. It would preserve the cohesion of European institutions, and it would support the relevance of European industry—and the European project—in a world of superpowers. Let us invest in a 21st-century Renaissance that starts in Europe but benefits every global citizen. Europe’s institutions, the heirs of the mecenati, can generate vast new wealth, and they can also ensure the benefits of prosperity for future generations. What illuminates this path to a bright future? Humanity, technology and ecology coupled to creative imagination.

Photo left: GROUPTHINK: The Internet of Neurons
Neuroscientist Shama Rahman and computer scientist Mick Grierson collaborated with Ali Hossaini, an AI and a cohort of artist-engineers to create GROUPTHINK. Performed at National Gallery X in London and broadcast via Ars Electronica Festival 2021. (c) 2021 Ali Hossaini