Human-Machine Interfaces, intelligent systems and cultural heritage
Maria Giulia Losi, PhD, Project Leader and Interaction Designer at RE:Lab; Silvia Chiesa, PhD, Project Manager in the R&D group of RE:Lab; Annalisa Mombelli, R&D group of RE:Lab, Reggio Emilia
Human-Machine Interfaces, intelligent systems and cultural heritage
Perception might be reality
In today’s context, related to health emergency, some aspects of technological design assume a peculiar role. The first is the theme of distance; the second is the theme of the extension of sensory capacity through digital technologies; and the third is the recognition of the state of the subjects. These three sets of factors, the need to manage distance, the need to extend technological sensoriality into the distance, and the need to recognise in the distance not only the sensoriality but also the state of the user—whether a patient, driver or viewer—thus constitute a new axis of technological design.
Picture left: Interfaccia suono, Copyright: RE Lab
A new anthropocentric setup for technologies
This new axis prefigures the attempt to fill the gaps, the attempt to relaunch an idea of technology that can bring people together, even in this current social framework of difficulty. Therefore, starting from this idea of rapprochement, a new anthropocentric approach to technological performance can be founded, which is undoubtedly one of the central axes of a New Renaissance, based on the integration between the anthropocentric culture of the Renaissance tradition and the potential 4.0 of the technologies mentioned above: those intended to extend the sensory framework; those intended to increase the recognition of the subject and respect of his emotional and cognitive state,; and finally those oriented to extend his ability to live experiences not necessarily in presence.Starting from this premise, RE:Lab, in the framework of the activities of the Emilia Romagna Region, wanted to compose this complex puzzle with diversified initiatives within several projects. First of all, in the midst of the pandemic, the experience of restitution of a phenomenon of intense bodily relations such as dance was sought. Subsequently, the idea of extending the sensory capacity in the relationship, born even before the health emergency took shape, was resumed and developed, and then relaunched during the pandemic experience through the concept of an Atelier, that is, a creative space that takes into account the management of sound manipulation. Finally, we added the idea of including in the processes of technological relations the recognition of the state of the user, not only cognitive but also emotional, so that his emotions allow to recognise the state of use and on the basis of this to orient and design systems of safer and more effective interactions.
A Living Lab for Novel Human-Machine Interactions – A New Grammar for Dramaturgy and Choreography in Digital Space
How can user experience design foster innovation processes in the dance and live performance sector?In a historical moment like the current one, where distance has become one of the hallmarks of experience, the performing arts and dance sector has been put to the test. At the beginning of 2020, the Aterballetto Dance Foundation asked itself some questions:
– Can the emotion of the performance exist without the performance?
– Is it possible for a dance performance to engage the viewer while the dancer is not physically present?
– Is there a safe-conduct to ensure that art can express its expressive power in the absence of contact and in the presence of „distancing“?
In the most uncertain months of 2020, marked by the closure of theatres, the Aterballetto Foundation has managed, through the project Virtual Dance for Real People1, in collaboration with RE:Lab, to imagine a different user experience, through the techniques of Cinematic Virtual Reality and the use of virtual reality visors (such as Oculus Quest).
The design of the user experience has led not only to the creation of a new format for viewers but also to the formulation of a new grammar involving choreographers, interaction designers, and video experts, in identifying a common language and a new way of thinking about choreography. The spectator in fact has been imagined in a central position, in which the viewer can dance together with the dancers.
The search for a relationship of closeness with the spectator has led us to investigate the immersive aspects of technology, also through the use of spatialised sounds, without making the dance abstract: the choreography, conceived for the final technological instrument, remains at the centre of the experience, able to release new and unexpected emotions.
Picture right: MicroDanza Meridiana, Fondazione Nazionale della Danza / Aterballetto, Coreografia: Diego Tortelli, Danzatrici: Casia Vengoechea, Annemieke Mooji, Foto Celeste Lombardi, Technological development and user experience design RE:LAB
Dynamic Mapping—Sound Atelier in distance environments
Many studies have analysed how much music and the sound environment that surrounds us can influence cognitive and linguistic development.How can we investigate, through new immersive audio technologies, the relationship between children and sound, considered as a compositional element of musical language but also of language itself?
From the encounter between engineering research in the area of the spatialisation of sound and the mission of Reggio Children (International Center for the Defense and Promotion of the Rights and Potentialities of All Children), the Sound Atelier2 was born, a laboratory in which children can experiment with a new and complete concept of sound fruition, for which the new child-friendly application EUPHONIA was designed.
The design of the user interface was developed for and with the children, who were involved as design partners to verify their understanding of a 3D interface. In fact, through the support of the figures of the atelierista in collaboration with the designer, it was possible to highlight the difficulties that children had to orient themselves in a virtual environment and a new UX design was redefined for the tablet application called EUPHONIA from the union of the ancient Greek words EU (true) and PHONIA (sound). The app’s interface reproduces the room that houses the atelier in order to give the user an idea of the position of the sound in the space. From a dynamic acoustic library it is possible to choose sounds, associated with representative images, and position them in the space, thus composing 3D soundscapes.
Technological-pedagogical research on sound in space has thus taken its most natural form as the „Atelier of Sound”, an immersive audio environment/laboratory in which children can experiment and dynamically explore the entity “sound“ in all its dimensions.
On the one hand, therefore, it is an expression of the Reggio Emilia Approach, a socio-constructivist educational philosophy based on research and the values of lifelong learning that strongly believes in the dialogue between theory and practice, and sees the child in his relationships with others, as an active part of the learning process. On the other hand, the interest of the multidisciplinary research group in the rendering of sound spatialisation has also approached the concepts of Acoustic Ecology described by the teacher/composer Murray Schafer in his great work The tuning of the world (R.Murray Schaefer, 1985): The soundscape is our acoustic environment, the ever-present noises we all live with. The author suggests that today we suffer from acoustic overload and are less able to hear the nuances and subtleties of sound. Our task, he argues, is to listen, analyse, and make distinctions despite the noise pollution.
The result of this Audio Interaction Lab is a dynamic 3D sound scene whose content and spatial organization is controlled in real time by the children, enhancing their awareness and sensitivity of sound in space.
Picture above: Euphonia, (particolare dell’App, schermata dell’ Area ambiente sonoro), progetto Atelier del suono/Reggio children, Technological development and user experience design RE:LAB
Spillover effects of Sound Design: Increasing driving safety and comfort through the recognition of one's emotional state: from the HU-DRIVE experience to the NextPerception project
Sound Design and its value-added in Human Machine Interaction are not limited to the Cultural and Creative Sectors and Industries—for illustration we review one project in the automotive sector: Several studies in the literature show a strong relationship between emotions and traffic accidents. In fact, negative emotions, such as fear, anger, disgust, or sadness, increase the likelihood of exhibiting dangerous behaviours while driving (Magaña et al., 20203).
In recent years, affective technologies have been developed that offer new opportunities to improve road safety by using the recognition of the driver’s emotions and state, and their regulation through appropriate interaction strategies on board the vehicle. Affective user interfaces are those interfaces that are capable of understanding through psychophysiological measures, conversational analysis, and facial expression recognition human emotions while interpreting, adapting, and potentially responding appropriately (Braun, Weber, & Alt, 2020).4
Within the HU-Drive5 project, the HUman Driver Assistance System presented at CES 2021 in collaboration with the startup Emoj, RE:Lab presents an innovative solution to increase safety, quality and comfort of driving. The HU Drive system, in fact, automatically recognises the driver’s emotional and cognitive state and adapts the vehicle interface in real time.
Novel Human-Machine Interactions based on trust and transparency
Recently, new trends are emerging in the topic of human-computer interaction. In order to achieve better levels of trust in automation, the „negotiation-based“ interaction approach has been studied (Koo et al., 20146; Castellano et al., 2018)7 which consists in providing the user with explanations instead of warnings, in order to increase the transparency of the system.
Taking this line of research as a basis, RE:Lab is carrying out the activity of developing an interface able to adapt to the state of the driver within the NextPerception project. The main objective of the project is to develop a Driver Monitoring System, able to classify both the driver’s cognitive states (distraction, fatigue, workload, drowsiness) and his emotional state (anxiety, panic, anger) as well as the driver’s position inside the vehicle. What can we learn here about the role of culture in digital societies and take spillover back to Cultural and Creative Industries?
In conclusion, RE:Lab’s research activity focused on the design of innovative strategies of interaction with emerging technologies, including artificial intelligence systems that do not want to replace people but put users, at best humanism, at the centre stage to amplify human capabilities.New conceptual models, new strategies, new metaphors will be born in future research about man and machine interaction, making the user more aware of what happens when interacting with these systems, and which are the expression of an interweaving between science and art, technology and cognitive and emotional languages: all elements at the basis of the redefinition of a New Contemporary Renaissance.
References
1 Trailer VIRTUAL DANCE FOR REAL PEOPLE (2021): https://youtu.be/h9u1Cs2GMf8
2 Federica Protti, Simone Fontana, Elena Maccaferri, Claudia Giudici, and Roberto Montanari. 2013. Audio-interaction lab: designing an immersive environment to explore the acoustic ecosystem with a tablet interface. In Proceedings of the Biannual Conference of the Italian Chapter of SIGCHI (CHItaly ’13). Association for Computing Machinery, New York, NY, USA, Article 19, 1–8. DOI:https://doi.org/10.1145/2499149.2499160
3 Magaña, Víctor Corcoba, et al. „The effects of the driver’s mental state and passenger compartment conditions on driving performance and driving stress.“ Sensors 20.18 (2020): 5274.
4 Braun, M., Weber, F., & Alt, F. (2020). Affective Automotive User Interfaces–Reviewing the State of Emotion Regulation in the Car. arXiv preprint arXiv:2003.13731.
5 EMOJ & RELAB present HU-Drive @CES2021, the innovative solution presented at CES 2021 by EMOJ and RE:Lab: https://www.youtube.com/watch?v=SQ_EWRAM23o
6 Koo, J., Kwac, J., Ju, W., Steinert, M., Leifer, L., & Nass, C. (2015). Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance. International Journal on Interactive Design and Manufacturing (IJIDeM), 9(4), 269-275.
7 Castellano, A., Landini, E., & Montanari, R. Un nuovo paradigma di interazione per la guida cooperativa: il progetto AutoMate.
Maria Giulia Losi
Maria Giulia Losi, PhD in „Humanities and Technologies: a research integrated path“. She is a project leader in RE:Lab with many years of experience in the design and development of Human-Machine Interaction systems and EU research projects. At RE:Lab since the beginning of 2011, she has been involved in several projects as Project Manager and Interaction Designer in different domains, from Automotive and Off-HIghway vehicles HMI and IoT Apps to Cultural Heritage interactive exhibitions design. Key publications are reported in the following:
1. Leandro Guidotti, Maria Giulia Losi, Roberto Montanari, Francesco Tesauri, “Motorsport Driver Workload Estimation in Dual Task Scenario”. IARIA Cognitive 2011
2. Daniele Pinotti, Fabio Tango, Maria Giulia Losi, Marco Beltrami. “A model for an innovative lane change assistant HMI”. HFES 2013 Conference Proceedings
3. Corazza F., Snijders D., Arpone M., Stritoni V., Martinolli F., Daverio M., Losi M.G., Soldi L., Tesauri F., Da Dalt L., Bressan S. „Development of a tablet app to optimize the management of pediatric cardiac arrest: a pilot study“. IPSSW2020 – 12th International Pediatric Simulation Symposia and Workshops
Annalisa Mombelli
Annalisa Mombelli, Graduated with a Master’s Degree in Conservation of Cultural Heritage at University of Parma, defending a thesis about the innovation in publishing by Bodoni with his illustrated masterpiece “Epithalamia”. She also graduated in Advanced European Studies at European College of Parma. She worked as a freelance coordinator for private and public cultural projects. Now, she works in the R&D group of RE:lab, following activities in the field of education and culture.
Silvia Chiesa
Silvia Chiesa, PhD in Psychological, Anthropological and Educational Science at the University of Turin, defending a thesis concerning the spatial cognitive abilities in disabled persons. She expanded her activities outside the academic field working in the context of Human-Machine Interaction and more in general in the User Experience Research. She is Project Manager in the R&D group of RE:Lab. Here, she led international projects and activities concerning usability and acceptance assessment, in the domains of transports, household appliances and healthcare.
From crisis in systems to crisis of systems!
Edina Soldo, Full Professor at IMPGT-Aix-Marseille University School of Public Managment and Territorial Governance and Co-Leader of OTACC Chair; Djelloul Arezki, Senior Lecturer at IMPGT and Co-Leader ot OTAC Chair, Marseille
From crisis in systems to crisis of systems!
How can a new culture of research and research policy contribute to a post-covid world?
Since 2020 we witness not only multiple climatic, social, economic, and democratic crises, but a new interconnectedness of crises: We believe these individual crises are indicators of a more general systemic crisis. This piece debates and presents what an adaptation of the academic world in this system of crisis could look like, and what academic institutional innovation can contribute to overcome it. We believe organisational innovations are a prerequisite to re-build better and empower the makers of a possible Next Renaissance.
Picture above: Copyright: OTACC Chair
Interwined crises : telling points of a societal model system failure
„The climate, energy (and more generally natural resources), biodiversity, agricultural and food, demographic, social, financial, economic, democratic and governance crises… all indicate that our development model systemic crisis is now showing its limits… Could the Covid-19 crisis be the first one of a series of interconnected crises? (College of Societal Transitions1).
Because of their characteristics, the created problems and the generated effects, those intertwined crises deeply weaken our contemporary societies and are causes of worrying imbalances: social, demographic, economic, environmental, increasing inequalities, etc. These crises fundamentally question values and strategic priorities on which our societies’ development models are based. They also seriously challenge our capacity to lastingly
integrate populations with various backgrounds (social, ethnic, linguistic, cultural, religious, etc.).
Faced with the urgency of this situation, can stakeholders involved in public policies (researchers, public structures and organisations, experts and professionals) still work in nonintegrated way and vision? Are compartmentalised approaches still relevant and efficient?
Today these questions become even more acute because over the last few decades the normative, individualistic, and competitive approach, linked to the domination of a neoliberal capitalist development model, has largely delayed and slowed the emergence of collective utopian and pragmatic proposals and solutions.
To design a complex, holistic way of thinking, the challenge of a Renaissance is more urgent than it has ever been. “Building a live-together society” necessarily implies the emergence of an inclusive societal transition project.
Integrated solutions: How can a university achieve a collective utopia?
Society is in need of an inclusive ecosystem in the Cultural and Creative Sectors and Industries (CCSI) in order to promote societal transition. The Organisations and Territories of the Arts, Culture and Creation (OTACC) Chair, led by the Public Management and Territorial Governance School, is a Training and Research Unit of Aix-Marseille University and could be viewed like a sandbox for the academic world proposing novel organisational answers to this wicked systemic crisis. To do so the OTACC Chair is designed with the following mission and especially tailored components and features:A Creative Industries sector in order to promote societal transitions.
The OTACC Chair mission is to operate as a laboratory for thinking, experimenting, and disseminating innovative management systems for artistic, cultural and creative projects.
OTACC: A participatory science programme to foster inclusive societal transition projects
« O » Organisations and inclusion: OTACC Chair supports CCI organisations to address the challenges of the 21st century: digitisation, new users’ practices, ecological transition, citizen participation. An inclusive environment with a wide variety of stakeholders can help to ensure the success of innovative artistic, cultural, and creative projects. As such, CCI organisations perform as a driving force for innovation, becoming a model for other sectors which can lead to cross-sectoral collaborations.« T » Territories and inclusion: CCI projects are deeply connected with territorial actors and resources contributing to a sustainable and local development and foster collaborative work practices. CCI projects are a strong part of territorial attractiveness. Its connections in all social, societal and economic dimensions can assume a real impact from those CCI projects. They act as civic laboratories by experimenting new democratic governance practices, encouraging citizen involvement and creating collective innovative dynamics.
« A » Arts and inclusion: Based on various disciplines and aesthetics, CCI projects reflect major social and societal transformations and are fully developed and designed with a transdisciplinary approach. Artistic works address citizenship issues and encourage expression of a broad range identities. By putting also into question standard thinking habits, artists extend societal and technological limits.
« C » Culture and inclusion: Because the Chair conceives Culture as a whole concept in its civilisational sense, its analysis and studies move beyond a basic approach of “cultural production”. It deals with all the parts of CCI value chain: from the creative act to its distribution. And all forms of cultural expression are considered, from the so-called mainstream to so-called subcultures. A plural and inclusive approach to culture is strongly advocated.
« C » Creation and inclusion: OTACC Chair supports cultural, creative, social, and solidarity-based entrepreneurship. OTACC Chair guides projects emergence and development through participatory research. Academics, professionals and students are all gathered in order to promote shared solutions and vision to the main issues of the CCI sector.
OTACC: 3 axes to foster and support inclusive societal transition projects
Chair scientific project is focusing on 3 axes: Training, Research and Development, and Events organisation.
AXIS 1. TRAINING
6 services/ actions
– MDOMC Master degree leading: “Direction of Cultural Projects or Establishments” (DPEC) Master degree and “Management and Law of Cultural Organisations and Events” (MDOMC) Master degree both provide to OTACC Chair a strong pedagogical expertise. Founded in 1998 and awarded by Eduniversal, MDOMC Master runs work-linked training courses. Junior executives, enrolled as apprentices, offer their expertise to all OTACC Chair members. This master’s degree, the oldest of IMPGT, has a large Alumni network. They are working in all types of CCI companies (local, national, and international).
– Incubator function: OTACC Chair operates as a cultural and creative, social, and solidarity-based entrepreneurial incubator. OTACC Chair Master students, throughout two years training, implement their elaborated inclusive CCI project.
– Serious game: as a last course, the serious game is conceived to put students in real professional situations. They have to submit a project proposal in response to a cultural operator order.
– Short training courses: from 1 to 5 days and open to all types of public, those training courses focus on cultural and creative industries management.
– Off-site training courses: customised thematic training courses are designed on request to fulfil the CCI sector professional operators requirements.
– Career development: skills assessment and professional support for CCI managers.
AXIS 2. RESEARCH AND DEVELOPMENT
4 services/ actions
– Worldwide watch of best and innovative inclusive practices in CCI
– Resources:Free Access to all OTACC Chair members studies and research
– „Experimental labs “ Evaluation: Evaluation and audit by OTACC Chair experts and students
– Research projects Leading/Participation/Coordination: proposals in respond to collaborative research projects calls (National, European and International fundings).
AXIS 3. FORUM PLACE
By offering a discussion, sharing and meeting forum, OTACC Chair gives an opportunity to Members to increase and improve visibility and audience. The Chair network allows members to know each other and gives possibilities of exchanging projects and experiences. 3 types of events including a major one are yearly scheduled
– „Journées Organisations et Territoires des Arts de la Culture et de la Création“ (OTACC Days): professional event supported by OTACC Chair and co-organised in partnership with Master’s students, Chair members and partner organisations.
– Throughout the year, physical and/or online thematic micro-events are planned
– Chair experts attending events organised by Chair members/partners.
OTACC Chair project : Anti-discrimination strategies in Evaluation
Every year, the Chair pilots two studies on inclusion topic: a „macro“ study analysing ICCs sector inclusion practices and a „micro“ action-research focusing on innovative inclusion systems.
The 2021 „micro“ study focuses on the anti-discrimination strategies evaluation of ICCs organisations current music sector. The Chair students were dispatched into evaluation teams supervised by the Chair experts. The teams have collaborated with contemporary music sector key operators producing „experimental laboratories“.
Picture right: Copyright: OTACC Chair
OTACC – a role model for a pluralistic fundamentally integrated organisation to teach innovative management to tackle crisis holistically.
Teaching management in a systemic crisis sounds on first sight like squaring the circle. We believe it is time that the academic world makes its own proposals to deal with uncertainty—for students in their future professional life or for courses and curricula of the academic world itself.
OTACC Chair, as a pluralistic fundamentally organisation and participatory science approach, brings together academic researchers, students, CCI sector professionals (public and private organisations representatives), artists, artists collectives, related sectors professionals (public and private organisations representatives) and offer them an non-linear learning environment with built-in cross-disciplinary surprises as well as the integrated skills for management in a world of multiple and continuing crisis.
References/Links
1 Collège des transitions sociétales: https://web.imt-atlantique.fr/x-de/cts-pdl/
OTACC presentation
Edina Soldo
Edina Soldo is Full Professor at IMPGT (Public Management and Territorial Governance School)-Aix-Marseille University (AMU) and AMU Special Advisor on CCI. Djelloul Arezki is Senior Lecturer. They are both researchers at the CERGAM, AMU research center and head “Management and Law of Cultural Organisations and Events” Master’s degree at IMPGT.They have founded with three major universities and the Provence Alpes Côte d’Azur Region a regional hub: MIN4CI, Mediterranean Innovative Narratives Competence Center for Cultural and Creative Industries.
Edina´s research topics focus on innovation in CCIs. She analyses inclusive approaches in co-developed CCI territorial projects by using Public management and strategic territorial management concepts and theories.
Picture © Gapore Tan
Djelloul Arezki
Djelloul Arezki´s research topics focus on inclusive practices in co-developed CCI organisational projects. He analyses inclusion through strategic human resource management and queer theory. They co-lead the OTACC Chair.
Picture © Gapore Tan
