CIMON-2 Makes Its Successful Debut on the ISS

Experimental technology for human-machine interaction in the space industry successfully masters the next stage of development in space

Scientific investigation into the effects of stress and isolation during long-term missions

Friedrichshafen, 15 April 2020 – CIMON-2, the updated version of the CIMON astronaut assistant, developed and built by Airbus for the German Aerospace Center Space Administration (DLR), has now demonstrated its capabilities during initial tests on the International Space Station (ISS). The free-flying, spherical technology demonstrator with artificial intelligence (AI) showed off a number of its features during interactions with ESA astronaut Luca Parmitano. CIMON-2 started its journey to the ISS on 05 December 2019, launching with the CRS-19 supply mission from the Kennedy Space Center in Cape Canaveral, Florida. It is scheduled to stay on the ISS for up to three years. Just shy of two months after the successful first use of CIMON-2, the project team has now received the analysis.

A number of tests have now been carried out on CIMON-2, for example on its autonomous flight capabilities, voice-controlled navigation, and its ability to understand and complete various tasks. It also managed to fly to a specific point in the ISS Columbus module for the first time. Thanks to absolute navigation capabilities, CIMON-2 was able to follow verbal commands to move to a particular location, regardless of where it was to begin with. For example, while starting up its new hardware and software, ESA´s Astronaut Luca Parmitano asked CIMON-2 to fly to the Biological Experiment Laboratory (Biolab) inside the Columbus module.

It was also given the task of taking photos and videos in the European ISS module on request – and then showing these to the astronaut. Using these capabilities, CIMON-2 will be able to help with future scientific experiments on the ISS.

The microphones of the current version of the technology demonstrator are more sensitive than its predecessor’s (CIMON), and it has a more advanced sense of direction. Its AI capabilities and the stability of its complex software applications have also been significantly improved. The degree of autonomy of the battery-powered assistant has been increased by around 30%. Astronauts can also activate a feature on CIMON-2 that allows it to analyse emotion in language and show empathy when interacting with the astronauts.

In addition, the project aims to research whether intelligent assistants such as CIMON could help reduce stress. As a partner and assistant, CIMON could support astronauts with their high workload of experiments and maintenance and repair work, thereby reducing their exposure to stress. CIMON lays the foundations for social assistance systems that could reduce stress resulting from isolation or group dynamics during long-term missions. Such systems could also possibly help to minimise similar problems back on Earth as well.

With the new improved hardware and complex software working so well, the CIMON team from DLR, Airbus, IBM, Ludwig Maximilian University in Munich (LMU) and the ESA User Support Centre Biotesc in Lucerne (Switzerland) are extremely satisfied with CIMON-2’s performance. This continued success of the CIMON project is yet another pioneering achievement in the use of AI in human space flight.

The CIMON ‘family’
The interactive astronaut assistant CIMON was developed and built by Airbus in Friedrichshafen and Bremen on behalf of the German Aerospace Center Space Administration (Deutsches Zentrum für Luft- und Raumfahrt – DLR) and funded by the German Federal Ministry for Economic Affairs and Energy. Watson AI technology from IBM Cloud provides voice-controlled artificial intelligence. Scientists from the Ludwig-Maximilian University Hospital in Munich (LMU) helped develop and oversee the human aspects of the assistance system. Biotesc at the University of Lucerne ensures that CIMON works perfectly in the Columbus module of the ISS and supports interaction of astronauts with CIMON from ground. Starting in August 2016, an approximately 50-strong project team from DLR, Airbus, IBM and LMU worked on the implementation of CIMON-1 for around two years. The prototype of the technology experiment flew on board the ISS from 02 July 2018 to 27 August 2019, and made its 90-minute debut – a world first – on 15 November 2018 with German ESA astronaut Alexander Gerst. It is no coincidence that CIMON’s name is reminiscent of ‘Professor Simon Wright’, the robotic assistant – or the ‘flying brain’ – from the Japanese science fiction series ‘Captain Future’. Following the successful CIMON-1 mission, the first European autonomous robot in human space flight was declared a German cultural asset and returned to Earth. The work on CIMON-2 was completed in less than a year by 20 employees from the CIMON ‘family’.

CIMON – the idea
Developed and built in Germany, CIMON is a technology experiment to support astronauts and increase the efficiency of their work. CIMON is able to show and explain information and instructions for scientific experiments and repairs. The voice-controlled access to documents and media is an advantage, as the astronauts can keep both hands free. It can also be used as a mobile camera to save astronaut crew time. In particular, CIMON could be used to perform routine tasks, such as documenting experiments, searching for objects and taking inventory. CIMON can also see, hear, understand and speak. CIMON can orientate itself using its ‘eyes’ – a stereo camera and a high-resolution camera that it uses for facial recognition – as well as two other cameras fitted to its sides that it uses for photos and video documentation. Ultrasound sensors measure distances to prevent potential collisions. Its ‘ears’ consist of eight microphones to identify directions, and an additional directional microphone to improve voice recognition. Its ‘mouth’ is a loudspeaker that it can use to speak or play music. At the heart of the AI for language understanding is IBM Watson AI technology from IBM Cloud. CIMON has not been equipped with self-learning capabilities and requires active human instruction. The AI used for autonomous navigation was provided by Airbus and is designed for movement planning and object recognition. Twelve internal rotors allow CIMON to move and rotate freely in all directions. This means it can turn towards the astronaut when addressed, nod and shake its head, and follow the astronaut – either autonomously or on command.

Original Source: Airbus.com

CIMON: The First AI Robot to Operate in Outer Space

Andrea Electronics, the pioneer of digital array microphones and noise reduction software that optimizes the performance of voice driven applications, has announced that its DA-250 array microphone is incorporated in to the CIMON virtual assistant robot that will operate as part of the “Horizons” project on the International Space Station.

This will be the first form of artificial intelligence (AI) on an ISS mission. CIMON is an experiment overseen by Space Administration at the German Aerospace Center (DLR) in cooperation with Airbus as the prime contractor as well as IBM, Reichert Design and LMU Munich.

CIMON (Crew Interactive Mobile CompanioN) is a mobile and autonomous assistance system designed to aid ESA astronaut Alexander Gerst with his everyday tasks on the International Space Station (ISS). Together they will experiment with crystals, solve the Rubik magic cube based on videos and conduct a complex medical experiment using CIMON as an ‘intelligent’ flying camera.

CIMON’s digital face, voice and use of artificial intelligence make it a “colleague” to the crew members. This collegial “working relationship” facilitates how astronauts work through their prescribed checklists of experiments, now entering into a genuine dialogue with their interactive assistant. The developers responsible for CIMON predict that this will help reduce astronauts’ stress and at the same time improve efficiency. In addition, CIMON helps enhance safety, because it can also serve as an early warning system in case of technical problems in the future.

Watson speech and vision technologies helped train CIMON to recognize Alexander Gerst, using voice samples and Gerst, as well as “non-Gerst” images. It also used the Watson Visual Recognition service to learn the construction plans of the Columbus module on the International Space Station to be able to easily move around. CIMON also learned all the procedures to help carrying out the on-board experiments. Experiments sometimes consist of more than 100 different steps, CIMON knows them all.

Learn More About CIMON

The Floating Robot With an IBM Brain Is Headed to Space
“The next shipment headed to the International Space Station packs nearly three tons of research and resupply materials. You know, the typical stuff: sediment studies, a plant thermometer, a replacement hand for the giant robotic Canadarm. Oh, and also a floating robot designed as a helpmeet for astronauts—scientifically, logistically, and emotionally….”