The problem of the population aging and the increase of the number of patients with Alzheimer's disease that people will face in the next 20-30 years, makes us think about its solution already now.

Since January 2017, Endurance, along with four US universities, has begun work on a project to create a robot-companion (chatbot) for senior people and patients with Alzheimer's disease.

The idea and problem

The main idea is to create a universal robot-companion for senior people.

The problem we are trying to solve is to give senior people and patients with Alzheimer's disease the opportunity to communicate: talk, share their experiences, memories.

At the moment, the mechanical part of the solution has been completed. This is a holder for a phone or tablet, which will rotate to follow the speaker. The holder is called SelfieBot and is fully 3D printed. The electronic components, program code, firmware are completely open source and are presented on the site EnduranceRobots.com.

We specially developed a model for 3D printing, so that anybody could create their own robot-companion.

What is needed to do?

Next, we face the challenge of creating an intelligent companion.

It is assumed that the robot will be able to conduct a conversation on more than 20 topics, such as: weather, nature, news, history, cinema, music, etc.

The chatbot faces the task not only to ask and answer questions but also to memorize the context of the conversation. This is crucial for conducting a cohesive dialogue.

No less important task is to recognize the tone of the user when he/she communicates with the robot. The robot should be able to start and conduct a dialogue at ease. At the moment, more than 10 thousand question and answer templates have been created. Our robot not only asks questions, it is also able to adequately respond to answers.

An interlocutor can give various answers to the question “How are you?” We consider three main scenarios:  

1. Good.

2. Normal.

3. Bad.

In each of the scenarios the robot's task is to ask the interlocutor to tell about this in more detail.

Here begins the most complicated part. It is almost impossible to determine now how the dialogue will develop further. An elderly person can tell what a wonderful dream he has dreamed, how well he feels, or, on the contrary, can start complaining.

For such cases, the robot must be provided with words of support that will allow a person to speak out.

Dr. Wallace took this into account when created his robot-interlocutor Alice in the 60's. Now Endurance has its own solutions and those of its partners, with the API of which we can determine the tone and context of the dialogue, find synonyms and much more.

It is also important to keep in mind that senior people often have speech and hearing problems. We are going to use ready-made API for speech recognition and synthesis. At the moment, we use solutions from Microsoft and Google, which we consider to be the best among the available technologies.

The main difficulty now is to understand how to teach the robot to shunt the conversation on another topic if needed. So far, we have not specified a sufficient number of criteria for this.

So, what is so unique about the Endurance chatbot?

1. Memory.  A chatbot needs to remember everything what the interlocutor tells it so that the conversation could be meaningful. It is important to remember not only the name of the interlocutor, but also his/her habits, likes and dislikes. The robot can ask the senior person about his/her plans for tomorrow, and ask tomorrow about the plans for today to compare the answer with the answer of the previous day. The therapeutic benefit of the robot will be manifested in revealing the adequacy of the interlocutor’s answers and whether he/she remembers what he/she said yesterday. It is very important for the dialogue to be meaningful and running.

2. Reactions. It is also essential that the chatbot should adequately respond to the intonation of the interlocutor and be able to distinguish between positive and negative connotations of the context. It is known that many senior people suffer from depression, so the chatbot will also face the task to improve the mood of the interlocutor and to switch complicated and difficult topics of a conversation, such as: illness or death to a positive course, so that the interlocutor could enjoy the communication.

3. Reminder. It is crucial to make a model for the chatbot, using which it will remind the interlocutor about the things the person can forget, such as: to take medicine or call relatives and friends.

4. Search for relevant and interesting information. As part of the intellectual development of the chatbot, we plan to make it more of a learner. That is to provide it with an automatic function to independently search for answers in Google if it does not know something. This will allow the chatbot to learn continuously. The more it communicates, the more knowledge it will have about the world around it.

5. Help and evaluation of the interlocutor’s health state. It is necessary to teach the chatbot to recognize the interlocutor’s state, so that in case of the person's bad health the chatbot could inform about it his/her relatives or a doctor.
   In general, the set of chatbot’s functions can be quite diverse. In the process of the chatbot developing and testing it will become clear what other tasks require a solution.

The chatbot key function will be its ability to store a communication history, making it available for viewing by the family/friends and a doctor if monitoring is necessary.

We hope that the project will become popular and help any interested person make his/her physical robot-companion, the creation of which will require a SelfieBot rotation mechanism, a phone/tablet and the Internet access.

In the long term, we plan to equip the chatbot with the image recognition function so that it could recognize the interlocutor, read his/her facial expressions and could give its interlocutor simple tasks such as: to draw something or tell.

This function will undoubtedly vastly improve the interactivity of the communication.

Microsoft Cognitive provides an API for emotions evaluation by a facial expression, which can become the first step to implementing this solution.

This project is non-commercial, and all the dialogues corps are open source.

We invite researchers and research centers to cooperate.

The design development of the SelfieBot phone/tablet holder is also open source and posted on EnduranceRobots.com

We believe that this technology will allow senior people and Alzheimer's disease patients to fully experience the joy of communication.

If you are interested in cooperation write to:  gf@EnduranceRobots.com

The content & opinions in this article are the author’s and do not necessarily represent the views of RoboticsTomorrow

Featured Product

Elmo Motion Control - The Platinum Line, a new era in servo control

Significantly enhanced servo performance, higher EtherCAT networking precision, richer servo operation capabilities, more feedback options, and certified smart Functional Safety. Elmo's industry-leading Platinum line of servo drives provides faster and more enhanced servo performance with wider bandwidth, higher resolutions, and advanced control for better results. Platinum drives offer precise EtherCAT networking, faster cycling, high synchronization, negligible jitters, and near-zero latency. They are fully synchronized to the servo loops and feature-rich feedback support, up to three feedbacks simultaneously (with two absolute encoders working simultaneously). The Platinum Line includes one of the world's smallest Functional Safety, and FSoE-certified servo drives with unique SIL capabilities.

More Products

Feature Your Product