Contributions to 3d data processing and social robotics

Resumen

1. Introduction The present thesis discusses three main topics. First, 3D object recognition over point clouds is discussed. Then, mapping of the environment is covered. Finally, other topics related to social robotics are included. Tridimensional object recognition methods are focused on classifying objects represented in 3 coordinates, just as in the real world. These methods differ from the 2D counterpart in that they focus on topological aspects of the data rather than their visual aspect. Although 2D recognition has a high hit rate and is perfectly applicable in many situations, there are situations where it does not work properly. For example, it has great difficulty in differentiating objects that look very similar in appearance, such as differentiating a photograph from a real object. That is why 3D object recognition can complement and overcome these circumstances. On the one hand, traditional methods that perform this task are very dependent on the conditions of the dataset, as they are largely handicapped by occlusions and noise, and sometimes lack generalisability. On the other hand, there are several methods that rely on deep learning to perform the classification. Despite their good results, they still suffer in the area of explainability, i.e. how easily their decisions can be understood by a human being. We will explore new methods to carry out this classification that can be more organic and understandable. Regarding to environment mapping, these systems address the problem of acquiring spatial models of physical environments that can be used by mobile robots. As the aim of this work is to be used by a social robot, we will focus only on indoor mapping techniques. There are good and robust methods that assume that the environment is static, structured and limited in size. However, working with large-scale, unstructured, dynamic maps of the environment is still a problem far from being solved. Historically, research has been divided between metric and topological maps. Metric maps store the geometric properties of the environment, while topological maps describe the connections between different places in the environment. In this thesis, we will try to combine the benefits of these two types of maps by mapping the environment to include semantic information about the objects in the scene. In this way, a robot will be able to navigate the environment and interact properly with those objects it needs to perform its programmed task. In regard to social robotics, in this thesis we include a comprehensive review of the state of the art of social robots, especially focused on interaction with elderly people and people with autism. According to the World Health Organization, people with disabilities are particularly vulnerable to deficiencies in services, such as health care, rehabilitation, support, and assistance. In this sense, recent technological developments can mitigate these deficiencies, offering less-expensive assistive systems to meet users’ needs. On the other hand, in this work, we propose an augmented reality system to help in the adherence and supervision of rehabilitation sessions at home, which can be carried out with low-cost sensors and can therefore be accessible to a large number of patients. This system stores user statistics and allows therapists to adjust exercises according to the results obtained. Finally, in this thesis, we present a visual recognition module that allows robots to identify the people they are interacting with in order to offer them a personalised treatment. The main idea of this work is that the robot is able to learn new identities in real time with a small interaction with the user, and from that moment to be able to recognise the user unequivocally on other occasions. 2. Theoretical Research In this thesis, different alternative methods for the realisation of 3D object recognition have been proposed. In the case of NurbsNet, similarity between free forms, nurbs in this case, has been introduced for the first time to determine the global classification of a cloud of points representing an object. To do this, research had to be done on a distance metric that would allow two surfaces to be properly compared in terms of shape similarity. Despite not surpassing the state of the art, this work opens up new, unpublished avenues of research for more organic object recognition that can be more easily understood by humans. In the case of VFD, the mathematical concept of the fractal dimension has been used for the first time for the recognition of three-dimensional objects. The results of this method are promising and invite further research on fractals applied to object recognition and other fields of 3D data processing. It is worth noting that the methods used make use of the point cloud as input, which is a plus since this is the representation chosen by the vast majority of three-dimensional sensors. Regarding the mapping of the environment, we have proposed the development of an automatic 3D environment mapping system in which not only the points are included directly in their corresponding location, but also additional processing has been carried out to give meaning to these points. In our research we have provided these maps of the environment with the recognition and segmentation of the objects present in the scene, as well as the detection of potentially dangerous areas for people. Likewise, we have proposed a planning system that allows to calculate routes between rooms using a semantic map of the environment, and returns the most optimal route between rooms taking into account connectivity criteria (corridors and open doors) and cost (distance, time or other circumstances). This system allows the navigation of a social robot between rooms. In addition, we have proposed a method for merging three-dimensional data from various sources, which allows us to take advantage of the benefits of each source and overcome their disadvantages. Regarding robotics, we have conducted a study on the state of social robotics concerning the care of the elderly, active aging and therapies with autistic people. In this way, we have been able to know where we are and propose solutions to existing problems. Based on this previous research, we have proposed an augmented reality system that allows to help in physical rehabilitation tasks within a home, and to be evaluated quantitatively with our metrics, so that the therapist can know the evolution of the therapy. Finally, we have presented a human recognition module capable of learning new identities in real time, which would allow robots to treat their interlocutors in a personalized way. 3. Conclusions The common nexus of this thesis is the research in the field of social robotics. To this end, great efforts have been devoted to the tasks of 3D object recognition and environment mapping, as these are two fields of vital importance for the interaction of the robot with a domestic environment. Additional research has also been carried out to provide the robot with the ability to recognise people, and a rehabilitation support system has been developed that can be incorporated into the robot and that can help with adherence to treatment. During this thesis, 7 works have been published in high impact journals, rated by the Journal Citation Reports (JCR). In addition, 3 contributions have been presented at international conferences.