Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Automation versus machine autonomy

Institute of Applied Physics “Nello Carrara” (IFAC), National Research Council of Italy, Florence, Italy ↗
Publication year
Digital Object Identifier10.54941/ahfe1004517

Abstract

When planning support to people in their living environments using technology based on AI, the first approach is to look for the possibility of automation of functionalities favoring independent living. For example, it is possible to install sensors to automatically switch on or off the lights, to turn on the television when a program of interest starts, to remind people the drugs to be taken at the right time. This is supposed to support people in maintaining their autonomy, i.e., their ability to live in their environment without unwanted external interference. When an Artificial Intelligence system detects abnormal behavior of the person, for example through physiological and/or environmental parameters, may be technologically capable to act: for example it can send an alert to the person, via a sound or a message on an electronic device, can proceed to warn relatives or caregivers, sending an appropriate signal through the TLC network, can also administer medicines, or make equally impactful decisions, via home robots or equivalent devices integrated into the home network.The main problem is that not all people, even if supported by automation, are able to live autonomously, due to physical or mental limitations. Therefore, it is necessary to consider the possibility that the supporting systems become partially autonomous, i.e., able to reach a predefined goal according to the current situation without recourse to human control. Such systems are supposed to perceive their environment via sensors, proactively create a plan of action according to the situation and related constraints and execute the planned actions safely and reliably via actuators.Different possibilities of autonomy are possible. The support system offers no assistance - the human must make all decisions and actions, offers a complete set of decision/action alternatives; narrows the selection down to a few alternatives; suggests one alternative. Then it may execute that suggestion if the human operator approves, or allows the human a restricted time to veto before automatic execution, or executes automatically, and necessarily informs the human, or informs the human only if asked etc. This implies several ethical problems, as shown in the following examples. It is necessary to decide who can check that the state of the person is such as to require more advanced autonomous decision-making systems based on AI? Which people and with what priority should those who have a relationship with the person in the house be allowed to contribute? Who becomes responsible for the malfunction of the equipment, in the event of a breakdown, which may cause personal injury? What is the level of decision that can be entrusted to the machine compared to that which is delegated to a caregiver? The answers are related to the person's capacity for autonomy, the responsibility that those who care for this person have, but also e.g., the person's will, the readiness to intervene, the type of warning. The problem will be discussed with reference to three applications implemented at IFAC: cooking in an intelligent kitchen (activity level), support for the solitude of people at home (social support); pedestrian mobility (activities in public areas).

INTRODUCTION

Technology is able to implement a vast series of actions normally performed by humans, making available devices capable of conducting pre-defined operations, with savings in fatigue and/or time. They normally implement procedures, which are uniform, regular, and cyclic. The process introduced by the adoption of such machines is an automation process, in which a machine, device or system takes the place of a person to perform a limited number of actions, which otherwise require a considerable expenditure of physical and mental energy.

With ‘Artificial Intelligence’ (AI) technologies, the level of complexity of the interventions can undergo significant increases, causing the need of del- egating to the system decision-making capabilities, which presently are the responsibility of the supervision of people. Thus, the concept of automation migrates towards that of the autonomy of the machine or in any case of the service.

This causes the emergence of important ethical issues (e.g., related to privacy and liability, and compatibility with current legislations) and is par- ticularly sensitive when the interventions are on human beings and their living environments.

AI AND SUPPORT SYSTEMS It is evident to all that, due to the use of ICT and information technology, the entire society is undergoing a transformation into an environment where people are surrounded by interconnected intelligent objects (Antona et al.,

© 2024. Published by AHFE Open Access. All rights reserved. 524

Automation Versus Machine Autonomy 525

2007) (Burzagli and Emiliani, 2014) (Burzagli et al., 2021). In the field of sup- port for people, especially elderly who are losing their autonomy or people with ability limitations (disabled people), Artificial Intelligence could repre- sent a core element in the decision making process of a support ecosystem, which, even without the need for specific Assistive Technology, can favor the emergence of complex applications for the well-being of people.

Among the person’s daily activities, self-care is one of the main ones. When the environment detects abnormal behavior, for example through physio- logical and/or environmental parameters, the support system, in addition to detecting the state of the person, is supposed to be able of progressively developing solutions about possible support interventions. For example, it can send an alert to the person, through a sound or a message in electronic format on the device most used by her, such as a tablet, a smart TV, a smart phone, an installed alarm system, or a virtual assistant, such as SIRI, Alexa, Google Assistant, and Cortana. It can warn relatives or caregivers, sending a special signal through the TLC network. It could also provide for the admin- istration of medicine, for example, through domestic robots or equivalent devices integrated into the home network.

Until now, these decisions have been delegated to the user herself, if she is able to do so, or to the caregiver. She can sense the situation, plan an intervention, and act to implement it.

Figure 1: Block diagram of an AI support system.
Original PDF Figure 1:

In principle an artificial intelligence system, as the one sketched in Figure 1 is technically able to replace the person or the caregiver in conducting these support operations. Now let’s see where and why it is necessary to make use of artificial intelligence, with reference to three prototypes developed in our IFAC laboratory:

  • cooking in an intelligent kitchen (activity level) (Burzagli et al., 2014);
  • support for the solitude of people at home (social support in geographi- cally limited contexts close to the user himself - condominiums, cohousing) (Burzagli and Naldini, 2018) (Burzagli and Nardini, 2020);

526 Burzagli

  • pedestrian mobility (activities in public areas according to paths that adapt to physical needs, sensory and cognitive aspects of the person, but also their preferences, to achieve a higher level of well-being) (Burzagli and Emiliani, 2021).

The cooking application has been implemented without the explicit use of artificial intelligence components (only equipment in the kitchen was con- sidered intelligent). Then, it became apparent that some functions would be easier with AI components: for example, the choice of meals for peo- ple with leukaemia or the planning of balanced diets as a function or age or present diseases. In the application for the support to solitude, AI was used to construct the profile of the user; to monitor her mood (voice analy- sis), to decide the type of support and to choose the person to be contacted, according to her availability and profile of interests, when the support by a person is considered the best solution. Finally, in the mobility appli- cation AI was used to construct the profile of the user; to monitor her mood (voice analysis); to collect and organise information about the route (from the perspective of a pedestrian); to suggest itineraries and to dis- cuss with the user about different possible itineraries (Burzagli and Naldini, 2023). This approach is widely used in the industrial and social spheres, where, however, it acquires a completely different value. At the industrial level, in fact, the scope of intervention often remains within the interaction between devices. On the contrary, when technologies are applied directly as a support to people, particularly people who are old and in a state of incomplete auton- omy, for example, in the home, the problem to solve takes on even greater importance. It concerns the person’s capacity for autonomy, the responsibility that those who care for her have, but also the person’s will, the readiness to intervene, the type of warning, and other parameters to be defined. And last, but not least, it must be considered that the conditions of the human being are often destined to change over time, in both perspectives both presenting improvements, but unfortunately also worsening. The technological system can follow these variations, but how can it do so while respecting dignity and rights of the person?

AUTOMATION VERSUS AUTONOMY When planning support to people in their living environments using tech- nology, the first approach is to look for the possibility of automation of functionalities favoring independent living. This is supposed to support people in maintaining their autonomy, i.e., their ability to live in their environment without unwanted external interference.

The main problem is that not all people, even if supported by automation, are able to live autonomously, due to physical or mental limitations. There- fore, it is necessary to consider the possibility that the supporting systems become partially autonomous, i.e., able to reach a predefined goal according to the current situation, to the point of deciding without recourse to human control.

Automation Versus Machine Autonomy 527

Automation As mentioned in the introduction, an automatic system is a system capable of performing a task previously carried out by a human being, using an existing technology or combination of technologies without being directly controlled. It is assumed here that such a system cannot interact in a way that is danger- ous to the person. An example can be a system that automatically turns on the lights, although the user can turn them off.

Autonomy When an AI system detects abnormal behavior of the person, for example through physiological and/or environmental parameters, it is supposed to be capable of planning an action and act. Such systems are supposed to perceive their environment via sensors, proactively create a plan of action according to the situation and related constraints and execute the planned actions safely and reliably via actuators.

In order to proceed with the analysis of applications in the light of the concepts of automation and autonomy, it is first necessary to focus on the concept of autonomy. For this reason, a series of definitions of autonomy reported in literature are summarized below (Müller et al., 2021), each of which has different nuances:

  • Autonomy is the ability to perform given tasks based on the system’s perception without human intervention.
  • Autonomy is an entity’s ability to structure its own actions and its environ- ment without unwanted external interference, i.e., it decides completely self-determined.
  • Autonomy is the ability of an autonomous system to make decisions and justify its actions based on its sensor measurements. The ultimate goal is to adapt to changes, which occur within the system itself, other systems it interacts with, its operation environment, or in the given task.

Therefore, a system is autonomous if it is able to reach a predefined goal, making independent decisions and adapting to new conditions.

In literature a deep analysis (FakhrHosseini et al., 2019) is available for the relationship between levels of autonomy and stages of information process- ing, dividing the latter into Sense (what are the parameters that is necessary to observe to decide about the possible problems and useful solutions), Plan (how a plan to support the person can be formulated), Act (how the plan can be implemented). This indicates how autonomy can take place at the level of observation, of identification of the problem, of decision-making and of the implementation of that decision. Different levels of intervention can be considered:

  • No autonomy (including actions made by a person and tele-operations)
  • Low autonomy (including assisted tele-operations, batch processing, deci- sion support)
  • High autonomy (including shared control with human initiative, shared control with robot initiative)
  • Full autonomy (including executive control, supervisory control).

528 Burzagli

It is evident that different possibilities of autonomy are possible. The sup- port system offers no assistance - the human must make all decisions and actions; offers a complete set of decision/action alternatives; narrows the selection down to a few alternatives; suggests one alternative. Then it may execute that suggestion if the human operator approves, or allows the human a restricted time to veto before automatic execution, or executes automat- ically, and necessarily informs the human, or informs the human only if asked.

This implies several ethical and legal problems as shown in the following examples. Who can check that the state of the person is such as to require more advanced autonomous decision-making systems based on AI? Which people and with what priority should those who have a relationship with the person in the house be allowed to contribute? Who becomes responsible for the malfunction of the equipment, in the event of a breakdown, which may cause personal injury? What is the level of decision that can be entrusted to the machine compared to those normally delegated to a caregiver? The answers are related to the person’s capacity for autonomy, the responsibility that those who care for this person have, but also e.g., the person’s will, the readiness to intervene, the type of warning.

Legal and Ethical Perspectives There are now possibilities of implementation of fully autonomous systems capable of supporting people. However, for their adoption and use it is not enough that they are able to meet the needs of people, but it is necessary to try to reconcile technological advancement with the legal system. More- over, it would be convenient to detect problems relating to the application of decision-making elements already at the design stage of the system, rather than to make use of a posteriori evaluation that can also invalidate the product.

This new possibility causes a situation that requires a new way of thinking about the support of people. In principle, due the complexity of the emerg- ing environments all people, not just some fragile categories, may need a technology help throughout the entire life cycle, changing in level depending on variations of the context and the situation of each person, such as the degradation of her abilities.

For example, aging is a concept that has been completely redefined in recent years, highlighting situations not related to particular and serious pathologies that lead to a complete lack of autonomy. In the society, there is an increasing number of elderly people, who are less and less autonomous. In an ideal situation, they should be helped. In the European Nordic countries, there are villages for elderly people. They are torn from their environment and cared for in communal residences. Due to the cost of the solution, in the rest of Europe the main attitude is to keep them at home, but in many cases a caregiver is necessary, who also costs too much. Therefore, the use of support environments based on intelligent machines may be the recommended option. (see European “Ageing Well in the Digital World” Programme) (https://www.aal-europe.eu/).

Automation Versus Machine Autonomy 529

From the point of view of law, the line is that no application or system can replace the decision of the person, except in emergencies. Actually, there are many figures who at successive levels are allowed to decide for the person: the guardian, the curator (diminished capacity) and the support administrator. Moreover, it is also necessary to consider people who make up the person’s network of relationships, from family members to the circle of friends and neighbors. These people can make some decisions and the legal system is modulated on any single role. An additional problem is the detection of the performance of dangerous actions by the person and of her status. Verifi- cation of the person’s status is possible by doctors, social workers, or other caregivers. Nevertheless, these professionals cannot always be present with people while a machine (application or system) can be. However, even in this situation, it is necessary to get people to express themselves. If the per- son concerned directly has not sufficient abilities, the persons responsible for their care must be involved. The problem arises to detect to what extent the person can decide, in other words what is his level of autonomy, both physical and cognitive.

General solutions do not yet exist. However, to understand the complexity of the problems when designing an advanced home automation system, it is interesting to go through a non-exhaustive list of questions that appear in the literature:

  • Who decides which types of interventions to include according to the abilities of a specific user?
  • Who can check that the state of the person is such as to require more advanced systems based on Artificial Intelligence with decision-making autonomy?
  • Who must check the pertinence of Artificial Intelligent decisions?
  • Which people, who have a relationship with the person in the house, should relate with her and with what priority?
  • Who becomes responsible for the malfunction of an equipment, in the event of a breakdown, which may cause personal injury?
  • What is the level of decision that can be entrusted to the machine compared to the level that is delegated, for example, to a caregiver?
  • What types of actions towards the person require special authorization? For example, the action of providing a drug versus choosing a movie to reduce effects of solitude?
  • If the service involves bringing in other people, such as neighbours, what data protection should be applied to their personal data?
  • What difference can there be, legally speaking, between a system that pro- poses an action and asks for confirmation, and a system that asks for confirmation to perform an action?
  • What types of actions towards the person require special authorization? For example, the same as before, the action of providing a drug versus choosing a movie?
  • How should we consider a suggestion given, because of the consequences it could have, with respect to an autonomous decision of the system?

530 Burzagli

Alongside this aspect, there is also the problem of the difference in valence of the actions that the machine can suggest or perform, for example, from a simple piece of advice for watching a movie, to an invitation to take a medication. The difference is substantial, because in the first case it seems like a suggestion without possible grave consequences, while in the second case it is an action related to the person’s health. However, the difference can be much more nuanced when the systems deal with a progressive worsening of the person’s abilities and are able to deal with conditions such as loneliness, a non-pathological condition.

CONCLUSION

Artificial intelligence applications seem in principle the most promising sug- gestion for the implementation of applications for supporting people, all people not only people with limitations of abilities, in conducting activities in emerging technological society. However, it must be taken into account that for the approval of a real adoption, problems of ethics and of compatibility with existing legislation must be taken into account.

ACKNOWLEDGMENT

The author would like to acknowledge Pier Luigi Emiliani for useful discus- sions on the problems considered.

REFERENCES

Antona, M., Burzagli, L., Emiliani P. L., Stephanidis C. (2007). The ISTAG scenarios: a case study, in: Roe, P. R. W. (Eds.). Towards an Inclusive Future: Impact and Wider Potential of Information and Communication Technologies, Section 4.1 of Chapter 4 “Ambient Intelligence and implications for people with disabilities” COST219ter, pp. 158–187. Burzagli, L., Di Fonzo, L., Emiliani, P. L., Boffi, L., Bak, J., Arvidsson, C., Kristaly,

D., Arteconi, L., Matrella, G., De Munari, I., Ciampolini, P. (2014). The FOOD Project: Interacting with Distributed Intelligence in the Kitchen Environment. Stephanidis, C., Antona, M. (Eds). Universal Access in Human-Computer Inter- action. Aging and Assistive Environments, volume 8515 of Lecture Notes in Computer Science, Springer, Cham., pp. 463–47. Burzagli, L., Emiliani, P. L. (2014). Implementation of Applications in an Ambient Intelligence Environment: A Structured Approach, in: K. Miesenberger, D. Fels, D. Archambault, P. Peˇnáz, W. Zagler, (Eds). Computers Helping People with Spe- cial Needs in Lecture Notes in Computer Science, volume 8548. Springer, Cham., pp. 172–179. Burzagli, L., Naldini, S. (2018). Elderly People and Loneliness: An Innovative

Approach, in: Miesenberger, K., Kouroupetroglou G. (Eds.). Computers Helping People with Special Needs, volume 10897 of Lecture Notes in Computer Science, Springer, Cham., pp. 387–390. Burzagli, L., Naldini S. (2020). Affective Computing and Loneliness: How This Approach Could Improve a Support System, in: Miesenberger, K., Kouroupet- roglou, G. (Eds.) Computers Helping People with Special Needs, Vol 10897 of Lecture Notes in Computer Science, Springer, Cham., pp 387–390.

Automation Versus Machine Autonomy 531

Burzagli, L., Emiliani, P. L. (2021). Criteria for the Adoption of a Support Information System for People’s Mobility, in: Antona, M., Stephanidis, C. (Eds). Universal Access in Human-Computer Interaction. Access to Media, Learning and Assistive Environments, Vol 12769 of Lecture Notes in Computer Science, Springer, Cham, pp 347–357. Burzagli, L., Emiliani, P. L., Antona. M., Stephanidis, C. (2022). Intelligent environ-

ments for all: a path towards technology-enhanced human well-being. Universal Access Information Society, 21(2), pp. 437–456. Burzagli, L., Naldini, S. (2023). GPT3 for Healthy Ageing?, Ital-IA. FakhrHosseini, M., Lee, C., Coughlin, J. F. (2019). Smarter Homes for Older Adults: Building a Framework Around Types and Levels of Autonomy. In: Zhou, J., Sal- vendy, G. (eds) Human Aspects of IT for the Aged Population. Social Media, Games and Assistive Environments. HCII 2019. Lecture Notes in Computer Science, vol 11593. Springer, Cham. https://doi.org/10.1007/978-3-030-22015- 0_24. Müller, M., Müller, T., Ashtari Talkhestani, B., Marks, P., Jazdi, N., Weyrich, M.

(2021). Industrial autonomous systems: a survey on definitions, characteristics, and abilities, at - Automatisierungstechnik, vol. 69, no. 1, pp. 3–13.

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-95-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →