Sinem Görmez, Carsten Röcker (2023). Exploring the Potential of Gestures for Controlling Doors and Windows in Smart Homes. In: Human Factors in Architecture, Sustainable Urban Planning and Infrastructure. AHFE Open Access, vol. 89, pp. 8–15. https://doi.org/10.54941/ahfe1003674
This paper explores the potential of gesture interaction as an alternative control concept for doors, windows and sliding systems in smart homes. In a first step, a technical prototype was built that enables to open and close door and window elements with a hand-swiping gesture. In a second step, a user study with N = 95 participants was conducted to explore the perceived usefulness of the developed solution using a questionnaire with 24 items. The results showed that 78 percent of the participants liked the concept of contactless gesture control of doors, windows and sliding systems. The reluctance of the remaining group could be traced back to a missing experience with smart control concepts (e.g., voice assistants) (t-test: Spearman’s rs = .27, p = .044) and the belief that gestures are hard to remember (chi-square test: α < .01, p = .007). The study also confirmed that the implemented control concept and gestures were perceived as natural and intuitively understandable.
As an effect of the Corona pandemic, the operation of devices and machines without direct physical contact has gained increased interest. In particular, mid-air gestures are a promising approach for contactless control, which could prevent the spread of germs and viruses in public usage scenarios. But already earlier, smart operating concepts were a matter of course in application areas such as the automotive industry or smart home devices and are therefore part of everyday interactions. For instance, autopilots can be controlled via projected keypads (holograms), and voice assistants are easily managed via simple voice commands. But what about expansions? Is it feasible to install such smart operating concepts in something as simple as a window? How do users feel about such enhancements?
Focusing on the aforementioned issues, this paper analyses the feasibility as well as the acceptance of gesture control concepts of doors, windows and sliding systems at the Schüco International KG. The exploration of gesture control is particularly interesting, as there is no comparable product there at the moment. Thus, such a system would serve a gap in the market.
Exploring the Potential of Gestures for Controlling Doors and Windows 9
Moreover, contactless control units could let the operation become invisible. This is especially interesting for architects, demanding for profile-integrated solutions. Both the actuators and wiring are already hidden, only the control panels are still visible.
SMART CONTROL CONCEPTS - STATE OF THE ART What is a “smart” control? Conventional solutions are all those, that require an operating item like a button, a switch or a door/ window handle at or near a so-called “element”. The term “element” is used in the following to summarize windows, doors and sliding systems. In contrast to such conventional solutions, “smart” approaches include contactless operations via for example gestural input, remote control or voice interaction, and many more. Such contactless solutions offer the advantage of reducing the contamination of the device surface, which is beneficial when navigating through screen content and menus in a dirty or clean environment (Basmaji et al. 2021; Gerba et al. 2016). Moreover, there are situations where a touch interface fails to accurately interpret user intentions, for example, with water on the surface (Wojtczuk et al. 2013). In such cases, a contactless interface can provide an alternative input method. These also offer the possibility of hiding them.
This paper focuses on the non-contact concept of gesture control. Whereas input devices inherently constrain human motion for meaningful human- computer dialogue (Foley et al. 1996), surface gestures are versatile and highly varied. Almost anything one can do with one’s hands could be a potential gesture (Wobbrock et al. 2009). Human-machine interfaces, that interpret hand gestures as a mode of input, provide a natural means of interaction (Wojtczuk et al. 2013). Especially, interaction devices such as the Microsoft Kinect sensor and other types of depth/ infrared cameras have facilitated the advancement of innovative and natural user interfaces for human-computer interaction (Yang et al. 2015). These mechanisms rely on generic hand tracking algorithms or recognition engines of a minimal set of hand gestures (Yang et al. 2015). For the mainstream, depth cameras became widely available with the Kinect revolution (Escalera et al. 2016). This was one of the anchor points that increased the public adoption of gesture recognition (Escalera et al. 2016).
DESIGN OF A GESTURE RECOGNITION DEVICE Indeed, a lot of sophisticated gesture recognition devices already exist. However, these solutions mostly do not fit to the problem of gestural control of doors, windows and sliding systems. And in addition, what is much more significant, these systems exceed the planned budget many times over. Therefore, the question arises whether such a gesture tracking device could not be built from scratch with simple means.
Thereby, the first prototype should only be able to recognize the most fundamental commands. The device should end up being as simple as possible. In the case of a sliding system, for instance, the most basic
10 Görmez and Röcker
commands are the “open fully”- and “close fully” -operations. To trigger these movements, the corresponding gestures are defined as a swipe with the hand from the left to the right or from the right to the left, respectively. Then, the task consists of replacing a sophisticated gesture recognition method and device with a comparatively simple solution. The first idea includes the usage of several infrared proximity sensors. Indeed, a single proximity sensor is just able to detect the presence of an object. But the more advanced idea is that if multiple sensors are connected in series, it may be possible to reconstruct a movement direction using the motion detection time history.
To realize this idea, tree shifted sensors are soldered into a pinhole board, resulting in a prototype as shown in Figure 1. This structure enables an elegant and small design with dimensions of 25 mm by 160 mm while costing less than ten euros. The data processing is done by an Arduino UNO.
Figure 1: Pinhole board of the first prototype.
This setup enables the recognition of wiping movements from left to right and vice versa. The Arduino UNO converts the gestures into movement commands. This gesture control system can be integrated into an existing building management system using Schüco’s BSC. As a result, a wiping movement from left to right can be used, to open a window, for example.
An acceptance study is conducted in order to evaluate to what extent this gesture selection meets user expectations.
ACCEPTANCE STUDY The main aim of the study is to validate the hypotheses listed in the following:
H1: There is a need for contactless operation.
H2: People like the idea of gesture control on windows, doors and sliding systems.
H3: A reliable operation is indispensable for control concepts.
H4: The lateral movement of a sliding system is associated with a hand wiping gesture.
H5: The rotation of a door is associated with a hand wiping gesture.
Methodology
The acceptance study is conducted as an anonymous online survey. Since the survey included 24 items, not all of which yielded relevant results, only the most important ones will be discussed. These are:
Item 18: Openness towards gesture control for doors and windows
Exploring the Potential of Gestures for Controlling Doors and Windows 11
Item 20 to 22: Suggestions for gestures to open/ close doors and sliding systems
Item 23: Belief that gestures are easy to remember
Item 5 to 8 and 11 to 13: Feedback on touchscreens and voice assistants
Item 9: Interest in smart home systems.
RESULTS
In total, 95 volunteers participated in the study. The gender of the test persons is predominantly male with a proportion of 78 percent. The participants are on average 34 years old, with a standard deviation of 12 years. The survey had a very technical audience (64 percent work in a technical profession).
Feedback on Touchscreens and Voice Assistants With regard to touchscreens, the respondents are most bothered by the fact that, depending on the quality, the operation is unsophisticated and the inputs are poorly recognized or not recognized at all (thirteen responses). This immaturity also leads to undesirably long response times, which tempts many users to impatiently press around somewhere and provoke incorrect entries. Frustration arises in such situations. In addition, the poor interpolation of some touchscreens leads to further usability issues of the system. With small characters, dots, etc., it is easy to press the wrong key (21 mentions) and, in the worst case, trigger unintentional security-relevant inputs with serious consequences without a confirmation prompt. Such typos happen less often with real buttons.
According to the respondents, a classic usage issue of touchscreens is that they quickly get dirty and then no longer respond properly (23 mentions). In addition to grease stains (four mentions), fingerprints (five mentions), bacteria (three mentions) and scratches spread quickly on the surface. Then, it is also not possible to operate the device with wet or oily fingers (seven answers). Therefore, touchscreens cannot be installed in damp rooms, for example, in bathrooms or in outdoor areas that are not protected from rain (nine mentions). This problem cannot be circumvented by wearing gloves, because then the touchscreen no longer reacts (eleven mentions). Such outdoor areas are also problematic because sunlight causes the touchscreens to become unreadable there (nine responses).
Moreover, the respondents mainly named public spaces as inappropriate locations for touchscreens, as the risk of bacterial contamination is higher there (seventeen answers). Especially in clinics, elevators or vending machines, the respondents perceive touchscreens to be hygienically questionable (four mentions).
Finally, touchscreens are problematic in situations that require permanent attention, like driving a car (thirteen responses). The system could distract the driver as it is not possible to operate touchscreens without looking at the screen (ten mentions).
The most frequently mentioned preferred application area for touchscreens are smartphones (22 mentions) and tablets (fifteen answers). They are
12 Görmez and Röcker
especially useful, because they allow an alternating usage between display functions and buttons (five answers).
To switch to voice assistants, the users appreciate that VAs allow a comfortable operation from the couch (five responses) without a need for physical interaction. Lights can be switched from anywhere in the house without looking for a remote control, a cell phone or reaching to the switch (five mentions). Additional operating devices like remote controls or touchscreens are no longer necessary (three mentions). Moreover, the respondents also like that the elimination of physical input saves typing and typos (five responses).
With regard to the negative aspects of voice assistants, the test persons mainly mention technical issues. In particular, the respondents criticize the reliability of the recognition, i.e. that spoken sentences are misunderstood (fourteen responses).
Openness on Gesture Control While 78 percent of the respondents like the idea of gesture control on windows, doors and sliding doors, this contrasts with 22 percent who do not like it. To understand the latter group, the research question asks: “Is there a difference between people with certain characteristics in terms of openness to gesture control?”
There is a weak positive correlation between the usage of voice assistants (VAs) and the openness to gesture control, Pearson’s r(95) = .27, p = .044. The difference is significant.
The frequency table for the two variables is shown Table 1 in order to investigate the actual distribution of the difference.
Table 1. Frequency table for the voice assistant usage and the openness on gesture control.
Table available in the original PDF paper.
Frequency Percent
VA usage Openness on gesture control no no 17 29 yes 42 71 yes No 4 11 yes 32 89
While 89 percent of the VA users like the idea of gesture control, only 71 percent of the non-users do so. The usage of VAs and the openness to gesture control correlate positively. Users of voice assistants are more open to gesture control than non-users.
Going further, a chi-square test for the relationship between the interest in smart home and the openness to gesture control looks as shown below in Table 2.
Those subjects who are interested in smart home like the concept of gesture control more than those subjects who are not interested in smart home, χ2(95) = 9.61, p = .008, α <.01. There is a meaningful dependency between the variables.
Exploring the Potential of Gestures for Controlling Doors and Windows 13
Table 2. Chi-squared test table for the relationship between the interest in smart home systems and the openness on gesture control.
Table available in the original PDF paper.
Value df p χ2 9.606 2 0.008 N 95
The attitude towards the memorability of gestures is also analysed. This includes the question whether the participants believe that it would be easy for them to memorize gestures. The data show that as the belief that gestures are easy to remember decreases, so does the openness to use gesture control, χ2(95) = 9.86, p = .007, α <.01. While 85 percent of the test subjects who assume that it would be easy for them to remember a handful of gestures are in favour of gesture control, only 40 percent of those who would find it difficult support it. The middle group, who is not sure if gestures would be easy for them to remember, supports it at 59 percent.
In summary, the openness to gesture control decreases with the distrust of being able to remember gestures.
The same test was applied to all other items, leading to the result that no statically significant relationship with openness to gesture control can be derived for any of the other characteristics asked in the questionnaire.
Suggested Gesture to Operate Doors and Sliding Systems A total of 95 suggestions were made for gestures to open a door. The two most popular options were firstly a swipe motion with 38 mentions (i.e. 40 percent) and secondly a push or pull motion with 32 mentions (i.e. 34 percent).
In addition to the type of gesture, a second dimension was also often described by the respondents, namely the direction of the movement. In 53 percent of the wipe proposals, the swipe direction was suggested to be the same as the opening direction. To be more precisely, this means moving the hand from the “doorknob” side to the opposite side.
Besides the type of gesture and the direction of movement, the subjects described the executing actuator in more detail as a third dimension. 91 percent of the proposals were hand gestures.
If a respondent had indicated a gesture for opening of a door in the previous question, he or she almost always indicated the same gesture, but in the opposite direction, for the closing gesture. Since the results are very similar, the suggestions for gestures to close a door are not discussed further.
For a sliding door, 94 percent of the suggestions are a hand wiping gesture. This is to be executed in the corresponding movement direction of the sliding door.
INTERPRETATION This chapter deals with the discussion of the previous described results of the acceptance study.
14 Görmez and Röcker
Need for Contactless Operation Contactless operation can make sense depending on the environmental conditions and what is to be realized technically. For compact devices, where operation and display must be realized at the smallest possible surface, contact-based operating concepts are indispensable. But in public contexts, non-contact concepts should rather be implemented. This reduces bacterial contamination. The same applies for dirty or humid environments and for applications where a blind operation is advantageous. Therefore, the hypothesis one: “There is a need for contactless operation”, can be confirmed and verified by the results from the study.
Openness on Gesture Control Since a total of 78 percent confirm that they like the idea of gesture control of doors, windows and sliding systems, gesture control is a solution that could fulfil this need for contactless operation. Thus, hypothesis two “People like the idea of gesture control on windows, doors and sliding systems” could be confirmed, too. Thereby, especially those test persons who already use other smart solutions such as voice assistants are particularly receptive. Such people are already familiar with smart control concepts and are therefore less influenced by prejudices and fears. In contrast, those consumers who fear that they will not be able to remember the control gestures easily should be especially supported; their openness is decreased. People are more likely to keep their currently functioning system if they are not sure whether a new system will work just as well. Nevertheless, the clarification that one basically only has to remember two gestures (one to open and one to close an element) should eliminate all fears.
Gesture Selection and Need for a Reliable Operation When it comes to the concrete design of the gesture control device, a handwipe gesture should be provided for both doors and sliding elements; even though swiping was only just ahead in the survey for normal doors. Since for a sliding door the gesture selection is more than clear, and the design for a normal door should be the same, this decision is still close to the customer. In detail, this wiping motion has to be carried out in the corresponding opening or closing direction. This does not only confirm hypotheses 4 and 5, but also the previously developed prototype; it processes exactly such swipe gestures.
Another important finding from the study is that a reliable operation is essential for control concepts. This is expressed above all in the criticism of users of touchscreens and voice assistants. What bothers consumers most is the fact that incorrect entries occur frequently, which leads to frustration.
CONCLUSION
Gesture control can be implemented for doors, windows and sliding systems. Furthermore, the market analysis reveals that it is accepted by potential users.
Exploring the Potential of Gestures for Controlling Doors and Windows 15
ACKNOWLEDGMENT
I would like to thank the Smart Building department of the Schüco International KG, which provided me with the necessary means and resources for this project.
REFERENCES
Basmaji, Tasnim. Tarabsheh, Anas Al. Zia, Huma. Hajjdiab, Hassan. Ghazal, Mohammed. (November 29, 2021). “Towards Urgent and Rapid Deployment of Contactless IoT Elevator Control Panel during Pandemics”, in: ArabWIC 2021: The 7th Annual International Conference on Arab Women in Computing in Conjunction with the 2nd Forum of Women in Research, Association for Computing Machinery, Sharjah, United Arab Emirates, pages 1-5, ISBN: 9781450384186, DOI: 10.1145/3485557.3485565. Escalera, Sergio. Athitsos, Vassilis. Guyon, Isabelle. (April 2016). “Challenges in
multimodal gesture recognition”, in: Journal of Machine Learning Research, volume 17, number 72, pages 1–54. Foley, James. Van Dam, Andries. Steven K. Hughes, John F. (1996). “The form and content user-computer dialogues”, in: Computer Graphics: Principles and Practice, pages 392-395, ISBN 0-201-84840-6. Gerba, Charles P. Wuollet, Adam L. Raisanen, Peter. Lopez, Gerardo U. (2016). “Bacterial contamination of computer touch screens”, in: American Journal of Infection Control, Volume 44, Issue 3, pages 358-360, DOI: 10.1016/j.ajic.2015.10.013. Wojtczuk, Piotr. Binnie, David. Armitage, Alistair. Chamberlain, Tim. Giebeler,
Carsten. (2013). “A Touchless Passive Infrared Gesture Sensor”, in: Proceedings of the Adjunct Publication of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST ‘13 Adjunct, Association for Computing Machinery, pages 67-68, ISBN: 9781450324069, DOI: 10.1145/2508468.2514713. Wobbrock, Jacob O. Morris, Meredith Ringel. Wilson, Andrew D. (2009). “User- Defined Gestures for Surface Computing”, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, series CHI ‘09, Association for Computing Machinery, pages 1083–1092, ISBN: 9781605582467, DOI: 10.1145/1518701.1518866. Yang, Lin. Zahng, Longyu. Dong, Haiwei. Alelaiwi, Abdulhameed. El Saddik,
Abdulmotaleb. (August 2015). “Evaluating and Improving the Depth Accuracy of Kinect for Windows v2”, in: IEEE Sensors Journal, volume 15, page 1, DOI: 10.1109/JSEN.2015.2416651.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Architecture, Sustainable Urban Planning and Infrastructure
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-65-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional