Gerhard Rinkenauer, Jai Prakash Kushvah, Marc Grosjean (2022). Psychophysics and user experience: Perceptual differences in the effort required to operate virtual push-buttons. In: Human Factors and Systems Interaction. AHFE Open Access, vol. 52, pp. 89–95. https://doi.org/10.54941/ahfe1002143
Perceptual physical aspects of controls, such as interaction forces during operation, are important for ergonomic design. However, controls with equivalent physical properties may be perceived as functioning differently depending on their visual or acoustic appearance. To address this issue, the present study investigated how the size, brightness and loudness of pushbutton switches affect the perceived operating force. Two simple pushbuttons (standard and test) were presented side by side in a virtual environment and actuated with a 3D haptic device. The simulated mechanical properties of the pushbuttons (force-displacement characteristics) corresponded to those of real switch buttons. Three experiments were conducted with different groups of subjects. Physical characteristics of the standard button were kept constant and physical size, brightness and loudness of the test button were systematically varied respectively in all 3 experiments. Participants were instructed to press the standard and the test push-button one by one and to judge the perceived force for test push-button compared to the standard one in a 2-alternative-forced-choice task. Based on these judgments, the required operating force of the test key was adjusted using a simplified adaptive staircase procedure until the force of the test key varied by the point of subjective equality. Based on the subjective equality, the perceived operating force for the experimental condition was calculated. The results showed main effects of key size, brightness and loudness on perceived operating force. Consistent with findings from basic research (size-weight illusion), perceived operating force was higher for smaller keys. Additionally, perceived operating force was reported higher for higher brightness or higher loudness. Overall, the results suggest that psychophysical methods are suitable for objectively measuring the user experience of interacting with controls in application contexts.
The ergonomic design of push-buttons aims to reduce mental and physi- cal stress, shorten execution times, and avoid errors (AS 4024.1901, 2014). Physical push-buttons have been extensively researched in the past (e.g.,
Ivergard, 1989, Oulasvirta et al. 2018), and ergonomic guidelines have been defined to manage and standardize their functional design and use (e.g., AS 4024.1906:2014). Beyond the purely functional and ergonomic design concepts, there is also the aspect of user experience in which the subjective and emotional evaluation by the user also plays an important role (e.g., Wellings et al. 2008). Predominantly, subjective criteria play a role in the estimation of user experience, based on self-reports about the user’s feelings when handling or engaging with a product (e.g., Laugwitz et al, 2006; Minge et al, 2016). In our study, we aim to investigate to what extent subjective perceptual differences of control elements can be measured with objective measures from the field of psychophysics. Little research has been done on how visual and auditory impressions affect the perceived functionality of con- trol buttons (e.g., Roesler et al, 2009). In our paper, we therefore address the perception of the operation of virtual buttons with identical mechani- cal properties when the visual or auditory appearance changes. To address this question, the present study conducted three experiments to investigate how size, brightness, and loudness of push-button switches affect perceived actuation force.
Concepts and methods from the field of weight perception serve as the the- oretical and empirical framework for our investigation. In these studies, it has been shown that judgments of object heaviness depend not only on mass but also on other properties such as object size, material properties, or brightness (e.g., Ellis & Ledermann 1993, 1999). For example, when lifting two objects of different volume but equal weight, people estimate the smaller object to be heavier. This size-weight illusion, first documented over 100 years ago, is both strong and robust (Flanagan & Beltzner, 2000). A leading hypothesis to explain this phenomenon is that the illusion is due to a discrepancy betw- een expected and actual sensory feedback regarding object weight (e.g., Ross, 1969). Recent research further suggests that the haptic system has a stronger influence on the size-weight illusion than the visual system (Ellis & Lederman 1993, 1999). Similar cross-modal effects are also found for object brightness. For example, dark objects are shown to be judged as heavier during purely visual judgments, but when lifted, a similar illusion to the size-weight illu- sion occurs and the darker objects are perceived as lighter (Walker et al., 2010). In general, it appears that perceptual expectations regarding weight are derived from the visual appearance of an object causing a sensory misma- tch between what is predicted and experienced. Such sensory mismatches can also be explained by current theoretical concepts of motor control (Flanagan & Beltzner, 2000). During lifting, the central nervous system generates a pre- diction of sensory feedback based on an internal forward model of the object to be lifted. Due to misleading visual cues, a inapropriate forward model of the object is built, and a mismatch occurs between the predicted and actual sensory feedback, resulting in erroneous weight perception.
Regarding the association of loudness and weight perception, there are fewer clear findings so far. There is apparently a relationship between lou- dness and perceived object length; more specifically, Hauck and Hecht (2019) reported that objects are perceived as longer with increasing sound pressure levels. However, it is unclear whether this change in length perception could
Psychophysics and User Experience: Perceptual Differences 91
also lead to a size-weight illusion and thus be indirectly related to loudness. In addition, there is evidence that weight perception is influenced by pitch rather than loudness, i.e., objects with high pitch were perceived as lighter than objects with low pitch (Takashima, 2018).
Independent of these findings on weight perception, i.e., where the sen- sory input for the effort to lift and hold a weight interferes with sensory information from other perceptual channels (e.g., Rinkenauer et al, 1999), there have been no direct studies on force perception and cross-modal influ- ences on the operation of real or virtual push-buttons so far. Therefore, in three experiments of this paper, we proceed analogously to the cross-modal studies on weight perception. Regarding the effects of visual appearance of the push-buttons, we are expecting similar effects of size and brightness as observed in the studies above, i.e., smaller and brighter buttons feel harder to operate compared to the control condition. Regarding loudness, we have no prediction of how the operability of louder keys will be judged.
METHOD AND PROCEDURE Data of forty-two participants was collected for Experiment 1 (N = 16, 7 female, Mage = 24.7 (SD = 4.4) years), Experiment 2 (N = 10, 6 female, Mage = 23.5 (SD = 2.9) years) and Experiment 3 (N = 16, 7 female, Mage = 24.1, (SD = 1.9) years). All 3 experiments were conducted under continuous vigilance of the experimenter. Participants were instructed to simply judge the force applied to the test push-button compared to the stan- dard one in a 2-alternative-forced-choice task. For this purpose, two simple push-buttons (standard button on the left and test button one the right, see Figure 1) actuated with a 3D haptic device (Phantom Desktop, see Figure 1, left panel), were presented in a virtual environment. The physical properties (size, brightness, loudness) of the standard button were kept constant in all 3 experiments, whereas the size (Experiment 1), brightness (Experiment 2), and loudness (Experiment 3) of the test button were systematically varied. Push- buttons could be operated with a 3D cursor (sphere) (Figure 1, right panels). Simulated mechanical properties of the push-buttons (force-displacement characteristics) were approximated to the properties of real switch buttons. For this, purpose the force-displacement curve (see Figure 2, left panel) was implemented as per the literature available. The force-displacement curve of our simulated push-buttons corresponded to an abrupt profile type, typi- cally found in mechanical ones (Gaspar et al, 2017). These push-buttons are characterized by a clearly contrasted actuation point. Figure 2 (left panel) shows the main design coordinates of the force-displacement curve. From the moment a key is touched with the virtual 3D cursor, the force increases linearly up to the actuation point and then drops abruptly by 1 N over a short distance towards the contact point, before the force increases linearly again. At the end of the actuation path, the force is then increased again sharply to simulate the mechanical end point.
Our experimental setup utilized a simplified adaptive staircase algori- thm to vary the forces of the actuation and contact points (see Figure 2, right panel). The difference between these two force coordinates was always
Figure 1: Experimental setup with screen and haptic display (left panel) for the size manipulation of the test key, as well as the two variants of the test key (large vs. small) and the virtual cursor (right panels).Figure 2: Force-displacement curve for the standard key (left panel) and examples of the actuation forces as a function of step of the test key, based on the stair-case algorithm used (right panel).
maintained at 1N. Experimental task trials were progressed by first pres- sing standard push-button and then the test one. The test push-button was either the same or different from the standard one. The actuation force of the standard push-button was 2.5N in all experiments. The initial value of the actuation force of the test pushbutton was randomly selected between 3 and 4 N. All further actuation forces depended on whether the participant judged the operating force of the test button to be higher. If the answer was “yes”, then the actuation force of the next step was reduced by 0.2 N, if the answer was “no”, then the force was increased by 0.2 N. Four runs were performed for each representation of the test button (same vs. different from the standard key). One pass consisted of 21 steps (see Figure 2, right panel)
Psychophysics and User Experience: Perceptual Differences 93
Figure 3: Results of the three experiments in which size (left), brightness (center) or loudness (right) of the test button were varied.
and the mean actuation force of the last 5 steps was used as an indicator of subjective equality (SE) of the actuation forces between the standard and test button. This corresponds to the actuation force of the test key that had to be set on average in order for it to be perceived as having the same actuation force as the standard key. From this value, the perceived force (PF) was then calculated as: PF = Standard + (Standard - SE) according to a suggestion by Rinkenauer et al. (1999).
The physical properties of the virtual push-buttons were varied in the fol- lowing way. The brightness of the standard push-button was defined by a gray value (color value = [0.6, 0.6, 0.6]) and the loudness of the click sound was approximately 50dB. In the experimental variations of the test push- button, the size manipulation presented half the width of the standard button, while the brightness manipulation presented the push-button in white color without resizing (color value = [1.0, 1.0, 1.0]), and the sound manipula- tion increased only the loudness of the click sound to approximately 60dB. Based on the participants’ judgments, the respective test button actuation force was adjusted as per the staircase algorithm until the test button force varied around the point of subjective equality.
RESULTS
For each experiment, a separate within-subjects ANOVA was performed for the factors Condition (same, different) and Trial (1-4). The Trial factor was included in the analysis to detect any learning or fatigue effects. For all three experiments, the main experimental effect of Condition (see Figure 3) was significant on perceived force. The main effect of Trial and the interaction effect of Condition and Trial were not significant (all ps > .17). As expected, the perceived force for the test push-button was greater (M = 2.51 N) when presented in a reduced size compared to the same size as the standard one (M = 2.36 N) [F (1, 15) = 5.05, p = .04, η2
G = .04]. The effect of brightness
was also in the expected direction, i.e., the perceived force for the test push- button was greater when it was brighter (M = 2.49 N) than the standard one (M = 2.33 N) [F (1, 9) = 5.69, p = .04, η2
G = .04]. The effect of sound was analogous to brightness. Participants perceived the force higher when the click noise of the test push-button was louder (M = 2.37 N) compared to the standard one (M = 2.28 N) [F (1, 15) = 5.25, p = .04, η2
Equation reproduced from the original PDF.
DISCUSSION & CONCLUSION Determining the subjective operating characteristics of push-buttons is an important basis of user experience for push-button design. In the three expe- riments of our study, we used a methodological approach from psychophysics to try to determine the perceptual differences in the operating force when the visual or auditory design of the buttons changes while the mechanical properties remain the same. Empirical studies from the field of weight perce- ption served as our conceptual starting point. In these studies, systematic perceptual differences are shown when the visual sensory information of the objects change (size or brightness of the objects) at constant weight, i.e., at constant effort. Analogous to these studies, it is shown that the operating force is perceived as higher when the button is displayed smaller or brighter. Analogous to brightness, sound manipulations also showed that louder keys are perceived as harder to operate. Overall, the results suggest that psych- ophysical methods are suitable to objectively measure the user experience when interacting with controls in applied contexts when the sensory context is changed. In principle, this approach can be extended to other dimensions than operator force, for example by assessing perceptual differences regar- ding perceived reliability, operator safety, or operator comfort as a function of the force-displacement curve with a psychophysical approach as used in this study.
ACKNOWLEDGMENT
This work was supported by the “Research Training Group 1855” fun- ded by the German Research Foundation (DFG). We would like to thank Ms. Johanna Renker for assistance in conducting and Mr. Oscar Opalinski for programming the experiments.
REFERENCES
AS 4024.1901 (2014). Safety of Machinery Displays, controls, actuators and signals - Ergonomic requirements for the design of displays and control actuators - General principles for human interactions with displays and control actuators. Ellis, R.R., Lederman, S. J. (1999). The material-weight illusion revisited. Percept.
Psychophys 61, 1564–1576. doi: 10.3758/bf03213118 Ellis, R.R., Lederman, S.J. (1993). The role of haptic versus visual volume cues in the size-weight-illusion. Perception & Psychophysics 53, 315–324. Flanagan J.R., Beltzner M.A. (2000). Independence of perceptual and sensorimotor
predictions in the size-weight illusion. Nature Neuroscience 3, 737–741.
Psychophysics and User Experience: Perceptual Differences 95
Gaspar, J., Fontul, M., Henriques, E., Silva, A. (2017). Haptics of in-car radio buttons and its relationship with engineering parameters. Int. J. Ind. Ergon. 59, 29–45. https://doi.org/10.1016/j.ergon.2017.03.005 Hauck, P., Hecht, H. (2019). The louder, the longer: Object length perce-
ption is influenced by loudness, but not by pitch. Vision, 3(4), 1–15. https://doi.org/10.3390/vision3040057 Ivergard, T. (1989). Handbook of Control Room Design and Ergonomics. Taylor & Francis, London. Laugwitz, B., Schrepp, M., Held, T. (2006). Konstruktion eines Fragebogens zur
Messung der User Experience von Softwareprodukten. [Construction of a que- stionnaire for the measurement of user experience of software products] In: A.M. Heinecke & H. Paul (Eds.): Mensch & Computer 2006 – Mensch und Computer im Strukturwandel. Oldenbourg Verlag, S. 125–134. Minge, M., Thuring M., Wagner, I., Kuhr, C.V. (2016). The meCUE Questionnaire. A Modular Evaluation Tool for Measuring User Experience. In Advances in Ergonomics Modeling, Usability & Special Populations, Proceedings of the 7th Applied Human Factors and Ergonomics Society Conference, edited by M. Soares, C. Falcao, and T. Z. Ahram, 115–128. Switzerland: Springer International Press. Oulasvirta, A., Kim, S., & Lee, B. (2018). Neuromechanics of a button press.
In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–13). Rinkenauer, G., Mattes, S., & Ulrich, R. (1999). The surface-weight illusion: On the contribution of grip force to perceived heaviness. Perception & Psychophysics, 61(1), 23–30. Roesler, F., Schuettler, F., Battenberg, G. (2009). Subjective perceptions and objective
characteristics of control elements. ATZ-Autotechnology, 48–53. Ross, H. E. (1969). When is a weight not illusory? Q. J. Exp. Psychol. 21, 346–355. Takashima, M. (2018). Perceived Weight Is Affected by Auditory Pitch Not Loudness.
Perception, 47(12), 1196–1199. https://doi.org/10.1177/0301006618808937 Walker P., Francis B.J., Walker L. (2010) The brightness–weight illusion. Experimental Psychology, 57 (6), 462–469. Wellings, T., Williams, M.A., Pitts, M. (2008). Customer Perception of Switch-feel in
Luxury Sports Utility Vehicles. Food Quality and Preference 19, 737–46.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors and Systems Interaction
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-28-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional