OBJECTIVE OF THE TUTORIAL This tutorial has the primary objective that participants will learn about an open and fully implemented approach to model interaction design with discourse models, which are at the highest level of the Cameleon Reference Framework, i.e., the Tasks & Concepts Level. These models focus on the spe- cification of (classes of) dialogues in contrast to tasks for modeling activities that can be performed by the user or the application (system). Participants will also get an overview of both automatically generating and customizing GUIs.
CONTENT OF THE TUTORIAL The content includes Communicative Acts as the basic elements, as well as Adjacency Pairs and RST relations for constructing the discourse models using these elements. It also includes an iterative-incremental process of user-interface generation using such models. This process also includes customization.
TUTORIAL OUTLINE
- Introduction - Brief introduction of the tutor - Brief introduction of the participants - Motivation and overview
© 2022. Published by AHFE Open Access. All rights reserved. 221
222 Kaindl
- Background - Interaction design - Ontologies - Speech acts - Conversation Analysis
- Interaction Design based on Discourse Modeling - Discourse example - Communicative Acts - Adjacency Pair - RST relations - Conceptual Discourse Metamodel - Exercise: Understand given model
- GUI Generation - Generation of Structural UI Model - Generation of Behavioral UI Model - Weaving of structural and behavioral models - Optimization for tailoring to device - Process of user-interface generation - Examples of generated user interfaces
- Customization - Custom rules - Custom widgets
- Conclusion
DESCRIPTION This tutorial is a combination of lectures, group discussions and exercises.
In the short Introduction, it turned out to be useful in the past to have all the people in the room briefly introduced. In particular, it is good to know the particular background and expectation of each participant, so that it becomes possible to focus parts of the following presentation on them. Then it also becomes easier to motivate the tutorial and to provide an overview.
Still, the group will not share the same Background required for digesting the subsequent material. So, very basic statements on underlying themes help in this regard. For this particular tutorial, sharing an essential view on Intera- ction design is important. Ontologies, Speech acts and Conversation Analysis may not necessarily be known by some participants at all, but these may not be ready to reveal that and ask. Therefore, it is important to say a few words, also on the historical background.
After that, the ground will be paved for the core material on Interaction design based on Discourse modeling. It starts with an example Discourse Model and its essential explanation by the tutor. Then the basic building blo- cks of such models, Communicative Acts (as a generalization of Speech acts), are explained, and the set of concrete ones is presented in some detail, always including examples. The Communicative Acts are connected via Adjacency Pairs, so that dialog structures can be modeled. Adjacency Pairs, in turn, are connected via Rhetorical Structure Theory (RST) relations. These are also explained, of course, as well as the set of RST relations adopted for discourse
Tutorial: Discourse Models for Interaction Design 223
modeling, always including examples. In addition, Procedural constructs are introduced, which serve for convenience in modeling the flow of a discourse. Based on all these building blocks, the Conceptual Discourse Metamodel can be explained, followed by another example of a Discourse Model, and an Exercise on interpreting the third overall example of a Discourse Model, this time by themselves.
Based on that, this tutorial explains how GUIs can be generated automati- cally and, in this course tailored to different devices (as specified). The tutor presents a Sketch of automated user-interface generation from such Discourse Models and Domain-of Discourse Models related to Ontologies (plus another kind of model and a device specification not treated in detail in the course of the tutorial).
This presentation includes a Process of user-interface generation, including its essential steps such as automated optimization for tailoring to a given device. It defines iterations dedicated to adding increments to the source model for automated GUI generation. This process is inspired by iterative and incremental software development. Still, this defined process only covers the inclusion of automated GUI generation and is, therefore, only a fragment of a comprehensive interaction design process. Finally, Examples of generated user interfaces are presented, in particular for a widely used smartphone.
This tutorial also shows how Customization can be integrated into such a generation approach, i.e., an adaptation of a generated GUI for the case at hand. These adaptations are supported through custom rules and custom widgets for making the changes persistent for re-generation (without any extra effort), in contrast to a manual modification on a lower level or even in the GUI code. Custom rules have absolute precedence over the other com- peting transformation rules in the course of the optimization search, which are not further investigated during this search. In addition, our approach provides the designer with the possibility of including custom widgets as alternatives to the widgets given in the specified toolkit.
Finally, a Summary and Conclusion wraps up and makes a few statements to carry home.
The references to papers published by this tutor given below cover this material.
TARGET AUDIENCE The target audience can be students or educators primarily of software or systems engineering, or information systems. They can also be practitioners creating software, embedded systems or information systems with a user interface.
BIOGRAPHY OF PRESENTER Hermann Kaindl is the head of the organizational unit entitled “Software- intensive Systems” of the Institute of Computer Technology and a Vice Chairman of the Senate at TU Wien. He joined this institute in early 2003 as a full professor. Prior to moving to academia in early 2003, he has gained
224 Kaindl
nearly 25 years of industrial experience in require-ments and software engi- neering as well as human-computer interaction at Siemens Austria. Kaindl is a Senior Member of the IEEE and a Distinguished Scientist Member of the ACM.
He has previously held more than 70 tutorials, e.g., at RE’03, CADUI- IUI’04, RE’04, INCOSE’05, AAAI’06, OOPSLA’06, IFIP Interact’ 7, OOPSLA’07, RE’08, IFIP Interact’09, ACM EICS’10, AAAI’11, RE’11, ACM CHI’12, BCS HCI’12, ACM SAC’13, IEEE APSEC’13, HICSS’14, ACM SAC’14, RCIS’15, RE’15, SMC’16, APSEC’16, RCIS’17, eics’17, ER’17, CAiSE’18, RE’18, ACM SAC’19, RE’19, SPLC’19, RE’20, SPLC’20, IUI’21, RE’21, SPLC’21 and ASE’21.
REFERENCES
Alonso-Ríos, D., Raneburger, D., Popp, R., Kaindl, H., Falb, J. (2014) A User Study on Tailoring GUIs for Smartphones. In Proceedings of the 29th Annual ACM Symposium on Applied Computing (SAC’14). ACM Press. Bogdan, C., Falb, J., Kaindl, H., Kavaldjian, S., Popp, R., Horacek, H., Arnautovic,
E., Szep, A. (2008) Generating an Abstract User Interface from a Discourse Model Inspired by Human Communication. In Proceedings of the 41st Hawaii Interna- tional Conference on System Sciences (HICSS’08), IEEE, Waikoloa, Big Island, Hawaii. IEEE. Bogdan, C., Kaindl, H., Falb, J., Popp, R. (2008) Modeling of interaction design by end users through discourse modeling. In Proceedings of the 2008 ACM Inter- national Conference on Intelligent User Interfaces (IUI’08). Maspalomas, Gran Canaria, Spain. ACM Press. Falb, J., Kaindl, H., Horacek, H., Bogdan, C., Popp, R., Arnautovic, E. (2006) A
discourse model for interaction design based on theories of human communica- tion. In Extended Abstracts on Human Factors in Computing Systems (CHI’06), pp. 754–759. ACM Press. Falb, J., Kavaldjian, S., Popp, R., Raneburger, D., Arnautovic, E., and Kaindl, H.
(2009) Fully Automatic User Interface Generation from Discourse Models. In Pro- ceedings of the 2009 ACM International Conference on Intelligent User Interfaces (IUI’09). ACM Press. Tool demo paper. Falb, J., Popp, R., Röck, T., Jelinek, H., Arnautovic, E., Kaindl, H. (2006) Using communicative acts in interface design specifications for automated synthesis of user interfaces. In Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering (ASE’06), pp. 261–264. Falb, J., Popp, R., Röck, T., Jelinek, H., Arnautovic, E., Kaindl, H. (2007) UI Prototy-
ping for Multiple Devices Through Specifying Interaction Design. In Proceedings of the 11th IFIP TC 13 International Conference on Human-Computer Interaction (INTERACT 2007), pp. 136–149. Rio de Janeiro, Brazil. Springer. H. Kaindl (2009) Model a Discourse and Transform It to Your User Interface. In Proceedings of the 12th IFIP TC 13 International Conference Human-Computer Interaction (INTERACT 2009), Part II, LNCS vol. 5727, pp. 948 – 949 Springer. Kaindl, H., Popp, R., Raneburger, D. (2014) Alternative Interaction Design Patterns
for Automated GUI Generation from Discourse-based Communication Models. In Proceedings of the 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC’14). San Diego, CA, USA. IEEE.
Tutorial: Discourse Models for Interaction Design 225
Kavaldjian, S., Bogdan, C., Falb, J., Kaindl, H. (2008) Transforming Discourse Models to Structural User Interface Models, Models in Software Engineering, MoDELS 2007 Workshops, LNCS 5002, pp. 77–88, Springer, Heidelberg. Kavaldjian, S., Falb, J., Kaindl, H. (2009) Generating content presentation according
to purpose. In Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics (SMC 2009). San Antonio, TX, USA. IEEE. Popp, R., Falb, J., Raneburger, D., Kaindl, H. (2012) A Transformation Engine for Model-driven UI Generation. In Proceedings of the 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS’12). Copenhagen, Denmark. Popp, R., Kaindl, H., Badalians Gholi Kandi, S., Raneburger, D., Paterno, F. (2014)
Duality of Task- and Discourse-based Interaction Design for GUI Generation. In Proceedings of the 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC’14), pp. 3323–3328. IEEE. Popp, R., Kaindl, H., Raneburger, D. (2013) Connecting Interaction Models and Application Logic for Model-driven Generation of Web-based Graphical User Interfaces. In Proceedings of the 20th Asia-Pacific Software Engineering Confe- rence (APSEC’13). ACM. Popp, R., Raneburger, D., Kaindl, H. (2013) Tool Support for Automated Multi-
device GUI Generation from Discourse-based Communication Models, in Proce- edings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS’13). Raneburger, D., Alonso-Ríos, D., Popp, R., Kaindl, H., Falb, J. (2013) A User Study with GUIs Tailored for Smartphones. In In Proceedings of INTERACT 2013. LNCS, vol. 8118, pp. 505–512. Springer, Heidelberg. Raneburger, D., Kaindl, H., Popp, R. (2015) Strategies for automated GUI tailo-
ring for multiple devices. In Proceedings of the 48st Annual Hawaii International Conference on System Sciences (HICSS-48). IEEE. Raneburger, D., Kaindl, H., Popp, R. (2015) Model Transformation Rules for Customization of Multi-device Graphical User Interfaces. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS’15), pp. 100–109. Duisburg, Germany. ACM. Raneburger, D., Kaindl, H., Popp, R., Šajatovi´c, V., Armbruster, A. (2014) A Process
for Facilitating Interaction Design through Automated GUI Generation. In Pro- ceedings of the 29th Annual ACM Symposium on Applied Computing (SAC’14), pp. 1324–1330. ACM. Raneburger, D., Popp, R., Alonso-Ríos, D., Kaindl, H., Falb, J. (2013) A User Study with GUIs Tailored for Smartphones and Tablet PCs. In Proceedings of the 2013 IEEE International Conference on Systems, Man and Cybernetics (SMC 2013). Manchester, UK. IEEE. Raneburger, D., Popp, R., Kaindl, H. (2013) Model-driven Transformation for Opti-
mizing PSMs: A Case Study of Rule Design for Multi-device GUI Generation. In Proceedings of the 8th International Joint Conference on Software Technologies (ICSOFT’13). SciTePress. Raneburger, D., Popp, R., Kaindl, H. (2021) A User Study to Evaluate the Customization of Automatically Generated GUIs. In Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021), Lecture Notes in Networks and Systems, pp. 683–690. Springer, Cham. Raneburger, D., Popp, R., Kaindl, H., Armbruster, A., Šajatovi´c, V. (2014) An Ite-
rative and Incremental Process for Interaction Design through Automated GUI Generation. In Human-Computer Interaction. Theories, Methods, and Tools. HCI 2014. Lecture Notes in Computer Science, vol. 8510. Springer, Cham.
226 Kaindl
Raneburger, D., Popp, R., Kaindl, H., Falb, J. (2011) Automated WIMP-UI Behavior Generation: Parallelism and Granularity of Communication Units, in Proceedings of the 2011 IEEE International Conference on Systems, Man and Cybernetics (SMC’11), pp. 2816–2821. IEEE. Raneburger, D., Popp, R., Kaindl, H., Falb, J., Ertl, D. (2011) Automated Genera-
tion of Device-specific WIMP-UIs: Weaving of Structural and Behavioral Models. In Proceedings of the 2011 SIGCHI Symposium on Engineering Interactive Computing Systems (EICS’11). Rathfux, T., Popp, R., Kaindl, H. (2016) Adding Custom Widgets to Modeldriven GUI Generation. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS’16), Brussels, Belgium. Rathfux, T., Thöner, J., Kaindl, H., Popp, R. (2018) Combining Design-time Genera-
tion of Web-pages with Responsive Design for Improving Low-vision Accessibility. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS’18), Paris.