Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Challenges of Designing a Life-Log Sharing System – The Pensieve

Research Institute for Serious Entertainment, Hanyang University, Seoul, South Korea ↗
Publication year
Digital Object Identifier10.54941/ahfe100595

Abstract

Although the privacy issue is still debated, an incessant interest and attention on the life-log could be explained by its potential applications. Though there are many challenges to realize its potentials, life-log as one’s memory aiding tool has been paid much attention. A series of images captured by the wearable camera are too voluminous and useless to keep the whole data to the collection owner. What the users really want to get from the device is not to review their own whole past life, but retrieve their meaningful moments, further share their meaningful experiences with others. In this regard, the author limited the role of life-log, as resourceful life hints that could give opportunities to reminisce one’s meaningful experience and exchange their experience with others. In this position paper we discuss what is necessary to consider to build a life-log data sharing system for our future research direction, (i) what one’s meaningful experience is; (ii) How the meaningful experience could be defined with life-log data; (iii) How the meaningful data conveys to the 3rd person to exchange and share one’s experience effectively.

INTRODUCTION

Wearable sensing technologies let a small device store the user’s personal information by tracking their daily life in the unobtrusive way. As the user wears the device, several built-in sensors simultaneously record what the user did and how she or he did it to keep personal data, so called life-log data.

The life-log data can be utilized in many applications. For instance, it can notify the user about their health and activity status immediately when it is recorded as an activity tracker (e.g., Fitbit™, Jawbone™, MIO alpha™). It also records every moment that the user faces in their daily life with captured images, audios or videos (e.g., Autographer™, Narrative™, Mecam™). More recently, many research offered the life-log data as a memory extender (“Memex”), which is similar to Vannevar Bush’s assertion (1945), by means of self-examination, reflection, and reminiscence (Byrne and Jones, 2009; Doherty and Smeaton, 2008; Gemmell et al., 2005; Hodges et al., 2006).

These two types of life-log devices have been of much interest in the commercialized application domains, in particular, Health. It led many commercialized products (shown in Figure 1). However, they are more or less used for personal purposes, and appropriate services on the wearable devices are not distinct. For instance, SenseCam™ and Autographer™ do not suggest any unique user experience, compared to the previous digital cameras. It only offers convenience to take a picture without distraction. We therefore discuss in this paper how the visual life-log data (e.g., photos, video) can be used for a new service, i.e., potential experience-sharing channel service, addressing the following three research questions: (i) what one’s meaningful experience is, (ii) what possible factors to define the meaningful experience are, and (iii) how we could convey the owner’s collection of experiences to the 3rd person effectively.

Figure 1. Wearable device as an activity tracker (Left: (a), (b), (c)) and a memory extender (Right: (d), (e), (f)) Personal life-log
Original PDF Figure 1.

As briefly discussed, activity tracker and memory extender have been widely employed in the personal use. As to the activity tracker, it monitors personal health status such as heart rate with the bio-physical sensor (Photoplethysmography; PPG) and informs their activity status with accelerometer, gyroscope sensor, GPS sensor that could figure out the user’s physical status (e.g., how fast and how long they exercise) (Fábián et al., 2008; Lee et al., 2011). Therefore, delivering the accurate and direct information at the right time is a key to meet the user’s needs on the activity tracker.

On the other hand, life-log as one’s memory aiding tool lets the user to recollect their whole life events with series of images, for instance, what they saw and met on July 23th 2014 at Gyeongbok Palace in Seoul, Korea. However, one’s captured daily images are much voluminous, and the opportunities to review their whole collection of images seem useless (Whittaker et al., 2010). Although the visual life-log data should be the primary medium to convey one’s experience, of less importance is to simply deliver the one-dimensional raw data without considering what the experiences mean to the user. As promising computational algorithms, various approaches to organize many images have thus been suggested, such as image recognition (Corcoran and Costache, 2005), time-based clustering techniques (Graham et a., 2002; Platt et al., 2003).

More important, as many researchers have acknowledged is to offer the user to collect life-log data and use it for self-exploration on the meaningful experience (Li et al., 2010). Nevertheless, the current momentum of life-log data application rests on the technical issues such as how to minimize the devices and improve the quality of the life-log data (Gurrin et al., 2008).

Sharing life-log with others Although there are incessant controversy over the privacy issue related to the life-log data, its application is inevitably extended to sharing one’s own experiences with the 3rd persons, following the success of Social Network Service (SNS). A lot of commercialized activity trackers thus connect their own life-log data with SNS such as Facebook™ and Twitter™ or build their own services to set a digital image-sharing playground. It takes a role as an implicit communication channel with others or a bullet board to share their exercise histories with others. It actually leads to boost up the user’s motivation to work out outside. Due to the characteristics of the collected data, which is quantitative, the experiences to share with others are just their records to show off (see Figure 2(a), (b)). Indeed, it does not reinterpret the original data but it directly shows how much and how long the user has exercised.

The visual life-log data is able to support experience-sharing in the community spaces for friends (Memon and Tanaka, 2013), families (Caprani et al., 2011) beyond and over the boundaries of the personal space. A series of empirical study carried out by Crete-Nishihata research team (2012) suggested that digital life-logging technologies Affective and Pleasurable Design (2021) should not be just confined as systems for extending an individual’s capacity for accurate recall of personal events, but instead should support the groups of people who are socially close and tell their life stories to share. Many researchers have thus tried to develop a system to support the way of sharing personal daily life (Caprani et al., 2013; Lee et al., 2008). For instance, “ShareDay” (Zhou et al., 2013) is an automatic life-log classification system based on the type of lifestyle events and derives the social interaction between families. Memon and Tanaka (2013) also suggested an approach to share one’s digital life-log with their friends based on the locational data. The form of the sharing system is not quite different from the personal photo album (except the way how it organized), however.

Figure 2. Left) Activity tracker (endomondo™) (a) personal workout history (b) newsfeed from the friends; Right) Memory Extender (Narrative™) (c) personal photo viewer (d) function to share one’s experience
Original PDF Figure 2.

Most of the commercialized wearable cameras do similar things, simply connecting a Social Network Service to give a way to share their moments with 3rd persons as shown in Figure 2(c), (d). Likewise, the life-log data sharing system is very similar to the personal life-log data album, showing or grouping the series of images in terms of the time and/or location that the images are captured. It does not motivate a significant interaction between the participating users to have empathy or intention to exchange (or share) experiences with others as we believe it is the future of the life-log sharing system. A note of Blažica research team (2013) is further needed here. They claimed that a lot of implicit meanings to the owner such as emotions, mood could not be delivered by a picture itself.

FUTURE RESEARCH DIRECTIONS : VISUAL LIFE-LOG, A PENSIEVE AS A SHARING EXPERIENCE CHANNEL

Figure 3. Pensieve in Harry Porter
Original PDF Figure 3.

Visual life-log data do not re-tread one’s whole experience (Petrelli et al., 2008). Its existing purposes should resort to a medium to keep one’s valuable moments and support to build an experience-sharing system like a pensieve in Harry porter (see Figure 3). It is not for simply viewing and navigating one’s experiences, but for doing reminiscence and understanding one’s and/or others’ past moments for the implicit experiences such as emotions and feelings (Byrne and Jones, 2009). Therefore, the visual life-log could offer new opportunities to the users to accept others’ experiences (i.e., mimesis) or extend it to their own experiences in the process of re-interpretation (e.g., alterity).

In this regard, we formed our main research directions to build an experience-sharing platform for both the experience-transferor who is the owner of the life-log data and the experience-transferee who likes to share the owner’s significant and meaningful moments. To achieve our research goals, we set three steps research questions shown in Figure 4 as below.

Figure 4. Future research directions to build the experience sharing system

Figure available in the original PDF paper.

1. What is one’s meaningful experience to share with others Defining a scene that could be meaningful and valuable for reminiscence or exchanging with others is the first thing to address in this research. Rather than the explicit traits that could determine the experiences in terms of such as location, and time, we want to focus more on the implicit meanings behind the captured moment with qualitative research methodologies. We assume that the sharable experiences are firstly meaningful for the experience owner. Based on the assumption, we would precede the open-ended interview to derive what their meaningful experience are by letting the interviewees recollect their past events (i.e., using thematic apperception test (Murray, 1943)). We would then define whether the meaningful moments are emotionally specific for either pleasant or unpleasant, or open experiences that could be interpreted in different perspectives.

2. Meaningful experience model (MEM)

To deal with the voluminous life-log data, prior studies have applied several approaches with various sensing information. However, they do not give enough implications to analyze one’s valuable life-log data. Based on the previous theme (i.e., what is one’s meaningful experiences that they want to share with others), we would build a computational model to filter out if the experience is meaningful. To do so, firstly, we carry out to identify the likely factors to pinpoint the value of one’s meaningful experiences. We currently apply the physiological data such as Heart rate variability (HRV), Galvanic Skin Response (GSR) and Functional Near-Infrared Spectroscopy (fNIR). In Affective and Pleasurable Design (2021) addition to this quantitative data, several lexical items that are able to represent the user’s intentions to express and convey their experience would be also considered at the verbal protocol sessions. We want to denote the factors that have an influence on generating one’s meaningful experiences regarding both explicit and implicit components.

3. How to convey the meaningful experience to others

In retrieving and building the meaningful experience model from an experience-transferor perspective, we would extend our research to consider the 3rd persons, (i.e., the experience-transferee) who would like to share other’s experiences through the experience-sharing system. The way to convey the experience to others would significantly affect on empathy and active interaction between the experience-transferor and experience-transferee when they exchange and share the experience. We would suggest a way to deliver the experience with two different perspectives, one is with different combinations of media (e.g., images, verbal, emotion) to present the moment (see Figure 5(a)), and the other is with different types of storytelling (e.g., images as an attention unit (Cohn, 2013), point of view, four steps in composition) to recompose the episodes to share (see Figure 5(b)). Depending on what the experience-transferor wants to share with the others, the way to visualize the events should be differentiated. If the experience-transferor wants to share their experience the same as what they felt at that moment, showing the images with additional information (e.g., sound or emotion) would be of greater value. On the other hand, if they want to get empathy with the meaningful happenings (episode), composing a story with the images would be worthwhile to deliver the experience to the transferee. To verify if the characteristics of the experience contents influences on the experience-transferee’s empathy with the experience-transferor, we will measure their physiological data such as fNIR, GSR, which are affected on the stimulus in terms of prefrontal cortex brain activity and emotions to be dimensionally determined, respectively.

Figure 5. Examples of the way to convey one’s meaningful experience; (a) different types of media; (b) storytelling with attention units
Original PDF Figure 5.

CONCLUSION

Although life-log data are suggestively used for the various fields, they are not properly utilized for the potential of experience sharing. We believe that life-log data should be a valuable resource to build an experience-sharing system, not just a memory repository, but for like more on social activity. That benefit is the only way to compensate privacy issue, so the experience holder can voluntarily give their private information for sharing experiences. Life-log itself is one’s personal episodes which is worthwhile to share and exchange with others. It would give opportunities to the others to do mimesis of others’ experiences and interpret it for their own experiential references.

Hence, the implicit meanings behind the life-log data should be regarded to do reminiscences of the meaningful moments and convey the moments to the others with empathy. In this regard, in this position paper, we address our research directions rather than empirical outcomes. Indeed, we are running an experiment on this research direction, and it is very likely to present what we have from this empirical study at the conference.

ACKNOWLEDGEMENTS

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Education, Science and Technology (MEST) (No. 201300000002341).

REFERENCES

Blažica, B., Vladušič, D., & Mladenić, D. (2013). A personal perspective on photowork: implicit human–computer interaction for photo collection management. Personal and ubiquitous computing, 17(8), 1787-1795. Bush, V. (1945). As we may think. Atlunlic Morn, 176(1), 101-106. Byrne, D., & Jones, G. J. (2009). Creating Stories for Reflection from Multimodal Lifelog Content: An Initial Investigation, In

CHI 2009 Workshop on Designing for Reflection on Experience, Boston. Caprani, N., O'Connor, N. E., and Gurrin, C. (2011). Motivating lifelogging practices through shared family reminiscence. Caprani, N., O'Connor, N. E., & Gurrin, C. (2013). Investigating older and younger peoples' motivations for lifelogging with wearable cameras. InTechnology and Society (ISTAS), 2013 IEEE International Symposium on, 32-41. Cohn, N. (2013). The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. A&C

Black. Corcoran, P., & Costache, G. (2005) Automated sorting of consumer image collections using face and peripheral region image classifiers. IEEE Trans Consum Electron 51. 3 (August 2005), 747–754. doi:10.1109/TCE.2005.1510478. Crete-Nishihata, M., Baecker, R. M., Massimi, M., Ptak, D., Campigotto, R., Kaufman, L. D., ... & Black, S. E. (2012).

Reconstructing the past: personal memory technologies are not just personal and not just for memory. Human–Computer Interaction, 27(1-2), 92-123. Doherty, A. R., & Smeaton, A.F. (2008). AutomaticallySegmenting Lifelog Data into Events, Proc. WIAMIS 2008. Fábián, Á., Győrbíró, N., & Hományi, G. (2008, February). Activity recognition system for mobile phones using the MotionBand device. In Proceedings of the 1st international conference on MOBILe Wireless MiddleWARE, Operating Systems, and Applications, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 41. Gemmell, J., Aris, A., & Lueder, R. (2005). Telling Stories with MyLifeBits, In Proc. ICME 2005, 1536-1539. Graham, A., Molina, H. G., Paepcke, A., & Winograd, T. (2002) Time as essence for photo browsing through personal digital

libraries. In JCDL ‘02: Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries. ACM. 326–335. Gurrin, C., Smeaton, A.F., Byrne, D., O'Hare, N., Jones, G., and O'Connor, N. (2008). An Examination of a Large Visual Lifelog. In Proc. AIRS 2008, Harbin, China. Hodges, S., Williams, L., Berry, E., Izadi, S., Srinivasan, J., Butler, A., ... & Wood, K. (2006). SenseCam: A retrospective

memory aid. In UbiComp 2006: Ubiquitous Computing ,177-193. Springer Berlin Heidelberg.

Lee, H., Smeaton, A. F., O’Connor, N. E., Jones, G., Blighe, M., Byrne, D., ... & Gurrin, C. (2008). Constructing a SenseCam visual diary as a media process. Multimedia Systems, 14(6), 341-349. Lee, M. W., Khan, A. M., & Kim, T. S. (2011). A single tri-axial accelerometer-based real-time personal life log system capable

of human activity recognition and exercise information generation. Personal and Ubiquitous Computing, 15(8), 887-898. Li, I., Forlizzi, J., & Dey, A. (2010). Know thyself: monitoring and reflecting on facets of one's life. In Proc. CHI 2010. CHI, ACM, New York, NY, 4489-4492. Murray, H. A. (1943). Thematic apperception test. Memon, M. A., & Tanaka, J. (2013). Sharing Life Experiences with Friends Based on Individual’s Locality. In Design, User

Experience, and Usability. Web, Mobile, and Product Design, Springer Berlin Heidelberg, 706-713.

Petrelli, D., Whittaker, S., & Brockmeier J. (2008). AutoTopography: what can physical mementos tell us about digital memories? In: Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (CHI ‘08). ACM, New York, NY, USA, 53–62. doi:10.1145/1357054.1357065 . Platt, J. C., Czerwinski, M., & Field, B. A. (2003). Phototoc: automatic clustering for browsing personal photographs. In:

Information. Communications and Signal Processing. 2003 and the Fourth Pacific-Rim Conference on Multimedia. Proceedings of the 2003 Joint Conference of the Fourth International Conference on. 1, 6-10. Whittaker, S., Bergman, O., & Clough, P. (2010). Easy on that trigger dad: a study of long term family photo retrieval. Personal and Ubiquitous Computing, 14(1), 31-43. Zhou, L. M., Caprani, N., Gurrin, C., & O’Connor, N. E. (2013). ShareDay: A Novel Lifelog Management System for Group

Sharing. In Advances in Multimedia Modeling, Springer Berlin Heidelberg, 490-492.

Cover of the proceedings volume Advances in Affective and Pleasurable Design

Explore the complete digital volume

Advances in Affective and Pleasurable Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2109-8

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →