Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evaluating Digital Library’s Service Concept and Pre-Launch Implementation

The National Library of Finland, & Aalto University, Helsinki, Finland ↗
Publication year
Digital Object Identifier10.54941/ahfe100253

Abstract

The paper presents a challenging online service concept for culture and science, namely the public interface of the Finnish Digital Library, known as Finna. Its most distinctive features originate from the fact that practically all Finnish libraries, archives, and museums are prospective partners through Finna. From the viewpoint of human-computer interaction (HCI), Finna’s greatest challenges are 1) to design and implement user experience for a heterogeneous target population, and 2) to design and implement a service that merges differing information structures and conventions on the use of libraries’, archives’, and museums’ materials. The paper focuses on four independent usability studies that evaluated the test version of Finna. The studies were planned to cover the most relevant questions and to reveal existing usability problems. Following the current principles of work organization, Finna’s research collaboration partners provided the resources for conducting the studies. The studies’ findings concerned the validity of Finna’s service concept and its implementation. The results indicate that the service may be accepted by its end-users if three issues are resolved: the coverage of content, the representation of materials, and the identity of the service. This will require collaboration between all project partners and Finna’s designers.

INTRODUCTION

Background

The world of public services is changing rapidly. Society requires productivity, and citizens require attractive ser- vices for their taxes. Consequently, libraries, archives, and museums (LAMs) are adjusting their functions to this economic, environmental and social transformation. These organizations have noticed that they share values and customers, which makes it desirable to cooperate across sectors in the delivery of their services.

For more than a decade, digitalization has generated forward-looking cooperation among LAMs. They consider it important to coordinate efforts on all fronts, in order to remain competitive in the evolving environment of online services (e.g., Covey, 2002). These organizations are sharing their expertise and joining forces to develop new types of information-facilitation services for citizens. ‘Digital library’ has become a common term for a range of infra- structures that support and preserve cultural contexts and research (Candela, 2010), regardless of the type of organi- zation providing the content.

This paper presents the concept and implementation of a digital library, which has been developed in nation-wide cross-sector cooperation between libraries, archives, and museums in Finland.

Finna – the public interface of the Finnish Digital Library Finna is part of the Finnish Digital Library (FDL) project, which is funded by the Finnish Ministry of Education and Culture. Its strong mandate ensures that the service can be developed with keen commitment and with consideration for all stakeholders.

Finna brings together the collections of Finnish libraries, archives, and museums. The National Library of Finland is responsible for the development and maintenance of the service. Representatives from participating organizations and stakeholders are involved in devising national digital library strategies, as well as in the operational work. (Hormia-Poutanen et al., 2013.)

Participating organizations can join Finna in two ways. They can publish their material in Finna’s National View, which provides online access to all restriction-free material within Finna’s metadata index. Or they can utilize the Finna software and build their own version of Finna’s interface, also called Organization View (see Image 1). Indi- vidual organizations’ own views can display all material within Finna, or just a limited set. So end-users can access Finna’s content via several interfaces available on the Internet.

Image 1: The family of Finna Views

Finna accomodates all sorts of material and various kinds of metadata formats approved in the FDL Standard Portfo- lio (The National Digital Library – Enterprise Architecture, 2010). Participants will still maintain their own back- end systems, but Finna will gradually replace their current online services. External services enhancing the user ex- perience, such as ontologies or online payment, will also be integrated into Finna (see Image 2). The final spectrum of Finna’s services will be very wide.

Image 2: Finnish Digital Library enterprise architecture

Implementation of Finna The Finnish Digital Library project begun in 2008. The first years were used for planning, i.e. defining the require- ments, establishing the governance model and development methods, and finding the suitable software for the Finna public interface. Following the common policy for LAM sectors, the most prominent library and discovery system vendors were invited to tender. The arduous procurement process was completed, and a choice was made. After the pilot testing phase was extended three times, it became obvious that the chosen software could not meet the require- ments. The contract was cancelled and another approach was taken. In January 2012 our in-house team at the Na- tional Library of Finland began to develop the public interface software.

We at the National Library have been building the Finna service on open source software (OSS) solutions. We are contributing to the OSS community by further developing the core interface module VuFind (see vufind.org/ about.php), and by publishing in-house-developed Finna-specific modules. We are managing the work using agile software development methodology. We consider that these approaches enhance flexibility and provide opportuni- ties for customer-driven innovation (Hormia-Poutanen et al., 2013). These elements also increase the complexity of the work and its outcome.

We published the first running test version of Finna in December 2012, and the first production version in October 2013. This paper discusses the status and actions taken during the 10 months of the test version phase.

Challenges from the viewpoint of HCI Finna, as a project as well as a service, is unique in many ways. The most distinctive features originate from the fact that practically all Finnish libraries, archives, and museums are prospective partners through Finna. The customer base of these organizations ranges from scholars in all academic disciplines, through the general public, to minority groups with special needs and requirements. Furthermore, Finna’s content includes digital materials accessible on- line, but also reference information for the physical collections of archives, libraries and museums.

Although LAMs have similar values, and preserve and distribute similar materials, there are significant differences in the way they serve their customers. Due to historical development, they have divergent methods and practices of collection management, metadata generation, material description, and even service objectives. These practices and policies influence the opportunities for cross-sector collaboration (Zorich et al., 2008). They also influence the expe- rience of a visitor to their premises or online sites.

As collaboration actors LAMs also form a heterogeneous group of resources and knowledge. The central agents in each sector can contribute to a challenging collaboration project on a wider scale than, e.g., a municipal one-man or- ganization managing its’ collections with a spreadsheet application. Collaboration cannot function on good will alone, it needs resources (Zorich et al., 2008). The same is true for collaboration in usability. Some LAM organiza- tions are well-accomplished in the user-centered design and implementation of their online services, but others have no resources to invest in these activities.

From the viewpoint of Human-Computer Interaction (HCI) research, public services may seem less attractive, be- cause governments and public-funding organizations can seldom pioneer ITC development. However, Buie and Murray (2012, xxxv) claim that government systems are an important area for usability and user experience re- search, because these systems affect the everyday life of tremendous numbers of government employees, not to mention citizens. For various reasons (as described in Buie & Murray, 2012), the usability of public systems is not as good as it should be. Further, the public sector is an important area for usability and user experience research, be- cause it can provide insight into profoundly complex systems and services, as well as divergent ways of organizing systems’ design and implementation.

This paper focuses on characteristics of Finna that were distinctive during the short period of development between December 2012 and October 2013. These characteristics can be of interest to the wider community of HCI practi- tioners and researchers in terms of the following:

  • Design and implementation of user experience for a heterogeneous target population.
  • Design and implementation of a service that merges differing information structures and conventions on the use of libraries’, archives’, and museums’ materials.

Although Finna will primarily serve users from its partner organizations, it should later attract new users, too. Finna should finally serve, in principle, every citizen in Finland who has access to the Internet. This means approximately 90% of citizens between the ages of 16 and 74 (Official Statistics of Finland, 2010), with their personal interests and contexts of use. It is acknowledged that designing multiuser systems or services for large populations requires a spe- cial approach (Krippendorff, 2006; Meroni and Sangiori, 2011).

Although organizations can build their own Finna interface, the core of all interfaces is a fusion of the underlying in- formation structures, metadata, conventions of use, and expectations of partners participating in the development. In terms of knowledge engineering (Sowa, 2000), Finna unites several domain models for one purpose.

This report is part of my longitudinal case research that focuses on the organization of usability work in the develop- ment of a digital library. My aim is to examine the change that has occurred in organizational settings, and for this purpose I use Cultural-Historical Activity Theory (see, e.g., Kaptelinin & Nardi, 2006). Following the principles of action research, I aim to study the existing reality at the same time as fostering change in it. My role as the manager and leader of usability activities at the National Library enables this. I hope that the documentation of phases and ac- tions will provide applicable information, firstly for actors involved in the case of Finna, and secondly for others facing similar challenges.

Questions and hypothesis Usability has been one of Finna’s essential development tracks. We have applied user-centered design methods and conducted studies on Finna’s usability from the beginning of the project (Hirvonen and Kautonen, 2011; Kautonen, 2013.) The new approach to build the software in-house at the National Library, using open source software compo- nents, raised the level of usability challenges. Instead of relying on software vendor’s efforts concerning, e.g., good interface usability, we at the National Library had to solve all problems by ourselves. Organizations taking part in Finna were also facing new challenges, because we involve our partners strongly in the development, including us- ability activities.

Between the publications of the test version and the first production version, we needed to study the validity of de- sign and implementation decisions. Our central questions were as follows: How do users perceive Finna as a service? Do they find that it answers to their needs and interests? How should the service be further developed?

The designers and developers at the National Library did not find it easy to define Finna’s service concept. Therefore we assumed that it was confusing for users, too. The implementation surely had some faults and usability is- sues, but the core idea of uniting material from Finnish LAMs to meet user’s needs was viable. We expected that the evaluations would provide proof of user satisfaction sufficient to further develop the service, and that they would point out the areas of conflicting user expectations.

Related research and services The challenge of heterogeneous audiences has long been recognized in the field of HCI. The principles of universal design have emphasized the importance of designing services for everybody, “services that can be used by people of all abilities, to the greatest extent possible, without adaptations” (Erlandson 2008, 5). The market potential of special audiences, e.g. elderly customers, has increased the attention on universal design as a desirable approach (e.g. Gassmann and Reepmeyer, 2011). Specific user groups can be considered through processes of globalization, inter- nationalization, and localization (Marcus and Gould, 2012).

There is ongoing interest in the field of HCI research into systems and services of high complexity. The reasons for complexity can arise from the system itself, its audiences, or the situation in which system is being used. When the system is complex, the needs of different user groups must be acknowledged independently (Albers, 2011a). Gener- ally, the aim is to reduce systems’ undue psychological complexity, and simplify users’ interaction with technology (Thomas & Richards, 2012). e-Research is one of the areas where large-scale, distributed, multidisciplinary, and multi-institutional – i.e. complex – user environments are being studied (Flor et al., 2010).

Users’ perception in the context of digital libraries have been actively investigated (Chowdhury, 2010). Many re- ports describe interesting cases, and point out challenges for digital library design.

There is good evidence and some unanimity concerning effective user experience design and evaluation practices in the context of modern digital libraries and equivalent services. For example, Agosti et al. (2009) have argued for us- ing a combination of implicitly and explicitly collected data for studying digital library users. Fagan et al. (2012) in- dicate similarities between the usability of discovery tools and digital libraries. As well, the subject of new view- points and methodologies has been raised. Khoo et al. (2012) have compared users’ and evaluators’ perceptions of usability, and their implications for usability evaluation. Recently, Kostkova and Madle (2013) have presented a framework for digital library impact evaluation. This discourse about digital libraries is founded on advancements within the HCI discipline.

When compared to other digital libraries, the closest to Finna is the European digital library Europeana (available at europeana.eu), which brings together the collections from European archives, libraries, museums, and galleries. An- other service uniting all types of a nation’s organizations is the Digital Public Library of America (DPLA, available at dp.la). However, both of these include only digital collections, whereas Finna also covers the reference informa- tion about physical collections.

Experiences from such projects and services have provided valuable information for the development of Finna. Eu- ropeana has published studies on its usability activities (Chowdhury, 2010; Dobrewa and Chowdhury, 2010) that are a relevant reference for practitioners in the field. Developers of DPLA are also sharing their everyday experiences of converging digital LAM initiatives in the USA (e.g., Marx & Frick, 2012). Nonetheless, little research has addressed the particular challenges this paper deals with, in the context of digital libraries or similar public services.

METHOD

Four evaluation studies in the spring of 2013 The usability of Finna was evaluated in four individual qualitative studies during the spring of 2013. The following sums up the basic facts of these evaluations, including the 5 dimensions of usability tests according to Dumas and Fox (2012): the purpose of the test, scope of the product tested, location of sessions, presence of test moderator, and level of functionality of the product. The common features describing the conducting of these evaluations are de- scribed first, and the evaluation-dependent facts later. In addition, the reasons behind the decisions are described in order to help in assessing the reliability and validity of these evaluations.

The studies focused on the most relevant questions for service development, and on revealing the existing usability problems. The researchers decided on their methods and the conducting their study independently from others. We at the National Library only provided background information and helped in contacting appropriate test persons.

By combining the results from the evaluation studies I aimed at deepening our understanding of relevant issues in the development of Finna. Each evaluator concentrated on the questions of his/her study, but in this paper I seek an- swers to the central questions.

The four actors conducting the studies The resources allocated for usability activities determined the execution of evaluations in 2013. We had to outsource all evaluations, and could purchase only one of them from a commercial procurement partner. One of the principles of the new approach has been the establishment of a strong collaboration network with different stakeholders. As a consequence, two Finnish universities hosting research in usability showed interest in Finna in 2013. Contracting students into Finna’s usability evaluation significantly reduced our expenses. We also valued the viewpoints and paradigm of their affiliation universities. Respectively, we took into account that the risks of the timing and quality of the students’ work might not equal that of professionals.

Two of the evaluations inspected the Finna’s National View, and two focused on the first running Organization View, namely the view of the Jyväskylä University Library (JYU View). Both views were tested online, by access- ing the test versions available on the Internet. There were no other operational views at that time. The tested views had three major differences. Firstly, the JYU View contained material only from the Jyväskylä University Library (JYU), whereas the National View displayed all released material. Secondly, the licensed electronic publications and databases of the JYU were integrated into their own view, not the National View. Thirdly, the appearance of each view was distinctive, although they both had the same basic layout and features.

We thought it was important to study both views. The National View was the root of all other views, and therefore examining it provided relevant information concerning the usability of all other views (see Image 3). On the other hand, the target users of the JYU View were the students, researchers and staff of the university. Together they rep- resent one of the main target groups of the Finna service. Therefore it was interesting to find out their expectations and experiences.

Image 3: The home page of Finna’s National View (left) and the Jyväskylä University Library View (right) Depending on the acquisition methods used, each evaluation was based on different kinds of data. The source mate- rial varied from interview scripts and notes to video recordings and screen shots. The researchers analyzed their data and used appropriate methods according to their own judgment. Only a fraction of the original field data was deliv- ered to the developers at the National Library. Each study report included a list of findings, and suggestions for overcoming the expressed usability problems. The fourth study also provided some exemplary source code for cor- recting user interface discrepancies.

Only two of the evaluation reports have been published (Heinänen, 2013; Kuuskoski, 2013), but all essential facts can be found in Table 1. Hereafter, I refer to each study with an identification code indicated in the table. The fol- lowing chapters give an overview of each study and its research settings.

Evaluation 1: Usability test of Finna’s National View The first test was conducted to evaluate the overall usability of the test version of the Finna’s National View. The study included all available functionalities on the interface. The evaluation was conducted by the experienced us- ability company Adage Ltd., which provides user-centered design and usability research services to its clients in business and the public sector. Adage is a permanent service supplier for the National Library.

The evaluation was a conventional usability test, which included interviews and user tests. The tests were carried out at the premises of Adage and monitored by usability experts from the company and the commissioner. The test con- sisted of roughly formulated tasks that followed participants’ individual interests, and tested the basic functionali- ties: search, narrowing the search, viewing the results, and saving the search results.

The participants were recruited by Adage. The National Library defined the criteria for the target group, and took part in planning the test and the relevant tasks. There were 6 participants of both genders, who represented elderly non-academic history researchers, e.g. family history hobbyists. None of them knew about Finna before the test.

Evaluation 2: User test of the JYU View The second evaluation was conducted to evaluate the first running Organization View of Finna, namely the test ver- sion of the JYU View. The aim was to find existing usability issues in the natural context of JYU View users. The study was planned and conducted by a graduate student from the University of Tampere. The data were collected in interviews and the user tests were conducted on the premises of the Jyväskylä University Library. The library helped in recruiting participants from among the University’s students and researchers.

Altogether 7 persons participated from three different disciplines of the university. All were experienced users of on- line library services, but they had no previous experience with Finna.

The participants were asked to complete the test following their own study or research interests. The assignments covered the natural flow of information retrieval, and explored the use of information. The test included interviews and discussions during the assignments, which gave the researcher more insight into the subjects’ thinking.

Evaluation 3: Heuristic evaluation of the JYU View The third evaluation was conducted to complement the other evaluation on the test version of the JYU View. The aim was to focus on the user interface and find all potential usability problems. Searching for licensed electronic publications was excluded from the evaluation. The evaluation was conducted as an expert evaluation. The evaluator was a graduate student from the University of Tampere, who had enough experience in informatics and online ser- vice development, as well as library informatics, to be considered an expert.

The evaluator used four sets of heuristics: three general online service heuristics and one designed for online public libraries. The reason for using this many was to test all features and detect all problems in the Finna interface.

Evaluation 4: Accessibility test of Finna’s National View The last evaluation was conducted to assess the accessibility and usability of Finna’s National View for visually im- paired users using a screen reader. The evaluation was conducted by a group of graduate students at Aalto Univer- sity as part of their studies. The Finnish Library for the Visually Impaired helped in recruiting the participants. Alto- gether 7 persons took part in either one or two user tests during the study. The tests were conducted in the users’ own environment, using their own devices to access Finna.

The user tests were complemented with a heuristic evaluation, background interviews, and questionnaires investigat- ing the user experience and validity of the test results. The evaluators also made a software model to evaluate the validity of their findings. The model was a valuable source of information for Finna’s developers.

Table 1: The facts of the four usability evaluation studies on Finna carried out in spring 2013 T I C ID Time when conducted Affiliation Target User group involved Location of sessions Applied data acquisition methods
Original PDF Table 1:

RESULTS

Overview The results of the evaluation studies were reported to the developers at the National Library and to Finna’s Usability Working Group.

The number of identified usability issues from all evaluations was nearly 200. However, many of these overlapped, and some were mentioned in all four reports. Each study indicated the severity of the findings, pointing to issues in need of prompt attention. The number of distinct individual issues of a ‘severe’ or a ‘critical’ nature was 31.

The three evaluations involving end-users (E1, E2, E4) included interviews or questions that inspired the test sub- jects to comment openly. The number of individual comments was 28. The comments provided important informa- tion about users’ thinking.

The heuristic evaluation of the JYU view (E3) revealed issues mostly concerning the user interface implementation. The descriptions included either suggestions or direct recommendations about how these issues could be resolved. As well, the three other studies indicated interface-related issues, and remedies.

Relevant findings The results from all four evaluation studies showed that the service concept is not clear to users. Several findings in- dicated that the way the search was targeted, or the way the search results could be filtered, was not intuitive enough (E1, E2, E3, E4). On the other hand, users familiar with online library systems were very effective in information re- trieval and could easily comprehend and even appreciate the basic functionality of Finna (E2). However, the expert evaluator expressed confusion in a case where the external service, i.e. the licensed material database search, functioned differently from Finna (E3).

The users’ prior knowledge influenced the way they used, or did not use the functionalities provided (E2). There were also some pleasant surprises (E4). On some occasions the evaluations revealed expectations that, in fact, were not dependent on Finna or its implementation. Such expectations concerned storing favorites (E1), searching the background database (E2, E3), or external services (E2). Results from evaluations of the JYU View (E2, E3) strongly suggested a need for effective use. The evaluators complained when quick and effective use was compro- mised because of some interface feature or functionality.

The results indicated that features originating from the back-end systems, integrated services or restricted materials influence users’ experience of Finna (E1, E3). These features are not part of or caused by Finna’s software, but rather users experience them as part of it. Also the look and feel of Finna was compared to the current systems. Some users expressed that the appearance of the interface related to a feeling of trustworthiness and reliability – or the opposite (E2).

The last evaluation pointed out that the usable accessibility of the system was dependent on users’ skills in the auxil- iary technology, i.e. the screen reader (E4). One particular finding from the fourth evaluation (E4) was that the inter- face code, which was more accessible with screen readers, was considered altogether better for sighted users as well.

All results contained indications of poor interface terminology. Some of these terms were user interface terminology (E1, E2, E3); others were abbreviations, keywords or other metadata originating from the back-end systems (E1, E4). Users found some field titles confusing. They found unexpected or even erroneous metadata in a particular type of field (E1, E4). There were also complaints that the available metadata was in discrepancy with a particular con- text of use (E1, E3). The quality of the metadata was considered as a meaningful element for use.

The findings included several notes concerning access to the material, in other words, Finna’s content. The users testing the National View (E1) expressed enthusiastic first impressions, but this changed when they discovered that the service included only a fraction of the material in Finnish libraries, archives, and museums. Users expected to get direct and easy access to the material that was relevant to them (E1). The users testing the JYU View (E2, E3) indicated wanting to see the collections from other libraries, if the desired publication was not accessible in their home library. The JYU View provided access only to the material of the Jyväskylä University Library. The users testing the National View indicated dissatisfaction when they found that all of the material was not available online (E1). However, users testing the JYU View did not indicate such a need and were satisfied with the reference infor- mation they found (E2).

CONCLUSIONS

Concerning the method and conducting the evaluation studies In the development team at the National Library of Finland we assumed that Finna’s service concept was confusing for its users. Four independent usability evaluation studies were conducted in order to validate the service concept and its implementation. These studies approached the usability of the Finna service from different viewpoints and used a variety of data acquisition and evaluation methods. The developers and designers at the National Library ap- preciated the results and quickly used them to improve the interface and other system features. Thus, the evaluation studies succeeded in showing further development options for the service.

We at the National Library had little control over how studies were conducted, because they were outsourced to our partners. This caused a degree of variation and discontinuity, and therefore their results were not comparable. None of the evaluations could look deep into the users’ needs and interests. These tests merely evaluated the access to the information, not its usefulness in real situations or for longer period of time. As Albers (2011a) argues, information salience is one of the necessary elements of an effective system.

Due to the long timespan of the project, as well as unforeseen changes of direction, limited resources, and changes in organization, there has been no opportunity to apply the fully controlled approaches. The user experience design, implementation, and evaluation practices of Finna may not be systematic, but similarities with the ecological approach defined by Still (2011) can be seen.

From the viewpoint of the usability activities’ management, I consider the results of the test phase to be satisfactory. Instead of a single evaluation we can benefit from the findings of four studies. The studies provided enough material to answer the central questions, and the users’ perceptions of Finna were mainly positive.

Concerning the challenge of a heterogeneous population One of the most important tasks in an innovative public service project, such as Finna, is expectations management. This requires constant dialogue with stakeholders, of whom the end-users should be among the first.

There will be different expectations about Finna as a service. Some users will only browse available contents, whereas others will demand a specialized tool for professional use. Interestingly, Chowdhury (2010) has noted that the same information in a digital library may be accessed, understood and used differently by users in different con- texts. Albers (2011b) emphasizes that the various needs of user groups should not be overlooked, and that the design should not be simplified to the lowest information need.

The evaluation studies reported in this paper were addressed to potential users with different backgrounds and dif- ferent needs. The results showed that the test version of Finna was not entirely what these people expected. Features familiar from existing systems were considered easy and quite usable, but there were also expressions of dissatisfac- tion. A negative experience may discourage further use of a new service. These conclusions are consistent with stud- ies on Europeana and The European Library (Dobreva and Chowdhury, 2010; Agosti et al., 2010). A good impres- sion of a service can be enhanced with refined implementation, but it is essentially an issue of design.

Concerning most parts – or elements, as Garrett (2011) says – user experience is created on a designer’s desktop. The evaluation of Finna’s accessibility for visually impaired users also showed the importance of implementation. Furthermore, the results implied that a service well implemented, in this case, produced with correct application code, serves all user groups better.

Independent of the context of use, it seems that people consider a digital library to be as good as its contents. The evaluation studies on Finna revealed many indications supporting this assumption. Dobreva and Chowdhury (2010) have also argued, that it is essential to resolve content issues alongside the usability issues. Thus, the coverage of content within Finna seems a crucial requirement for the success of the service.

Concerning the challenge of merged services The service concept of Finna is a combination of current library, archive, and museum use scenarios. Finna’s partner organizations know their patrons well, and therefore Finna relies on their specifications about information structures and conventions. However, the results of our evaluation studies proved, that the new interface revealed the short- comings of these presumptions. It seems obvious, that something has to be changed in the way the metadata, the in- terface terminology, and other conventions are represented.

LAMs in Finland apply different metadata standards that are described in the NDL Standards Portfolio (The Na- tional Digital Library – Enterprise Architecture, 2010). Although harvesting the diverse metadata and mapping them to the system requires resources, it is feasible. But it will require much effort before the material from LAMs can be displayed via one interface in a usable manner.

The results of Finna’s evaluation studies also indicated a need to reconsider the terminology and expressions dis- played to users. It is a challenge for interaction designers to abandon the conventions inherited from established or- ganizations and existing systems. According to Krippendorff (2006), it is a designer’s fundamental duty and right to adjust the design thinking towards users’ actual and future contexts of use.

New policies and the transformation of entire work processes are required in order to change the representation of material. The current cooperation through the Finna consortium provides solid ground for such change. According to Buxton (2007), a cooperative approach could be desirable in design work. This suggests partners being involved in activities that are now the sole responsibility of the development team at the National Library. To cover all aspects of representation, Finna should also involve proprietary software partners in collective design problem-solv- ing.

These findings could be used as an argument against the original idea of Finna, i.e. merging services from divergent application domains into one interface. However, the option to implement an Organization View and modify it to lo- cal needs actually enables such an aspiration. Library professionals in America have expressed similar concerns about hybrid services for academic and public use, addressing the case of the Digital Public Library of America (Rothman, 2011; Sydell, 2013). A user-centered approach, however, criticizes hasty conclusions. If there are users who desire primary material from different content providers, they should have an opportunity to access the material using some kind of a single search (Palmer et al., 2009; Perescott & Erway, 2011). Too often, users must face the different representation models and conventions, and adjust their actions accordingly. Once Finna has been launched, the users’ actual behavior together with this challenge will be studied further.

Finally, one more aspect concerning merged services was indicated in the evaluation studies. The participants did not necessarily see, which feature or functionality was part of Finna. They did not identify the boundaries of the Finna service. This concern is related to the reliability of the service, which was also indicated in the evaluation re- sults. The users’ trust in libraries, archives, and museums must not be lost. According to Kuniavsky (2003), person- ality and attraction are one of the central aspects of creating a good user experience. On the basis of this assertion, the identity of the Finna service should be further designed and defined.

Future work

The four evaluation studies revealed findings pertaining to the validity of Finna’s service concept and its implemen- tation. Most of the issues concerning the implementation that the findings raised were rather straightforward, and could be resolved before the launch of the first production version. However, there were also findings that pointed out more fundamental issues in the information design and service concept of a cross-sector digital library.

The results indicate that the service may be accepted by its end-users, if the following three issues are resolved:

  • Coverage of content,
  • Representation of materials, and
  • Identity of the service.

It is important to recognize that none of these challenges will be overcome only with better design and implementa- tion, i.e., the work done at the National Library of Finland. They require the involvement of LAM organizations tak- ing part in Finna, other partners, and stakeholders. Sandford and Doulton (2012) state that improving users’ experi- ence of government services requires a profound transformation of delivery processes. The same will be required from services provided by other public organizations, such as LAMs. Shifting the focus from organizations’ cultures to users’ cultures can ultimately change the policymaking and lead to a deeper digital convergence of LAMs.

This paper was limited to a short period of the Finna project, focusing on two challenges. Along with the progress made, more work and studies will be done, and other user groups will become involved. We will focus on Finna’s actual use once the service has been launched for the public. We also intend to review the process of how user expe- rience design and an agile software development method have been intertwined. The development of Finna contin- ues until 2016, and so does the work on its usability.

My research on organizational settings will hopefully also reveal other kinds of transformations. By paying attention to the case of Finna, its actors, activities, distribution of work, and other elements of the activity framework, I hope to introduce the available opportunities to developers of public services in their challenging circumstances.

ACKNOWLEDGMENTS

First and foremost I want to thank Marjo Haahtela, Hanna Heinänen, Aino Jakobsson, Reeta Kuuskoski, Piia Sevón, Emil Virkki and Jari Väänänen. This paper is based on their contributions. I also thank my colleagues at Aalto Uni- versity, as well as Adage Ltd., the University of Tampere, the National Library of Finland, the Usability Working

Group, and all others with whom I have had the honor to collaborate with in this undertaking.

REFERENCES

Agosti, M., Crivellari, F., & Di Nunzio, G. M. (2009). Evaluation of Digital Library Services Using Complementary Logs. ACM Special Interest Group on Information Retrieval 2009. Agosti, M., Crivellari, F., Di Nunzio, G. M., & Gabrielli, S. (2010). Understanding user requirements and preferences for a digi-

tal library Web portal. International Journal on Digital Libraries, 11(4), 225–238. Albers, M. J. (2011a). Usability and Information Relationships: Considering Content Relationships and Contextual Awareness When Testing Complex Information. In M. J. Albers & B. Still (Eds.), Usability of Complex Information Systems. Evalua- tion of User Interaction (pp. 109–131). Boca Raton, FL, USA: Taylor & Francis. Albers, M. J. (2011b). Usability of Complex Information Systems. In M. J. Albers & B. Still (Eds.), Usability of Complex Infor-

mation Systems. Evaluation of User Interaction (pp. 3–16). Boca Raton, FL, USA: Taylor & Francis. Birrell, D., Dobreva, M., Ünal, Y., & Feliciati, P. (2010). Constituencies of Use: Representative Usage Scenarios in International Digital Library User Studies, a Case Study on Europeana. Proceedings of The International Conference on Electronic Pub- lishing 2010. Buie, E. & Murray, D. (2012). Usability in Government Systems. User Experience Design for Citizens and Public Servants.

Burlington, MA, USA: Elsevier. Buxton, B. (2007). Interactive Technologies. Sketching User Experiences. Getting the Design Right and the Right Design.

Burlington, MA, USA: Elsevier. Candela, L. (Ed.). (2010). The Digital Library Reference Model. DL.org Project Deliverable. http://www.dlorg.eu/uploads/ DL%20Reference%20Models/The%20Digital%20Library%20Reference%20Model_v1.0.pdf Chowdhury, G. (2010). From digital libraries to digital preservation research: the importance of users and context. Journal of

Documentation, 66(2), 207–223. Covey, D. T. (2002). Usage and Usability Assessment: Library Practices and Concerns. Washington D.C., USA: Digital Library Federation. http://www.clir.org/pubs/reports/pub105/pub105.pdf Dobreva, M., & Chowdhury, S. (2010). A User-Centric Evaluation of the Europeana Digital Library. International Conference

on Asian Digital Libraries 2010, Lecture Notes in Computer Science, 6102, 148–157. Dumas, J. S. & Fox, J. E. (2012). Usability Testing. In J. A. Jacko (Ed.), The Human-Computer Interaction Handbook. Fundamentals, Evolving Technologies, and Emerging Applications (3rd ed., pp. 1221–1257). Boca Raton, FL, USA: Taylor & Francis. Erlandson, R. F. (2008). Universal and Accessible Design for Products, Services, and Processes. Boca Raton, FL, USA: Taylor

& Francis. Fagan, J.C., Mandernach, M., Nelson, C.S., Paulo, J.R., & Saunders, G. (2012). Usability Test Results for a Discovery Tool in an Academic Library. Information Technology and Libraries, 31, 83–113. Flor, de la G., Jirotka, M., Lloyd, S., & Warr, A. (2010). Embedded e-Research Applications: Designing for Usability. In W. H.

Dutton & P. W. Jeffreys (Eds.), World Wide Research. Reshaping the Sciences and Humanities, (pp. 135–152). Cambridge, MA, USA: MIT. Garrett, J. J. (2003). The Elements of User Experience. User Centred Design for the Web. New Riders, Indianapolis, USA: New Riders. Gassmann, O., & Reepmeyer, G. (2011). Universal Design – Innovations for All Ages. In F. Kohlbacher & C. Herstatt (Eds.) The

Silver Market Phenomenon. Marketing and Innovation in the Aging Society (2nd ed., pp. 125–140). Heidelberg, Germany: Springer. Heinänen, H. (2013). Digitaalisen kirjaston käytettävyyden arviointi: JYKDOK-Finnan käytettävyystutkimus. (Master’s thesis.)

Tampere, Finland: University of Tampere. Hirvonen, V. & Kautonen, H. (2011). Focus on Users in the Open Development of the National Digital Library of Finland. Paper presentation at the Digital Humanities Conference 2011, Hamburg, Germany. Hoek, van der A. & Petre, M. (2014). Introduction. In A. van der Hoek & M. Petre (Eds.), Software Designers in Action. A Hu-

man-Centric Look at Design Work (pp. xvii–xxviii). Boca Raton, FL, USA: Taylor & Francis. Hormia-Poutanen, K., Kautonen, H., & Lassila, A. (2013). The Finnish National Digital Library: a national service is developed in collaboration with a network of libraries, archives and museums. Insights, 26 (1), 60–65. Kaptelinin, V. & Nardi, B. A. (2006). Acting with Technology: Activity Theory and Interaction Design. Cambridge, MA, USA:

MIT. Kautonen, H. (2013). Making It Everyone’s Finna – Cross-Sector Collaboration and User Experience Design in a Digital Library.

Ext. Abstracts, HCI International 2013, Communications in Computer and Information Science, 374, 440–444. Khoo, M., Kusunoki, D., & MacDonald, G. (2012). Finding Problems: When Digital Library Users Act as Usability Evaluators.

Hawaii International Conference on System Sciences, 1615–1624. Kostkova, P. & Madle, G. (2013). What impact do healthcare digital libraries have? An evaluation of a national resource of infection control at the point of care using the Impact-ED framework. International Journal of Digital Libraries, 13, 77–90. Krippendorff, K. (2006). The Semantic Turn. A New Foundation for Design. Boca Raton, FL, USA: Taylor & Francis. Kuniavsky, M. (2003). Observing the User Experience. A Practitioner’s Guide to User Research. San Francico, CA, USA: Mor-

gan Kaufmann.

Kuuskoski, R. (2013). Yliopistokirjastojärjestelmän käytettävyyden arviointi. Heuristinen evaluointi ja heuristiikkajoukkojen vertailu. (Master’s thesis.) Tampere, Finland: University of Tampere. Marcus, A. & Gould, E. W. (2012). Globalization, Localization, and Cross-Cultural User-Interface Design. In J. A. Jacko (Ed.),

The Human-Computer Interaction Handbook. Fundamentals, Evolving Technologies, and Emerging Applications (3rd ed., pp. 341–366). Boca Raton, FL, USA: Taylor & Francis. Marx, M. & Frick, R. (2012). Large scale digital convergence - the practical level: the Digital Public Library America as an example. Working Group presentation at the IFLA Conference 2012, Helsinki, Finland. Meroni, A. & Sangiori, D. (2011). Design for Services. Farnham, Surrey, GB: Gower Publishing. Official Statistics of Finland (OSF) (2010.) Use of information and communications technology by individuals. Helsinki,

Finland: Statistics Finland. http://tilastokeskus.fi/til/sutivi/2010/sutivi_2010_2010-10-26_tie_001_en.html Palmer, C. L., Teffeau, L. C., & Pirmann, C. M. (2009). Scholarly Information Practices in the Online Environment. Themes from the Literature and Implications for Library Service Development. Dublin, Ohio: OCLC Research. http:// www.o- clc.org/programs/publications/reports/2009-02.pdf Prescott, L. & Erway, R. (2011). Single Search: The Quest for the Holy Grail. Dublin, Ohio: OCLC Research. http://www.o-

cls.org/research/publications/library/2011/2011-17.pdf Rothman, D. (2011, April 4). On Robert Frost, Fences, and Electrons: Why We Need Two Separate Digital Library Systems for Academics and The Rest of America—And Content Exchanges and Other Neighborliness. [Online article.] LibraryCity. http://librarycity.org/?p=945 Schaffner, J. (2009). The Metadata is the Interface. Better Description for Better Discovery of Archives and Special Collections,

Synthetized from User Studies. Dublin, Ohio: OCLC Research. http://www.oclc.org/programs/publications/reports/2009- 06.pdf Sowa, J. F. (2000). Knowledge Representation. Logical, Philosophical, and Computational Foundations. Pacific Grove, CA, USA: Brooks/Cole. Still, B. (2011). Mapping Usability: An Ecological Framework for Analyzing User Experience. In M. J. Albers & B. Still (Eds.),

Usability of Complex Information Systems. Evaluation of User Interaction (pp. 89–108). Boca Raton, FL, USA: Taylor & Francis. Sydell, L. (2013, August 19). Combining The Nation's Digitized Libraries, All In One Place. [Online article.] NPR All Tech Considered. http://www.npr.org/blogs/alltechconsidered/2013/08/19/213498478/combining-the-nations-digitized-libraries-all- in-one-place The National Digital Library – Enterprise Architecture. (2010, v1.0). [Project report.] Helsinki, Finland: The National Digital Li-

brary project / Ministry of Education and Culture. http://kdk.fi/images/stories/tiedostot/NationalDigitalLibrary_ EnterpriseArchitecture.pdf Thomas, J. C. & Richards, J. T. (2012). Achieving Psychological Simplicity. Measures and Methods to Reduce Cognitive Complexity. In J. A. Jacko (Ed.), The Human-Computer Interaction Handbook. Fundamentals, Evolving Technologies, and Emerging Applications (3rd ed., pp. 491–513). Boca Raton, FL, USA: Taylor & Francis.

Cover of the proceedings volume Advances in The Human Side of Service Engineering

Explore the complete digital volume

Advances in The Human Side of Service Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2091-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →