Building High Usability Consumer-oriented Linked Data Mobile Apps

Ying Wang and Rob Brennan

ADAPT Centre, Trinity College Dublin, College Green, Dublin 2, Ireland.

{yiwang, rob.brennan}@scss.tcd.ie

Abstract. This paper identifies good practice for building high usability Linked Data mobile apps, demonstrates these practices through a prototype app for a commercial service and evaluates the usability achieved. The state of the art in mobile Linked Data technologies and design guidelines for mobile search user interfaces were reviewed and analyzed to define good practice. This paper addresses the lack of research on mobile applications using Linked Data and consumer-oriented applications of Semantic Web technology. The user study consisted of a lab-based experimental investigations and surveys of real customers. The experiments showed that it is possible for a consumer-oriented Linked Data mobile application to achieve high usability, comparable to a commercial web-based solution on desktop devices. This provides evidence that our categorization of good practices in mobile Linked Data app design has broader applicability for mobile Linked Data applications.

Keywords: Linked Data, Human Computer Interaction in AI.

Introduction

Mobile applications (apps) continue to dominate consumer use of computers, however advanced computing fields tend to focus on the needs of technical users and ignore concerns such as usability [21]. Consumers need easy to use services like product matching to find products related to other products, e.g. product accessories like power supplies. Recommender systems based on past purchases are not enough to get high quality matches since the details of matching technical specifications are key. However, collecting and maintaining sufficient relevant data to support these product matching applications is challenging.

The leading web-based, standards-based data publishing and management technology is Linked Data based on the Semantic Web. Linked Data provides a large source of structured knowledge on the web, but it is still very hard for non-specialists to consume [7] and mobile apps based on Linked Data are rarely studied [20]. The Linked Data approach can be applied in mobile application development, but it also implies unique challenges for how to design a well-integrated mobile system with a high usability interface that is suitable for typical web users.

This paper investigates the research question “To what extent can consumer-oriented, product matching mobile applications based on Linked Data achieve high usability?”. The challenge here is to hide the complexity of the underlying data while providing a high usability interface on a limited resource mobile application. The technical approach is to survey the literature to find the recommended architectural patterns for Linked Data apps and good practice mobile interface design elements for searching and matching and to apply these recommendations for a real-world use case provided by MyVolts Ltd., an Irish online retailer, to build a prototype product matching app based on Linked Data and then to evaluate it through lab trials, usability interviews and questionnaires. The performance of the prototype app developed for the android platform is compared to a commercial web-based product matching application already deployed by MyVolts Ltd.

The contribution of this paper is a description of a real-world Linked Data mobile application, good practice guidelines for designing high usability Linked Data apps and experimental evidence of the effectiveness of the app design.

The rest of this paper is structured as follows: Section 2 describes the MyVolts Linked Data app use case, Section 3 surveys good practice in Linked Data apps and usability design, Section 4 describes the iterative design of the MyVolts app built to test the guidelines, Section 5 evaluates the app through three experiments and finally Section 6 provides some conclusions.

Use Case: MyVolts Power Supply Matching App

MyVolts is an online power supply retailer which provides a wide range of replacement power supplies and cables for consumer electronics devices. Making it easy to find the right power supply for a customer’s device is key to increasing sales. However, finding the correct power supply for a customer’s device is complex due to the large number of consumer electronics devices, the number of variations in power supplies (e.g. tip shape, diameter, length and polarity is just one of 26 dimensions tracked) and the way manufacturers produce many variations of each device over time based on performance improvements and component costs. As the underlying datasets are large, complex, evolving and based on distributed updates from individual manufacturers, theoretically a Linked Data approach to data management can provide a way to unify this sparse, heterogeneous data. A native Linked Data client mobile app could then improve efficiency of access to the combined datasets for data consumers. However most Linked Data and Semantic Web applications are designed for technical users rather than typical users on the web so such an app is unlikely to have high usability for the target users.

MyVolts currently provides a web-based solution which offers users the ability to find the correct power supplies and cables through a web browser. It provides the functionality to allow users to find the matched product based on their device information by entering queries in the search box or selecting items from drop down boxes. Users are also able to refine the search on the search results page. However the complexity of the datasets and search filters requires user interface elements that are unsuited to mobile devices and a dedicated app would have the benefits of acting as a communications channel to customers and could host a dedicated shopping cart optimized for mobile use.

The requirements for the current work are to build a high usability consumer-oriented Linked Data mobile app for power supply matching on a native android mobile platform. This builds on a Linked Data infrastructure for power supply data collection which uses the Ontop [1] platform to dynamically create a Linked Data dataset from the MyVolts relational database of power supplies and device technical specifications. This Linked Data is fused with other market intelligence such as “top seller” lists from Amazon and the resultant datasets are available to the mobile app. Figure 1 presents the deployment context for the app from the Ontop documentation [1]. Ontop is an open source OBDA (Ontology-Based Data Access) system, which exposes relational databases as virtual RDF (Resource Description Framework) graphs by linking terms in the ontology to the data sources through sets of declarative mappings. It provides a SPARQL remote endpoints to allow the MyVolts mobile client to explore and query datasets by sending a SPARQL HTTP request.

Fig. 1. Overall Architecture of Ontop- the Ontology-Based Data Access Platform used to
support the MyVolts Linked Data, based on Calvanese et al. [1]

Good Practice in Mobile Linked Data Usability Design

Considering there was no existing Linked Data mobile apps related to product matching [20], this section discusses existing Linked Data mobile applications to define the design of MyVolts mobile app technical architecture. In addition, the state of the art mobile search design patterns is also reviewed and analyzed in order to identify the app search user interface design patterns and requirements.

Linked Data Mobile Apps

We review typical examples of web-user oriented Linked Data mobile apps in this section. DBpedia Mobile [2] is a location centric client application which extracts information from DBpedia [3] and presents it on a map. It builds on a client-server architecture, which provides the functionality for the client application to search, retrieve, and store data. mSpace Mobile is a Semantic Web application that provides relevant information in a selected knowledge domain to mobile users based on their location. It builds on a distributed client-server-based architecture, in which the mobile client accesses the knowledge of domains and relevant information by querying the corresponding repositories using SOAP (Simple Object Access Protocol) and HTTP protocol. OntoWiki Mobile [4] is an open source HTML5 application which supports a wide range of usages. It provides offline usage by implementing a persistent local storage through caching data at the client side. Who’s Who [5] provides the functionality to allow users to access the Linked Data cloud. It uses a client-server architecture. The client retrieves and stores the result triples in a local RDF store, then the interaction can be done by executing local SPARQL queries for the further filtering of the available information. GetThere [6] provides real-time information about public transport for rural areas. GetThere also followed client-server architecture and it doesn’t provide local storage and filter functionality or offline usage.

Table 1. Comparison of Native Linked Data Mobile Apps.

Features	DBpedia Mobile	mSpace Mobile	OntoWiki	Who’s Who	GetThere	MyVolts Mobile
Client-Server	Yes	Yes	Yes	Yes	Yes	Yes
SPARQL	Yes	Yes	Yes	Yes	Yes	Yes
Local Storage & Filter	No	No	Yes	Yes	No	Partial
Offline Usage	No	No	Yes	No	No	No

Table 1 presents the comparison of these Linked Data mobile apps. It shows that all the presented Linked Data mobile apps adopted a client-server architecture, so this approach will be also selected by MyVolts mobile app. OntoWike Mobile and Who’s Who supports the local storage and filters in order to improve the performance of the mobile applications by resolving the latency and network unreachable problem. However, considering the mobile memory is limited, so the large amount of local storage and filters should be avoided when designing MoVolts mobile app. Therefore, MyVolts mobile application will only design and implement the cache function for a small amount of information. In addition, OntoWiki mobile also supports offline usage. However, an offline search may not be able to locate the expected record even if it exits, which means this approach may not be appropriate for commercial use. Therefore, MyVolts mobile app won’t implement offline usage feature.

Search User Interface Design Patterns

Search is the fundamental activity of the mobile application, good search patterns can allow users to search effectively and effectively [7]. So there is no doubt that it is beneficial to review the good practices regarding to the mobile search design patterns. Through browsing from Google Scholar, there are three top search design guidelines recommended to be applied into mobile user interface design including Nudelman’s android search design patterns [8], Neil’s search design patterns [9], and Morvillie & Callender’s search design patterns [10]. As these search design patterns may have different terminologies for the same pattern, so the terminology for each specific pattern was unified. Table 2 illustrates the comparison between three design patterns, the right column presents the design patterns which are selected for MyVolts mobile application.

Table 2. Comparison of Search UI Design Patterns from Literature and Identification of Patterns Selected for the MyVolts Mobile App.

Patterns	Nudelman’s	Neil’s	Morvillie & Callender’s	MyVolts
Auto-Completion	Yes	Yes	Yes	Yes
Pull to Refresh	Yes	No	No	No
Tap-Ahead	Yes	No	No	No
Voice Search	Yes	Yes	No	Yes
Barcode Search	No	Yes	No	Yes
Search History	No	Yes	Yes	Yes
Search Form	No	Yes	No	No
Refine Search	Yes	Yes	Yes	Yes
Scoped Search	No	Yes	No	No
Lazy Loading	No	Yes	Yes	Yes
Personalization	No	No	Yes	No
Best First	No	No	Yes	No
Faceted Navigation	No	No	Yes	No
Structured Results	No	No	Yes	Yes
Popular Search	No	No	No	Yes

According to the comparison table, it is clear that the auto-complete pattern and refine search pattern are highly recommended to be applied into the mobile search feature design. It means these features are able to help the user to improve search experience, so they should be adopted to the design of MyVolts mobile app. The tap-ahead pattern is actually related to the auto-complete pattern as it supports the suggestions field by providing a guessing process and allows the user to tap the diagonal arrow icon on the right side of the query to populate to the search box. This pattern as well as the search form pattern, faceted navigation pattern, and scoped pattern could work well if there are many different categories of the product because it is able to improve the search efficiency by instructing users to construct their queries. However, MyVolts mobile app only focuses on selling power supplies, so it is not necessary to implement these pattern as it may make the app more complicated.

There are two different patterns introduced for loading more items. The Pull to refresh pattern is suitable for social media mobile apps or any others which require high timeliness. The other one is the lazy loading pattern which can be widely used in any type of mobile applications to allow results to be displayed progressively and immediately. So the MyVolts app will only implement the lazy loading pattern.

The Voice search pattern and barcode search pattern are recommended to reduce users’ search effort by allowing quick responses. These two patterns will be implemented as alternative ways for user to search. Structured results are very important to allow users to navigate and verify the target product quickly and simply, which is significant to improve the user experience. However, the personalization pattern and best first pattern require the algorithms to be implemented on the server side, which is not a case in the current project.

However, no these research indicated the popular search pattern. As recreational users are attracted to popular trends, implementing popular search functionality is able to instruct them with the highest rated search items [11], which should also be considered in product matching app.

Application of Best Practice to the MyVolts App

This section presents the design and implementation of the MyVolts mobile app prototype based on the needs of the use case and findings from the best practice review.

System Architecture

Figure 2 presents the overall system architecture of the MyVolts mobile application. The MyVolts Linked Data dataset is hosted in the ADAPT Research Center. As a proof of concept and to enable usability experiments, a prototype MyVolts mobile app was built on the android platform.

As is typical of Linked Data apps, the MyVolts app uses a client-server architecture. The app queries the remote SPARQL endpoint where it is passed to Ontop which translates the query in real-time to retrieve data from the MyVolts relational database. The results are translated into a SPARQL XML response format and returned to the mobile client. All local data is cached in a SQLite database. An instrumentation and timing layer was added to enable timing of user actions and tracking of screen interactions for experimental purposes.

The MyVolts app was implemented using the MVC (Model View Controller) architecture. The model component is responsible for managing the business logic. In MyVolts project, the model component handles network status and data from HTTP response or data from the local SQLite database. The view component is responsible for handling users’ input or output and visualizing data for updates from the model component (see below for user interface design). The controller component contains the logical functions of the system, which is responsible for handling users’ actions.

Fig. 2. MyVolts Overall System Architecture

User Interface Design

This section introduces how the search patterns as discussed in section 2 were applied to the app user interface design.

Search User Interface. The search user interface was designed for users who choose to search products using keyword search. It consists of a search box, a popular search list, and a search history list. The voice search button was placed near the search box, to make it more clickable and visible to users. The search history list was designed to provide the functionality for users to remove a single item or all items at one. There were two options for designing popular search list. One is to implement grid view, and the other one is to adopt tag cloud approach. Through the research, it was found Nielson indicated that “Tag clouds were a huge fad in 2009 and have actually been a fad for several years. Even so, usability studies show that most normal users don’t know what they are and don’t know how to deal with them.” So considering the potential usability issues from the tag cloud approach, the grid view was adopted to implement the best-selling list. Figure 3 displays the screenshot of the search user interface.

Search Results List User Interface. The result list user interface was designed for displaying user search results. There were two designs for displaying search results. One is to present results on the list view, and the other is grid view. A key factor to make a decision between the list view and the grid view is how much information a user needs in order to choose between items. Generally, details in Lists, and pictures in Grids. Considering it can be very difficult to allow users to verify power supplies or cables mainly through pictures, especially for users without previous purchase experience. Therefore, it was decided to adopt list view to display the search results. Figure 4 displays the screenshot of the search results user interface

Fig. 2. Search UI	Fig. 3. Search Results UI

Evaluation

The MyVolts app usability evaluation was carried out in three experiments including two lab-based experiments and one web-based online questionnaire. Four types of participants were used: (1) domain experts - employees from MyVolts Ltd, (2) colleagues and friends who either have CS (Computer Science) background or (3) without a CS background, and (4) potential users from the MyVolts customer base were recruited for the user survey (experiment 3 only). Ethical clearance for these studies was secured from the School of Computer Science and Statistics Ethics Committee.

Experiment 1

The first experiment aimed to observe and measure user behavior after the first full development cycle of the prototype app.

Hypothesis. The MyVolts mobile app has high usablity.

Experimental Methodology. This experiment was conducted as a lab-based experiment with a total 12 participants involved, four task scenarios were designed to evaluate the usability of the MyVolts mobile app based on the list of suggestions [12]. During the investigation, a task list was provided to the participants using plain language and the minimum instructions were given before the tasks being performed.

Table 3. Task Scenarios for Experiment 1.

Task	Scenarios
1	Use app search box to find the correct MyVolts cable “1 to 4 DC Power split cable 2.1mm”.
2	Use app drop down boxes to find a power supply which matches the device “Router” and made by “2wire” with the model reference “2700HG”.
3	Use app search box to find a yellow Ethernet cable which made by 2wire.
4	Use search history list to find the power supply/cable which was searched in task 1.

Each scenario was designed to investigate different search methods of the application. After the participants finished the tasks, each of them was asked to fill out the SUS (System Usability Scale) questionnaire [19] and encouraged to provide oral feedback about the app. Table 3 illustrates task scenarios used in this experiment. All tasks are based on the same Linked Data database.

Usability Metrics. The usability of the MyVolts mobile app was measured by applying the ISO 9241-11 [13] usability model, which standardizes the evaluation approach. Then a set of usability metrics (see Table 4) were selected to quantify the usability of the MyVolts app in this experiment. In order to qualify the experimental results, 95% confidence interval [14] was applied when calculating the usability metrics from small samples using the Wald method [15]. Table 4 lists the usability metrics used to evaluate the MyVolts mobile application and how they can be calculated.

Table 4. Usability Metrics and Measuring Methods.

Metrics	Measuring Methods
Effectiveness	Calculated by measuring the task completion rate [18].
Efficiency	Measured in terms of task time [18]. This time can be calculated by subtracting the start time from the end time.
Satisfaction	User satisfaction is measured through SUS questionnaire.
User Error Occurrence Rate	Calculated based on the number of error opportunities in each task.

Data and Analysis. This section discusses whether the MyVolts mobile app is usable or not by analyzing users’ performance of the tasks as recorded by the data collected.

Effectiveness. Every participant completed all tasks during the experiment, and all tasks achieved a mean 92.3% completion rate with a 95% confidence interval (as per our ethical guidelines participants were allowed to abandon tasks at any time during the experiment). According to a study from Sauro, the average task completion rate is 78. Therefore, it can be concluded that the MyVolts app has a high effectiveness due to this high rate.

Efficiency. Table 5 illustrates the mean completion time for each task specified by each group of participants. We can find from the data that people with a CS background are able to spend less time on completing tasks than others. In other words, people who have a technical background feel more confident using the mobile app.

Table 5. Mean Completion Time for Each Task by Different Groups.

Groups	Task 1	Task 2	Task 3	Task 4
MyVolts	38.38	38.32	47.47	12.32
CS	15.92	17.16	21.40	5.91
Non-CS	35.68	36.82	35.24	18.50

Satisfaction. By analyzing the SUS score from each participant and the overall frequency distribution, it was found that 91.7% of users rated the app usability between 80 and 100. After calculation, the overall satisfaction with 95% confidence interval is 85.3%, which can be mapped to the “excellent” usability according to the methodology described by Bangor et al. where they examined the distribution of SUS scores for over 200 usability studies [17].

Error Occurrence Rate. There are many ways to calculate error rates, in this study the THERP (Technique for Human Error Rate Prediction) was used as it provides detailed instructions for calculation [16]. The rate for each task as shown in table 6.

Table 6. Experiment 1 Error Occurrence Rate by Task.

Task 1	Task 2	Task 3	Task 4
# of Errors Occurred	0	2	1	0
# of Error Opportunities	66	44	55	33
Error Occurrence Rate	0	5%	2%	0

By manually analyzing the errors that occurred, one notable issue was when a user did not see a target item when scrolling the screen. This indicated that even when items are displayed in alphabetic order, users can still miss the target one.

Conclusion. By analyzing these usability metrics, it can be seen that the MyVolts mobile app has achieved high effectiveness. Users were very satisfied with the app usability, and the error occurrence rate is low. It can be concluded that the app is usable and the hypothesis is correct to the limit of the number of participants examined.

Experiment 2

The purpose of this experiment was to compare the difference between the MyVolts app and the standard web browser application developed by MyVolts before this study, in order to find out whether the mobile app could achieve the equivalent usability as the web application on a desktop or laptop device. There were 6 people involved in this experiment with no previous experience with the MyVolts mobile app. In this experiment, two sample t-test was applied to analyze the difference between two means in order to indicate there was a significant difference between two samples of users (). The evaluation was conducted as a lab-based experiment using the same usability evaluation methodology and metrics as experiment 1. Each participant was given the first three tasks from the experiment 1 as the MyVolts web browser application doesn’t implement the search history functionality used for task 4.

By comparing the usability of the MyVolts mobile app and the web browser application (Table 7), the MyVolts mobile app and the web browser application have achieved equivalent effectiveness and user error occurrence rates and the user satisfaction levels for the two applications are very close. Therefore, we can find that it is possible for MyVolts mobile app to achieve the equivalent usability as the MyVolts web browser application on desktop devices. This is significant as the large and complex datasets and filter conditions used would suggest that the larger screen size and device computational resources would be important in determining usability.

Table 7. Usability Comparison between MyVolts mobile app and web browser application.

Effectiveness (%)	Efficiency (sec)	Satisfaction (%)	User Error Rate (%)
Mobile App	86.7	31.82	85.3	2.33
Web App	80	29.54	84.6	2.33

Experiment 3

The third experiment was done by analyzing users’ subjective impression and preferences on MyVolts mobile app features based on the answers and comments from a web-based online questionnaire. The questionnaire included app screenshots and solicited feedback on specific features. In this experiment, MyVolts contacted 204 recent customers by sending them a web-based questionnaire and got 14 responses. As the customers all had previous experience on purchasing power supplies or cables online, their feedbacks are significant to measure whether the app usability is engaging to the target customers. Due to space limitations we only present the overall results of the survey here.

Table 8. Survey Responses: Overall Subjective Impression of MyVolts App.

Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
# of Responses	5	7	2	0	0

The responses (Table 8) have shown that 12 users out of 14 agreed that the app has a good usability and there was no disagreement of the app usability. In addition, the user comments are positive. They complemented the simplicity of the mobile application with the right functionality. Users had no problem with app consistency.

Conclusions and Future Work

The research question was answered by showing in experiment 1 that by following the good practice guidelines (Tables 1 & 2) a high usability Linked Data app could be created for a realistic use case (MyVolts). It was also shown that the app could attain equivalent usability to a commercial web-based product matching solution. Finally a demand for product-matching applications and a strongly positive reaction (12 out of 14 agree) to the app features and design was shown by a survey of actual customers. However the numbers of participants in our studies was small (24) and do not support generalization of these results beyond the specific application, but they do indicate that this is a fruitful line of research. The next stage of evaluation should involve more participants and targeted evaluation of each design recommendation to estimate the relative effectiveness and contribution of each pattern to overall usability.

Acknowledgement

The authors wish to thank the staff of MyVolts Ltd. for their advice and participation during this study. This research was partially supported by Science Foundation Ireland and co-funded by the European Regional Development Fund through the ADAPT Centre for Digital Content Technology [grant number 13/RC/2106].

References

Calvanese, D.: Ontop: Answering SPARQL queries over relational database. International Journal on Semantic Web and Information Systems. 5(3), 1–22 (2017).

Backer, C., Bizer, C.: DBpedia mobile: A location-enabled linked data browser. In: Proceedings of World Wide Web 2008 Workshop (LDOW2008), (2008).

Wilson, M., Russell, A., Smith, D.: A browser-independent presentation vocabulary for RDF. In: Proceedings of End User Semantic Web Workshop and ISWC2005. (2005).

Ermilov, T., Heino, N., Tramp, S., Auer. S.: OntoWiki mobile – Knowledge management in your pocket. In: Proceedings of the ISWC2011. (2011).

Cano, A.E., Dadzie, A., Hartmann. M.: Who’s who – a linked data visualization too for mobile environments. In: Proceedings of the ISWC2011. (2011).

Corsar, D., Edwards, P., Baillie, C.: GetThere: A rural passenger information system utilizing linked data & citizen sensing. In: Proceedings of the ISWC2013. (2013).

UX Planet, https://uxplanet.org/mobile-ux-design-user-friendly-search-51e5f78f5a1e, last accessed 2017/08/26.

Nudelman, G.: Android design pattern gallery: interaction design solutions for developers. Wiley, Mar. (2013).

Neil, T.: Mobile design pattern gallery: UI patterns for mobile applications. O’Reilly Medioa, Inc. (2012).

Morville, P., Callender, J.: Search patterns: design for discovery. Sebastopol, CA. (2010).

Adpearance, https://adpearance.com/blog/designing-for-ecommerce-the-first-impression, last accessed 2017/08/17.

Shaw, D.: Handbook of usability testing: How to plan, design, and conduct effective tests. Journal of the American Society for Information Science. 47(3), 182–192 (1996).

ISO/IEC. 9241-14 Ergonomic requirements for office work with visual display terminals (VDT)s – part 14 menu dialogues. (1998).

Sauro, J., Lewis, J.R.: Estimating completion rates from small samples using binominal confidence intervals: comparisons and recommendations. In: Proceedings of Human Factors and Ergonomic society 49th Annual metting. Orlando, Florida. (2005).

Ferre, X., Juristo, N., Windl, H., Constanine, L.: Usability basics for software developer. IEEE Software. (2001).

Swain, A.D.: Technique for human error rate prediction (THERP). (1964).

Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction. 24, 574-594 (2008).

Tullis, T., Albert, B.: Measuring the user experience. Morgan Kaufman. (2013).

Brooke, J.: A “quick and dirty usability” scale. In: Jordan Pw, Thomas, B, Weerdmeester BA, McClelland AL, es. Usability Evaluation in Industry. Abingdon, UK. (1996).

Yum, R., Pappachan, P.: Are apps going semantic? A systematic review of semantic mobile applications. In SEMWEB. (2011).

National Research Council: The new global ecosystem in advance computing: implications for U.S. competitiveness and national security. The National Academies Press. (2012).

http://myvolts.com/

https://www.nngroup.com/articles/tag-cloud-examples/

http://www.measuringu.com/blog/task-completion.php