Keywords

1 Introduction

Taiwan is an island nation with the highest density of scooter-riding populations in the world. For improving air quality and public health, the Taiwanese government proposes an eco-friendly environmental protection policy to replace scooters with electric two-wheelers (E2Ws) in the Taiwanese two-wheeler market. Providing E2W trial ride services has a positive impact on the user’s acceptance level of the E2W product [1,2,3]. Direct product or service experiences may create stronger associations in memory, given its inherent self-relevance [4]. Experiential benefits generally correspond to product-related attributes, and the experience of using the product or service, and satisfying needs such as sensory pleasure, variety, and cognitive stimulation [5]. Therefore, E2Ws industries are becoming increasingly interested in applying experiential marketing, to positively stimulate people’s willingness to accommodate E2Ws. Thus, an experience system using virtual reality (VR) devices and applying 360-degree panoramic formats for the experiential service of vehicle is proposed. In this experimental study, the research goal is to analyze the effectiveness of using head-mounted display (HMD) virtual reality (VR) devices and applying 360° panoramic formats in eliciting positive user perceived usability and to compare these to usability evoked in physical scooter ride settings. Within the usability evaluation methods described in the literature [6], questionnaires assume a significant importance for qualitative self-reported data collection about the characteristics, thoughts, feelings, perceptions, behaviors or attitudes of users [7]. Questionnaires have the advantage of being low budget techniques, they do not require test equipment, and their results reflect the users’ opinions. They also provide useful information about what are the strengths and weaknesses of a product or service. The System Usability Scale (SUS) is one of the usability evaluation questionnaires most widely used [8]. The SUS is a widely used self-administered instrument for the evaluation of usability of a wide range of products and user interfaces. The principal value of the SUS is that it provides a single reference score for participants’ view of the usability of a product or service. Therefore, SUS has been adopted in this study.

2 Literature Review

2.1 Virtual Reality

Virtual reality (VR) has typically been portrayed as a medium. Computer hardware and software and its peripheral devices, that are used to create a VR system, are designed to replicate the information available to the sensory/perceptual system in the physical world and to produce outputs that impinge upon the body’s various senses, resulting in convincing illusions for each of these senses and thus a rich, interactive multimedia facsimile of real-life [9]. With regard to hardware, VR is a particular collection of technological hardware, including computers, head-mounted displays, headphones, and motion-sensing controllers. With regard to content, two major types of VR content are realistic images or videos, in 360-degree and three-dimensional (3D) digital representations. In the past, the format most used has been 3D. The 3D format is created digitally through computer vision software, the navigation is continuous, and it must be connected to a computer. Now, panoramic videos are a new and rapidly growing approach that can display the power of sight, sound, and motion in an entirely new way, and allows viewers to sense action from all angles and directions. Experiences created by the VR system, which including computer hardware and software and video contents, are expected to provide sensory, emotional, cognitive, and behavioral values that replace functional values. Related simulations have been verified to evoke user responses, similar to those in physical environments [10].

2.2 System Usability Scale

The System Usability Scale (SUS) is one of the most widely adopted usability evaluation questionnaires to measure users’ subjective assessments of a system’s usability [11]. The SUS was developed by John Brooke more than 25 years ago as part of a usability engineering program (1986) as a “quick and dirty” survey scale that would allow the usability practitioner to quickly and easily assess the usability of a given product or service. The main advantage of SUS is that it comprises only 10 items to be rated on a five-point scale ranging from strongly disagree to strongly agree, among which five are positive statements and the rest are negative. The SUS items have been developed according to the usability criteria defined by the ISO 9241-11, including the ability of users to complete tasks using the system, and the quality of the output of those tasks, i.e., effectiveness; the level of resource consumed in performing tasks, i.e., efficiency; how easily users can learn to use the system, i.e., learnability; and the users’ subjective reactions using the system, i.e., satisfaction. In addition, the SUS score is single and ranges from 0 to 100. The results of the SUS can be comprehensible even to non-experts. A SUS score above 68 would be considered above average and anything below 68 is below average [12]. SUS has a remarkably robust measure of system usability, even with a small sample size [13].

3 Methods

This study conducted with two experiential services, including 360° immersive video service and panoramic video service. We investigated paths through which VR and/or 360° panoramic video technologies impacted user perceived usability. The service presented actual situations of riding scooters on the road, which allowed us to make valid comparisons. The responses from participants were collected to understand user’s usability all in using 360° immersive video service, using panoramic video service, and riding a scooter on the road.

3.1 Participants

Fifty-six individual scooter commuters took part in the experiment and completed surveys. There were 36 males and 20 females aged 18–62 years (\( \bar{\rm{X}} \) = 23.53; \( \upsigma \) = 7.31). No participant experienced simulator sickness or had technical difficulties with the VR system.

3.2 Procedures

At the beginning of the experiment, participants were required to read and sign an IRB-approved consent form, which provided details regarding the experiment. They were allowed to opt out of the study at any time during their participation. Then, participants were instructed regarding the use of the VR equipment. In the experiment, participants experienced a scooter ride on the road. The overall experience of the participant is based on the VR using HMD of VR for transmission of panoramic video data. At the end of the experiment, participants completed the SUS.

3.3 Data Analysis

Analyses were conducted using SPSS software, Version 22.0. The two-tailed significance level was set at p < .05.

4 Results

4.1 Descriptive Statistics

The results of each question item for SUS is listed in Table 1.

Table 1. Descriptive statistics of SUS scores
Full size table

4.2 T Test

The T-test results indicated that SUS items for riding a scooter had significantly different findings for ‘I would like to use this system frequently’ (F = 4.63, p < .05; t = 3.24, p < .05) between men (\( \bar{\rm{X}} \) = 3.5, \( \upsigma \) = 1.00) and women (\( \bar{\rm{X}} \) = 2.55, \( \upsigma \) = 1.19) and ‘easy to use’ (t = 2.49, p < .05) between men (\( \bar{\rm{X}} \) = 3.5, \( \upsigma \) = .56) and women (\( \bar{\rm{X}} \) = 2.95, \( \upsigma \) = 1.1). In addition, SUS items for using VR service had significantly different findings for ‘easy to use’ (F = 6.35, p < .05; t = 2.19, p < .05) between women (\( \bar{\rm{X}} \) = 3.05, \( \upsigma \) = .76) and men (\( \bar{\rm{X}} \) = 2.5, \( \upsigma \) = 1.11) and ‘needed to learn a lot of things before I could get going with this system’ (t = 2.02, p < .05) between men (\( \bar{\rm{X}} \) = 2.53, \( \upsigma \) = 1.06) and women (\( \bar{\rm{X}} \) = 1.9, \( \upsigma \) = 1.21).

4.3 Wilcoxon Signed-Rank Test

The Wilcoxon signed-rank test revealed that there were statistically significant difference for SUS1, SUS2, SUS3, SUS4, SUS8, and SUS9 items (P 2 0.05). These results indicate that:

  • Users are perceived themselves more frequently to ride the scooter (Mdn = 3) than to use the VR system (Mdn = 2)

  • Users are perceived more easy to use when riding a scooter (Mdn = 3) than when using VR system (Mdn = 3)

  • Users are perceived more confident when riding a scooter (Mdn = 3) than when using VR system (Mdn = 3)

  • Users are perceived themselves more needs to have support from technical person when riding a scooter (Mdn = 3) than when using VR system (Mdn = 1)

  • Users are perceived more cumbersome to use when riding a scooter (Mdn = 3) than when using VR system (Mdn = 2.5)

  • Users are perceived more complex when riding a scooter (Mdn = 3) than when using VR system (Mdn = 2)

5 Discussion

In this study, a quantitative analysis of the usability while riding a scooter and usability while using VR service to watch the panoramic scooter ride video was carried out using a SUS scale. SUS is an easy scale to administer to participants. Also, SUS can effectively and quickly differentiate between usable and unusable systems. The results revealed that the average score of SUS for riding a scooter is 68.75, which is above average. This means that most of the participants consider the usability of riding a scooter is acceptable. In other words, riding a scooter created a positive impression concerning usability. However, the 68.75 score expresses somewhat negative subjective usability impressions about scooters as well. In addition, the results revealed that the average score of SUS for using the VR system is 56.47, which is below average. This means that most of the participants consider the usability of using VR system is marginally acceptable. Such a score expresses a few negative subjective usability impressions about VR service applying in scooter products as well. The results verified that both the scooter product and the VR scooter ride service are acceptable and useful systems for users. However, the functionality provided by scooter and VR service to users is different. Riding a scooter is mainly based on the usefulness of transportation vehicles, while the VR scooter ride service is mainly about to create like the reality of riding a scooter.

The influence of a scooter ride and a scooter VR service on user-perceived usability when riding a scooter or watching 3600 scooter ride videos was investigated. With regard to riding a scooter, easy to use, confident, and intention of using frequently play a big part in positively influencing user’s perception of scooter usability. In addition, the values of these three factors for user-perceived usability when riding a scooter are significantly higher than using VR service. With regard to using VR service, easy to learn, easy to use, and confident play a big part in positively influencing user’s perception of VR service usability. Compared with riding a scooter, user perceived fewer needs of the support for a technical person and lower feel of cumbersome and complex when using VR service.

There is a gender-based variation in usability while riding a scooter and using VR service. This gender-based difference has been found to be due to the user’s general response toward the scooter as a product and the VR service as an experience tool to create reality-like scooter riding experiences for users. The results indicated that most of the male participants tended to use frequently and feel easy to use for scooter products, while female participants tended to have slightly negative feelings. This means that male users perceived better usability of riding a scooter than female users. In addition, the results indicated that most of the female participants tended to feel easy to use for VR service, while male participants tended to have slightly negative feelings. Men participants tended to need learning a lot of things before they could get going with VR system, while female participants tended to have slightly less feelings. This means that female users perceived better usability of using VR service to create reality-like scooter riding experiences than male users.

6 Conclusion

The main aim of this study was to use SUS evaluation methods to investigate the usability and the differences in user-perceived usability between riding a scooter and using VR service. Results of the surveys, conducted with scooter commuters in the country of Taiwan, provide evidence to understand that users perceived usability for both scooter product and VR service are acceptable and tend to be marginally acceptable. Comparing with VR service, user-perceived easy to use, confident, and intention of using frequently play a big part in positively influencing user’s perception of scooter usability. In contrast, using VR service to create an experience of scooter ride may have fewer needs of the support for a technical person and lower user’s feel of cumbersome and complex then riding a scooter. Hence, VR service is an innovating idea applying to two-wheeler products for creating a rider experiences. Furthermore, female users perceived better usability of using VR service to create reality-like scooter riding experiences than male users. Finally, designers and developers of VR systems for experiencing scooter products would need further research to improve the effectiveness of applying on two-wheeler products and implement permanent versions of the public VR two-wheeler service.