171

New Method For Mobile Application Testing Using

Lean Canvas To Improving The Test Strategy
Padmaraj Nidagundi
Faculty of Computer Science and Information
Technology, Institute of Applied Computer Systems
Leonids Novickis
Riga Technical University Head of the Division of Software Engineering
Riga, Latvia Riga Technical University
padmaraj.nidagundi@gmail.com Riga, Latvia
lnovickis@gmail.com

Abstract— The rise of mobile smartphones and its usage application can raise many challenges in the testing [5] to
of mobile applications made the human life more productive make sure app will work and fulfills the user expectations.
and boosted several activities in everyday routine jobs and
business areas. In another side now a day, most of the One of the main reason the mobile application testing
business and personal applications transferred to the mobile may become more challenging day today because of
apps. The one hand mobile application providing more growing number of platforms, operating system, and
features than traditional desktop applicationas and proving device fragmentation for example, in android only in the
endless opportunities for new ventures. Day to day mobile
year 2014 about 24,000 separate android device models
application are becoming a part of several model device
with different mobile operating system such as Android, available in the market and mobile compatibility issues
iOS, Windows, BlackBerry, Symbian, etc. and it becomes are rising day to day with developed mobile apps. In
challenging to development and quality assurance teams to another hand, mobile, functional testing must the full fills
verify and validate mobile app that works on end-user the business goal of the application and non-functional
mobile with no error. testing to make sure it works in different conditions. To
overcome such situations of mobile application failure, it
With growing technology, mobile application must be well tested in all possible scenarios, and before
development companies adopted new tools and work in the test team must have a better test strategy, plan that
different development approaches such as waterfall, agile, fulfill the stakeholder business requirements.
prototyping [1], [2], [3] but still, challenges are same for the
quality assurance team to test the application and make II. THE PROBLEM WE TACKLE
application error free in the dedicated short time. The
testing mobile application becomes more challenging due to Mobile application testing is can be complex and time-
the growing number of devices by the different vendor and consuming comparing desktop and web based application.
test on all possible devices sometimes impossible for the Most of the time desktop application must be tested on
quality assurance team. The mobile application testing team
one platform, but the mobile application works on
most of the time face challenges related to the usability,
interrupts, poor connections, device settings with functional multiple platforms and several devices. With mobile app
and non-functional testing issues. In such situation, test testing, quality assurance team must make sure all
planning and building more proper test strategy to cover all possible kinds of tests such as functionality, performance,
aspects will lead to making application error free. From network, security, compatibility, conformance, usability,
many years, lean canvas boards are used for the developing installation, and provisioning did before publishing the
business plans, strategy and for risk analysis. This paper is
focuses on the re-design and develop lean canvas design
application in the app stores. Most of the time quality
using lean principles to use it for the mobile application ssurance team must work in parallel with the development
testing process and it can affect precisely on app testing team to make sure the app is error free.
quality, resource, cost and time.
The most common mobile application software testing
Keywords— lean canvas for testing; mobile app testing; challenges are [15], [16], [17], [18]:
mobile application test planning; test process; test strategy
x Mobile screen size and different OS fragmentation.
I. INTRODUCTION x Manufacturer fragmentation.
The mobile app works all the time error free in
different conditions, then it increases the customer x Application localization.
satisfaction and provides great business value to the end x Mobile network operators and user’s mobility for
customer. To make any mobile app error-free, quality connection types (edge, UMTS, 3G, 4G, 5G).
assurance team must be well verified and validated
complete application according to the business needs and x Different application types (Native, Hybrid, Web).
different circumstances. Growing complexity of mobile

CSIT 2017, 05-08 September, 2017, Lviv, Ukraine

Authorized licensed use limited to: Escuela Superior de Ingeneria Mecanica. Downloaded on October 27,2022 at 15:48:06 UTC from IEEE Xplore. Restrictions apply.
 172
x Power consumption and battery life for application x Describing the collaboration possibilities on lean
usage. canvas for the test team.
x Usability of the application. x Analyzing and visualizing the blocks with titles.
x Core application functionality and its UI.
x Tools selection for automation & manual testing.
x Memory leakage & interrupt testing.
x Knowledge sharing with testing & QA teams and
documenting test process.
x Test planning and test strategy building.
According to world quality report 2016-2017
challenges [19] in testing mobile and multi-channel
application are not having the right testing
process/method, not having mobile application testing Fig. 2. Overview on mobile app development & test process.
experts, not having have in-house testing environment, not
identified right tools to test, not enough time to test all A. Related Research
application, not having the have the devices readily In the year 2008 first time, Alex Osterwalder with his
available, not do in-house mobile testing. co-authors talked about the possibilities of the business
model canvas and later it is known as a lean canvas. Till
nowadays, it is used by entrepreneurs, universities,
accelerators and organizations in the outlining business,
business plan and building business strategy on one page.
The life cycle of the business model canvas consists of
six phase start with an idea, build, product, measure, data
and learn and giving feedback to each other.

Fig. 1. World quality report 2016 – 2017 for challenges in the testing
mobile and multi-channel application.

III. SCIENTIFIC NOVELTY OF PAPER

From many years startups and business development Fig. 3. Business model canvas life cycle overview.
companies using lean canvas board for the test a business
plan, build the strategy and identifying unique value Business model canvas life cycle helps to identify the
propositions. Lean canvas is a single page light weight objects of the business.
document that contains the many segments on it. This x It starts with an idea and constant connect with
main paper contribution is to find well optimized lean learning.
principles adopted lean canvas board for the mobile
application testing [5]. x It removes the unwanted things from the process in
the life cycle and gives improved feedback to each
Particularly: other.
x Determining possible best fit, lean canvas design x It provides a constant feedback in the life cycle.
for the mobile application testing.
B. Mobile application testing and Lean canvas life cycle
x Test planning improvement.
The fig. 1. app development & test process shows the
x Simplifying the mobile test strategy. overview how the app development & testing done in any
software development company [4]. Now considering the
x Adopting lean principles, develop a lean canvas business model canvas as a base model for the mobile app
design. development and testing process, we able to identify and
x Investigating possible blocks for the lean canvas gather the similar terms for mobile application testing.
design.

CSIT 2017, 05-08 September, 2017, Lviv, Ukraine

Authorized licensed use limited to: Escuela Superior de Ingeneria Mecanica. Downloaded on October 27,2022 at 15:48:06 UTC from IEEE Xplore. Restrictions apply.
 173
x Motion – Distributing beta versions is hard,
principally in iOS where the device UDID limit of
is 100.
x Waiting – Updating mobile OS takes time or tester
wait a long time for until developer fixes the
applications errors. Tester not able to test the
application until the developer approves for testing.
x Overproduction – Testing finished early and not
identified potential bugs in test automation.
x Over processing – Testing direct app instead of
testing designed mock-ups.
x Defects – Rise in the number of bugs due to poor
Fig. 4. Prototype design for mobile application testing and lean canvas test coverage.
life cycle.
A. Mobile App Test Metrics for Lean Canvas Board
x Ideas - Mobile application functional & non- Considering the lean principles, we can gather several
functional requirements. testing metrics for the lean canvas board and later creating
x Build - Error free mobile app for end users. blocks we can add them. In testing, metrics play the key
role in identifying the software quality in each cycle [9].
x Product - Mobile app as a solution.
x Measure - App Bugs related to the application
notifications, updates, gestures, network, user
scenarios, location, platform, orientation,
interrupts, guidelines - iOS HIG, android design,
etc.
x Data - Sample data used for application testing.
x Learn - Error, bugs, app crash bugs and logs.
IV. MOBILE APPLICATION TESTING PROCESS AND
POSSIBLE WASTE IDENTIFICATION TEST METRICS
USING LEAN PRINCIPLES Fig. 5. Transformation process to find the lean test metrics.
Rethink on test planning and test design of the mobile As shown in above figure from the input we gather
application testing to bring a high-quality app for the end several lean matrices and ultimately add optimized test
user [7]. In traditionally till most companies are using metrics on the lean canvas board.
long test documentation and test planning for the mobile
application testing such as test plans to test the mobile B. Design Lean Canvas Board for Mobile App Test
app, these are proven more time-consuming in modern Using the Design Principles
agile software development methods. Once we have a list of test metrics, now able to
generate the lean canvas board and fill each using the
Mobile testing teams must decide on the mobile notes, pictures, handwriting, links etc. to collaborate with
device selection, emulators & physical devices, mobile the team. This visual board allows the team to see the big
app testing on cloud, network connectivity, manual & picture of the mobile testing process.
automated testing, performance testing, security testing
and other types of testing with new app version release to C. Prototype design of lean canvas for mobile
make application error free, in such situation testing application testing challenges cosndering the mobile
complexity grow and quality assurance team must work in and multi channel (Mobile,Wearables, Social, and
more collaborative way to make sure app is bug free for Traditional) - world quality report 2016- 2017
end user [12]. If a quality assurance team does not care all Considering the recent survey data [19], as an input for
possible scenarios, then it leads towards the increasing the prototype of lean canvas design for mobile application
cost of the project and the lead towards the project failure. testing show us for the mobile application test planning.
In such situation, lean core principles we can adapt to get
different test metrics for different tests [8], [9], [14]. This porotype addresses the main problem of the
world quality report 2016- 2017 about the “don't have the
x Transport – Doing non-functional testing with, not right testing process/method”. It gives test team to set an
developed features of the app. appropriate test process, methods to with mobile
application and testing and bring quality assurance/testing
x Inventory – Many tests not able to test on the same
team on a single page. It improves the software quality
mobile device.

CSIT 2017, 05-08 September, 2017, Lviv, Ukraine

Authorized licensed use limited to: Escuela Superior de Ingeneria Mecanica. Downloaded on October 27,2022 at 15:48:06 UTC from IEEE Xplore. Restrictions apply.
 174
and also addresses the scope, out of scope, resources, generate more new ideas and collected ideas used further
people, test environment, tools and risk mitigation ideas. to do more research.
REFERENCES
[1] S. Dorairaj, J. Noble, and P. Malik. Effective communication in
distributed Agile software development teams. In Agile Processes
in Software Engineering and Extreme Programming, volume 49 of
Lecture Notes in Business Information Processing, pages 102–
116. Springer Berlin Heidelberg, Madrid, Spain, 2011.
[2] S. Jalali and C. Wohlin, “Agile practices in global software
engineering a systematic map,” in International Conference on
Global Software Engineering (ICGSE). IEEE, 2010.
[3] S. Soundararajan, «A Methodology for assessing Agile Software
Development Approaches,» CoRR, vol. abs/1108.0427, 2011.
[4] C. Treude and M.-A. Storey, “Work item tagging: Communicating
concerns in collaborative software development,”Software
Engineering, IEEE Transactions on, vol. 38, no. 1, pp. 19 –34,
jan.-feb. 2012..
Fig. 6. UI prototype design of lean canvas design to address mobile
testing challenges world quality report 2016 - 2017. [5] D. G. Rex Black, Erik Van Veenendaal, Foundations of software
testing: ISTQB certification, 3rd Edition. Cengage Learning
V. CONCLUSION AND FUTURE RESEARCH EMEA, 2012..
[6] X. Zhang and T. Hoshino, “A trial on model based test case
The lean canvas board for the mobile app testing can extraction and test data generation,” in 3rd Workshop on Model-
help in improving the test strategy, cover possible testing based Testing in Practice (MoTiP 2010), 2010, pp. 51–60.
type and increase the collaboration within the team [11], [7] N. Padmaraj, L. Margrita, “Introduction to adoption of lean canvas
[13]. This is one-page light white board with several in software test architecture design”. Computational Methods in
Social Sciences. Vol. IV, Issue 2, 2016.
blocks can help to divide big component in the small
[8] Padmaraj N, Leonids N, “Introducing Lean Canvas Model
component. This board can help to do app verification and Adaptation in the Scrum Software Testing”, Procedia Computer
validation as early possible and it can effect on reducing Science, 2017, Volume 104, pp 97–103..
the cost of testing in save the time. In next level to create [9] D. Raffo, M. Mehta, D. Anderson, R. Harmon, “Integrating Lean
well optimized visualized lean canvas for mobile app principles with value based software engineering” Picmet
technology management for global economic growth, 2010, pp. 1–
testing must carry out more following listed research 10.
activities [10]. [10] X. Wang, “The Combination of Agile and Lean in Software
Development: An Experience Report Analysis”, Agile Conference
x In a mobile development environment, must do the (AGILE), 2011, pp. 1–9.
practical analysis of the lean canvas test metrics. [11] J. Kasurinen, “Elaborating Software Test Processes and
Strategies”, Third Int. Conf. on Software Testing, Verification and
x Types of testing with mobile a application to be Validation, 2010, pp. 355–358.
segmented. [12] A. C. Barus, D. I. P. Hutasoit, J. H. Siringoringo, Y. A. Siahaan,
“White box testing tool prototype development,” in 2015 Int.
x Need to find an appropriate test matrix with Conf. Electrical Engineering and Informatics, ICEEI, 2015, pp.
description. 417–422.
[13] M. Ide, Y. Amagai, M. Aoyama, Y. Kikushima, “A Lean Design
x Need to work on the ongoing live project and Methodology for Business Models and Its Application to IoT
develop a prototype that shows the mobile app Business Model Development,” in Agile Conference, AGILE,
testing lean canvas visualization. 2015, pp. 107–111.
[14] U. Viswanath, “Lean Transformation How Lean Helped to
x Need to develop algorithms that identify possible Achieve Quality, Cost and Schedule: Case Study in a Multi
mobile test metrics. Location Product Development Team,” in IEEE 9th Int. Conf. on
Global Software Engineering, ICGSE, 2014, pp. 95–99.
x Need to develop a prototype of UI for lean canvas. [15] C. Valdez, “What to include in an effective mobile app test plan”
(2017).
x Needs to carry out impact analysis with a real [16] “The Essential Guide to Mobile App Testing” uTest.
project to prove the proposed model. [17] P. Gajos, “What is Agile Development for Mobile Apps?
“Sourcebits”, 2015.
This paper is focused on the utilization of lean canvas
[18] A. Lozancic, “Mobile Testing Challenges in 2017”, 2016.
design in the mobile application testing process and the
[19] “World Quality Report 2016--2017” Capgemini, Sogeti, and HP,
author wishes this introductory new method paper will 2017.

CSIT 2017, 05-08 September, 2017, Lviv, Ukraine

Authorized licensed use limited to: Escuela Superior de Ingeneria Mecanica. Downloaded on October 27,2022 at 15:48:06 UTC from IEEE Xplore. Restrictions apply.