Optimizing user interface layouts via gradient descent

P Duan, C Wierzynski, L Nachman - … of the 2020 CHI Conference on …, 2020 - dl.acm.org
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020dl.acm.org
Automating parts of the user interface (UI) design process has been a longstanding
challenge. We present an automated technique for optimizing the layouts of mobile UIs. Our
method uses gradient descent on a neural network model of task performance with respect
to the model's inputs to make layout modifications that result in improved predicted error
rates and task completion times. We start by extending prior work on neural network based
performance prediction to 2-dimensional mobile UIs with an expanded interaction space …
Automating parts of the user interface (UI) design process has been a longstanding challenge. We present an automated technique for optimizing the layouts of mobile UIs. Our method uses gradient descent on a neural network model of task performance with respect to the model's inputs to make layout modifications that result in improved predicted error rates and task completion times. We start by extending prior work on neural network based performance prediction to 2-dimensional mobile UIs with an expanded interaction space. We then apply our method to two UIs, including one that the model had not been trained on, to discover layout alternatives with significantly improved predicted performance. Finally, we confirm these predictions experimentally, showing improvements up to 9.2 percent in the optimized layouts. This demonstrates the algorithm's efficacy in improving the task performance of a layout, and its ability to generalize and improve layouts of new interfaces.
ACM Digital Library