You Are How You Use Apps: User Profiling Based on Spatiotemporal App Usage Behavior

Mobile app usage behavior is defined as a set of activity records generated by mobile users using mobile apps on smartphones [17]. Generally, when one user launches an app in the foreground of a smartphone, that launch will be regarded as a use of that app. In particular, an app usage record involves four key features: who, what, where, and when. In other words, a usage record is aggregated into a tetrad—that is, \(r = \, \lt u, a, l, t\gt\) , where u denotes the user, a denotes the app used, l denotes the user’s location, and t indicates the timestamp of that record. The location information of mobile users can be gathered from the GPS sensors of mobile phones. Alternatively, network operators can infer the coarse location from network-level data, such as the location of associated base stations. As a result, app usage behavior, as a kind of spatiotemporal data, contains the interior relationships among users, apps, locations, and time [18]. By investigating such relationships, we can infer user attributes, improve app usability, and provide context-aware app recommendations and usage predictions [19, 20, 21]. This article works on the problem of automatic user profiling from spatiotemporal app usage behavior by exploring hidden relations.

Our crucial user profiling problem is to infer user attributes from their app usage behavior. Specifically, existing studies have presented the existence of relations between user attributes and the apps they use [10, 22]. For example, Malmi and Weber [10] exhibited that the presence of period-tracking apps is a good predictor for gender. However, existing works did not take full advantage of the hidden relationships in usage behavior as they ignored spatiotemporal features [7, 23]. Therefore, inspired by the limitations of existing studies, in this article, instead of only considering app adoptions, we tackle the user profiling problem by thoroughly leveraging users’ spatiotemporal app usage behavior.

We formulate this problem as a task of multi-label classification. Essentially, we first construct a heterogeneous graph from spatiotemporal mobile app usage records, representing co-occurrence relations between users, apps, locations, and time. We then infer the labels of user nodes from both the local multi-relational graph structure and features of heterogeneous nodes.

Figure 1 presents an overview of our proposed system. Specifically, there are three critical steps illustrated as follows.

Construction of the Mobile App Usage Graph. In this step, we detect the co-occurrence of users, apps, locations, and time from mobile app usage datasets and construct a normalized weighted app usage graph. Figure 1 exhibits a schematic diagram of the heterogeneous app usage graph, where U represents user nodes, A represents app nodes, L represents location nodes, and T represents time nodes.

Automatic Representation Learning of User Nodes. In this step, we incorporate the heterogeneous neighbor sampling, relational graph convolutional operation, and multi-relational attention operation into the graph learning framework to develop a representation learning model that can leverage both multi-relational graph structure and heterogeneous node features.

User Profiling. In this step, we exploit the learned representations of user nodes to infer user profile labels. Explicitly, we set the dimension of a user’s representation vector as the number of labels in the user profiling task. We then predict user profile labels by applying the softmax function to users’ representations.

Notably, our proposed system is an end-to-end solution. The notations we will use throughout the article are summarized in Table 1.

To represent the interior relations between users, apps, locations, and time, we construct a multi-relational heterogeneous graph to encode the associations between different entities in mobile app usage records. The structure of the graph is shown in Figure 2, where vertexes stand for the four-type objects and edges reflect the co-occurrence of different objects in app usage records. Specifically, since time is a continuous variable, we evenly divide 1 day into T time slots and use time slots to represent time instead. For simplicity, we still use the term time in the following explanations. For a multi-relational heterogeneous graph, \(G=(V, E, O_V, R_E, H)\) , V and E respectively denote the vertexes and edges in the graph. \(O_V\) and \(R_E\) represent the set of vertex types and relation types, respectively. H is the set of the feature vector of each vertex. In our case, the vertex types \(O_V\) include user, app, time, and location. The edge types \(R_E\) include user-app edges \(E_{ua}\) that reflect the app adoptions of users, user-location edges \(E_{ul}\) that describe the trajectories of users, user-time edges \(E_{ut}\) that reveal the active time of users, app-time edges \(E_{at}\) that represent the temporal features of apps, app-location edges \(E_{al}\) that indicate the spatial features of apps, and location-time edges \(E_{lt}\) that express the temporal dynamics of locations. Notably, the heterogeneous app usage graph is undirected.

To model the strength of connections between vertexes, we assign each edge a weight. In practice, the method to compute the weight of edges is described as follows. Initially, the weight of each edge is set as 0. For an app usage record \(r = \, \lt u, a, l, t\gt\) , the weights of edges \(e(u, a)\) , \(e(u, t)\) , \(e(u, l)\) , \(e(a, t)\) , \(e(a, l)\) , and \(e(l, t)\) will increase by 1. Traversing all records, we will get the weights of all edges. However, as app usage records are unevenly distributed in all user, location, time, and app dimensions, we need to normalize the edge weights across different edge types. In practice, we normalize the weight for each edge type separately. For example, for user-app edges \(E_{ua}\) , we set the normalized edge weight aswhere \(\hat{\omega }(u, a)\) denotes the normalized weight of edge \(e(u,a)\) , U is the set of user vertexes, and A is the set of app vertexes. We then apply a similar computation to other edge types (i.e., \(e(u, t)\) , \(e(u, l)\) , \(e(a, t)\) , \(e(a, l)\) , and \(e(l, t)\) ) and finally obtain the normalized weights for all types of edges.

The mobile app usage graph can be partitioned into six subgraphs based on the types of edges: user-app subgraph \(G_{ua}\) , user-location subgraph \(G_{ul}\) , user-time subgraph \(G_{ut}\) , app-time subgraph \(G_{at}\) , app-location subgraph \(G_{al}\) , and location-time subgraph \(G_{lt}\) . Each subgraph is a normalized weighted bipartite graph, and different subgraphs describe different semantic relations between heterogeneous vertexes.

To leverage the rich side information of vertexes, we assign a feature vector \(\boldsymbol {h}\) to each vertex \(v \in V\) .

For app vertexes, we use the app category information as app features. In each app category, the apps are usually of similar functionality and have an inherent semantic meaning [24]. The apps’ categorical information is available in app stores, like Google Play and Apple Store for Android and iOS apps, respectively. Assuming the number of app categories is C, for an app vertex a, we use a one-hot vector to express the feature and denote the categorical information; thus, the feature vector is \(\boldsymbol {h}_a \in \mathbb {R} ^{1\times C}\) ,where \(c_a\) is the category label of app a.

For location vertexes, we leverage the density and category information of nearby POIs of that location. A POI refers to a point location with specific functions, such as residence, workplace, and theater. Previous studies have shown that locations with similar POI distributions have similar urban functions [25, 26]. Hence, we use the POI distribution within the location as the semantic feature of the location vertex. In practice, the POI information can be crawled from map service providers like Tencent Maps and Google Maps. Generally, a POI is recorded with a tuple consisting of a POI category, a name, and a geo-position (i.e., latitude and longitude). For each location, we count the number of POIs in each category accordingly. Assuming the number of POI categories is P, for an arbitrary location \(l \in L,\) where L is the set of location vertexes, we have a POI distribution vector \(\boldsymbol {p}_l = [p_{l1}, p_{l2}, \ldots , p_{lP}],\) where \(p_{li}\) stands for the number of POIs of POI category i in location l. In particular, the popularity of POI categories varies a lot. For instance, restaurant POIs are far more popular than tourist spot POIs. Therefore, we normalize the POI features to eliminate the influence of the uneven popularity distribution across different POI categories. In this work, we apply TF-IDF [27]. For a location vertex \(v_l\) , its feature vector \(\boldsymbol {h}_l \in \mathbb {R} ^{1\times P}\) can be calculated aswhere \(\frac{p_{li}}{\sum _{i=1}^P{p_{li}}}\) is the term frequency and \(\log \frac{|L|}{|\lbrace \boldsymbol {p}_l:p_{li}\gt 0\rbrace |+1}\) represents the inverse document frequency.

For time vertexes, we use one-hot vectors to express their features. For a time vertex \(t \in T\) , where T is the set of time vertexes, the feature vector is \(\boldsymbol {h}_t \in \mathbb {R}^{1 \times |T|}\) ,

Similarly, we use one-hot vectors to express user vertexes’ features as well, and the feature vector of a user vertex \(u \in U\) is \(\boldsymbol {h}_u \in \mathbb {R}^{1 \times |U|}\) ,

The critical idea of Graph Neural Networks (GNNs) is to aggregate features from a node’s neighbors [28]. Typically, the computation of GNN is carried out in two phases: (1) the message passing phase and (2) the aggregating and updating phase. Specifically, in the message passing phase, a node passes its representation vector to its first-order neighbors. In the aggregating and updating phase, a node first aggregates the received representation vectors with its own representation. Then, the node updates its own representation vector with the aggregated one. By increasing the number of network layers, each node can incorporate information from higher-order neighbors and thus learn richer node features. For example, as shown in Figure 3(b), in layer 1, nodes C, D, F will pass their representation vectors to their first-order neighbors (e.g., node E). In layer 2, node E will then pass the aggregated representation to node D. In this way, node D can incorporate the feature information from its second-order neighbors (i.e., nodes C and F). However, applying this approach to a mobile app usage graph may raise several issues:

To solve these issues, we apply a heterogeneous neighbor sampling strategy based on a bootstrapping approach. Specifically, for each node, we randomly sample a fixed-size set of neighbors with probability proportional to the edge weights. Mathematically, we use \(\hat{\mathcal {N}}(v)\) to denote the sampled fixed-size neighbors of node v, drawn from the set \(\lbrace u \in V: e(v,u) \in E \rbrace\) . We then use the neighbors sampled to approximate the aggregation of the total neighbors. For example, as depicted in Figure 3(c), we set the sample size as 2. Therefore, by dropping the edge \(e(D, A)\) , node D only passes its representation vectors to nodes B and E in layer 1. Correspondingly, in layer 2, node D will only aggregate the representations from nodes B and E as well. We also note that the app usage graph is heterogeneous, having various node types. Moreover, the degree distribution of different node types varies greatly. Thus, for different types of nodes, we implement the sampling strategy separately. In other words, given a node v, we sample its neighbors of user nodes, app nodes, location nodes, and time nodes with the fixed size of \(\hat{\mathcal {N}_u}(v)\) , \(\hat{\mathcal {N}_a}(v)\) , \(\hat{\mathcal {N}_l}(v)\) , and \(\hat{\mathcal {N}_t}(v)\) , respectively.

The heterogeneous neighbor sampling strategy can avoid the issues mentioned earlier due to two principal reasons. (First, only a small set of the most relevant neighbors with large edge weights are selected, thus mitigating the issues of neighborhood expansion and oversmoothing. Second, for each node, all types of neighbors are collected, and the sample size is fixed by leveraging bootstrapping, which solves the issue of varying node degrees.

Because of the heterogeneity of nodes, as illustrated in Section 3.2, the nodes with different types have different feature spaces. Hence, we first project the features of different node types into the same space using a type-specific transformation. Mathematically, the projection process can be expressed as follows:where \(\boldsymbol {M}_{o_v}\) denotes transformation matrix and \(o_v\) represents the type of node v. \(\boldsymbol {h}_v\) and \(\hat{{\bf h}}_v\) denote the original and projected feature vectors of node v, respectively. After the node type-specific projection operation, the downstream operations (e.g., graph convolution) can cope with arbitrary types of nodes.

The app usage graph has different edge types, revealing different relations between nodes and having different semantics. For example, user-app edges can describe the app co-using relationships between users, whereas user-location edges can depict the relationships between users visiting the same location. However, the conventional graph convolutional operation treats graph edges equally and cannot explore the different semantics of various edge types. As a result, it cannot be applied to the mobile app usage graph directly.

In this work, we leverage a relational graph convolutional operation consisting of relation-specific propagation and aggregation phases. For each edge type, we implement a corresponding layer. Therefore, the relational graph convolutional operation on the app usage graph exploits six layers: the user-app, user-location, user-time, app-location, app-time, and location-time layers. These layers use the edges of corresponding relations, which is equivalent to information propagation and aggregation in the respective bipartite subgraphs. Given a node i, by the relational graph convolutional operation of relation \(r_e \in R_E\) , we will obtain a learned relation-specific feature vector \(\boldsymbol {h}^{r_e}_i\) of node i, which can be calculated as follows:where \(\hat{\mathcal {N}}_{r_e}(i)\) denotes the set of sampled neighbors of node i under relation \(r_e \in R_E\) , \(\hat{\omega }(j,i)\) is the normalized edge weight of edge \(e(j, i)\) , \({\bf W}_{r_e}\) represents the relation-specific transformation weight matrix of relation \(r_e\) , \(\hat{\boldsymbol {h}}_j\) stands for the projected feature vector of node j, and \(\sigma (\cdot)\) is an activation function.

Intuitively, (7) accumulates transformed feature vectors of neighbor nodes through a set of relation-specific edges, which have homogeneous semantics. Due to the symmetry of the app usage graph, all types of nodes have three types of edges. Given an arbitrary node i, after feeding the node feature into the relational graph convolutional layer, we can obtain a group of relation-specific node feature vectors, denoted as \(\lbrace \boldsymbol {h}^{r_e^0}_i, \boldsymbol {h}^{r_e^1}_i, \boldsymbol {h}^{r_e^2}_i\rbrace\) . To better understand the relational graph convolutional operation, we briefly explain the processes in Figure 4.

We next fuse together the multiple relation-specific feature vectors to update new features of nodes. Importantly, we need to ensure that the new feature vector of a node can also be informed by the corresponding previous feature vector. Therefore, as depicted in Figure 4, we add a self-loop of a specific relation type to each node. Given a node i, we can calculate its self-relation feature vector \(\boldsymbol {h}^{s}_i\) aswhere \({\bf W}_{0}\) is the weight matrix of self-relation and \(\hat{\boldsymbol {h}}_i\) is the projected feature vector of node i. In this way, for an arbitrary node i, it has four relation-specific feature vectors—that is, \({\bf H}_i = \lbrace \boldsymbol {h}^{r_e^0}_i, \boldsymbol {h}^{r_e^1}_i, \boldsymbol {h}^{r_e^2}_i, \boldsymbol {h}^{s}_i\rbrace\) .

Inspired by the vanilla attention approach [29], we first use a one-layer Multi-Layer Perceptron (MLP) with the activation function of \(\tanh\) to transform relation-specific feature vectors. We then compute the importance of different relation-specific feature vectors by multiplying an attention vector \(\boldsymbol {c}\) . Given a node i, the importance of relation \(r_e \in R_E\) can be calculated aswhere \(\bf W\) is the weight matrix, \(\boldsymbol {b}\) is the bias vector, and \(\boldsymbol {c}^{T}\) is the attention vector. Note that we have added the self-relation into the set of \(R_E\) . Next, we normalize the importance across different relation-specific features with the softmax function. By denoting the normalized weight as \(\beta _{i, r_e}\) , we can compute \(\beta _{i, r_e}\) asThe higher the \(\beta _{i, r_e}\) , the higher the contribution of \(\boldsymbol {h}^{r_e}_i\) toward the new feature vector of node i. Therefore, by using the learned weights as coefficients, we can fuse these relation-specific feature vectors and update the new feature vector \(\boldsymbol {h}^{\prime }_i\) as follows:In this way, the updated feature vectors of nodes aggregate all semantics hidden in multiple relations.

Since we formulate user profiling as a multi-label classification task, the last layer of our model is responsible for predicting the labels of users based on the representations of user nodes. Given the set of user profile labels Y, we employ the softmax function on the users’ representation matrix and obtain \({\bf Z} \in \mathbb {R} ^{|U|\times |Y|}\) (i.e., the predicted probability distribution of users’ labels). We then adopt the cross entropy as the loss function to carry out end-to-end training for the model. Mathematically, the loss function over all of the users labeled is defined aswhere \(\boldsymbol {y}_u\) and \(\boldsymbol {z}_u\) are the ground truth and the predicted probability distribution of user u, respectively. Guided by the labeled data, we can optimize our proposed model through the back-propagation method.

We first evaluate the classic models on three scenarios: the app-based scenario, location-based scenario, and app-location-joint scenario. For the app-based scenario, we exploit the used apps for gender prediction on the MNO dataset and age group prediction on TalkingData. We treat each app as a dimension and represent the input feature of each user as an app-based vector. For each app, the corresponding value is the normalized frequency of usage. Similarly, for the location-based scenario, each user is represented as a location-based vector to indicate the normalized frequency that the user visits the corresponding location. For the app-location-joint scenario, we jointly explore users’ used apps and visited locations for user profiling. In detail, for each user, we concatenate her or his app-based feature vector and location-based vector together. The performance of classic models for user profiling under three different scenarios is presented in Tables 4 and 5. Additionally, we evaluate graph-based models on the heterogeneous app usage graph and depict the results in Tables 6 and 7. GCN-Sampling and GAT-Sampling refer to the GCN and GAT models with our proposed neighbor sampling operation. From the results, we have the following key observations.

First, MRel-HGAN performs best among all methods, including both classic models and graph-based models. Specifically, as shown in Table 6, MRel-HGAN outperforms the best baseline by 5.26%, 4.38%, 3.99%, and 3.49% in terms of AUC, PRE, Macro-F1, and Micro-F1 on the MNO dataset for predicting users’ gender. In addition, MRel-HGAN outperforms the best baseline by 4.22%, 2.48%, and 2.98% in terms of PRE, Macro-F1, and Micro-F1 on TalkingData for predicting users’ age groups.

Second, compared with visited location information, the used app information is more useful for the task of both gender prediction and age group prediction. As shown in Tables 4 and 5, all classic models’ performance in the app-based scenario is better than that in the location-based scenario. Moreover, classic models perform poorly on the app-location-joint scenario, implying that simple concatenating operation is insufficient for coping with heterogeneous features and cannot explore the hidden relationships between different types of features.

Third, graph-based models generally outperform classic models. The main reason is that the graph structure can capture spatiotemporal app usage behavior across various users very well. By traveling through the local graph structures, graph-based models can learn the hidden relations between users, apps, locations, and time, which are helpful for user profiling.

Fourth, GCN performs the worst for the task of gender prediction compared to other graph-based models. As stated in Section 4.1, the reason may be twofold: oversmoothing and varying node degrees, which can be solved by the neighbor sampling mechanism. By applying our proposed neighbor sampling operation, GCN-Sampling achieves satisfactory performance.

Fifth, GAT outperforms GCN because GAT applies the attention mechanism to automatically estimate the importance of neighbors, which will mitigate the issues of varying node degrees and oversmoothing. In addition, GAT still obtains performance gain from the neighbor sampling mechanism.

Sixth, HAN achieves the best performance among all baselines. This is because HAN formalizes meta-path-based homogeneous graphs. Such a meta-path-based structure can leverage the semantics of different types of edges to enhance the performance of learned embeddings of nodes. However, HAN discards intermediate nodes along the meta-path when constructing meta-path-based homogeneous graphs. Therefore, compared with MRel-HGAN, HAN cannot leverage the node features of intermediate nodes, which leads to information loss and performance degradation.

In this section, we compare the performance of MRel-HGAN with the following four variants:

The results of MRel-HGAN(L), MRel-HGAN(A), MRel-HGAN(No-S.), MRel-HGAN(No-A.), and MRel-HGAN are presented in Figures 5 and 6. The following can be observed:

The results demonstrate that the modules we design, including heterogeneous neighbor sampling, relational graph convolutional operation, and multi-relational attention operation, are necessary to integrate the multi-relational data in the heterogeneous network.

The relationships between various user profiles and variations in mobile app usage behavior have been the subject of several studies. Zhao et al. [44], for instance, analyzed the user-group level. They identified 382 distinct categories of users based on user behavior in mobile apps. They then assigned user groups semantic labels, such as night communicators, financial users, and evening learners. In a descriptive examination of how gender and age affect app usage, Andone et al. [9] found that both factors are influenced. Teenagers between the ages of 12 and 17 years are found to spend more than 40 minutes each day, on average, using communication, social networking, and gaming applications. However, those older than 30 years only use these applications for less than 10 minutes at a time. Peltonen et al. [45] examined how consumers’ cultural backgrounds affect their usage patterns. The researchers identified three main categories of cultural affiliations with corresponding use patterns: European, English-speaking, and mixed cultures. They did this by using app category usage as a feature using the hierarchical clustering technique. Zhao et al. [46] conducted a good survey to summarize recent studies on user profiling from their use of smartphone applications.

Predictive analytics, or the inference of user profiles using characteristics retrieved from app usage logs, have been done in certain studies [47]. Seneviratne et al. [22], for instance, collected data on 200 users’ mobile app usage and used SVM to infer gender classifications based on the users’ app adoptions. Additionally, Malmi and Weber [10] used a bigger dataset encompassing 3,760 individuals to validate the research of Seneviratne et al. [22] and forecast additional demographics, such as income and race. They found that while wealth is difficult to forecast, gender is the factor that is most difficult. Zhao et al. [7] used SVM and MLP to estimate the gender of users after extracting specified subject characteristics from app descriptions. Additionally, personality qualities are a crucial aspect of users, and considerable work has been done in this area. For instance, Peltonen et al. [48] and Montjoye et al. [49] used metrics based on mobile phone usage and app usage traces to infer personality aspects of consumers. A federated learning framework was also suggested by Brandão et al. [50] to safeguard user privacy when doing user profile activities. These studies, however, do not examine the spatiotemporal characteristics of app usage and are restricted to the app dimension. Additionally, because these systems cannot automatically extract relationships between people, applications, places, and time, they must either build handmade features or fusion methods, which restricts their applicability and efficiency. Notably, some earlier studies may attain competitive performance with carefully specified elements, such as app start and shut events, mobile phone brands, and battery drain patterns. However, on a broad scale, service providers and network operators find it challenging to gather the characteristics they utilize.

Graph-based representation learning aims to learn low-dimensional vectors to represent vertices in a graph by exploring the local structure of vertices. Traditional approaches are based on matrix factorization using an adjacency matrix [51] or Laplacian matrix [52]. Inspired by the word embedding model [37], some algorithms were proposed to learn vertex representations based on random walks [36, 38]. Generally, they applied a random walk algorithm to traverse the graph and generated a series of node sequences. They then obtained node embeddings by treating node sequences as the equivalent of sentences and feeding them into the skip-gram model. In particular, Perozzi et al. [36] employed the conventional random walk algorithm, whereas Grover and Leskovec [38] used a biased random walk procedure to explore different graph structures. Moreover, to cope with the heterogeneous graph structure, Dong et al. [39] proposed a meta-path-based random walk approach, which can leverage the semantics of different types of nodes to enhance the performance of node embeddings.

Many graph embedding approaches based on GNNs have recently emerged. The GNN, which learns vertex representations by taking into account both local graph structure and node attributes, has performed well in a variety of tasks, including recommendation [53] and node classification [40]. Kipf and Welling [40] presented a powerful model: GCN. GCN is a kind of GNN that uses the graph Laplacian matrix to conduct convolutional operations in the graph Fourier domain. Furthermore, by substituting the convolutional operation with an attention operation, Veličković et al. [41] introduced GAT. During the aggregation operation, the attention mechanism decides the relative relevance of neighbor information for the target node. GAT, however, discards edge weight information, limiting its effectiveness on weighted graphs. Furthermore, Wang et al. [42] suggested HAN to deal with the problem of heterogeneous graphs by integrating meta-paths and GAT. They initially turn a heterogeneous network into several meta-path-based homogeneous graphs. They then employed an attention method to merge the meta-path-specific embeddings of each node. However, HAN failed to account for intermediate nodes along the meta-path, resulting in information loss and performance limitations.

This article studied the problem of user profiling from their spatiotemporal mobile app usage behavior based on real-world datasets. Notably, our proposed framework is a general model for various user profiles. However, limited by the available datasets, we can only evaluate it with gender labels and age groups, which is a major limitation of our work. Additionally, in this work, we only take app categories as the feature of apps, which may limit our model’s performance. Some previous studies suggested that app icons [7], app descriptions [54], and user comments on apps [55] are also helpful in deriving app features. Hence, enriching app features with cross-domain data is a future step of this work.