Home Is Where the Smart Is: Development and Validation of the Cybersecurity Self-Efficacy in Smart Homes (CySESH) Scale

An overarching literature survey on cybersecurity self-efficacy was conducted to gain a general review data set on the topic (file link: preregistration of literature survey). Multiple simultaneous literature reviews originate from this data set each with a different scope, such as a psychometric quality assessment [19], discussion on implications for practitioners, or as in this case a contextual overview of self-efficacy measures. The review data set has two main contributions to this scale development work. First, it reinforces the posited need of low-cost, standardized, and validated research methods in the field of cybersecurity self-efficacy by indicating the current extent of measurement heterogeneity. Second, it serves to identify a candidate for convergent scale analysis by assessing the context fit to CySESH and general representativeness of previously published scales. Consequently, it was the goal of the literature review to analyze current heterogeneity or item overlap in measures and identify relevant scales for convergent analysis.

Methods for gathering and extracting data of the literature were preregistered on OSF. Items were collected by three trained coders. Training was completed when an inter-rater agreement coefficient of ι > 0.6 for main variables was reached. Each coder extracted data from 2/3 of the publications, so that all publications were coded twice to ensure high quality review data. Given the interdisciplinary nature of cybersecurity self-efficacy research, we systematically queried nine scientific databases (EBSCOhost, IEEE Xplore, ACM Digital Library, Science Direct, dimensions.ai, arXiv, Scopus, Web of Science, and Wiley Online Library). To decrease bias in our keyword selection, we implemented a quasi-automated text mining and keyword co-occurrence network method as introduced by Grames et al. [54] (R version: 4.0.3; litsearchr version: 1.0.0). As a result, we used the following keyword string:

We identified 173 different self-efficacy measures across 174 studies published in the past decade (list available at: data of literature review). Measures relied on ad-hoc development and different conceptual understandings of self-efficacy stemming from multiple theories, such as reactive control [97] or as a general trait [112]. Hence, we did not use the listed scales of the literature review as sources for our item development. We coded scales for their item-based heterogeneity. Specifically, we collected all provided items and then subjectively coded the mutual content of each scale’s items to categorize the scale’s overall context. For that reason, heterogeneity of scales is determined by the content of the respective items. A thematic synthesis strategy, involving open coding with emergent categories, was applied to identify three contexts [17]: (a) computer or internet usage, (b) coping or compliance behavior, and (c) privacy or IT security implementation. Computer or internet usage scales (4 scales) focus rather on the primary tasks of usage [67, 127]. Coping or compliance behavior scales (56 scales) aim at emotional, cognitive, or behavioral processes used in stressful situations [23, 69]. Privacy or IT security implementation scales (65 scales) use indicators of an individual’s cybersecurity performance or application [100, 129]. Consequently, we focus on scales from the last context that would concern cybersecurity self-efficacy in smart homes.

To determine whether CySESH is sufficiently close to content-similar constructs and in consequence a meaningful measure of latent cybersecurity self-efficacy, we analyzed its convergent validity. For this purpose, we selected a suitable self-efficacy scale from the context of privacy or IT security implementation: the Self-Efficacy in Information Security (SEIS) scale by Rhee, Kim and Ryu [100]. The scale’s representative and general character is highlighted by recurring references within other ad-hoc scale development works [39, 75, 94] and significant correlations (ranging between r = .60 and r = .85) with a number of alternative cybersecurity self-efficacy measures. Rhee, Kim and Ryu [100] report several steps taken to ensure validity in item construction. The SEIS items involve skills related to the protection of information or systems more generally and hence, overlap with our own construct definition. Therefore, we expected a moderate correlation between SEIS and CySESH suggesting evidence for convergent construct validity.

However, the SEIS lacks context specificity. This would raise the question to which degree the SEIS scale is reliable and valid for the domain of cybersecurity in smart homes. One reason for difference is that the SEIS scale, published in 2009, was not designed to include present-day security issues relevant to smart homes, such as linkage, cookies, and cloud services. Items of the SEIS scale focus on more abstract protection skills related to personal computers and browser usage [100]. Some SEIS items refer to behaviors not broadly applicable to smart home devices, or they may even tap into other psychological traits that should be distinguished from self-efficacy [12, 14], e.g. outcome expectation or hypothetical assumptions about future skill acquisitions. Based on results from an expert workshop on smart home security stages, CySESH provides new domain-appropriate and contemporary items that are intended to reflect the recommended construct specificity. Items were exclusively generated by deductions from SCT theory as well as the workshop results. We piloted the scale in two expert studies (N = 105) and evaluated the remaining items with another two studies (N = 1, 137). Results indicate CySESH’s psychodiagnostic quality in terms of reliability and validity.

Given the importance of content validity, its several sources (representativeness [35], relevance [118], domain clarity [96], and technical quality [57]) were consulted in three consecutive pilot studies to balance the approaches. We included a mix of content experts (workshop study), test development experts (first pre-testing study), and native English speakers (second pre-testing study) to generate items that provide good domain coverage with adequate psychometric performance and comprehensibility.

In the workshop study, security experts discussed potential indicators of CySESH’s operationalization with the goal to achieve content representativeness and relevance [35, 118]. That is, the expert workshop was used to design the scale, not the literature review. This included six steps following Bandura’s guide for scale construction [12]: (1) insights about multicausality, i.e. what dynamic factors influence the successful configuration of security in smart homes; (2) phases of pursuit; i.e. what chronological stages exist over which users can exercise control; (3) content validity of latent psychological constructs - behavioral, cognitive, affective - involved in cybersecurity; (4) definitions, response scales, and practice items; (5) smart home security tasks and activities, i.e. what abilities, skills or knowledge can be applied by users to succeed; and (6) challenges and impediments hindering regular security behavior. The five workshop participants were security experts in different areas of smart home cybersecurity: one software engineer with security expertise, two HCI designers with security expertise, one security expert, and one legal practitioner that is a privacy expert. We recruited experts through our affiliated partners of the funding consortium to identify crucial aspects of secure smart home technology use and adoption that are tied to self-efficacy. To achieve psychometric validity, the scale is required to cover user actions actually relevant to the implementation and maintenance of smart home security – including important actions and behaviors that lay users may simply not be aware of. Further, lay people might have misconceptions about actions they frequently take but that do not actually improve security (e.g., scheduled password changes with complicated password rules). Lay people were involved in the pre-testing study 2 to make sure that the items that cover relevant behaviors identified by security experts are phrased comprehensibly.

The qualitative analyses of the unstructured workshop data, which were recorded through written notes, combined multiple synthesis schemes. Descriptive grouping contributed to analyzing the cybersecurity stages. Results revealed five stages in which users can apply abilities, knowledge, and skills to increase the cybersecurity of their smart home devices: (1) purchase, (2) commissioning, (3) daily use, (4) maintenance tasks, and (5) decommissioning. As a primary goal, participants generated a concept map to collect relevant security tasks and activities (file link: concept map of expert workshop). We used a thematic synthesis of the security tasks and activity data to examine the views of involved experts and identify common topics. The resulting 16 common topics were: data deletion, device connection, data collection, investments in devices, privacy policies, third party communication, setup assistant, settings, installation, password management, professional help, meaning of settings, understanding manufacturer declarations, updates, dark patterns, and threats. Other common themes of the discussion focused on: (a) performance demands, such as self-regulatory processes of users, (b) limitations of user control because their security would rely on technological solutions, and (c) limitations of circumstantial user control, i.e. people that did not choose to use a smart home device, such as children or guests.

Items were derived from the resulting five stages and 16 key topics of the discussion using the framework synthesis method, which is a structured technique that fits key topics to the cybersecurity stages. For each stage and each topic one or more items were developed. This analysis is only one of three aspects on which we based the generation of items (the other two being of theoretical nature: definition and assumptions of CySESH). Data of the framework analysis and resultant items are accessible on OSF (file link: data of expert workshop).

Self-efficacy items should be domain-specific and discriminant, i.e. allow distinguishing them from other, related psychological processes [62]. Discriminant validity is tested by assessing CySESH’s distinction from constructs that should be closely related to self-efficacy but have established differences on a definitional dimension or by theoretical considerations [8, 79, 81]. This so-called nomological network of construct relations [22, 36, 62] puts forward trait-level differences from which insights can be best obtained.

Consequently, we expected small correlations between CySESH and selected discriminant constructs: self-esteem, optimism, and outcome expectations. Self-esteem is typically defined as a self-reflective overall evaluation of one’s value or worth as a person [26, 102]. If a person believes to be capable of a certain behavior (i.e., high self-efficacy) and if that capability is valued, then the self-efficacy belief can positively impact one’s self-esteem [79]. Optimism is the dispositional expectancy that positive outcomes will prevail throughout one’s life [106, 108]. Even though optimism influences persistence behavior similarly to self-efficacy, there is no agency sensitivity to attain the positive outcome state [81]. Relations between both constructs are still expected since highly self-efficacious people tend to think more optimistically [14]. Outcome expectations are relatively defined as “believed consequences of a person’s behavior” [42] and can be loosely categorized in physical, affective, and social outcomes that are either positive, negative, or neutral [42, 77]. These concern the results of performing a behavior while self-efficacy is the believed capability of performing it [8].

In September 2021, we conducted two pilot studies to thoroughly test the developed item sets. The objective of the first study was to refine content validity (domain clarity and technical quality of the generated items [57, 96]) for which experts on test development - with at least a Bachelor’s degree in psychology - were sampled. Expertise on theory-based, latent psychological constructs and their measurement is of high relevance to e.g., reassure the fit of generated items and the conceptual definition of CySESH, recognize missingness in structure, test instructions, or trial items, and be aware of deviations in scoring or other flaws. Participation advertisements were distributed on social media and university-internal job platforms. The experts received 15 EUR as compensation.

In the second study, native English speakers who preferably owned smart homes devices were recruited to ensure the comprehensibility of items and instructions as the final part of technical quality. This sample was acquired through Prolific and rewarded with the recommended payment rate of 7.50 GBP per hour (i.e., 1.75 GBP for participation in this study). The minimum required age for participation in both studies was 18 years.

Both survey flows began with an informed consent form, followed by sociodemographic questions, sample-specific instructions (focus on validity vs. comprehensibility), and the pilot version of CySESH. After each CySESH pilot item, experts were asked to provide remarks on the item in a text box.

A total of 23 psychologists (13 with a Bachelor’s degree, 10 with a Master’s degree) participated in the first study (17 female, 5 male, and one non-binary person with a mean age of M = 25 (SD = 2.32)). Participants also reported their English proficiency (lowest B2).

In the second study, we recruited 82 native English speakers (55 female, 26 male, one non-binary person). The sample’s mean age was M = 33.16 (SD = 10.47). For the educational level, 34 participants stated that they had a Bachelor’s degree, 33 finished the secondary school, 9 had a Master’s degree, and 6 graduated from trade school. Three participants indicated they have or pursue a degree in psychology. Most participants (91%) owned at least one smart home device.

Items with negative remarks in the two pilot studies (N = 105) were either adjusted (7 items: A1, A5, A11, A12, A16, A28, A33), marked for exclusion (9 items: A3, A14, A15, A21, A23, A24, A32, A37, A39) pending further evidence of quantitative data, or in fact excluded (10 items: A17, A20, A22, A25, A27, A30, A34, A40, A41, A46) regardless of quantitative results. 19 items without negative remarks were eligible for continued inclusion (A2, A4, A6 - A10, A18, A19, A26, A29, A31, A35, A36, A38, A42 - A45).

Generally, participants commented on potential ambiguities, unknown technical terms, or perceived misfits of items and the provided CySESH definition, e.g. “maybe specify [...]” (participant #17) or “I think I would find it good if similar terms were always used [...]” (participant #12). Accordingly, we aligned terms (“devices” and “attack”), reworded items to reduce ambiguity (“safety” and the source of collection for A5), and simplified sentence structures (A11 and A5). Items with multiple remarks or remarks that required exhaustive changes were marked for exclusion, e.g. “the term resources may be unclear in this context” (participant #13). Strong ambiguity was perceived with regard to the terms “arrange” and “things” (A14) as well as “disadvantageous” and “trick me” (A23 and A24). This applies also to item A21 (setting up multiple devices), where the extent of the statements seemed unclear to participants. Comments on A3, an item about deletion of local data, were resolved by specifying the instructions of the scale on referred devices. The item was kept after confirming results of the quantitative decision stage to balance the content of A2, which regards the deletion of cloud data. Another item about the deletion of cloud data (A15) had remarks on the ambiguous meaning of the type of data and the definiteness of deletion. Instead of rewording the item, A15 was removed because A2 outperformed A15 and both statements cover the deletion of cloud data. Severe negative remarks, for example “fine, but motivation =/= CySESH. Don’t know if the definition above is confusing?” (participant #08) or “what exactly are bad settings?” (participant #60), lead to the exclusion of the respective pilot item. In hindsight, we realized that A17, which has to do with open ports, used terms that might be too technical for lay users. A27 (concerning the protection of information) was formulated very vaguely and pointed rather towards successful consequences of the behavior. Severe concerns were also voiced for item A34, which is about the detection of bad settings. Participants remarked that responses could refer to different subjects; either themselves, the functionality of the device, or the manufacturer.

Due to the small sample size, the quantitative analyses were only performed on the data collected from the native English speakers in the second pilot study (N = 82). Results can be accessed on the OSF (file link: results from CySESH pilot studies). We calculated an initial factor estimate using Horn’s parallel analysis. It demonstrated a one factor solution, which supported our goal of developing a unidimensional scale. An exploratory factor analysis with a promax rotation partly confirmed this finding with mostly satisfactory factor loadings, but not yet exclusively acceptable model fit indices (TLI = 0.734;RMSEA = 0.070;RMSR = 0.09;BIC = −2831.60;χ²(945) = 1332.75, p < 0.001).

Table 2 shows the three psychometric criteria that items had to satisfy to be considered for subsequent scale validations. Item selection based on these criteria leads to scales that achieve a centralized distribution of responses, best differentiate between high-scoring and low-scoring users, and include homogeneous items. Item analysis results for all criteria are shown in Table 1. We dropped 22 items (A16, A19, A20, A22, A25, A26, A29 - A33, A35 - A41, A43 - A46) because responses to those items indicated issues with ceiling effects (possibly due to low complexity of the security task). We removed another item (A42) that did not meet the criteria for minimum item-total correlation coefficients. This was also the only item in the pilot version which was inverted. No remaining items showed lower standard deviations than defined. Consequently, 22 items (A1 - A12, A14, A15, A17, A18, A21, A23, A24, A27, A28, A34) were left for further selection processes.

Items were required to pass both analyses perspectives to be eligible for the scale validation stage. Of the 13 eligible items (A1, A2, A4 - A12, A18, A28), 2 were redundant (A18, A28) in content with qualitatively slightly better performing items (A4, A12) and therefore excluded for reasons of test economy. However, the remaining items only tangentially covered the forth stage of cybersecurity (maintenance tasks). To compensate this imbalance, we developed a new item about software updates (B13). Another imbalance was created by including A2 (deletion of cloud data) and the ineligibility of items on the deletion of local data. As a solution, we decided to include A3 and revise its qualitative issues by including more specific scale instructions. The final 13 items included in the validation version of CySESH are provided in Table 5.

Data were collected in October 2021 via an online questionnaire designed in Qualtrics. An a priori power analysis (using the R package pwr) indicated a minimum sample size of 138 participants for an assumed correlation of r = .30, power of .95, and α = .05. We recruited participants (at least 18 years old, native language English) from Prolific and paid them the recommended hourly rate of 7.5 GBP (i.e., 1 GBP for participation in this study).

The objective of the initial validation study was to estimate CySESH’s discriminant validity from outcome expectation. The survey flow began with an informed consent form, followed by the validation version of CySESH (items B1 - B13 listed in Table 5) paired with outcome expectation items, and questions about participant demographics. We included two attention check items (see CySESH scale validation version and measures of initial validation study). In Table 3, we provide an overview of the measures used in the initial validation study.

The Outcome Expectation (OE) scale was adopted from Maddux, Norton and Stoltenberg [80] and consists of 2 generic items. Those two generic items are specified as regards content for each CySESH item (B1 - B13), making the scale a total of 26 OE items. This pairing of self-efficacy and outcome expectation is inevitable because CySESH items each describe a certain skill of which the effectiveness of doing so is being assessed by the OE items. Example pairings are for B1: “For those who can use devices’ privacy policies for a risk assessment of their privacy, it is a very effective way to improve their IT security or privacy” (OE1-B1) and “If I were able to use devices’ privacy policies for a risk assessment of my privacy, it would improve my IT security or privacy” (OE2-B1). The measure was primarily chosen due to the applicability of the generic items and its comprehensive discussion on the role of OE in SCT and self-efficacy theory.

A total of 229 people participated in the initial validation study. Participants were excluded from analysis if they stated to not own a smart home device or failed at least one attention check, which reduced the sample size to 166 participants (119 female, 45 male, and 2 non-binary). The mean age of participants was M = 33 (SD = 11.54) years. Participants had a relatively high level of education with 74 people reporting a Bachelor’s degree, 46 having graduated from secondary school, 23 with a Master’s degree, 19 trade school graduates, 2 PhD level participants, 1 primary school graduate, and 1 participant who preferred not to say. 84 participants stated being full-time and 23 part-time employees, 26 were students, and 33 declared other statuses (e.g., retirement).

The main validation study was pre-registered on the OSF prior to data collection with detailed methodology records and an R script for statistical analyses plans (file link: CySESH preregistration). Data were collected in January 2022 using Qualtrics. Participants (at least 18 years old, native language English) were recruited from Prolific and received the recommended payment rate of 7.5 GBP per hour as reward (i.e., 1.25 GBP for participation in this study). We calculated an a priori power analysis in R (package: semPower) to plan the sample size needed to adequately power our registered structural equation model. Results indicated a minimal sample size of N = 461 with a power of .95, an α = .05, and an expected model fit of RMSEA = .04 for a unidimensional model with 13 items. To account for dropouts, missingness, and additional exploratory analyses, we aimed to approximately double the required sample size.

The primary objectives of this study were to: (1) select items for a final CySESH version, (2) test the reliability of the CySESH items, (3) demonstrate its convergent validity to a closely related measure, and (4) verify its discriminant validity to theoretically related but distinct psychological traits. The survey began with an informed consent form, followed by four scales as well as three attention check items that were all presented in a randomized order, and ended with demographic questions (see Table 4). All items and scale instructions are accessible on the OSF (file links: CySESH scale validation version and measures of main validation study). First, cybersecurity self-efficacy in smart homes was measured with the validation version of CySESH (items B1 - B13 listed in Table 5). Second, for self-efficacy in information security, we used the SEIS scale [100]. It is a representative scale of its field and demonstrates acceptable scale quality criteria [100]. Third, self-esteem was measured with the Rosenberg Self-Esteem (RSE) scale [101]. We selected the RSE scale because of its long history and significant popularity in psychological self-esteem research [61], as well as its high quality [31, 34]. Lastly, we used the Life Orientation Test Revised (LOT-R) [107] to measure optimism. The LOT-R is a highly established measure of dispositional optimism [25, 109] that repeatedly demonstrated its psychometric quality [30, 51].

In the main study, a total of 1,068 survey responses were collected. We excluded 97 participants because they either indicated to not own a smart home device, failed at least one attention check, had missing responses on CySESH, or did not complete the demographic questions. The remaining 971 participants (613 female, 352 male, 4 non-binary, and 2 preferred not to say) had a mean age of M = 37.9 (SD = 13). 397 people had a Bachelor’s degree, 357 graduated from secondary school, 125 had a Master’s degree, 56 were trade school graduates, 22 had a PhD, 11 participants preferred not to say, and 3 were primary school graduates. For employment status, the sample included 532 full-time as well as 177 part-time employees, 83 students, and 179 people stated other statuses (e.g., retirement).

We applied the same psychometric criteria (see Table 2) as in the pilot studies. The aim was to select items with moderate difficulty that sufficiently differentiate self-efficacy strengths and homogeneously align with the overall scale. Based on these criteria, one item (item number B13) was excluded, see Table 5. Its mean score was higher then the defined limit (\(M = 5.2 > M_{max} = 5\)). The exclusion of item B13 was calculated to affect reliability marginally; Cronbach’s α dropped from 0.91 to 0.90. The other items (B1 - B12) performed well within the set thresholds (Table 5). Thus, the 12 final items of CySESH are: B1 - B12.

The distribution of CySESH mean scores is shown in Figure 2. The overall mean scale score was M = 4.15 (SD = 1.10) with a skew of g₁ = 0.05 and a kurtosis of g₂ = −0.06. These values imply a symmetric distribution that is mesokurtic. No violation of normal distribution assumptions or biased tendencies of the sample to agreement or disagreement are indicated.

We conducted four analyses to test unidimensionality. Inter-item correlations indicated a moderate to strong relatedness, r ranging between .29 and .61 (OSF file link: scale analysis of main validation study). A confirmatory factor analysis (CFA) using a maximum likelihood (ML) estimator showed mixed results for a one factor solution (TLI = 0.938;CFI = 0.949;RMSEA = 0.068;χ²(54) = 295.84, p > 0.001). We complemented this with another CFA model using a diagonally weighted least squares (WLSMV) estimator, which is specifically designed for ordinal data [74, 85]. This yielded better standard (TLI = 0.997;CFI = 0.998;RMSEA = 0.020;χ²(54) = 75.15, p < 0.05) and robust fit indices (TLI = 0.944;CFI = 0.954;RMSEA = 0.052;χ²(54) = 194.61, p < 0.001). Lastly, the path diagram presented in Figure 3 shows the standardized parameters of a unidimensional structural equation model. Loading coefficients of the 12 items reflect the assumed conceptually distant cybersecurity stages that were identified by experts in our first pilot study and still indicate strong relationships to a latent self-efficacy belief. A meta-analysis conducted by Peterson [93] found that for empirical factor loadings the average loading obtained is only λ = 0.32. The results from all four techniques support our validation approach.

For reliability analyses of CySESH items, we report two coefficients: Cronbach’s α as the lower limit of reliability [53] and McDonald’s ω as a better representation of the means [60, 119]. Both reliability estimates demonstrate excellent reliability; with α = 0.90 and ω = 0.90.

To assess convergent construct validity of CySESH, we compared the correlation between CySESH and SEIS against a defined threshold. The said threshold represents a correlation limit that is aimed to be reached or surpassed and is determined by the attenuation correction formula [70]. As described by Kristof [70] the attenuation correction formula is given by:

For each comparison i, we compute the attenuation correction ρ_{limit, i} via Eq. 1 to obtain specific thresholds. Here, rel_CySESH and rel_i represent the reliability estimates of CySESH and SEIS respectively. For convergent validity, we set ρ_ideal = 0.8 because we assumed similarity but domain-specific differences between CySESH and SEIS beliefs. The reported ρ_ideal = 0.8 deviates from our preregistered ρ_ideal = 1 because a correlation of r = 1 would mean a perfect positive relatedness between CySESH and SEIS, which was not anticipated. A correlation of r = 0.8 still signifies a very large effect size in human-related research [32, 45] and was therefore chosen as ρ_ideal for convergent validity.

The SEIS items had a reliability coefficient of α = 0.94. This resulted in a convergent limit of ρ_{limit, SEIS} = 0.73. The correlation between CySESH and SEIS was r = 0.64, p < .001 and thus, fell below the defined threshold of convergent validity with a difference of r_diff = −0.09. However, the expected trend of the relationship between CySESH and SEIS was supported.

For each discriminant validity analysis, we also calculated the attenuation correction ρ_{limit, i} via Eq. 1. To reflect the theoretical differences and yet relatedness between CySESH, OE, RSE, and LOT-R, we set ρ_ideal = 0.2. In the preregistration, ρ_ideal for discriminant validity was set to 0.1. Taking into account the meta-scientific discussion that nonzero correlations are to be expected between any given variables [90], we heightened ρ_ideal to signify the theorized relationships between CySESH and its discriminant constructs. Discriminant validity between two scales is affirmed if their correlation coefficient is equal to or below the computed threshold.

Results of the three discriminant validity analyses are shown in Table 6. The comparison of the correlation coefficients with their specific discriminant limits substantiate the validity of CySESH. Overall, the assumed discrimination between CySESH and its discriminant constructs was sustained, but outcome expectation did not reach the defined threshold. The study demonstrated CySESH’s excellent discriminant validity for self-esteem and optimism. Figure 4 shows the inter-construct correlations from both validation studies (left panel: main study; right panel: initial study) to illustrate the reported evidence.

An exploratory multiple regression was estimated to determine whether the CySESH scores were systematically affected by demographic variables (i.e., age, gender, educational level, and employment status). The reference group of the regression analysis are female participants of average age, who are full-time employed and have a Bachelor’s degree. Age and male gender significantly predicted CySESH scores; \(\beta _{\mathrm{age}} = -0.017, p < .01, 95\%\ \mathrm{CI}\ [-0.25, -0.09]\) and \(\beta _{\mathrm{male}} = 0.28, p < .01, 95\%\ \mathrm{CI}\ [0.15, 0.42]\). The other computed standardized regression coefficients were non-significant and ranged from β = −0.01 to β = 1.12 (OSF file link: exploratory analysis of main validation study). The demographic regression accounted together for \(R^2 = .05, 95\%\ \mathrm{CI}\ [0.01, 0.06]\). Reasons for significant estimates could for instance be that our large sample size over-powers otherwise null effects or that there is an imbalance between the different manifestations of the demographic variables.