Search Results (237)

Background/Objectives: The potential and limitations of chatbots in medical education and clinical decision support, particularly in specialized fields like psychiatry, remain unknown. By using the Rasch model, our study aimed to evaluate the performance of various state-of-the-art chatbots on psychiatry licensing exam questions to explore their strengths and weaknesses. Methods: We assessed the performance of 22 leading chatbots, selected based on LMArena benchmark rankings, using 100 multiple-choice questions from the 2024 Taiwan psychiatry licensing examination, a nationally standardized test required for psychiatric licensure in Taiwan. Chatbot responses were scored for correctness, and we used the Rasch model to evaluate chatbot ability. Results: Chatbots released after February 2024 passed the exam, with ChatGPT-o1-preview achieving the highest score of 85. ChatGPT-o1-preview showed a statistically significant superiority in ability (p < 0.001), with a 1.92 logits improvement compared to the passing threshold. It demonstrated strengths in complex psychiatric problems and ethical understanding, yet it presented limitations in up-to-date legal updates and specialized psychiatry knowledge, such as recent amendments to the Mental Health Act, psychopharmacology, and advanced neuroimaging. Conclusions: Chatbot technology could be a valuable tool for medical education and clinical decision support in psychiatry, and as technology continues to advance, these models are likely to play an increasingly integral role in psychiatric practice. Full article

The Bayesian information criterion (BIC) is a widely used statistical tool originally derived for fully observed data. The BIC formula includes the sample size and the number of estimated parameters in the penalty term. However, not all variables are available for every subject in planned missingness designs. This article demonstrates that a modified BIC, tailored for planned missingness designs, outperforms the original BIC. The modification adjusts the penalty term by using the average number of estimable parameters per subject rather than the total number of model parameters. This new criterion was successfully applied to item response theory models in two simulation studies. We recommend that future studies utilizing planned missingness designs adopt the modified BIC formula proposed here. Full article

Background: Children often present to primary and specialty care clinics with multiple somatic symptoms of nonspecific origin that can be highly distressing and prompt significant health service use. We evaluated the psychometric properties of the eight-item Children’s Somatic Symptoms Inventory (CSSI-8) as a brief measure of somatic distress that could be easily integrated into clinical systems. Method: Eight items from the 24-item CSSI were selected based on their representation of multiple bodily systems, association with high base rates, and ability to maximize the separation of the items’ Rasch measure scores. The psychometric quality of the eight-item scale was evaluated in 876 pediatric patients with chronic abdominal pain and a nonclinical sample of 954 school children using methods from three psychometric models (the classical test theory, Rasch modeling, and confirmatory factor analysis). Results: The CSSI-8 showed good measurement properties on an extensive array of psychometric criteria, had adequate Rasch person separation reliability for a brief instrument (r_sep = 0.74–0.75), and distinguished between clinical and nonclinical youth. Girls in both groups had significantly higher CSSI-8 scores than boys. Norms for the clinical sample are presented. Conclusions: The CSSI-8 is a psychometrically sound measure suitable for use as a brief dimensional assessment of pediatric somatic distress. Full article

Background/Objectives: Resources to cope with maternity and women’s participation are essential modulators of maternal well-being. Therefore, it is relevant that the psychosocial factors of woman be monitored during maternity to promote adequate healthcare. This study involved the design and the validation of two new tools that identify women’s knowledge of healthcare rights (MatCODE) and perception of resource scarcity (MatER) during pregnancy, labor and early postpartum; Methods: The content validity was carried out using the Aiken’s V coefficient and the content validity index (CVI-i) based on five experts. In addition, for the face validity, the pilot cohort was considered the INFLESZ scale. Finally, the questionnaires were applied to 185 women, which allowed to assess the construct validation by factorial and Rasch analysis. The divergent validity was also studied with validated psychological questionnaires; Results: MatCODE and MatER questionnaires received CVI-i and Aiken’s V > 0.80 values, and the INFLESZ demonstrated acceptable semantic understanding. The analysis confirms the unidimensionality of the questionnaires, with fit values for MatCODE of RMSEA = 0.113 [0.105; 0.122] and for MatER of RMSEA = 0.067 [0.063; 0.072]. The divergent validity showed significant and consistent correlations with the constructs assessed. For MatCODE, ω = 0.95 and α = 0.94, and for MatER, ω = 0.79 and α = 0.78; Conclusions: MatCODE and MatER are useful new tools for monitoring maternal healthcare, with adequate psychometric characteristics in the Spanish context. Full article

Figural matrices tests are frequently used to measure fluid intelligence. The HeiQ—an operation-oriented figural matrices test—was developed to tackle limitations of previous matrices tests, mainly the possibility of excluding distractors based on superficial features instead of actively solving the items. However, allowing for a total administration time of 60 min for the assessment of one construct is not feasible in many study designs. Thus, the goal of this study was to develop three short forms of the existing HeiQ. Two parallel 20-item short forms (the HeiQ-S A and HeiQ-S B) that are comparable in content as well as on a psychometric basis and a 6-item short form (the HeiQ-XS) were generated. All tests showed good internal consistency (Cronbach’s Alpha ranging from α = 0.82 to α = 0.86) and good criterion-related validity (correlations with high school grade (Abitur) ranging from r = −0.34 to r = −0.38); construct validity (correlations with the global intelligence scores of the Intelligence Structure Test 2000R were between r = 0.58 and r = 0.71). Further, all test versions showed to be Rasch-scalable, implying a uniform underlying ability. Thus, we conclude that all three newly developed short versions are valid tools for assessing fluid intelligence. Full article

Background: This study aimed to translate, culturally adapt, and psychometrically validate the PROMIS Anxiety Short Form 8a item bank into Arabic for the general population of Saudi Arabia. Methods: The PROMIS Anxiety Short Form was translated according to the FACIT group method. Second, psychometric validation was conducted on a convenience sample of 322 participants (mean age, 26 ± 10.4 years; predominantly female) from the general population in Saudi Arabia. Rasch analysis (Winsteps^® version 5.6.1) was used to examine category functioning, item fit, the person separation index, item difficulty, unidimensionality, and local dependency. Results: Translation and cultural adaptation demonstrated that most of the items were culturally suitable and conveyed the same underlying concepts as the original scale. The five response categories of the scale satisfied the category functioning criteria, and all items fit the underlying construct, with the exception of one item that demonstrated a misfit. The item difficulty demonstrated poor targeting for the sample population; however, the person separation index and reliability were good (2.67 and 0.88, respectively) and no local dependency was noted. Conclusions: The Arabic translation of PROMIS-A SF8a demonstrated good structural validity and psychometrics, making it a valuable tool for screening anxiety in Arabic-speaking populations. The application of this outcome measure shows promise for healthcare professionals and patients alike, as it contributes to the provision of high-quality care and formulation of appropriate treatment plans. Full article

Including motivational variables such as interest in the cognitive load framework is an ongoing process. Of particular interest is the question of how motivational variables influence the investment of mental effort. In this study, we investigated how topic interest affects the investment of mental effort in simple tasks. A total of 1543 students’ judgments regarding invested mental effort, perceived task difficulty, and topic interest for 32 tasks of a chemistry test were analyzed at the task level based on item response theory parameters. Additionally, objective task difficulty was calculated. The Rasch parameters were used for correlation and moderated regression analyses. The results indicated that when perceived task difficulty was low, students invested more mental effort in solving tasks of low topic interest compared to tasks of high topic interest. With increasing perceived task difficulty, the amount of invested mental effort rose for tasks of low as well as high topic interest. However, the difference between tasks of low and high topic interest in the amount of invested mental effort decreased as perceived task difficulty increased and even vanished when perceived task difficulty roughly corresponded to students’ performance capability. These results are in line with flow theory and the expectancy-value-cost model of motivation. When solving tasks that match their performance capability, students can experience a flow situation. However, when solving rather easy tasks of low interest, students can experience motivational costs in terms of additional effort, such as an increased need for motivational self-regulation. The results of this study provide a basis for systematically investigating and better understanding the relationship between interest, task difficulty, invested mental effort, flow experience, and emotional costs. Full article

Previous research on providing feedback on public speaking has investigated the effectiveness of feedback sources, namely teacher feedback, peer feedback, and self-feedback, in enhancing public speaking competence, predominantly individually. However, how these sources of feedback can be collectively harnessed to optimize learner engagement and public speaking performance still warrants further investigation. Adopting a pre- and post-test quasi-experimental design, this study randomly assigned four classes to four feedback conditions: Group 1 received teacher feedback, Group 2 self-feedback and teacher feedback, Group 3 peer and teacher feedback, and Group 4 feedback from all three sources. Both student engagement, measured using the Public Speaking Feedback Engagement Scale (PSFES), and their public speaking performance ratings, assessed using the Public Speaking Competency Instrument (PSCI), were validated using Rasch analysis. The inferential statistics revealed that Group 3 showed significant improvements across nearly all three dimensions of engagement, whereas Group 2 experienced significant declines in all dimensions of engagement except behavioral engagement. Group 3 demonstrated significantly greater engagement gain compared to Groups 2 and 4, indicating the synergistic effect of peer and teacher feedback in contrast to the limited impact of self-feedback. Additionally, all groups demonstrated significant improvements except for Group 2, which showed significantly lower improvement compared to Group 4. The following correlation analysis identified a significant correlation between the gain of students’ behavioral engagement and the gain of public speaking performance, whereas such association was absent between cognitive or emotional engagement and public speaking competence. This study suggests that peer feedback should be preceded by group discussion and supplemented with teacher feedback in classes for enhancing the teacher–student dialog, while self-feedback should be conducted after class to improve student engagement and public speaking performance. Full article

Background: Current and former athletes are one of the most at-risk population groups for disordered eating (DE), impacting their dietary practices, body composition, performance and health during and following their athletic careers. Few comprehensive DE screening tools exist for this group. To help address this, the current study utilised a mixed-methods approach of Classic Test Theory (CTT) and Item Response Theory (IRT) to develop and validate a DE screening tool suitable for current and former athletes. Methods: Novel scale development methodologies were used to develop and assess the validity (content, face, cross-cultural, construct), test-retest reliability, internal consistency reliability, factor analysis and Rasch analysis of a new DE scale. Results: A new validated Athletic Disordered Eating (ADE) screening tool was created, with 17 items and four subscales (food control, bingeing, body control, body discontent), with an internal consistency reliability of 0.91, excellent content and construct validity, an Intraclass Correlation Coefficient of 0.97 and excellent Rasch model fit. Conclusions: The ADE screening tool has been dually developed for research purposes and as a clinically applicable screening tool to detect DE in current and former athletes and is suitable for a global use across sporting categories, diverse genders and levels of competition. Full article

The study evaluates the impact of behavioral changes resulting from climate awareness on energy consumption and carbon emissions in European households based on the results of a two-stage survey addressed to individuals manifesting pro-ecological attitudes. In the first stage, the author analyzed 67 pro-environmental behaviors declared by the participants, identified a set of new sustainable choices, and compared them to the conservation habits used in Rasch and Campbell’s models. The 10 most popular initiatives undertaken by over 50% of participants were selected for further analysis. The influence of these initiatives on energy consumption and CO₂ emissions was assessed. A total of 24 impact indicators were identified at the building scale. Energy and carbon savings were calculated for 500 participants from 28 European countries and compared to the results computed for the 100 households used as a reference. The main conclusions from the research concern the significance of individual decisions at the building scale in the context of their actual environmental impact calculated for a larger scale. The comparative analysis showed that the highest annual energy (2292.1 MWh) and emission (267.02 tons of CO₂) savings resulted from the car-to-bicycle (or walking) transition on short-distance trips (declared by 79%) and from the transition from non-renewable to renewable energy sources (PV panels installed by 65% of respondents). Annual energy and emission savings reached, respectively, 1300 MWh and 262.6 tons of CO₂. The research findings help explain the critical importance of transforming the built environment towards renewable energy sources and supporting pedestrian and sustainable transportation. Full article

Background/Objectives: To evaluate the clinical and patient-reported outcomes (PROMs) obtained with an optimized version of a previously investigated trifocal IOL. Methods: Prospective non-comparative single-center study enrolling 29 patients (55–71 years) undergoing bilateral cataract surgery with implantation of the trifocal diffractive IOL Liberty 677CMY (Medicontur Medical Engineering Ltd., Zsámbék, Hungary). Visual and refractive outcomes as well as PROMs were evaluated during a 3-month follow-up: measurement of uncorrected and corrected distance (UDVA, CDVA), intermediate (UIVA, DCIVA) and near visual acuities (UNVA, DCNVA), defocus curve, patient satisfaction, photic phenomena perception, spectacle independence, and difficulty in performing some vision-related activities. Results: A total of 100%, 92%, and 80% of patients achieved a postoperative binocular UDVA, UIVA, and UNVA of 20/25 or better, respectively. Likewise, 100%, 80%, and 84% of patients achieved a postoperative binocular CDVA, DCIVA, and DCNVA of 20/25 or better, respectively. In the defocus curve, all mean visual acuity values were better than 0.15 logMAR for all defocus levels. A total of 95.8%, 95.8%, and 91.7% of patients referred to be satisfied with their distance, intermediate, and near visual vision, respectively. Mean overall Catquest Rasch calibrated score was −3.12 ± 0.98. Most of the patients were spectacle independent: far (95.8%), intermediate (95.8%) or near vision (91.7%). No bothersome or minimal to moderately bothersome halo, starburst, and glare was perceived by 83.3%, 83.4%, and 83.3% of patients, respectively. Conclusions: The trifocal IOL evaluated provides a visual acuity improvement, with high levels of spectacle independence, patient satisfaction, and perceived visual quality associated. Full article

Background: Abdominal pain is a common and often debilitating issue for children and adolescents. In many cases, it is not caused by a specific somatic condition but rather emerges from a complex interplay of bio-psycho-social factors, leading to functional abdominal pain (FAP). Given the complex nature of FAP, understanding its origins and how to effectively manage this condition is crucial. Until now, however, no questionnaire exists that targets knowledge in this specific domain. To address this, the Abdominal Pain Knowledge Questionnaire (A-PKQ) was developed. Methods: Two versions were created (one for children and one for parents) and tested in four gastroenterology clinics and one specialized pain clinic in Germany between November 2021 and February 2024. Children between 8 and 17 years of age (N = 128) and their accompanying parents (N = 131) participated in the study. Rasch analysis was used to test the performance of both versions of the questionnaire. Results: The original questionnaires exhibited good model and item fit. Subsequently, both questionnaires were refined to improve usability, resulting in final versions containing 10 items each. These final versions also demonstrated good model and item fit, with items assessing a variety of relevant domains. Conclusion: The A-PKQ is an important contribution to improving assessment in clinical trials focused on pediatric functional abdominal pain. Full article

Buildings exert a profound influence on the environment, with the design phase recognized as the pivotal determinant of a building’s overall performance. Green building design, in particular, introduces heightened complexity, where the attributes of the design team play a pivotal role in shaping performance outcomes. Consequently, the characteristics of the design team emerge as crucial factors in the enhancement of both green building design performance and client attributes. This study aims to empirically examine a model formulated to gauge the extent to which Effective Design Team Attributes contribute to the enhancement of performance in designing green buildings and influencing client attributes. To achieve this objective, a comprehensive questionnaire survey was administered to professionals within the architecture and engineering domains actively engaged in the design and consulting sectors of the building industry. The collected data underwent meticulous scrutiny for authenticity and dependability using the WINSTEPS 5.2.5 software before undergoing subsequent analysis. Statistical analyses were conducted using SPSS version 19, with Principal Components Analysis (PCA) and the Structural Equation Modeling (SEM) approach implemented through Amos version 18 to derive the most robust model. The findings underscore the pivotal role of an adeptly managed design team in significantly improving both the performance of green building designs and the qualities of clients. Rasch’s analysis confirmed the validity of our 5-point Likert scale for Design Green Building Performance (DGBP), Effective Design Team Attributes (EDTA), and Client Qualities (CQ). All items demonstrated excellent reliability, separation, and discrimination, ensuring robust data quality. Dimensionality tests revealed the appropriateness of response categories, indicating satisfactory scale performance. The Effective Design Team Model, validated through Principal Components Analysis (PCA), exhibited a satisfactory fit, supported by significant chi-square statistics, high goodness-of-fit indices, and acceptable root mean square residual values. Client attributes displayed a strong association with effective design team management, validating key model elements. The intricacies inherent in the design process can be mitigated by adopting the green design charrette approach. Consequently, the establishment of an effective design team, coupled with green design leadership, active participation, and clarity in roles and responsibilities, emerges as a potent strategy for elevating the performance level of green building designs. Full article

This study investigated Earth science students’ experiences with online education during the COVID-19 pandemic at the University of Tennessee, Knoxville, in the US. We used an existing survey from the online education literature, the Online Learning Environment Survey (OLES), which consists of three instruments: (a) community of inquiry (CoI), (b) Institutional Support (IS), and (c) Self-Directed Online Learning Scale (SDOLS). The survey rating subscales ordered from highest to lowest are autonomous learning, asynchronous online learning, institutional support, teaching presence, social presence, and cognitive presence, respectively, indicating interest for the online learning environment. Among all of the subscales, the asynchronous online category was rated the highest by the students. The data were then analyzed using Rasch modeling. According to the Rasch analyses, asynchronous online teaching represents the most favorable course delivery technique for geoscience education. Overall, the survey data show a general interest in online delivery and the effectiveness of the modality, thus indicating potential for evolving into an online Earth science program. Finally, also discussed are possible future extensions of the research (e.g., extending the research to other introductory online geoscience courses). Full article

Knowledge tracing plays a crucial role in effectively representing learners’ understanding and predicting their future learning progress. However, existing deep knowledge tracing methods, reliant on the forgetting model and Rasch model, often fail to account for the varying rates at which learners forget different knowledge concepts and the variations in question embedding covering the same concept. To address these limitations, this paper introduces an enhanced deep knowledge tracing model that combines the transformer network model with two innovative components. The first component is a multiband attention mechanism, which comprehensively summarizes a learner’s past response history across various temporal scales. By computing attention weights using different decay rates, this mechanism adaptively captures both long-term and short-term interactions for different knowledge concepts. The second component utilizes a quantized question embedding module to effectively capture variations among questions addressing the same knowledge concept. This module represents these differences in a rich embedding space, avoiding overparameterization or overfitting issues. The proposed model is evaluated on popular benchmark datasets, demonstrating its superiority over existing knowledge tracing methods in accuracy. This enhancement holds potential for improving personalized learning systems by providing more precise insights into learners’ progress. Full article