Validation and modification of existing bleeding complications prediction models for percutaneous renal biopsy: a prospective study

Introduction

Percutaneous renal biopsy (PRB) is a pivotal procedure in nephrology, instrumental in diagnosing the etiology and prognosis of renal diseases, guiding treatment decisions, and monitoring kidney transplant function and viability (Dhaun et al., 2014). Since its introduction roughly 70 years ago, PRB combined with histological analysis has become the gold standard in nephrology practice due to its ability to provide definitive insights into kidney pathology (Cameron & Hicks, 1997). However, despite its diagnostic significance, PRB is not without risks, with bleeding being the most frequent and concerning complication (Corapi et al., 2012). Bleeding complications occur in 11% to 37.7% of cases (Corapi et al., 2012; Li et al., 2024; Varnell, Stone & Welge, 2019), influenced by factors such as biopsy technique and patient characteristics. Severe bleeding can lead to prolonged hospital stays, increased treatment costs, and additional interventions (Hogan, Mocanu & Berns, 2016). In particular, the need for blood transfusions introduces immunological challenges, as it can increase sensitization and raise the risk of future immunologic complications for patients requiring kidney transplants (Lim et al., 2021). Given these risks, accurately predicting post-PRB bleeding is essential for patient management (Bhadauria et al., 2022). Effective risk prediction allows clinicians to tailor post-procedure care, including determining the appropriate length of bed rest and whether prophylactic treatments are necessary to prevent complications (Torigoe et al., 2022). Numerous studies have explored factors that may influence bleeding following PRB (Wang et al., 2022; Xu et al., 2021). Patient characteristics such as age, body mass index (BMI), and coagulation status have been consistently associated with bleeding risk (Li et al., 2024; Patel et al., 2021). Furthermore, impaired kidney function, low pre-biopsy hemoglobin levels, and elevated clotting times—such as activated partial thromboplastin time (aPTT) or prothrombin time (PT)—have also been identified as significant predictors of bleeding complications (Pombas et al., 2020).

In 2020, Schorr et al. (2020) developed a bleeding risk calculator that offers clinicians a practical tool for estimating post-biopsy bleeding risk based on pre-procedural patient characteristics. This model integrates variables such as age, BMI, platelet count, hemoglobin concentration, and maximum kidney size to calculate the individualized risk of bleeding. While this model has shown good predictive performance, it has limitations, particularly in not accounting for procedural factors that may influence bleeding risk. These factors are challenging to assess comprehensively in a predictive model.

The use of real-time ultrasound guidance during PRB has improved procedural safety and reduced complication rates. Ultrasound guidance not only aids in precise needle placement but also allows for the immediate detection of periprocedural complications, such as hematomas or arteriovenous fistulas (Zhan & Lou, 2023). In fact, performing an ultrasound immediately after the PRB has become standard practice to detect these complications early, thereby enabling timely intervention if needed. In 2021, Granata et al. (2021) emphasized the importance of performing a follow-up ultrasound after the PRB, even before moving the patient, to check for any adverse events such as hematomas, which is the most common periprocedural complications. Immediate ultrasound findings, such as the presence of a perinephric hematoma, could enhance the predictive accuracy of post-PRB bleeding risk by providing real-time information on complications that may have already developed.

Recognizing the limitations of existing models and the potential benefit of incorporating post-procedural data, we hypothesized that integrating both pre-procedure patient characteristics and immediate and 6-h post-PRB ultrasound findings would enhance the accuracy of bleeding risk prediction. To test this hypothesis, we validated the existing KBRC-5 model and compared its performance to modified models incorporating ultrasound data. We employed decision curve analysis (DCA) to evaluate the clinical utility of each model by assessing net benefit (NB) across a range of risk thresholds. DCA provides a comprehensive measure of a model’s practical applicability, offering valuable insights for clinical decision-making (Liu et al., 2024).

This study aimed to quantify the improvement in predictive performance achieved by incorporating ultrasound findings into bleeding risk models. We hypothesized that models combining patient characteristics with real-time ultrasound findings would outperform those relying solely on pre-procedural factors. The models were assessed using the area under the receiver operating characteristic curve (AUROC) and DCA. Furthermore, the added value of ultrasound data was evaluated through net reclassification improvement (NRI) and integrated discrimination improvement (IDI).

Materials and Methods

Results

Kidney biopsy risk calculator model for the prediction of bleeding complications

To evaluate the predictability of bleeding complications, the AUROC was calculated. Variables that yielded significant results in the multivariable analysis were included, alongside additional factors such as age, platelet count, hemoglobin levels, and kidney size on ultrasound. Although these additional factors did not reach statistical significance, they were included due to their potential clinical relevance in predicting bleeding risk.

Three models were developed and compared. The first model, termed KBRC-5, included the following variables: age, BMI, platelet count, hemoglobin, and kidney size on ultrasound. The second model, IKBRC, was an extension of KBRC-5 that added immediate ultrasound findings. The third model, SKBRC, also built upon KBRC-5 but included the size of the hematoma identified at the 6-h ultrasound. The AUROCs for the models were 0.683 (p < 0.001; 95% CI [0.635–0.731]) for KBRC-5, 0.786 (p < 0.001; 95% CI [0.744–0.827]) for IKBRC, and 0.867 (p < 0.001; 95% CI [0.833–0.902]) for SKBRC (Fig. 1).

Predictive nomogram and calibration curve for three models

Three nomograms combining significant predictors and clinically relevant variables were developed (Fig. 2). Each variable was assigned a score on a point scale, and by summing the total score, clinicians can reference the total point scale to draw a straight line down to determine the estimated probability of bleeding complications. Risk factors included immediate ultrasound hematoma, ultrasound hematoma size at 6 h. Protective factors included age, platelet count, hemoglobin concentration and BMI. In addition, calibration plots (Fig. 3) demonstrated good agreement between predicted and observed probabilities for all models, with Brier scores of 22.5 (KBRC-5), 18.9 (IKBRC), and 14.2 (SKBRC), confirming their reliability.

DCA for three models

DCA was conducted to assess the clinical utility of the three models (Fig. 4). For patients with a risk threshold between 25% and 77%, IKBRC provided greater NB than either treating all patients or treating none. Similarly, SKBRC showed higher NB when the risk threshold ranged between 10% and 95%. Both IKBRC and SKBRC, which incorporated ultrasound findings, outperformed KBRC-5 across all risk thresholds. This analysis underscores the improved clinical decision-making potential when incorporating real-time ultrasound data into bleeding risk prediction models following PRB.

Incremental predictive value of ultrasound data

NRIs and IDIs were calculated to evaluate the risk reclassification capability of the models. The addition of immediate (p = 0.000) or 6-h ultrasound (p = 0.000) increased NRI and IDI in the KBRC-5 model (all p < 0.05; Table 3, Fig. 5), indicating that inclusion of immediate or 6-h ultrasound information improved the risk reclassification ability of the prediction model. For IKBRC the NRI was 0.69 $(P(\uparrow |event)=44.5\mathrm{\%},P(\downarrow |non-event)=90.2\mathrm{\%})$. For SKBRC, the NRI was 1.30 $(P(\uparrow |event)=79.2\mathrm{\%},P(\downarrow |non-event)=85.9\mathrm{\%})$. These results suggest that the 6-h ultrasound data provided better discrimination between events and non-events than the immediate ultrasound data.

Table 3:

Performance metrics of multivariate models with and without ultrasound information to predict bleeding complications after PRB.

Model	Continuous NRI			IDI
	Estimate	p value	95% CI	Estimate	p value	95% CI
KBRC-5 model	Reference			Reference
KBRC-5 model with immediate ultrasound information	0.694	0.000	[0.559–0.840]	0.135	0.000	[0.105–0.166]
KBRC-5 model with 6 h ultrasound information	1.304	0.000	[1.095–1.455]	0.316	0.000	[0.276–0.357]

DOI: 10.7717/peerj.18741/table-3

Note:

PRB, Percutaneous renal biopsy; NRI, net reclassification improvement; IDI, integrated discrimination improvement.

For IKBRC, the IDI was 0.14, reflecting an improvement in mean predicted probabilities for events and a reduction for non-events. For SKBRC, the IDI was 0.32, indicating even greater enhancement in discrimination between events and non-events compared to IKBRC. These results confirm the incremental value of adding ultrasound findings to the KBRC-5 model, as both IDI values were statistically significant (p < 0.05).

Discussion

In this study, we prospectively collected and analyzed data on bleeding complications in 471 patients who underwent PRB. We assessed the impact of incorporating immediate and 6-h post-biopsy ultrasound data into bleeding risk prediction models. Our findings demonstrate that adding ultrasound data substantially enhanced the predictive accuracy and risk reclassification capacity of the models. Furthermore, we compared the clinical utility and predictive power of three models: a base model (KBRC-5) and two enhanced models incorporating ultrasound data (IKBRC and SKBRC). The results showed that models with ultrasound data not only achieved higher predictive accuracy but also provided a greater NB across specific risk thresholds, highlighting their potential for more precise and clinically useful bleeding risk assessment.

The results demonstrate a marked improvement in the risk reclassification capability of the KBRC-5 model when immediate or 6-h ultrasound data is integrated. This enhancement, measured through significant increases in the NRI and IDI, underscores the model’s enhanced precision in assigning patients to appropriate risk categories—an essential factor for effective clinical decision-making (Pencina et al., 2008). Specifically, the incorporation of ultrasound findings significantly boosts the model’s accuracy in identifying high-risk patients, enabling clinicians to prioritize these individuals for closer monitoring and timely intervention (Van Den Bosch et al., 2021). This allows clinicians to better target individuals requiring closer monitoring or timely intervention. This enhancement is particularly relevant in a clinical context where accurate stratification of bleeding risk can guide monitoring intensity and therapeutic decisions, ultimately contributing to better patient outcomes (Antonopoulos et al., 2022). These findings support the clinical value of integrating real-time imaging data into predictive models to refine risk stratification and inform personalized patient management strategies.

Our study’s models offer several advancements over existing bleeding risk prediction models for PRB, particularly in terms of predictive accuracy and AUROC values. The KBRC-5 model, which included pre-biopsy variables such as age, BMI, platelet count, hemoglobin, and kidney size on ultrasound, demonstrated moderate predictive accuracy with an AUROC of 0.683. This is consistent with the results of earlier studies, which reported similar performance metrics for the original KBRC-5 model (Schorr et al., 2020). By incorporating immediate post-biopsy ultrasound findings, the IKBRC model showed a significant improvement in predictive accuracy, with an AUROC of 0.786. This immediate ultrasound assessment provides critical information about the initial impact of the biopsy on kidney tissue, including the detection of small hematomas or other abnormalities before the patient is moved (Bhattacharya et al., 2024). The ability to assess these early complications in real time allows for a more nuanced prediction of subsequent bleeding events. The SKBRC model, which includes hematoma size identified at the 6-h ultrasound, further enhances predictive performance, achieving an AUROC of 0.867. This substantial improvement highlights the value of delayed imaging in capturing the full extent of post-biopsy changes, as hematomas may enlarge or become more apparent over time (Prasad et al., 2015). However, while the SKBRC model demonstrated the highest accuracy, it is worth noting that obtaining 6-h ultrasound results may not always be feasible in routine clinical practice due to logistical challenges. Therefore, the IKBRC model, which uses more readily available immediate ultrasound data, may offer a more practical solution for many healthcare settings. In comparison with prior models, the integration of real-time ultrasound data (immediate and 6-h post-biopsy) in the IKBRC and SKBRC models led to higher AUROC scores, indicating improved accuracy in identifying patients at risk of bleeding complications. This improvement highlights the added value of dynamic, real-time assessment, which many previous models lack.

DCA provides a clearer understanding of how predictive models can inform clinical decisions by evaluating their NB across different risk thresholds (Vickers & Holland, 2021). In our study, both ultrasound-enhanced models outperformed the original KBRC-5 model across all risk thresholds. The IKBRC model offered a positive NB for patients with a bleeding risk threshold between 25% and 77%. Similarly, the SKBRC model, with its inclusion of 6-h ultrasound data, showed an even broader range of NB, from 10% to 95%. For patients with a bleeding risk threshold between 10% and 95%, the SKBRC model provides guidance that balances benefit and risk more effectively than either treating all patients or treating none. This broad applicability makes SKBRC a valuable tool for individualized patient care, as it allows clinicians to tailor their monitoring and intervention strategies based on specific risk levels (Whittier, Sayeed & Korbet, 2016). It can be seen that the combination of real-time ultrasound data enables personalized patient management and has the potential to reduce the incidence of serious complications by facilitating early identification and intervention of high-risk patients.

Similar to previous studies (Shidham et al., 2005; Trajceska et al., 2019), our univariable logistic regression analysis showed that specific vital signs, age, laboratory test results, ultrasound information were associated with bleeding complications after PRB. However, in the multivariable logistic regression analysis, variables associated with bleeding complications were the BMI immediate ultrasound information, and hematoma size at 6 h. This reduction in significant predictors suggests that certain variables identified in univariate analysis may act as confounders or mediators. For instance, lower BMI and hemoglobin levels are likely indirect markers of underlying patient frailty (Huynh et al., 2022; Zhang et al., 2023), which might be more directly linked to an increased bleeding risk following PRB. These insights underscore the importance of refining predictive models to focus on the most robust indicators of risk.

The nomograms derived from each model, particularly from the SKBRC model, significantly enhance clinical applicability by providing a rapid, visual tool for assessing bleeding risk. To improve the usability of the SKBRC model specifically, we developed a nomogram that integrates key predictors—hematoma size at 6 h, BMI, hemoglobin levels, platelet count, kidney size, and age. Each predictor is assigned a weighted score, allowing clinicians to quickly calculate a patient’s overall bleeding risk by summing these values. This accessible tool facilitates efficient and informed decision-making in busy clinical environments, as it enables practitioners to easily assess the likelihood of bleeding complications based on individualized patient profiles. The prominence of hematoma size at 6 h as the most heavily weighted predictor within the SKBRC nomogram highlights the importance of delayed ultrasound imaging in capturing clinically significant post-biopsy changes. This finding aligns with a recent studie suggesting that delayed imaging offers valuable insights into evolving hematomas, which may not be fully evident in immediate post-biopsy scans (Montoya et al., 2021). Unlike previous models that focused on pre-biopsy and immediate post-biopsy factors, our nomogram underscores the added predictive value of later imaging data, which enhances accuracy in stratifying bleeding risk.

Additionally, the nomogram emphasizes the contribution of other variables such as BMI, hemoglobin, and platelet count, which add essential context for assessing overall risk. Studies by Palsson et al. (2020) and Hasegawa et al. (2022) similarly highlighted the role of these parameters in bleeding risk, but did not integrate them as intuitively into clinical tools. By translating complex statistical models into a user-friendly, accessible format, our nomogram bridges this gap, making advanced risk prediction easily implementable in clinical workflows.

Overall, the integration of real-time ultrasound findings in the SKBRC model offers a new standard for bleeding risk prediction following PRB. By enhancing predictive accuracy and supporting timely clinical interventions, this model has the potential to improve patient outcomes and optimize resource allocation in healthcare settings.

However, some limitations should be considered. First, this study was conducted at a single tertiary-care institution, which may limit the generalizability of our findings. External validation in larger, multi-center cohorts is needed to confirm the utility of these modified models across diverse populations. Additionally, patients initially expected to undergo PRB but later excluded for various reasons were not part of the analysis, which may introduce selection bias. Moreover, certain relevant variables—such as biopsy needle size, operator experience, and biopsy depth—were not included in our dataset, potentially limiting the comprehensiveness of the predictive models.

Despite these limitations, our study provides valuable insights into predicting bleeding complications after PRB. The integration of real-time ultrasound data into the IKBRC and SKBRC models represents a significant advancement in risk prediction, with potential to enhance patient safety and improve clinical outcomes. Future research should focus on validating these models in varied clinical settings and identifying additional risk factors that could further refine bleeding risk prediction, thereby enhancing the models’ utility in routine clinical practice.

Conclusions

This study demonstrates the value of ultrasound data as a predictor of bleeding risk following PRB. By incorporating ultrasound information after PRB, the predictive performance of the KBRC model for bleeding risk significantly improves. This enhanced model provides clinicians with a more accurate tool to guide patient management decisions, including the duration of supine positioning, monitoring requirements, and the appropriate use of hemostatic interventions. These findings underscore the importance of ultrasound imaging in observing hematomas post-PRB and highlight its potential to support clinical decision-making and facilitate integration into medical practice.

Supplemental Information