Step-By-Step Guide to Building Clinical Prediction Models
STEP 1 Purpose, Team, Review, and Plan
1
Clarify the Purpose of the Prediction Model
Define the objectives of the prediction model, including:
Target population: Specify the patient group the model is aimed at, such as HIV patients in South Africa, individuals with a history of diabetes, or postmenopausal women.
Health outcomes: Define the health outcomes to be predicted, such as overall survival, progression-free survival, or specific adverse events.
Healthcare setting: Determine the healthcare environment in which the model will be used, for example, a community hospital or a tertiary comprehensive hospital.
Users: Specify who will use the model, such as primary or secondary care physicians, patients, or researchers.
Decision context: Describe how the model predictions will be applied in the clinical decision-making process, such as identifying patients requiring further diagnostic testing, deciding on treatment strategies, or guiding personal decisions.
2
Literature Review
Conduct a literature review to identify existing relevant prediction models and studies on significant risk factors. This process helps understand established predictors and the limitations of existing models, as well as the interactions between predictors, the nonlinear relationships between predictors and outcomes, the reasons for missing data, and the distribution of predictors in the target population.
3
Develop Research Protocol
Refer to the TRIPOD reporting guidelines and PROBAST tool to ensure methodological transparency and structure.
STEP 2 Redevelop or Upgrade?
1. Literature Review and Model Evaluation:
Determine whether there are existing prediction models related to your research question through the previous literature review.
2. Assess Bias Risk of Existing Models:
If the bias risk of existing models is low (can be assessed using the PROBAST tool) and applicable to your research question, consider external validation instead of developing a new model. External validation assesses the effectiveness of existing models by testing their performance with a new dataset.
3. Consider Updating and Adapting Models:
If existing models perform poorly in your research environment, consider updating and adapting those models instead of starting from scratch. Common model updating strategies include:
Recalibration: Adjust the intercept in the model to better fit new data.
Revision: Re-estimate certain model parameters.
Extension: Add new predictors to the existing model.
4. Determine the Development Path for the Model:
If the literature review indicates no existing models applicable to your research question, or if the existing models have a high bias risk, consider developing a new model. In this case, it is necessary to define predictors from scratch, collect data, and conduct model development.
5. Considerations of Literature and Sample Size:
Whether validating externally or updating existing models, considerations of literature and sample size are critical. If updating a model, it is essential to understand how the existing model performs in different environments. If developing a new model, sufficient sample size must be considered to avoid overfitting issues.
STEP 3 Determine Outcome Indicators
STEP 4 Identify Candidate Predictors
STEP 5 Data Collection and Checking
STEP 6 Determine Sample Size
STEP 7 Handle Missing Data
STEP 8 Fit the Prediction Model
STEP 9 Evaluate Model Performance
STEP 10 TO 13
Public Account Statement
“The Digestive Disease Department of Wangjing Hospital, China Academy of Chinese Medical Sciences” WeChat public account serves as a public welfare platform aimed at popularizing health and wellness knowledge for patients. If there are any inappropriate content or illustrations in this article, please contact us promptly, and we will handle it immediately. Thank you for your efforts in public welfare!
Text and Images | Zhang Naiwen
Editor | Zhang Mengjia
Reviewed by | Yan Ningjuan
Thank you for your attention
If you find this article beneficial, feel free to share and like it below. Thank you for supporting health initiatives; your encouragement is our motivation!