Home
|
Inbox
|
Search
|
View Abstract
Is this an Efficacy to Effectiveness Trial? Comparing Two Tools to Classify Trials Along the Efficacy to Effectiveness Continuum
Description
.abstract img { width:300px !important; height:auto; display:block; text-align:center; margin-top:10px } .abstract { overflow-x:scroll } .abstract table { width:100%; display:block; border:hidden; border-collapse: collapse; margin-top:10px } .abstract td, th { border-top: 1px solid #ddd; padding: 4px 8px; } .abstract tbody tr:nth-child(even) td { background-color: #efefef; } .abstract a { overflow-wrap: break-word; word-wrap: break-word; }
A4901 - Is this an Efficacy to Effectiveness Trial? Comparing Two Tools to Classify Trials Along the Efficacy to Effectiveness Continuum
Author Block: S. M. Nyenhuis1, A. Apter2, M. Schatz3, S. Bandi4, C. Codispoti5, H. Kumar6, J. Moy7, J. Tirumalasetty1, B. Yu4, J. Krishnan8; 1Medicine, Univ of Illinois at Chicago, Chicago, IL, United States, 2Hosp of the Univ of Pennsylvania, Philadelphia, PA, United States, 3Kaiser Permanente Medical Ctr, San Diego, CA, United States, 4Medicine, Rush University, Chicago, IL, United States, 5Department of Immunology and Microbiology, Rush University Medical Center, Chicago, IL, United States, 6Pediatrics, Univ of Illinois at Chicago, Chicago, IL, United States, 7Rush University Medical College, Chicago, IL, United States, 8Univ of Illinois, Chicago, IL, United States.
Rationale: There is increasing interest in studies using effectiveness designs that address the expressed needs of stakeholders to accelerate uptake of new health-related information into clinical practice.
Objectives: The objective of this study was to evaluate the feasibility, acceptability, and inter-rater reliability of the Pragmatic Explanatory Continuum Indicator Summary (PRECIS) tool and an adaptation of the PRECIS tool that includes an assessment of stakeholder engagement and a scoring algorithm (PRAgmatic Clinical Trial Assessment Scale (PRACTAS)) to classify clinical trials along the efficacy to effectiveness continuum.
Methods: Six academic clinicians were randomly assigned to use either the PRECIS tool (n=3 clinicians) or PRACTAS tool (n=3 clinicians) and asked to evaluate the same two peer-reviewed published clinical trials in asthma. We assessed feasibility (time to complete the assessment), acceptability (3-item questionnaire), and inter-rater reliability (spider graphs and intraclass correlation coefficient (ICC)) of both tools.
Results: The PRACTAS tool took twice as long to complete compared to PRECIS (22 vs. 11 minutes per study; p=0.07), but users reported similar acceptability (median score 4 and 3.3, respectively, p=0.2). Inter-rater reliability was poor for both the PRECIS (ICC= 0.32) and PRACTAS (ICC= 0.38) tools. Spider graphs suggested inter-rater reliability may be greater for certain design elements (e.g., Eligibility criteria and Practitioner adherence), but larger studies are needed to confirm findings.
Conclusions: The PRECIS and PRACTAS tools had similar acceptability but limited inter-rater reliability among clinicians. Clinicians using either tool are likely to disagree about the extent to which clinical trials employ efficacy or effectiveness designs.
Author Block: S. M. Nyenhuis1, A. Apter2, M. Schatz3, S. Bandi4, C. Codispoti5, H. Kumar6, J. Moy7, J. Tirumalasetty1, B. Yu4, J. Krishnan8; 1Medicine, Univ of Illinois at Chicago, Chicago, IL, United States, 2Hosp of the Univ of Pennsylvania, Philadelphia, PA, United States, 3Kaiser Permanente Medical Ctr, San Diego, CA, United States, 4Medicine, Rush University, Chicago, IL, United States, 5Department of Immunology and Microbiology, Rush University Medical Center, Chicago, IL, United States, 6Pediatrics, Univ of Illinois at Chicago, Chicago, IL, United States, 7Rush University Medical College, Chicago, IL, United States, 8Univ of Illinois, Chicago, IL, United States.
Rationale: There is increasing interest in studies using effectiveness designs that address the expressed needs of stakeholders to accelerate uptake of new health-related information into clinical practice.
Objectives: The objective of this study was to evaluate the feasibility, acceptability, and inter-rater reliability of the Pragmatic Explanatory Continuum Indicator Summary (PRECIS) tool and an adaptation of the PRECIS tool that includes an assessment of stakeholder engagement and a scoring algorithm (PRAgmatic Clinical Trial Assessment Scale (PRACTAS)) to classify clinical trials along the efficacy to effectiveness continuum.
Methods: Six academic clinicians were randomly assigned to use either the PRECIS tool (n=3 clinicians) or PRACTAS tool (n=3 clinicians) and asked to evaluate the same two peer-reviewed published clinical trials in asthma. We assessed feasibility (time to complete the assessment), acceptability (3-item questionnaire), and inter-rater reliability (spider graphs and intraclass correlation coefficient (ICC)) of both tools.
Results: The PRACTAS tool took twice as long to complete compared to PRECIS (22 vs. 11 minutes per study; p=0.07), but users reported similar acceptability (median score 4 and 3.3, respectively, p=0.2). Inter-rater reliability was poor for both the PRECIS (ICC= 0.32) and PRACTAS (ICC= 0.38) tools. Spider graphs suggested inter-rater reliability may be greater for certain design elements (e.g., Eligibility criteria and Practitioner adherence), but larger studies are needed to confirm findings.
Conclusions: The PRECIS and PRACTAS tools had similar acceptability but limited inter-rater reliability among clinicians. Clinicians using either tool are likely to disagree about the extent to which clinical trials employ efficacy or effectiveness designs.
|
Home
|
Inbox
|
Search
|