CLINICAL TRIALS - Faithful Migration: Shifting Patient Reported Outcomes From Paper to Electronic

INTRODUCTION

Patient reported outcomes (PROs) are one of the four types of clinical outcome assessments (COAs) that play an important role in the evaluation of key study objectives often critical to the success of clinical trials. Most PRO instruments were originally de­signed to be administered in paper format, but according to re­cent market research from Industry Standard Research, 89% of respondents now report a preference for using electronic data capture over traditional paper methods. Given this shift, organi­zations are now focusing on electronic PROs (ePROs).

Not only do participants prefer the electronic method, but implementing ePROs minimizes data loss and errors, as well as increases response rate. However, before an instrument is imple­mented electronically, it’s important that faithful migration is per­formed. This important step ensures the original properties of the paper PRO remain intact as much as possible when migrating from the existing paper PRO to an ePRO format.

This process needs to be completed very carefully as current guidance requires equivalence when migrating paper PROs to ePROs – essentially ensuring all information shared and question phrasing is of equal meaning, value, and implication in all ap­plications. Faithful migration ensures the content, meaning, and measurement properties of the questionnaire are retained when migrating it from paper to electronic format. However, some modifications may be necessary to align with the electronic modality. The level of modification determines what methodology may be required to demonstrate equivalency. Prior to discussing ways to establish equivalence per modification level, it is best to understand the recommended steps of faithful migration.

FAITHFUL MIGRATION STARTS WITH THESE THREE STEPS

Organizations should closely adhere to the following:

1. Contact the instrument’s copyright holder or developer to de­termine if there are any requirements for an electronic migra­tion, and obtain permission or licensing as needed.
2. Identify any changes that need to be made by reviewing the original version of the instrument. It may be necessary to once again contact the developer or copyright holder to obtain ap­proval for the changes necessary to migrate to the electronic version. Exact words and the order of the questions and re­sponses should be kept where possible. There may be spacing constraints, so evaluation of the instructions and where they are placed, whether that is on the same page or the following page, is necessary. Aesthetic elements should also be incor­porated, such as the spacing between questions and re­sponses and font size. The navigation of the ePRO should be intuitive and indications should be clear on moving forward and back­ward.
3. Sending screenshots of the ePRO back to the developer or copyright holder for both review and approval may be a re­quired third step.

SUCCESSFULLY ADDRESSING MODIFICATION LEVELS

Within the migration process, there may be modifications necessary to ensure equivalency. There are three distinct types of modifications: minor, moderate, and substantial. Minor modifications are com­mon and not expected to change any meaning or interpretation of questions or responses. A specific example of a minor modification would be updating the phrase Please Circle on a written form to Please Select on an electronic form.

Moderate modifications may have an impact on the patient’s ability to interpret and therefore respond to questions. These modifications are typically more format re­lated, such as splitting a question across multiple screens. While minor and moder­ate changes are mostly related to marking responses or formatting, a substantial modification is most likely to impact the patient’s ability to interpret and respond to questions. Things such as removing items or removing recall periods would be con­sidered a substantial modification. More impactful modifications (moderate and substantial) should be avoided, when pos­sible, but when they are unavoidable, it is key to provide evidence of equivalency, es­pecially to regulatory bodies.

EQUIVALENCY TESTING FOR EACH LEVEL OF MODIFICATION

Evidence of equivalency may be re­quired by global authorities if data is to be used in a regulatory setting. FDA guidance dictates that when a PRO used to support labeling claims has been modified, evi­dence of equivalency is required. For each level of modification previously discussed, there are specific forms of evidence needed. For minor modifications, a “think aloud” approach to interviewing, known as cognitive debriefing, is required. A par­ticipant is asked to complete both the orig­inal PRO measure and the migrated ePRO measure in the presence of an interviewer while they “think aloud.” This means inter­viewers ask participants to describe the meaning of the questions and responses in their own words. This is a form of a qualitative assessment to determine if the migration has affected the meaning of the questions or responses. Between 5-10 par­ticipants in the target population should participate in the cognitive debriefing.

For moderate modifications, a quan­titative assessment of the comparability of responses from the original measure to the migrated measure, known as equivalence testing, is necessary. The individuals who are part of this testing complete both the original PRO and the migrated ePRO in random orders. Statistical methods are then used to assess the level of agreement between the two measures.

Substantial modifications require full psychometric testing as the psychometric properties of the measure could change with migration. This testing is required to demonstrate test and re-test reliability, in­ternal consistency, and validity to ensure the ePRO is statistically robust. Usability testing, which examines how well partici­pants understand, navigate, and submit responses through the ePRO measure, is a constant between all three types of mod­ifications. Major difficulties seen during us­ability testing must be addressed and reassessed through additional usability testing. This loop continues until all the substantial issues are rectified.

Regulators and other health organi­zations encourage electronic modes of data collection from patients. It assists in minimizing data entry error or missing data and is a convenient method of col­lection for both patients and sponsors. The migration from paper PROs to ePROs should aim to remain as true to the origi­nal measure as possible with minimal modifications when necessary. By follow­ing the aforementioned steps to faithful migration, the shift to electronic measures is efficient and reliable, especially as there is an increasing body of evidence showing that when minor changes are made to an original measure, the resulting ePRO is likely to be considered equivalent.

Melissa Mooney is Director, eCOA Solutions Engineering, eCOA Technologies at IQVIA. She has more than 17 years of experience in the development of eCOA solutions for use in clinical trials. Her area of expertise is eCOA solution design, in which she has supported clients and eCOA vendors in developing robust and usable eCOA software solutions that meet eCOA protocol requirements. She also brings a plethora of experience in eCOA requirement gathering, leading eCOA user acceptance testing, eCOA data management, and business development support.