Electronic patient-reported outcome

The two main methods currently used for ePRO are computers (including smartphones) and telephone systems.

Computers are most often touch-screen devices, ranging from wearables, hand-held devices and smartphones, through iPads and other Tablet computers. The smaller devices are often used as electronic diaries, designed to be used for symptom reporting on a daily basis. Larger devices are generally used in a clinical setting. Computers generally run dedicated ePRO applications - the use of the web for ePRO is not yet widespread. Typically a single question at a time is presented on the screen, with a set of possible response options. The user taps on the appropriate response with a finger or stylus, then moves on to the next questions.

Telephones normally use an interactive voice response system (IVR). The user calls into a dedicated phone line, and hears a spoken script that details the question, and the possible responses. Each response option is given a number, and the user presses the corresponding number key on the phone keypad to record the choice. IVR systems are more often used for diaries, with the patient phoning in e.g. from home, but they can be used in a clinic setting.

There are also a number of custom devices designed specifically for use as ePRO data collection devices.

Diaries are used when it is desirable to obtain frequent assessments over a period of time, for example when a condition fluctuates in severity. In such cases recall of severity over a period of time is unlikely to be accurate. Research has shown substantial bias in such summary recall, with ratings unduly influenced by how the patient is feeling at the time of making the rating, and by maximum severity rather than average severity during the assessment interval.[1] Diaries can overcome this problem by recording severity either on a momentary basis ("How bad is your pain right now?") or by recalling over short periods ("How bad has your pain been today?"). However, when diary data is collected on paper, it is not known when the ratings are actually made, and there is evidence that compliance may be quite poor. In one study, patients were given an instrumented paper diary that recorded covertly when it was opened. The study showed frequent cases of "back-filling", filling in a batch of entries sometimes, often days, after they were due, and even in some cases of "forward filling", completing entries before they were due.[2]

Electronic diaries automatically time-stamp all entries and can be set up to only allow entry within specified time windows. This improves compliance and ensures that true compliance can be documented. Documenting compliance is important if ePRO data are to be used to support regulatory applications.[3] ePRO applications typically achieve compliance rates of over 80%, often over 90%.[4][5] Electronic diaries also have benefits in that they only allow valid, in-range entries to be made. Devices such as PDAs allow reminders to be given to patients when entries are due. Systems generally transfer data promptly to a central server, allowing tailored feedback to be given to patients. This can also improve compliance. Electronic diaries also eliminate the need for manual editing and entry of data, time-consuming and error-prone processes.

The other main setting for ePRO is the clinic, with questionnaires completed when patients come in for their scheduled visits. In this supervised situation, compliance is less of an issue. The questionnaires used in site-based ePRO are often longer and more complex than those used in diaries, assessing quality of life and activities of daily living, for example, in some detail. They more often include branching logic ("if YES continue with the next question, if NO, go to question 34"). Such branching logic can be handled automatically by the ePRO application, and it is often not necessary for the patient even to know that branching is taking place. This makes it easier for the patient to use.

As with electronic diaries, the prevention of out-of-range or inconsistent entries, and the elimination of manual editing and data entry are important features of site-based ePRO. Missing data within a questionnaire can be reduced or eliminated, and this is important as missing data has been identified as a crucial quality issue in questionnaire data.[6]

From the early years of ePRO there has been concern about whether all patients can cope with computer technology. This is important, as if significant numbers of patients refuse to take part in clinical trials because of dislike of computers then there will be bias in the study population. One of the earliest ePRO studies used a LINC-2 minicomputer to collect patient data. The majority of patients preferred the computer to paper data collection.[7] Similar findings have been reported from many later studies.[8][9]

Elderly patients, and those not familiar with computers, might be expected to have more problems. But these groups also show high acceptance of ePRO, and again often prefer it to paper.[10][11][12] Thus there seems to be no great barrier to recruiting representative patient samples in ePRO studies.

Establishing the validity of an ePRO instrument is in principle no different from that for other methods, such as paper. However most of the instruments in current use have been validated in paper form, and we need to ask (1) whether the paper data can be used to establish validity of the electronic version, and (2) whether data from paper and electronic versions can be used interchangeably.

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has issued guidelines on establishing an equivalence between modes of administration.[13] Their approach is hierarchical and depends on the degree of change made during the process of migration from paper to electronic format. Three levels are suggested. At the lowest level, where the least change has been made, cognitive interviewing of patients as a check that they construe ePRO and paper, in the same way, is sufficient. This level includes both trivial changes (touch rather than circle a response choice) as well as changes that are supported by empirical findings in the literature. At the second level, equivalence studies comparing the scores obtained from the two modes should be carried out. At the third level, where most change has occurred, the ePRO instrument must be treated as a new instrument, and complete psychometric validation carried out.

There is a great deal of evidence supporting the general equivalence of paper and ePRO methods. Gwaltney and colleagues[14] have reported a meta-analysis in which they included 46 studies evaluating 278 scales. They concluded that there was good agreement between paper and ePRO, and no evidence of systematic bias. This general finding, of course, does not guarantee that any specific migration will lead to equivalence, and each case must be reviewed and documented.

It is not always necessary to validate an ePRO measure against a pre-existing paper version. In some cases, an instrument may be developed and validated from the beginning in electronic form. More commonly, perhaps, new instruments will be developed in parallel for paper and electronic use, as is the case with the PROMIS (Patient-Reported Outcomes Measurement Information System) initiative.[15]

Standards of validity must be maintained throughout every target language population. In order to ensure that developmental standards are consistent in translated versions of an ePRO instrument, the translated instrument undergoes a process known as Linguistic validation in which the preliminary translation is adapted to reflect cultural and linguistic differences between diverse target populations. This process also accounts for any formatting errors that may occur in languages using non-Roman fonts.

Several successful regulatory approvals have used ePRO data in recent years, including ketorolac for ocular pain, eszopiclone for insomnia, milnacipran for fibromyalgia, estradiol/levonorgestrel for post-menopausal symptoms, and ruxolitinib for myelofibrosis.[16] In the case of estradiol/levonorgestrel, detailed ePRO data on bleeding/spotting from a one-year clinical trial are presented in the patient information leaflet.[17]

As well as clinical trial use, ePRO methods may be used to support patients in regular care. An example of this is the collection of symptom data from patients undergoing chemotherapy, using handheld diaries. This allows clinic staff to monitor outpatients, and to identify the occurrence of adverse reactions that may require intervention.[18]

• Byrom B, Tiplady B (eds.) ePRO: Electronic Solutions for Patient-Reported Data (2010) Farnham: Gower. ISBN 978-0-566-08771-4.
• Byrom B, Mühlhausen W Electronic Patient-reported Outcome Measures: An implementation handbook for clinical research(2018) ISBN 978-1-72028-110-8
• Stone AA, Shiffman S, Atienza AA, Nebeling L (eds.) The Science of Real-Time Data Capture (2007) Oxford: Oxford University Press. ISBN 978-0-19-517871-5.

• International Society for Pharmacoeconomomics and Outcomes Research
• Patient-Reported Outcomes Measurement Information System