There are multiple methods for gathering electronic and web-based data for research purposes including crowdsourcing [1,2], registries [3,4], web-based questionnaires, and patient-reported information [1–13]. These methods can recruit large cohorts of diverse samples in a cost-efficient and convenient manner [1,8]. Patient-reported data and medical records agreement for clinical information on comorbidities can be up to 90% [9], although agreement between patient-reported data versus what is reported in the medical record can depend upon variables such as race [10], sociodemographics [11], complexity of disease [11], comorbidities [13], and age [11]. For example, in one study when subjects were older and reported depression and anxiety, only fair agreement was noted between participants and their health records when reporting on their hypertension and diabetic management [13].
Inheritable bleeding disorders (BDs) are a group of rare diseases that increase the tendency to bleed, and include coagulation factor deficiencies like haemophilia, von Willebrand disease, and platelet dysfunctions. Historically, efforts to collect data in the United States (US) from persons with inheritable bleeding disorders (PIBD) have centred on clinical information entered by the healthcare team. Since 1998, the Centers for Disease Control and Prevention (CDC) has collected surveillance data from federally supported specialised haemophilia treatment centres (HTCs), initially, via the Universal Data Collection [14] initiative, progressing to Community Counts in partnership with the American Thrombosis & Hemostasis Network (ATHN) [15]. Additional clinical and diagnostic data collected by HTCs through ATHN provides a national research infrastructure [16–17].
The National Bleeding Disorders Foundation (NBDF; formerly known as the National Hemophilia Foundation) recognised the need to amplify the patient voice in research. PIBD are appropriately deemed ‘lived experience experts’ (LEEs) because they know better than anyone else how BDs affects their lives, what they go through to get a diagnosis and treatment, how treatments affect them, how symptoms or side effects impact daily life, and the challenges of interacting with the health care system. The term LEE also includes immediate family members (parents, partners, siblings, and grandparents) because they develop similar knowledge through their own lived experience, gained from another important viewpoint. Including LEEs’ perspectives in research ensures that research questions are relatable, recruitment and retention strategies are optimised, and study designs are less burdensome. Dissemination of results in lay language provides valuable and clear evidence to incorporate in shared decision making, promoting LEEs as active members of their health care team [18].
NBDF, founded in 1948, is the largest patient advocacy organisation in the US serving LEEs. Its mission is “finding cures for inheritable blood and bleeding disorders and addressing and preventing the complications of these disorders through research, education, and advocacy enabling people and families to thrive” [19]. NBDF has supported research for over five decades via grants and fellowships available to multidisciplinary HTC team members [19–21]. The NBDF Research Department’s focus is three-fold:
address gaps in knowledge to improve the lives of LEEs;
ensure that the lived experience drives and influences the research process;
promote equity, diversity, and inclusion so that research findings include all.
In this paper, we:
Document the evolution of CVR as a LEE-centric means to complement other community research efforts
Highlight mechanisms to incorporate LEEs in different stages of the research process
-
Identify LEE-centred gaps in research based on CVR
data.
EVOLUTION OF CVR
Established in 1975, the Nurse’s Health Study stands among the largest prospective investigations into the risk factors for major chronic diseases in women, with over 275,000 participants [22]. Motivated by this extensive longitudinal study, NBDF recognised a lost opportunity within the inheritable BD community to better understand LEE experiences, including those during the HIV and hepatitis crisis of the 1970s and 1980s, when at least 50% of LEEs with haemophilia contracted HIV [23] and 80% contracted Hepatitis C [24]. Consequently, the idea to establish a patient-reported, web-based registry to gather the experiences of PIBD and their families as LEEs was born, enabling timely data collection and reporting of their experiences and concerns.
MyBDC was NBDF’s first effort to comprehensively describe the LEE perspective longitudinally through electronic surveys and place it at the center of research. Initial goals were to transform the lived experience into evidence, identify trends and patterns, and develop LEE-inspired research priorities. This would include input from LEE subgroups who had not previously participated in data collection efforts, e.g. those receiving care outside of HTCs.
The development of MyBDC, which evolved into CVR, included input from key stakeholders and diverse LEEs. Initial stakeholders were identified through NBDF chapters and other advocacy organisations, and then invited to focus groups held during large national meetings. Later, stakeholder groups were built around identifying and meeting certain diversity metrics in order to ensure a wide range of perspectives. These metrics include race, ethnicity, gender, sexual orientation, age, and roles within the community (e.g., LEEs with various diagnoses, HTC clinicians, researchers, NBDF chapter and Hemophilia Federation of America representatives, government and industry partners, payors, and other research entities, such as ATHN).
METHODS
Aims
CVR aims to:
comprehensively describe the experience of living with an inheritable BD;
recognise relationships between health outcomes and variables such as bleeding, treatment, access to care, pain, anxiety, depression and social determinants of health;
develop research questions and prioritise protocols based upon survey responses;
provide participants with other research opportunities, research findings from data collected, and tailored educational resources;
identify opportunities to improve health behaviours and quality of life through research, education, and advocacy.
Design
Recruitment
Recruitment began in March of 2019 (Figure 1) and focused on LEEs with self-reported inheritable BDs and their immediate family members over the age of majority living in the US. In-person educational and advocacy events, such as NBDF’s Washington Days and Bleeding Disorders Conferences, and social media campaigns were targeted for recruitment.
Achieving a diverse and representative sample is more likely to happen with registries when compared to more traditional methods [1]. To ensure ongoing equity, diversity, and representation in CVR, recruitment efforts focus on participants seeking care outside the HTC network and on historically marginalised and understudied communities. The availability of a Spanish version of the surveys and dashboard, and Spanish speaking Hispanic recruiters support efforts focused on this traditionally underrepresented ethnic group. In 2020, the COVID-19 pandemic forced the research team to pivot to virtual settings where recruitment continued solely through webinars and social media.
In 2020, MyBDC was rebranded to “Community Voices in Research” (CVR), reflecting NBDF’s increased emphasis on research and reaffirming its mission. Both MyBDC and CVR protocols have been reviewed and classified as ‘exempt’ with each major revision by a centralised Institutional Review Board. A brief enrolment hiatus occurred in 2023 while the CVR registry was transferred to a new platform with expanded capabilities focused on enhancing an intuitive and impactful participant experience (Figure 1).
Surveys
CVR collects data through web-based electronic self-reported surveys containing validated psychometric patient reported outcome measures (PROMs), and expert developed questions to assess demographic information and other relevant topics, see Table 1. The inclusion of LEEs since inception ensures that the data collected are relevant to them and encourages participation and continued engagement. Enrolment begins by creating an account and taking a short survey that collects basic demographic and diagnosis information (Figure 2).
Table 1.
Validated tools included in CVR surveys
The Baseline Survey, which is repeated annually, gathers self-reported perspectives including diagnostic journey, bleeding history and treatment, pain, physical and mental health, comorbidities, functional status, access to specialised care and treatment, and social determinants of health - topics not being systematically collected elsewhere (Figure 3). Additional questions regarding reproductive health and menstrual bleeding are asked to women and persons who have or have had the potential to menstruate.
The experiences of immediate family members (mothers, fathers, siblings, grandparents, and other relatives) are explored via a separate Family Baseline Survey. CVR serves as an instrument for both internal NBDF staff and external researchers interested in LEEreported information (Figure 4).
Personalised Dashboard
In addition to information via electronic surveys, CVR provides direct benefits to participants through a Personalised Dashboard (PD), which was launched in November 2019 once there was sufficient data for display. The PD allows participants to:
create a health summary to share with health care providers and others;
take available surveys;
access aggregated, de-identified data, allowing them to follow their responses over time and compare them with others;
connect with additional research opportunities (both internal and external);
access research publications using CVR data in easyto-understand language;
explore vetted educational information.
Virtual and In-person Advisory Panels
CVR is used to recruit to NBDF-supported virtual and in-person advisory panels (VAPs and PAPs), providing the opportunity for LEEs to participate in research study design, and share their knowledge in the development of educational and market research materials (Figure 4). Invitations are based on predetermined inclusion and exclusion criteria. One goal of VAPs/PAPs is to diversify and expand the range of geographic and sociodemographic characteristics of LEE consultants.
NBDF holds approximately 8-10 advisory panels per year facilitated by experts in the field. VAPs are usually two to three hours long and PAPs are around six to eight hours. Panels are composed of 6-10 LEEs to ensure that everyone can share their perspective. When outside researchers or industry sponsors are involved, participants’ personal or identifying information is kept private. Spanish and additional language translators are available during the eligibility process and panel discussions as needed. NBDF pre-screens questions and scripts with researchers to ensure session goals. Participants are reimbursed for their time. A final comprehensive written report is available to the sponsor/partner.
Examples of past advisory panel topics include:
improving the relationship between researchers and the BD community
understanding emergency room experiences
exploring health equity, diversity, and inclusion issues
defining challenges with diagnosis and treatment
clinical trial design
development of educational programs and materials.
Data Collection, Storage, and Analyses of CVR
Data are safely gathered and stored on a secure server using the strictest of industry standards. NBDF staff do not have access to participants’ identifiable information. The information collected remains confidential and is only de-identified and aggregated data is reported.
Data are de-identified by a professional statistician with extensive research experience, in collaboration with the platform vendor via a secure file transfer protocol (SFTP). The data are then analysed, summarised, and aggregated by the statistician before results are shared. Raw data and individual responses are not accessible to NBDF, the community, or external researchers to protect data security and participant privacy – this is a priority to NBDF.
CVR has a certificate of confidentiality from the National Institute of Health (NIH) protecting the privacy of enrolled participants’ identifiable and sensitive information [25]. With limited exceptions, this certificate also protects NBDF from having to disclose information gathered to legal authorities.
Results
Since the launch of CVR, enrolment has continued to increase (Figure 5). Data on topics collected (Figure 3) provide information to stimulate ideas for research and answer descriptive or correlational research questions.
Starting in early 2020 and in partnership with NBDF, external researchers have been able to develop and launch surveys via CVR with specific research questions based on the literature and pre-existing data. Examples include the validation of a Pain PROM and a discrete choice experiment. Table 2 shows publications to date. Surveys can be tailored and sent to participants conforming to specific inclusion and exclusion criteria. Results are provided to external researchers in a deidentified and aggregated manner.
Table 2.
Publications and abstract presentations using CVR data to date
| YEAR | TITLE | PUBLICATION | MEETING |
|---|---|---|---|
| 2024 | The relationship between different pain measures, depression, and social support and race and ethnicity in persons with hemophilia | AJH | THSNA |
| Glanzmann Thrombasthenia beyond bleeding: Insights from lived experience experts | Haemophilia | WFH | |
| Pain attitudes and pain outcomes among people with bleeding disorders: Results from community voices in research [27] | Haemophilia | ||
| 2023 | Gene Therapy for Hemophilia A: A Mixed methods study of patient preferences & decision-making [28] | PPA | |
| Development of a haemophilia A gene therapy shared decision making tool for clinicians [29] | Haemophilia | ||
| Gene therapy preferences & informed decision-making: Results from NHF CVR survey [30] | Haemophilia | ||
| 2022 | Relationship between perceived social support, mental health, activity, & chronic pain in PIBD [31] | Blood | ASH |
| 2021 | Community Voices in Research (CVR): A patient-centric approach moving the future of IBDs forward [32] | RPTH | HTRS |
| Community Voices in Research (CVR): A patient-centric approach to study design phase through Community Research Network | DIA | ||
| Poor outcomes in people with hemophilia: Physician & subject matter expert perspectives | ATHN | ||
| Bleeding disorder data registry reveals racial/ethnic disparities that could significantly impact patient journey [33] | Blood | ASH | |
| Utilizing a community registry to analyze pain limitations in PIBDs | ASPMN | ||
| The relationship between self-reported physical activity, treatment, regimen, mental health, & pain in persons with hemophilia [34] | Haemophilia | WFH-MSK | |
| Patient preferences & priorities for haemophilia gene therapy in the US: A discrete choice experiment [35] | Haemophilia | ||
| Evidence of a disability paradox in patient-reported outcomes in haemophilia [36] | Haemophilia | ||
| Preferences of people with hemophilia A and B for treatments including gene therapy in the US: A discrete choice experiment [37] | RPTH | THSNA | |
| 2020 | Developing My Bleeding Disorders Community (MyBDC): A community-powered registry to provide a 360-view of living with a bleeding disorder [38] | Haemophilia |
Through the PD, participants continue to have access to pending and completed surveys. They may also view aggregated survey responses in chart format, with the option to custom filter (e.g., by age and diagnosis) to further understand how their answers compare to others like them. NBDF staff regularly develop and curate new content aimed at continuing engagement.
CVR’s ability to swiftly query participants on relevant topics and gauge the community's stance in real time enables a “rapid research cycle”. A five-question Vaping Survey, aiming to understand the impact of lung injury associated with the use of e-cigarettes and launched in August 2019, was the first test of this capability. Upon survey completion, respondents were directed to the CDC website and offered additional vetted education [37]. Another example is the ‘COVID-19 Questionnaire’, launched in April 2020, aimed at understanding how the pandemic was initially affecting participants. Based on the results, NBDF developed and launched COVID-19-specific resources to support LEE needs.
As CVR evolved and expanded, it became necessary to change platform vendors to improve its capabilities for both LEEs and researchers. A transition and re-development period in 2023 resulted in a more attractive and intuitive tool, offering enhanced functionality and safety measures.
CVR surveys are voluntary and, generally, uncompensated. Periodically, however, incentives are offered to encourage enrolment and participation in additional research surveys. During the COVID-19 pandemic, recruitment through social media resulted in rapid enrolment, raising concerns about potential bot activity and the creation of false accounts. A complex algorithm based on multiple criteria (e.g., mismatches between enrolment and baseline demographic information, survey completion time impossibly short, patterns of consecutive surveys with systematic numerical sequences, etc.) was developed to identify potential invalid entries. If more than two criteria were met, the statistician reviewed the individual responses, and a clinical expert validated the final determination for inclusion or exclusion from analyses. Entries that did not meet these criteria were permanently deleted. To mitigate the chances of this reoccurring, the CVR team implemented a stricter enrolment process, including the Completely Automated Public Turing Test to Tell Computers and Humans Apart (CAPTCHA), email addresses verification and use of pre-established clinical and demographic criteria to flag or deny enrolment. The team also monitors enrolment trends, addressing suspicious activity quickly, and financial incentives are used judiciously.
Discussion
CVR is a nimble, versatile, and powerful community tool with the primary aim of incorporating the LEE firsthand experience and perspective into research (Figure 6).
During the inception of CVR, NBDF identified an unexplored opportunity within the inheritable BD community to gain deeper insights into the LEE experience. With nearly five years of implementation, CVR is making substantial strides in comprehensively gathering the experiences directly from those most impacted by inheritable BDs and their family members, devoid of filters from clinicians or researchers. The current interaction of CVR facilitates swift responses to societal events that may impact the lives of LEEs by furnishing timely and pertinent data for research purposes.
CVR has enriched researchers' understanding of the nuanced needs of LEEs and their families directly from them, enabling their unique viewpoints to be incorporated into research, education, and advocacy. It ensures that the information collected is relevant and accurately represents participants’ views and experiences. As CVR’s enrolment and vision expansion outpaced the initial platform's capabilities, a new vendor was engaged in 2023 to accommodate growth and analytical needs, paving the way for ongoing adaptation to the evolving research landscape. Collaboration with state NBDF chapters, other non-profits, government entities, and ATHN remains integral to the research process, fostering regular interactions both virtually and in person.
The CVR PD serves as a bridge between clinicians, researchers, and LEEs, facilitating effective information exchange. It provides education and research findings and enables LEEs to link with additional research opportunities. CVR also furnishes a robust cohort for research purposes and integrates participants' input into pre-clinical trial designs, as well as the development of educational and promotional materials. Its infrastructure facilitates rapid research cycles, monitors longitudinal changes, and identifies forthcoming research inquiries pertinent to all stakeholders.
CVR represents a pivotal starting point for researchers to actively engage with LEEs through the VAPs/PAPs programmes. Advisory boards now include LEEs in the discussion of future research proposals, enabling a unique, timely, and personal perspective to research questions and relevant findings. Including LEEs early in the research process as members of the team and utilising CVR data helps identify and pursue community-prioritised research gaps.
CVR has accelerated the movement to engage LEEs at regional and national levels, in rural communities, and outside the HTC network. Strategies to augment LEE engagement with CVR include the development of the personal ‘health summary’ (summarising participants’ health history and current health concerns), and an updated family baseline survey (including self-reported caregiver burden items), and a direct linkage to additional research opportunities beyond CVR surveys and clinicaltrials.gov.
CVR data has enabled assessment of several LEEcentred gaps in research, including pain issues [27,31,34], gene therapy preferences [28–30,37], perceived social support and mental health [31,34], racial and ethnic disparities among PIBD [33], and the disability paradox when comparing persons with haemophilia to the general population [36]. It has also increased awareness of how timely issues affect LEEs participating in CVR and has resulted in educational initiatives focused on vaping and COVID-19. As more participants enrol and the data matures, the opportunities to identify additional gaps in research will increase further.
Challenges, Barriers, and Limitations
The data collected through CVR represents the perspectives of self-selected enrolled participants and may not fully represent that of all LEEs. LEEs without access to a computer, smartphone, or Wi-Fi may be excluded from the registry.
Reliance on web-based recruitment during the COVID-19 pandemic highlighted issues associated with this method and the importance of close monitoring to ensure data validity.
Protection of data remains a top priority. During the 2023 hiatus, the CVR research team established a strict enrolment protocol including CAPTCHA and two-factor authentication and ensured that only entries meeting predefined criteria were enrolled.
While some clinicians have expressed concerns that self-reported data may not be as accurate as data collected directly from the clinician, many studies have shown that self-reported data [40–42] and patient-reported outcomes are accurate and valid [43,44]. Their use is recommended to better understand LEE views and measure outcomes relevant to them [45].
Conclusion
The inclusion of LEEs to enrich research has been effectively highlighted through the implementation of CVR. This approach not only offers a mechanism for engagement, but also allows for a deeper insight into their perspectives. It is imperative that LEEs become active partners in every phase of research. Such involvement presents LEEs and researchers with the opportunity to develop more relevant studies and educational initiatives in a timely manner.