Reassess User Needs

Along with evaluating and possibly adjusting your goals, you will want to continually reassess external needs to accommodate changing industry and research priorities.

Adapt to Evolving Research Priorities

Continuing to engage with key research stakeholders is critical to the ongoing success of the registry. Stay up to date with current research interests and consider how they have changed since you first set your registry goals. Assessing changing external priorities could reveal important developments, such as:

• Need for transition to a new technology platform
• Changes in your funding sources
• New or shifting stakeholders depending on the status of current research or industry focus
• Introduction of new regulatory requirements
• Changes in Common Data Elements (CDEs)

Continue to Engage with Registry Participants

Within an ongoing, voluntary registry, the ability to recruit and retain participants depends on the relevance of the registry. If participants feel that the registry no longer meets their needs, they will likely stop participating and your registry can lose momentum. It is critical to stay connected with participants and understand their expectations of the registry.

Surveys or other methods of collecting feedback from current and potential registry participants can be useful tools. They can expose recruitment or retention barriers and identify potential improvements to maintain your registry’s relevance to their rare disease community.

Overview

Review whether you’ve met your goals, make adjustments based on that assessment, determine whether you’re ready to move into the next phase of your registry and to begin to plan for the move. This iterative evaluation of your progress will help you to improve the effectiveness and relevance of your registry and provide another opportunity to engage with your stakeholders.

Plan for Next Steps

Look ahead and determine what’s next for your registry. You’ve fostered productive communication and received input from key stakeholders. Use the information gathered from patients, researchers, and industry to inform your next steps.

Next steps could include:

• Forming a Scientific Advisory Board for your registry with key researchers
• Establishing a data governance program
• Securing funding sources to further grow the registry
• Building your network to include other groups or organizations
• Creating a non-profit disease advocacy organization

The scope of your registry can grow to:

• Cover different geographical regions (e.g., national to international)
• Address new research questions
• Include other types of patient data (e.g., medical history, biobanks)
• Transition to a more advanced registry platform

Use the RaDaR Tool: Registry Evaluation Checklist to determine whether the registry is ready to move to the next phase. The resources included in the Resources section of this step are also helpful.

The next phase of RaDaR will include step-by-step information on collecting participant medical history. Read more about future plans for the RaDaR website.

Revisit Registry Goals

Periodically evaluating your registry goals helps you verify that you’re achieving what you committed to―with both your registry participants and researchers. When you set up your registry, you developed a set of measurable metrics to help you monitor and evaluate the registry’s success and help you determine whether you’re accomplishing your goals. If you have not determined the components of your monitoring and evaluation approach, including developing a set of metrics, do this now.

Measure Your Success

The following questions can help you evaluate your progress in meeting your goals. Each question is followed by a sample goal and an associated metric that could help you monitor whether you are achieving the goal.

• How many participants have joined the registry?
  • Sample goal: meet recruitment target
  • Sample metric: total number of participants
• What is the quality of the data you have collected?
  • Sample goal: complete data set
  • Sample metric: number of fully completed registrations
• Are participants updating their data?
  • Sample goal: high retention and active participation
  • Sample metric: number of times participants visit/update their data
• Is your data being used?
  • Sample goal: data used for research studies
  • Sample metric: number of inquiries from researchers and number of studies in which data were used

Identify Problems and Solutions

If you are not meeting your goals, ask yourself:

• What is working well? What isn’t?
• What are the challenges in meeting the registry’s goals? Examples might include:
  • Not finding researchers to use the data
  • Not able to recruit participants
  • Participants not keeping their information up to date
• What are possible solutions to these challenges?
  • Are there easy fixes or do they require longer-term implementation?
  • What resources do you need to accomplish these solutions?
  • Are these solutions feasible given your current resources?
    • If yes, revisit your timeline/calendar of events and registry plan.
    • If not, what are some ideas for how you can obtain the resources you need?

Depending on your answers to these questions, you might want to revise your goals and metrics to assess your registry’s success.

Promote Your Registry

Find researchers who are studying your registry’s rare disease and reach out to them to discuss ways that your registry can support their research projects.

Consider these suggestions for promoting your registry from FasterCures, “Expanding the Science of Patient Input: Building Smarter Patient Registries .”

• Publish periodic data summaries.
• Promote and advertise the ability for end users to direct queries or surveys.
• Exhibit at professional society/trade conferences and demonstrate the availability of registry data.
• Contract with external researchers to conduct studies that you and your participants consider important.
• Issue funding opportunities (if financially feasible) for projects that use registry data.
• Post data sets on open research platforms.
• Issue data challenges or prizes to stimulate data analysis.

Refer to the Resources section of this step to learn about other resources that discuss methods for promoting your registry.

Manage and Share Data

To access your registry’s data, researchers will need to request permission to acquire the data. Provide a means for these potential end users to contact you (via email, your website, social media, or other means). Before you grant this access, determine how you will:

• Control the type of data provided to end users
• Manage requests for the data
• Prioritize data security and participant privacy

Create a data management plan to address the process for receiving data requests:

• How will researchers make their requests?
• Who will respond to data requests and answer any data-related questions?
• In what format will researchers receive data?
• Will researchers be able to contact registry participants anonymously via surveys, or only access data you have already collected?
• How will you ensure the data are used responsibly?

Draft a data user agreement (DUA), which must be entered into before any data is used or disclosed. The DUA should include, at minimum, the following:

• Permitted uses and disclosures of the limited data set
• Who may use or receive the information
• Statement of no use or further disclosure of the information other than as permitted by the agreement or as otherwise permitted by law
• Use of appropriate safeguards to prevent an unauthorized use or disclosure
• Reporting any use or disclosure of the data not allowed by the agreement
• Understanding that registry participants will not be identified or contacted

Find Researchers

The strategy for reaching your target audience will depend on your registry’s rare disease and on the status of the research focused on this disease.

To find academic and commercial researchers:

• Review biomedical company portfolios to learn about their focus areas.
• Attend conferences and meetings that target rare diseases in general or your specific rare disease in particular.
• Partner with related rare disease organizations.
• Explore these online resources:
  • PubMed: A free search engine that primarily accesses the MEDLINE database of references and abstracts on life sciences and biomedical topics; the United States National Library of Medicine at the National Institutes of Health (NIH) maintains the database.
  • ClinicalTrials.gov : The largest clinical trials database holding registrations from more than 230,000 trials from 195 countries; it is managed by the United States National Library of Medicine at NIH.
  • NIH RePORTer: Research Portfolio Online Reporting Tools provides access to reports, data, and analyses of NIH research activities, including information on NIH expenditures and the results of NIH supported research.
  • GARD Guide – How to Find a Disease Specialist: The Genetic and Rare Diseases Information Center (GARD), a program of the National Center for Advancing Translational Sciences (NCATS), provides tips for finding researchers and healthcare professionals with expertise in a particular rare disease.

Overview

Let researchers know your registry exists—if they do not know about it, they will not use it. Continue to prioritize data stewardship and your registry participants’ privacy by entering into data use agreements with registry end users that establish permitted uses of the data.

Clean Data Collected

After reviewing your registry data, you may find you need to contact participants to correct data entry errors. If data are missing, explore why the fields are not completed. It is important to determine whether the data entry error is a user error or an issue with a question or the data collection tool.

Questions to help identify the cause of data entry errors:  

• Did the participant feel uncomfortable completing the field? Why?
• Did the participant understand the question?
• Did the participant attempt to complete the field?
• Did technical difficulties arise? What were the technical difficulties?
• What other explanation might describe the data entry error?

If you identify problems in the data collection process, fix the issues and then contact the participants and ask them to enter the missing information. If many participants have the same issues with entering information (and it’s not a data collection problem you can resolve), develop a frequently asked questions (FAQ) page that clarifies the data input process, and make it easily accessible on the registry website. 

Perform Data Quality Control

After participants have entered their information into your registry, review the data to identify and correct entry errors. Identify any problems that may affect the data collection process, such as issues with converting addresses to the standard U.S. Postal Service format or other glitches that prevent participants from completing certain fields. If participants commonly skim over specific questions, assess whether the question can be rephrased. Consider whether the question asks for information that participants are uncomfortable providing.

Examples of common data entry errors:

• Incomplete fields
• Incorrect data
• Missing data
• Duplicate submissions

Use the RaDaR Tool: Data Quality Control Checklist to help you access the quality of the data you’ve collected for your registry.