Project Grant Competition
Part 2: Overview of the Peer Review Process

Title

Welcome to this learning module in the Project Grant competition series: Part 2: Overview of the Peer Review Process. In this module, reviewers will learn about the Project Grant review activities to ensure they feel prepared to effectively participate in the peer review process.

Playbar buttons

This course is designed to be self paced.

Use the playbar below to resume playback, navigate between slides, mute and unmute audio, and toggle closed captions. You can also browse the full table of contents, and collapse or move the playbar.

The Project Grant program

The Project Grant program is open to applicants in all areas of health research that are aligned with the CIHR mandate. "To excel, according to internationally accepted standards of scientific excellence, in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened health care system."

The Project Grant program 2

The Project Grant program is designed to capture ideas with the greatest potential for important advances in fundamental or applied health-related knowledge, the health care system, and/or health outcomes, by supporting projects with a specific purpose and a defined endpoint. The best ideas may stem from new, incremental, innovative, and/or high-risk lines of inquiry or knowledge translation approaches.

The Project Grant program 3

Project Grant applications follow a single-stage committee-based peer review evaluation process. This process involves the evaluation of applications by a group of reviewers, who have the required experience and expertise to assess the quality and potential impact of the proposed research and research related activities, within the context of the program's objectives. These reviewers are grouped into Peer Review Committees based on their expertise and the topics of applications submitted to these committees.

Peer Review Committees (PRC) Responsibilities

Peer Review Committees (or PRCs) are responsible for:

• evaluating individual applications based on adjudication criteria and interpretation guidelines;
• providing written reviews for each application;
• discussing applications at the committee meeting;
• rating each application according to a consensus score;
• and lastly, recommending a budget and term to support the proposed research if the application is approved.

Assigning applications

In this section, reviewers will learn about the process of assigning applications for the Project Grant competition.

Assigning applications 2

The competition process begins with the applicant submitting a registration followed by a full application. Within the ResearchNet registration process, applicants are required to indicate their suggestion of up to two most relevant peer review committees and to provide justification for the committees selected.

Following submission of registration, the Chairs, Scientific Officers, and CIHR staff, review the registrations assigned to their committees. Together, they are responsible for ensuring that their committees have the appropriate expertise and upon accepting applications for review by their committees, they accept responsibility for ensuring that their committees conduct a fair review.

In some cases, registrations may not meet a particular committee's mandate and may need to be reassigned. CIHR will make the final decision on which peer review committee will review each application based on the summary of proposed research received at registration.

After the list of applications is compiled following the submission of full applications, committee members are given access to the application summaries to declare any conflicts of interest and indicate their level of expertise which are categorized as High, Medium, Low, or Not enough expertise.

Note: A reviewer is not automatically in conflict if they are from the same institution as the applicant, but does not know or work with the applicant.

Assigning applications 3

Next, the Chairs, Scientific Officers and CIHR staff then assign each application to three committee members based on optimal alignment between the application content and the reviewer's declared expertise.

It is possible that once access to the full applications is granted to reviewers, a reviewer may decide that they do not have the appropriate expertise to review one or more of the applications assigned to them. If this happens, reviewers are to contact CIHR immediately so that the application can be re-assigned appropriately.

Before the peer review committee meeting, all committee members, excluding external referees, are given access to the full applications assigned to their committee.

While the Project Grant competition has no limit on number of times an application can be resubmitted, the review of the application is conducted as per the evaluation criteria within a given competition. Every competition is distinct, so reviewers are instructed to treat all applications, including resubmissions, as new applications.

Reviewing applications

In this section, reviewers will learn about the process of reviewing applications for the Project Grant competition.

Principles of peer review

The integrity of the peer review process relies on well-established principles and policies that ensure fair and effective evaluation, and support CIHR's mandate and objectives.

CIHR's principles of review are: confidentiality, absence of conflict of interest, fairness, and transparency.

Review quality

In addition to incorporating the principles of review, the success of the peer review system is critically dependent upon the willingness and ability of all reviewers to exercise rigorous scientific judgement. They must be fair and reasonable, and take into account, in a balanced way, the particular context of each application.

Reviews must be written in the reviewer's own words and must not be directly copied from elsewhere. This would include comments from other reviewers, their own previous reviews, or those generated by artificial intelligence tools (the use of which constitutes a breach of confidentiality).

A constructive quality review that helps the applicant by pointing out the strengths and weaknesses to be improved upon in a future submission will help demonstrate to an applicant that a fair assessment of the proposal was provided.

Broadening your assessment of research contributions and impacts

The San Francisco Declaration on Research Assessment (DORA) recognizes the need to improve the ways in which research is evaluated. It is therefore crucial that peer reviewers look broadly, beyond the traditional indicators of productivity, when assessing contributions and impacts. For example, rather than using metrics such as h-index to assess productivity, consider the context of the applicant and focus on assessing quality and impact directly. Some examples of historically valued indicators and additional valuable research outputs are listed here. Visit our website to learn more about broadening your assessment of research contributions and impacts.

Summary of Progress

When accessing an application for review, one of the first items presented will be the Summary of Progress. The Summary of Progress is a mandatory task on all applications. It supports the proposal by allowing Nominated Principal Applicants to describe how the application being submitted fits into their overarching research program.

This two-page document supports the research proposal by allowing applicants to:

• Outline and contextualize any activities, contributions and impacts that support the current application;
• Describe how the application fits within their overarching research program and why the requested funds are needed. This should include a clear outline of how the current budget request is distinct from funds currently held or distinguishes between applications submitted to other funding agencies or organizations, pending grants.
• At their discretion and where relevant, outline the impact of specific factors on their research progress. For example, leave history, career stage, pandemic impact, family responsibilities or other circumstances.
• Lastly, ensure the Summary of Progress is a narrative and not a detailed accounting of progress and funding. Details on funding can be found in the applicant's CV.

Resubmission applications

Applicants are asked to identify whether their application is a resubmission of an unsuccessful application submitted to a previous Project Grant competition. This information will be available in the peer review version of the application PDF.

Reviewers are instructed to treat all applications, including resubmissions, as new applications. This is done in an effort to ensure that all applications are reviewed relative to each other. For additional information, please consult the Resubmissions webpage.

Response to previous reviews

For applicants who are resubmitting a previously unsuccessful application, they may provide a response to previous reviewers' comments. In this response, the applicants must include all the reviews and Scientific Officer (SO) Notes (if available) received in that round of submission in a new attachment called Previous Reviews.

Reviewers should note that the Response to Previous Reviews remains an optional task and both these attachments are independent of the resubmission question. Reviewers will only see the Response to Previous Reviews in an application if the Previous Reviews attachment was included.

Applicant profile CVs

All academic applicants (regardless of their role on the application) must continue to use their CIHR Biosketch CV. Non-academics, Indigenous organizations, knowledge users and international applicants have the option of uploading either the CIHR Biosketch CV or the Applicant Profile CV as an attachment to the Participant Information section in ResearchNet. Although the Applicant Profile CV may not exceed three pages, there are no section restrictions, therefore, each applicant can choose what to emphasize. For non-academics, it is possible that not all sections are applicable.

Reviewers are to review the applicant CV along with the information provided in the Most Significant Contributions subtask to assess the complement of expertise, experience and resources as it relates to the ability to collectively deliver on the objectives of the project. Additionally, reviewers are to assess productivity broadly by taking into consideration a range of contributions and impacts as well as the context of the applicant and how it may have affected their progress.

Other application material

Reviewers must read all Other Application Materials that are provided.

The mandatory appendices include the following:

• The Sex and Gender-Based Analysis (SGBA) certificate;
• If the Nominated Principal Applicant (NPA) has a pending appointment, the applicant must include a letter of support from the dean of the faculty indicating the date the appointment is expected to take effect. The appointment must begin by the effective date of funding.
• If someone on the application is an international researcher who will be paid from the grant, the applicant must include a letter from their employer attesting that that individual is not being compensated for time spent on the grant-funded research activities.
• If the proposal relates to Indigenous health research, the applicant must include letters of community support from Indigenous partners.

Should an application include appendices that are not eligible, this should be brought to the attention of CIHR staff as soon as possible. To learn more about attachments that are mandatory, optional, and no longer accepted, consult Task 8 of the Project Grant application instructions.

Reviewing applications

Each application will be assigned to three reviewers. Although the applicants can structure their research proposal as they see fit, the reviewers must assess the application based on the specific adjudication criteria.

For each application, in ResearchNet, reviewers must provide a summary of the project to demonstrate their understanding of the research work that is being proposed.

Reviewers must provide their overall initial ratings to one decimal place in advance of the peer review meeting. However, reviewers are not bound by the initial rating and can change it at the peer review committee meeting.

Each of the reviewers' assigned applications must be designated as either top-ranked (competitive) or bottom-ranked (non-competitive) in its current form.

Sex and Gender-Based Analysis needs to be evaluated in the strengths and weaknesses section. It will be part of the reviewers' overall initial score.

Reviewers must provide the strengths and weaknesses of the application based on the evaluation criteria.

Finally, reviewers are required to verify the requested budget and justification. Reviewers may recommend that the budget remains as requested or recommend a reduction. Budget adjustment recommendations must be justified and be specific to item and amount.

For additional information, review the conducting reviews section of the Peer Review Manual.

Formatting requirements for applicant prepared attachments

While CIHR has simplified its attachment formatting requirements for applications, all applicant-prepared attachments must continue to use a minimum of 12 point, Times New Roman font in black type. Other fonts and font sizes may be used for text in tables, charts, figures, graphs and legends only, as long as it is legible when the page is viewed at 100%. If these are not legible when viewed at 100%, reviewers are not required to read them or account for them as part of their assessments.

Should a reviewer be assigned an application that may not have followed instructions, reviewers should bring it to the attention of CIHR staff as soon as possible.

Submitting Reviews

The deadline for uploading the reviews to ResearchNet is five business days before the committee meeting date. This includes the assessment of overall quality (top and bottom groups), the initial rating, and the initial budget recommendations. Once reviewers have submitted their reviews, they will be able to see the reviews of the other reviewers assigned to the same applications and access all applications they are not in conflict with, to prepare for the peer review meeting.

Once reviews are submitted, reviewers will no longer be able to modify them prior to the committee meeting. Reviewers who wish to revise their reviews can do so during, or after the committee meeting. They will have 5 business days after the committee meeting to complete their changes in ResearchNet.

It is advised that reviewers should save their work often in ResearchNet or copy and paste from a word processor as ResearchNet times-out often.

During the meeting

In this section, reviewers will learn what happens during the peer review committee meetings.

Streamlining applications

For the Project Grant competition, only a portion of the Project Grant applications submitted to a competition will ultimately receive funding. Therefore, it is important that during the meeting, committee members focus their discussions on the most competitive applications to ensure and achieve an accurate comparative evaluation.

To help support this goal, a streamlining process is used to remove applications that meet the streamlining criteria from the discussion process, thereby allowing committee members more time to judge and discriminate between potentially successful applications. Applicants whose proposals are streamlined still benefit from the review process, as they still receive written reviews from the assigned reviewers but no detailed Scientific Officer notes.

It is important to note that applications are identified for streamlining both before, and at the meeting itself. Applications with less than three reviews cannot be streamlined.

An application is streamlined if it meets the following conditions:

1. The average of the reviewers' scores places the application in the bottom 60% of all applications in the committee.
2. At least one reviewer placed the application in their non-competitive (bottom) group. And,
3. There is no objection from any committee member to streamline the application.

Committee Discussion

The assessment of applications begins with the assigned reviewers announcing their initial ratings to one decimal place. The reviewers may revise their initial ratings considering prior committee discussion. The committee is also encouraged to use gender-neutral and gender-inclusive language when presenting reviews and engaging in discussions. If an application is not streamlined, the committee meeting discussion proceeds as follows:

• The first reviewer gives a brief synopsis of the proposal and shares their assessment, describing strengths and weaknesses of the proposal, including how the applicants have addressed sex and/or gender considerations.
• The other two reviewers follow, focusing on points of agreement or disagreement with the other assigned reviewers, and elaborating on points not already addressed.
• Next, if external reviews have been provided by external referees, one of the three reviewers assigned to that application reads these reviews.
• The Chair then opens the discussion to all committee members.
• After, the Scientific Officer reads back the Scientific Officer notes, capturing the key elements of the discussion to be considered by the committee when rating the application.
• The Chair then seeks a consensus rating from the three assigned reviewers. If a consensus cannot be reached, the Chair determines the consensus rating, usually by using the mean value of the assigned reviewers' ratings after committee discussion.
• All committee members not in conflict (excluding the Chair and Scientific Officers) are asked to vote, to plus or minus 0.5 of the consensus rating, for the final rating of each application.
• Finally, CIHR will seek recommendations from the reviewers on the budgets and terms requested. If reviewers propose a budget cut, they need to be specific about which item, by how much, and why.

After the meeting

In this section, reviewers will learn what happens after the peer review committee meetings.

After the meeting 2

Upon completion of the committee meeting, for all applications discussed, CIHR will collect the final ratings and recommendations on funding level and grant term from all the committee members. This information will be used for funding decision-making.

Next, all applicants will receive a Notice of Recommendation, along with a copy of all reviews and the Scientific Officer notes for the applications discussed by the committee. After, a letter of decision and a Notice of Decision will be sent indicating whether or not their application was approved. If approved, it will also contain information on the allocated funds.

Applications that have been flagged for special attention and followed up by CIHR staff are withheld as "pending". The applicants are notified if further information is required.

Additional resources

The resources listed on screen will provide you with additional details to prepare you for reviewing applications in the Project Grant competition. Before concluding this module, please complete the survey to assist CIHR in tracking the uptake and improving the quality of the learning.

You may choose to exit the module and return to the learning page or continue to the next part of the Project Grant competition series, Part 3: Adjudication criteria and rating scale.

Date modified:

2024-10-03