Validating the quality of proposed and ongoing research

Research is a critical component of any nation’s competitive position. We share an understanding of the progression of science, from exploratory research to economic outcomes. Like any large-scale, globally competitive enterprise, research involves risks. Often the stakes are high–large investments in equipment, academic preparation of researchers, and national strategy are all based on the expectation of research success.

How do funders deal with the uncertainty involved with investing in research? How does the community of science work nationally and globally to optimize return on investment? Expert evaluation (aka merit review or proposal evaluation) directly addresses these risks by preventing initial funding miscues and refining the direction of established programs.

Peer review is an established tradition with a practical purpose and many benefits to science and scientists. Funders receive valuable information to support decisions. Scientists receive valuable feedback on proposed ideas. Panel members learn about exciting new research directions. Peer reviews of articles provide checks and balances that help avoid missteps. In short, it’s simple:  quality peer reviews improve science quality.

Quality peer review is built on three pillars:

  1. Reviews must be independent
    Independence means to speak without bias or inappropriate influence. Funding agencies disseminate calls for proposals that include criteria to align proposals with research strategies. The review exists to evaluate the degree to which each proposal advances that agenda. Reviews must be managed to avoid conflict of interest or expression of vested interests or undue bias.
  2. Reviewers must be qualified
    The nature of peer review is the expectation that evaluators are at least as qualified as the proposer. A thorough vetting process is required to identify and engage highly qualified experts as reviewers. Often the search is global and demands some understanding of the proposed research.
  3. Reviews must be conducted with integrity
    Protecting review integrity is important to everyone involved. Quality peer review requires vigilant management of a well-designed systematic process. Reviews must be conducted in a manner that will withstand critical scrutiny. Any stakeholder has the right to contest or appeal sponsor decisions. Poorly conducted reviews that are not challenged potentially pose an even greater risk.

Adherence to these basic principles ensures peer-review outcomes are respected and make a meaningful impact.

The Oak Ridge Institute for Science and Education provides peer review services and is interested in engaging with scientists, funders, and other peer review professionals to help advance science.