NSF to Update Merit Review Procedures – Impact on Broader Impacts?

NSF is offering a draft of a new Proposal and Award Policies Procedure Guide. The guide is expected to ‘go live’ in January 2017.  An active leader in the National Alliance for Broader Impacts recently noted that this proposed guide has some subtle yet rather interesting changes in the merit review wording. The biggest question is whether or not the new wording actually clarifies PI options for leveraging existing programs (see red font portion below) versus maintaining a high standard of unique innovation for each proposal’s outreach. Also, there is no readily available list of recommended ways to achieve Broader Impacts per these guidelines.

Read this excerpt from the proposed guidelines to consider how you can best manage these upcoming changes.

______Excerpt Starts Here___________

Merit Review Principles and Criteria The National Science Foundation strives to invest in a robust and diverse portfolio of projects that creates new knowledge and enables breakthroughs in understanding across all areas of science and engineering research and education. To identify which projects to support, NSF relies on a merit review process that incorporates consideration of both the technical aspects of a proposed project and its potential to contribute more broadly to advancing NSF’s mission “to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense; and for other purposes.” NSF makes every effort to conduct a fair, competitive, transparent merit review process for the selection of projects.

Merit Review Principles These principles are to be given due diligence by PIs and organizations when preparing proposals and managing projects, by reviewers when reading and evaluating proposals, and by NSF program staff when determining whether or not to recommend proposals for funding and while overseeing awards. Given that NSF is the primary federal agency charged with nurturing and supporting excellence in basic research and education, the following three principles apply:

  • All NSF projects should be of the highest quality and have the potential to advance, if not transform, the frontiers of knowledge.
  • NSF projects, in the aggregate, should contribute more broadly to achieving societal goals. These broader impacts may be accomplished through the research itself, through activities that are directly related to specific research projects, or through activities that are supported by, but are complementary to, the project. The project activities may be based on previously established and/or innovative methods and approaches, but in either case must be well justified.  [The NABI rep asks, “Does this mean that they are specifically indicating that it is okay to leverage EXISTING BI programming and there is not always a need for ‘unique and innovative BI?’”]
  • Meaningful assessment and evaluation of NSF funded projects should be based on appropriate metrics, keeping in mind the likely correlation between the effect of broader impacts and the resources provided to implement projects. If the size of the activity is limited, evaluation of that activity in isolation is not likely to be meaningful. Thus, assessing the effectiveness of these activities may best be done at a higher, more aggregated, level than the individual project.

With respect to the third principle, even if assessment of Broader Impacts outcomes for particular projects is done at an aggregated level, PIs are expected to be accountable for carrying out the activities described in the funded project.

Thus, individual projects should include clearly stated goals, specific descriptions of the activities that the PI intends to do, and a plan in place to document the outputs of those activities. These three merit review principles provide the basis for the merit review criteria, as well as a context within which the users of the criteria can better understand their intent.

  1. Merit Review Criteria All NSF proposals are evaluated through use of two National Science Board approved merit review criteria. In some instances, however, NSF will employ additional criteria as required to highlight the specific objectives of certain programs and activities. The two merit review criteria are listed below. Both criteria are to be given full consideration during the review and decision-making processes; each criterion is necessary but neither, by itself, is sufficient. Therefore, proposers must fully address both criteria. (GPG Chapter II.C.2.d.(i) contains additional information for use by proposers in development of the Project Description section of the proposal.) Reviewers are strongly encouraged to review the criteria, including GPG Chapter II.C.2.d.(i), prior to the review of a proposal.

When evaluating NSF proposals, reviewers will be asked to consider what the proposers want to do, why they want to do it, how they plan to do it, how they will know if they succeed, and what benefits could accrue if the project is successful. These issues apply both to the technical aspects of the proposal and the way in which the project may make broader contributions. To that end, reviewers will be asked to evaluate all proposals against two criteria: • Intellectual Merit: The Intellectual Merit criterion encompasses the potential to advance knowledge; and • Broader Impacts: The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.

The following elements should be considered in the review for both criteria: 1. What is the potential for the proposed activity to: a. Advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and b. Benefit society or advance desired societal outcomes (Broader Impacts)? 2. To what extent do the proposed activities suggest and explore creative, original, or potentially transformative concepts? 3. Is the plan for carrying out the proposed activities well-reasoned, well-organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success? 4. How well qualified is the individual, team, or organization to conduct the proposed activities? Proposal & Award Policies & Procedures Guide III-3 NSF 17-1 5. Are there adequate resources available to the PI (either at the home organization or through collaborations) to carry out the proposed activities?

Looking for more information and practical tips about Broader Impacts? See these additional posts in this blog.

 

Leave a Comment