> For the complete documentation index, see [llms.txt](https://docs.researchhub.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.researchhub.com/welcome/the-problem/research-grants-incentivize-waste/lack-of-accurate-metrics.md). # Lack of accurate metrics The main problem of academic science can be summed up in one question: "How does one judge the quality of a scientific researcher?\ \ Put yourself in the shoes of any individual or organization that has prioritized investing in scientific research - you have a $1m grant earmarked for quantum gravity research and two different physicists have applied for the funding. How would you go about choosing which lab will produce the most scientific bang for your buck?\ \ Funders have traditionally relied on bibliometrics to help guide their grant-making decisions. To determine if a project is likely to produce a safe return on investment, they judge the productivity of the applying researcher, and the impact of their previous publications. * **Productivity**: A quantitative metric of the number of publications a researcher has produced * **Impact**: A qualitative measure of a publication’s importance, based on the number of citations it generates. The most commonly used impact metric is the “*impact factor*” of the academic journal in which a paper is published. “Impact factor” is calculated based on the number of times a journal is cited by academic publications within the previous two years. It has been well-established that the use of productivity and impact are not effective tools to objectively determine the best recipient of grant funding. A 2017 study found that there was no agreement between National Institute of Health reviewers in their qualitative or quantitative evaluation of grant-applications. > *“It appeared that the outcome of the grant review depended more on the reviewer to whom the grant was assigned than the research proposed in the grant.”* - [Pier et al., 2017](https://pubmed.ncbi.nlm.nih.gov/29507248/) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.researchhub.com/welcome/the-problem/research-grants-incentivize-waste/lack-of-accurate-metrics.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.