Research evaluation

Sharing and developing best practice for evaluation of research in organic food and farming: The results of work package 5 of the Core Organic project

1. Organisation of organic farming research, with emphasis on the question, whether an organic research programme exists or whether organic farming research is integrated in a general scheme.

2. Organisation of organic programmes and the priority setting process.

3. Different steps of open calls.

4. Reporting and monitoring of the projects and programmes.

5. Organisation of ex-post evaluation.

6. Dissemination activities.

7. Proposals on how the procedures could be optimised

8. How do the countries deal with interdisciplinarity, grass root research, and scientifically controversial methods.

The results of this survey are presented in a project report, finalized in March 2006. This report is available at the Core Organic intranet. For each of the eight sections the main findings are summarized in a table (see example) supplemented by an explanatory text.

• Four countries have their organic farming research integrated in a general scheme and have no specific organic programme. Specific calls are issued just occasionally. Another important difference between the countries is the frequency of the calls. Five partners do not issue their calls regularly. The Scandinavian countries and France launch calls regularly. However the frequency varies between once every five years (DK) up to four times a year (NO).
  
  
• The description of the priority setting process and of the actors involved is very similar in the participating countries. There might be a difference in the level of formality. Some countries describe the process as rather informal, others use a more formal approach (two-step-consultation).
  
  
• At first sight, there seems to be no big difference in how the countries handle the organisation of open calls. With some exceptions, in most countries there is the possibility to submit a pre-proposal, and the evaluation process includes also a panel discussion. However, there is an important difference regarding the duration of the evaluation, which can last between 3 and 40 weeks. This indicates that there might be more differences in the organisation of the open calls than it appears from this survey.
  
  
• The most frequently named criterion is scientific excellence. Furthermore specific competence of the applicants as well as relevance and innovation for organic farming are important criteria for the ex-ante evaluation. None of the partners uses specific criteria for organic farming or suggests criteria that could be used. Also nobody expresses the need to enlarge the set of criteria specifically suited to evaluate organic farming projects.
  
• The countries deal differently with anonymity and payment of the evaluators. Potential conflicts are avoided in different manner ranging from open discussion to the exclusion of experts. Matching funding is requested by some partners and is regarded by most partners as positive.
  
  
• The requirements of reporting and monitoring are similar in all countries.
  
  
• The ex-post evaluation follows basically the same criteria as the ex-ante evaluation.
  
  
• Dissemination activities are part of the contract in nearly all countries. Publishing in Organic Eprints is compulsory for some countries, whereas others leave it to their researchers to choose adequate dissemination tools.
  
  
• Participants make only a few suggestions on how to improve the procedure: to have more external experts and to minimise administration load by simplifying the workflow are the most important aspects. Developing specific criteria for the evaluation process are not mentioned. UK suggests reinforcing the internal expertise of the programme owner, i.e. to develop an internal intelligent customer function.
  
  
• The opinion on whether and how to stimulate inter-disciplinarity is controversial. Some participants see all agricultural research as multi-disciplinary, whereas others stimulate it through methodological debates and encourage the inclusion of social scientists. None of the countries uses explicitly criteria, which makes grass-root research and scientifically controversial methods eligible. However, nearly all say that such research may be accepted if methodologically sound.
  
  

Download

Relevant parts of the report can be downloaded as an PDF

Reference

Thomas Alföldi, Urs Niggli and Bertil Sylvander (2006): Sharing and developing best practice for the evaluation ofresearch in organic food and farming = Deliverable 5.1 of the ERA-NET project Core Organic CORE Organic - Coordination of European Transnational Research in Organic Food and Farming, Project no. 011716. Available at the Core Organic Intranet

Contact

Thomas Alföldi, Research Institute of Organic Agriculture (FiBL), Ackerstrasse, CH-5070 Frick, Tel. +41 62 8657272, Internet www.fibl.org, E-Mail http://www.fibl.org/english/mail.php?id=1