Recent developments in multi-criteria evaluation of regulations

Main Article Content

F. Bartolini
D. Viaggi


Multi-Criteria Analysis, policy evaluations, decision support, indicators, multiple impact


Introduction Agricultural activity is affected by a wide set of regulations and prescriptions due to agricultural and non-agricultural policies. In agricultural policy literature, growing attention is being paid to the issue of policy evaluation. Objectives The objective of this paper is to provide a literature review of the MultiCriteria Analysis applied in the evaluation of policy and regulations. Methodology The paper proceeds by firstly identifying the main issues of the policy evaluation process and then discussing the potential contribution of Multi-Criteria Analysis as a decision support tool in a policy-making context. Results Multi Criteria Analysis was created as a decision support tool for the selection of alternative projects when decisions are effected by trade-offs between different criteria. However, existing applications to policy evaluations still show significant room for improvement. In particular, an improved ability to support decision making processes can be expected if Multi-Criteria Analysis is included in a consistent process of monitoring and gathering of data, and both Multi-Criteria Analysis and monitoring are developed in such a way as to provide a structured interaction with the decision maker in a participative approach. Conclusion A cautious integration of Multi-Criteria comparison techniques and policy-analysis concepts should be sought in order to better contribute to policy evaluation processes.

Abstract 32 | PDF Downloads 38


Aly S., Vranal I. (2008) Evaluating the knowledge, relevance and experience of expert decision makers utilizing the Fuzzy-AHP. Agricultural Economics-Zemedelska Ekonomika, 54, 529–535.
Banville C., Landry M., Martel J.-M., Boulaire C. (1998) A stakeholder approach to MCDA. System Research and Behavioral Science,15, 15–32.
Bartolini F., Gallerani V., Raggi M., Viaggi D. (2010) Water management and irrigated agriculture in Italy: multicriteria analysis of alternative policy scenarios. Water Policy,12,135–147.
Bartolini F., Gallerani V., Samoggia A., Viaggi D. (2005) Methodology for multicriteria analysis of Agri-Environmental Schemes. ITAES Project, DEIAgra, Bologna, 126pp.
Bernasconi M., Choirat C., Seri R. (2010) The analytic hierarchy process and the theory of measurement. Manage Science 2010 April 1,56, 699–711.
Bouyssou D. (1996) Outranking relations: do they have special properties? Journal Multi-Criteria Decision Analysis,5, 99–111.
Brans J.P. (1982) L’ing ?eni ?erie de la d ?ecision. Elaborationd’instruments d’aide a`la d ?ecision. M ?ethode PROMETHEE.In:L’aide a`la d ?ecision: Nature, instruments et prospectivesd’avenireds Nadeau R., Laundry M. pp. 183–214, Presses del’Universit ?e Laval, Qu ?ebec, Canada.
Brans J.P., Mareschal B. (1992) Promethee-V. – Mcdm Problems with Segmentation Constraints. INFOR Journal,30, 85–96.
Brans J.P., Vincke P. (1985) A preference ranking organisation method: (the PROMETHEE method for multiple criteria decision-making). Management Science,31,647–656.
Bryson N. (1995) A goal programming method for generating priority vectors. Journal of Operational Research Society, 46,641–648.
Cheneyah S., Takeda E. (2008) Exploitation Procedeure Based on Eigenvector Revisited: The Concept of Weighted Preference Flows in Multicriteria Outranking Analysis. Cybernetics and Systems: An International Journal,39, 61–78.
Cornelissen A.M.G., Van den Berg J., Koops W.J., Kaymak U.(2002) Eliciting Expert Knowledge for Fuzzy Evaluation of Agricultural Production Systems. Erasmus Research Institute of Management, Erasmus University. ERIM Report Series Reference No. ERS-2002-108-LIS.
De Lange H.J., Lahr J., Van der Pol J.J.C., Wessels Y., Faber J.H.(2009) Ecological vulnerability in wildlife: an expert judgment and multicriteria analysis tool using ecological traits to assess relative impact of pollutants. Environmental Toxicology and Chemistry, 28, 2233–2240.
Dubois D., Prade H. (1980)Fuzzy Sets and Systems: Theory and Applications. Didier Dubois, Henri Prade Academic Press, New York.
Dye T.R. (1972) Understanding public policy. Prentice-Hall: Englewood Cliffs, New Jersey.
European Commission. (2004)Evaluating EU Activities – A Practical Guide for the Commission Services. (Ed Office for Official Publications of the European Communities. European Commission, Brussels, p. 114.
Evalsed. (2009)The Evaluation of Socio-Economic Development –the Guide (Ed Policy D-GfR). Evalsed, Brussels, p. 176.
Fazil A., Rajic A., Sanchez J., McEwen S. (2008) Choices, choices: the application of multi-criteria decision analysis to a food safety decision-making problem. Journal of Food Protection, 71, 2323–2333.
Fernandez E., Navarro J., Bernal S. (2009) Multicriteria sorting using a valued indifference relation under a preference disaggregation paradigm. European Journal of Operational Research,198, 602–609.
Figueira J., Mousseau V., Roy B. (2005) ELECTRE methods. In: Multiple Criteria Decision Analysis: State of the art surveyseds Figueria J., Greco S., Ehrgott M. Springer, New York: pp.133–162.
Figueira J.R., Greco S., Roy B. (2009) ELECTRE methods with interaction between criteria: an extension of the concordance index. European Journal of Operational Research,199,478–495.
Finn J.A., Bartolini F., Bourke D., Kurz I., Viaggi D. (2009) Expost environmental evaluation of agri-environment schemes using experts’ judgements and multicriteria analysis.Journal of Environmental Planning and Management,52, 717–137.
French S. (1993) A manifesto for the new MCDA era.Journal of Multi-Criteria Decision Analysis,2, 125–127.
Funtowicz S., Martinez-Alier J., Munda G., Ravetz J. (1999) Information tools for environmental policy under conditions of complexity. In: Environmental Issues Series eds European Environmental Agency, 34 pp., Copenhagen, Denmark.
Funtowicz S.O., Ravetz J.R. (1994) The worth of a songbird: ecological economics as a post-normal science. Ecological Economics,10, 197–207.
Gallerani V. (2008) Nuove problematiche per la valutazione dellepolitiche agricole. In:Riforma della PAC, evoluzionitecnologiche e trasformazioni ambientali: aspetti economici, estimativi, giuridici e urbanisticieds Marone E. University Press, Firenze: pp. 37–49.
Gass S.I., Rapcsak T. (1998) A note on synthesizing group decisions. Decision Support System,22, 59–63.
Georghiou L., Roessner D. (2000) Evaluating technology programs: tools and methods. Research Policy,29, 657–678.
Gibbons M., Georghiou L. (1987).Evaluation of Research: ASelection of Current Practices. Organisation for Economic Co-operation and Development. OECD Publications and Information Centre, Paris.
Gonzalez-Zapata L.I., Alvarez-Dardet C., Ortiz-Moncada R.,Clemente V., Millstone E., Holdsworth M., Sarri K., Tarlao G.,Horvath Z., Lobstein T., Savva S. (2009) Policy options for obesity in Europe: a comparison of public health specialists with other stakeholders. Public Health Nutrition,12, 896–908.
Gonzalez-Zapata L.I., Diaz C.A.D., Clemente V., Davo M.C.,Ortiz-Moncada R. (2008) Assessment criteria for public policies on obesity: the view of Spanish stakeholders. Gaceta Sanitaria,22, 309–320.
Guitouni A., Martel J.M. (1997) Tentative guidelines to help choosing an appropriate MCDA method.European Journal of Operational Research,109, 501–521.
Hajkowicz S., Collins K. (2007) A review of multiple criteria analysis for water resource planning and management. Water Resource Management, 21, 1553–1566.
Hayashi K. (2000) Multicriteria analysis for agricultural resource management: a critical survey and future perspectives. European Journal of Operational Research,122, 486–500.
Hinloopen E., Nijkamp P. (1990) Qualitative multiple criteria choice analysis - the dominant regime method. Quality and Quantity, 24, 37–56.
Howlett M., Ramesh M. (2003) Studying Public Policy: Policy Cycles and Policy Subsystems. Oxford University Press, New York.
Janikowski R., Kucharski R., Sas-Nowosielska A. (2000) Multi-criteria and multi-perspective analysis of contaminated land management methods. Environmental Monitoring and Assessment,60, 89–102.
Kahraman C., Kaya I. (2010) A fuzzy multicriteria methodology for selection among energy alternatives. Expert Systems with Application, 37, 6270–6281.
Kaufmann A. (1975) Introduction to the Theory of Fuzzy Subsets. Academic Press, New York.
Keeney R.L., Raiffa H. (1976) Decisions with Multiple Objectives: Preferences and Value Tradeoffs. John Wiley, New York.
Krieger S., Schiefer G., da Silva C.A. (2007)Costs and Benefits inFood Quality Systems: Concepts and a Multi-Criteria Evaluation Approach. (Ed Agricultural Management MaFSA, Division RIaA-I). FAO (Food and Agriculture Organization), Rome,p. 56.
Kuzma J., Paradise J., Ramachandran G., Kim J.A., Kokotovich A., Wolf S.M. (2008) An integrated approach to oversight assessment for emerging technologies. Risk Analysis, 28, 1197–1219.
Laininen P., Hamalainen R.P. (2003) Analyzing AHP-matrices by regression. European Journal of Operational Research,148,514–524.
Laudien R., Pofagi M., Roehrig J. (2010) Development and implementation of an interactive Spatial Decision Support System for decision makers in Benin to evaluate agricultural land resources-Case study: AGROLAND. International Journal of Applied Earth Observation and Geoinformation,12,S38–S44.
Li W.X., Li B.Y. (2010) An extension of the Promethee II method based on generalized fuzzy numbers.Expert Systems with Application, 37, 5314–5319.
Maystre L.Y., Pictet J., Simos J. (1994)M ?ethodes multicrite`res ELECTRE. Politechniques et Universitaires Romandes, Lausanne, Switzerland.
Mouron P., Aubert U., Heijne B., Naef A., Strassemeyer J., Hayer F., Gaillard G., Mack G., Hernandez J., Avilla J., Sol ?eJ.,Saupanor B., Alaphilippe A., Patocchi A., Samietz J., H ?ohn H., Bravin E., Lavigne C., Bohanec M., Bigler F. (2010) A multicriteria decision method assessing the overall sustainability of new crop protection strategies: the case of apple growing in Europe. In 9th European IFSA Symposium, Wien, Austria, 2010, pp. 1–11.
Munda G. (2000) Conceptualising and Responding to Complexity. Environmental Valuation in Europe and Cambridge Research for the Environment.
Munda G. (2004) Social multi-criteria evaluation: methodological foundations and operational consequences. European Journal of Operational Research,158, 662–677.
Munda G., Nijkamp P., Rietveld P. (1994) Qualitative multicriteria evaluation for environmental management. Ecological Economics,10, 97–112.
Munda G. (2009) A conflict analysis approach for illuminating distributional issues in sustainability policy. European Journal of Operational Research, 194, 307–322.
Nachmias D. (1979). Public Policy Evaluation: Approaches and Methods. St. Martin’s Press, New York.
Nijkamp P., Borzacchiello M.T., Ciuffo B., Torrieri F. (2007)Sustainable urban land use and transportation planning: acognitive decision support system for the Naples metropolitan area. International Journal of Sustainable Transportation,1,91–114.
Nijkamp P., Vindigni G. (1998) Integrated multicriteria analysis for sustainable agricultural policy evaluation. Rivista di Economia Agraria, LIII, 9–40.
O’Connor M. (2000) The VALSE project - an introduction. Ecological Economics,34, 165–174.
Onate J.J., Andersen E., Peco B., Primdahl J. (2000) Agri-environmental schemes and the European agricultural landscapes: the role of indicators as valuing tools for evaluation. Landscape Ecology,15, 271–280.
Pohekar S.D., Ramachandran M. (2004) Application of multi-criteria decision making to sustainable energy planning - A review. Renewable and Sustainable Energy Review,8, 365–381.
Primdahl J., Peco B., Schramek J., Andersen E., Onate J.J. (2003) Environmental effects of agri-environmental schemes in Western Europe. Journal of Environmental Management, 67,129–138.
Rao R.V., Patel B.K. (2010) Decision making in the manufacturing environment using an improved PROMETHEE method. International Journal of Production Research,48, 4665–4682.
Roy B. (1985).M ?ethodologie multicrite`re d’aide a la d ?ecision.Economica, Paris.
Roy B., Bouyssou D. (1993)Aide Multicrite`re a`la D ?ecision:M ?ethodes et Cas. Economica, Paris.
Ruzante J.M., Davidson V.J., Caswell J., Fazil A., Cranfield J.A.L., Henson S.J., Anders S.M., Schmidt C., Farber J.M. (2010) Amultifactorial risk prioritization framework for foodborne pathogens. Risk Analysis, 30, 724–742.
Saaty T.L. (1980)The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. McGraw-Hill, New York.
Sevkli M. (2010) An application of the fuzzy ELECTRE method for supplier selection. International Journal of Production Research, 48, 3393–3405.
Siciliano G. (2009) Social multicriteria evaluation of farming practices in the presence of soil degradation A case study in Southern Tuscany, Italy. Environmental Development and Sustainnens,11, 1107–1133.
Stewart T.J. (1992) A critical survey on the status of multiple criteria decision-making theory and practice. OMEGA-International Journal of Management Science, 20, 569–586.
Stirling A. (1998) Risk at a turning point? Journal of Risk Research,1, 97–109.
Themelin A., RaoultWack A.L., Lebert A., Danzart M. (1997) Multicriteria optimization of food processing combining soaking prior to air drying. Dry Technology,15, 2263–2279.
Tsiporkova E., Boeva V. (2006) Multi-step ranking of alternatives in a multi-criteria and multi-expert decision making environment. Information Science,176, 2673–2697.
Turner R.K., Morse-Jones S., Fisher B. (2010) Ecosystem valuation a sequential decision support system and quality assessment issues. Ecological Economics Review,1185,79–101.
Van den Hove S. (2006) Between consensus and compromise: acknowledging the negotiation dimension in participatory approaches. Land Use Policy, 23, 10–17.
Viaggi D., Finn J., Kurz I., Bartolini F. (2010) Using information from mid term evaluations of rural development programmes for the multicriteria analysis of agri-environmental schemes. Agricultural Economics Review; in press.
Vincke P. (1992) Multicriteria Decision-Aid. John Willey & Sons, New York, USA.
Witters N., Van Slycken S., Ruttens A., Adriaensen K., Meers E., Meiresonne L., Tack F.M.G., Thewys T., Laes E., Vangronsveld J. (2009) Short-rotation coppice of willow for phytoremediation of a metal-contaminated Agricultural area: a sustainability assessment. Bioenergy Research,2,144–152.
Yoe C. Trade-off analysis planning and procedures guidebook. Prepared for Institute for Water Resources, U.S. Army Corps of Engineers, Alexandria, VA, 2002, p. 155.
Zadeh L.A. (1965) Fuzzy sets. Information Control, 8, 338–353.
Zhang K.J., Kluck C., Achari G. (2009) A comparative approach for ranking contaminated sites based on the risk assessment paradigm using fuzzy PROMETHEE. Environmental Management,44, 952–467.