How is Citizen Science different from Science?

How does Citizen Science actually work? - Impact models and goals

Citizen Science enables citizens to participate in the research process. But how does this participation actually affect the Citizen Scientists? Are you learning about the topic of the Citizen Science project or about scientific working methods? Do you change your interest in scientific topics or even your behavior through doing research? In the first post of our new blog series, we lay the foundation for our insight into impact research, on which we will build in the coming posts, by explaining impact models and goals.


Click here for the overview page of the blog series!


Impact models and goals

When developing a project or when writing a funding application, one is often confronted with the task of formulating clear objectives for a project and describing methods of how one would like to find out whether these objectives have actually been achieved. It is no different with citizen science projects. The Citizens create knowledge-Survey showed, for example, that over 60% of the participating citizen science projects evaluate their programs with regard to different aspects of the effectiveness of citizen science. But where do you start? What concepts and resources are there to support this process?

It can be very helpful for developing a project Impact model to be formulated for the specific project. In this way, the structures and relationships of a project become clear in a clear way. An example of such an impact model is the logic model. This model differentiates between input, output, outcomes and impact and shows how these are related.

  • Inputs: all resources that are required or were available to implement the measure
  • Outputs: Services provided, use of services by target groups, satisfaction of the participants
  • Outcomes: new knowledge, skills, abilities, formation of opinions, changes in the actions or in the life situations of the participants
  • Impact: Changes within society as a whole

We recommend this to deepen your knowledge Course book effect by PHINEO. It is available free of charge as an online learning tool and as an eBook.


A logic model for citizen science

In the area of ​​citizen science, a logic model for projects in the area of ​​biodiversity and environmental research was specified. In this so-called PPSR (Public Participation in Scientific Research) logic model (Figure 1) by Jennifer Shirk and colleagues, it is shown how and on which levels projects can have an impact.

Figure 1: PPSR (Public Participation in Scientific Research) logic model translated from Shirk et al. 2017

The outcome level can be divided into different target areas:

  1. Science: e.g. concrete research results, answers to research questions, scientific publication on methods and results
  2. “Socio-ecological” systems: This includes new legislation and other political measures (e.g. new environmental protection zones, measures to reduce noise or improve air quality), but also the networking of relevant stakeholder groups.

The model here relates to projects in the field of environmental protection and biodiversity, so the designation of this category may have to be adapted for other research fields.

  1. Individuals: This includes new or in-depth knowledge and skills, but also aspects such as motivation, interest and attitudes as well as changes in behavior.

In addition, there are other models and frameworks for describing and investigating the effects of Citizen Science in a structured manner. For example:

  • The very complex CPI framework by Mohammad Gharesifard and colleagues (2019), with 5 categories and 22 sub-categories, should enable an analysis of the context, a process evaluation and an impact analysis.
  • The Framework by Rebecca Jordan and colleagues (2012) focuses not only on the individual and the program level, but also on the impact on the community level.
  • in the model Till Bruckermann and colleagues (2020) emphasize the learning opportunities offered and the real use of learning opportunities in order to then be able to describe the cause of an effect more precisely.

Models can therefore be selected or adapted accordingly with regard to the focus of the project.


In general, the PPSR logic model is a good starting point for formulating the goals of a project and locating them. To support this process, the Outcome Spaces Framework-Method that our colleagues from the Participatory Science Academy Zurich also particularly recommend in collaborative and co-designed projects and processes. Based on the Description of the method in the td-net toolbox we offer a worksheet here to support your thoughts on the objectives of your Citizen Science project. In a next step, these goals can then be classified in the logic model. In this way, in the overall concept, it is possible to consider in a structured manner how the goals are to be achieved and which resources are available or necessary for this. We also offer a worksheet to support this (translated into Phillips et al., 2014)

Formulate goals

Project goals should be formulated according to the SMART criteria (Flinkerbusch et al., 2017):

  • S.specific
  • M.edible
  • A.accepted
  • R.ealistic
  • Terminable

They should be phrased positively and as a result, not as an intention. So rather "XY is reached " instead of "XY should be reached ". In addition, it is important to clearly identify the target group for which the impact goals are to be achieved.

Are the goals being achieved?

After the target has been formulated, the target achievement of the corresponding measure must be checked. This can fulfill various functions (Stockmann, 2002):

  • Cognitive function
  • Control function
  • Legitimation function
  • Learning function

Impact research aims primarily to gain knowledge about effects and mechanisms of action in order to generate evidence-based knowledge about these processes and to expand our understanding of them. Evaluations, on the other hand, usually also fulfill very specific project-specific functions, as control measures can be developed from the findings or the resources (input) used and the continuation of the programs can be legitimized, for example, for funding agencies.

There are also practical guides to help you check your goals, for example the User’s Guide for Evaluating Learning Outcomes from Citizen Sciencefrom the Cornell Lab of Ornithology or the Guide to the evaluation of projects in the federal program for biological diversity from the Federal Agency for Nature Conservation. These can help to define indicators that show the achievement of a goal in an evaluation, or to develop an investigation design with appropriate survey methods (e.g. with this worksheet, translated into Phillips et. al. 2014). The book chapter by Teresa Schaefer and colleagues is also helpful (2021) Evaluation in Citizen Science: The Art of Tracing a Moving Target and for more information on evaluations in the field of science communication in general, we refer to the work ofImpact unit of Science in dialogue.

Effect on individuals

In this blog series we will focus on the effects of Citizen Science on the individuals, in our case the participants. That's why we're zooming a little deeper into this area, because it can be structured in more detail. For example, the framework by Tina Phillips and colleagues is a good choice (Figure 2), which differentiates between different areas of activity at the individual level.

Figure 2: Different aspects of outcomes for individuals translated into Phillips et al., 2018

This detailed examination makes it clear how diverse the goals can be with regard to the effects of Citizen Science on individuals. In our blog series, we will provide an overview of published research findings in these areas and present useful resources for a deeper analysis. We will process findings from review studies, from the most frequently cited publications on the respective areas according to our Scopus research and from selected current publications and publish new articles twice a month.

Literature:

Bruckermann, T., Lorke, J., Rafolt, S., Scheuch, M., Aristeidou, M., Ballard, H., Bardy-Durchhalter, M., Carli, E., Herodotou, C., Kelemen-Finan , J., Robinson, L., Swanson, R., Winter, S., & Kapelari, S. (2020). Learning opportunities and outcomes in citizen science: A heuristic model for design and evaluation. In O. Levrini & G. Tasquier (Eds.), Electronic Proceedings of the ESERA 2019 Conference. The Beauty and Pleasure of Understanding: Engaging With Contemporary Challenges Through Science Education (pp. 889-898). University of Bologna.

Gharesifard, M., Wehn, U., & van der Zaag, P. (2019). What influences the establishment and functioning of community-based monitoring initiatives of water and environment? A conceptual framework. Journal of Hydrology, 579, 124033.

Flinkerbusch, E., & Nowack, C. (2017): Guidelines for the Evaluation of Projects in the Federal Program for Biological Diversity. Federal Agency for Nature Conservation. https://biologischerevielfalt.bfn.de/fileadmin/BfN/service/Dokumente/Evaluationsleitfaden_LNV_2017_PH5_Final__BITV_.pdf

Jordan, R. C., Ballard, H. L. & Phillips, T. B. (2012). Key issues and new approaches for evaluating citizen-science learning outcomes. Frontiers in Ecology and the Environment, 10(6), 307–309. https://doi.org/10.1890/110280

Mitchell, Cynthia, & Fam, Dena. (2020, April 30). Outcome spaces framework. Outcome Spaces Framework. Zenodo. http://doi.org/10.5281/zenodo.3717200

Phillips, T., Ferguson, M., Minarchek, M., Porticella, N. & Bonney, R. (2014, December). User’s Guide for Evaluating Learning Outcomes from Citizen Science. Cornell Lab of Ornithology. https://www.openchannels.org/sites/default/files/literature/Users%20Guide%20for%20Evaluating%20Learning%20Outcomes%20from%20Citizen%20Science_0.pdf

Phillips, T., Porticella, N., Constas, M. & Bonney, R. (2018). A Framework for Articulating and Measuring Individual Learning Outcomes from Participation in Citizen Science. Citizen Science: Theory and Practice, 3(2), 3. https://doi.org/10.5334/cstp.126

Phineo: Course book effect.https://www.ffekt-lernen.de

Schaefer T., Kieslinger B., Brandt M., van den Bogaert V. (2021) Evaluation in Citizen Science: The Art of Tracing a Moving Target. In: Vohland K. et al. (eds) The Science of Citizen Science. Springer, Cham. https://doi.org/10.1007/978-3-030-58278-4_25

Shirk, J.L., H.L. Ballard, C.C. Wilderman, T. Phillips, A. Wiggins, R. Jordan, E. McCallie, M. Minarchek, B.V. Lewenstein, M. E. Krasny, and R. Bonney. 2012. Public participation in scientific research: a framework for deliberate design. Ecology and Society 17 (2): 29.http://dx.doi.org/10.5751/ES-04705-170229

Stockmann, R. (2002). What is a good evaluation? Introduction to functions and methods of evaluation procedures (CEval working papers; 9). Center for Evaluation. https://ceval.de/modx/fileadmin/user_upload/PDFs/workpaper9.pdf


Click here for the overview page of the blog series!