Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

COVID-19 vs. stakeholder engagement: the impact of coronavirus containment measures on stakeholder involvement in European energy research projects

COVID-19 vs. stakeholder engagement: the impact of coronavirus containment measures on... version 3 The coronavirus (COVID-19) pandemic has affected societies and economies around the world, and the scientific community is no (revision) exception. Whereas the importance of stakeholder engagement in 14 Oct 2021 research has grown quickly the consequences of the pandemic on this version 2 has so far not been empirically studied. In this paper, we investigate the effects of the COVID-19 crisis on European energy research, in (revision) report report particular the stakeholder work, during the first wave of the 27 Sep 2021 coronavirus in spring and summer 2020. We pose the research questions: (i) How much of a problem are the coronavirus version 1 containment measures for stakeholder engagement? (ii) How have report report 25 May 2021 researchers coped with the situation, and (iii) How do they evaluate alternative stakeholder activities implemented? We conducted an 1. Walter Leal Filho, Hamburg University of online survey among European energy research projects with stakeholder engagement between June and August 2020. We found Applied Sciences, Hamburg, Germany that only one of six engagement activities could be implemented as Amanda Lange Salvia, Hamburg University planned, whereas almost half were cancelled or delayed. The most of Applied Sciences, Hamburg, Germany common coping strategies were changing involvement formats – mainly to webinars or online workshops – or postponement. Whereas 2. Michael J. Fell, University College London, respondents are largely satisfied with one-to-one and unidirectional London, UK online formats, such as webinars, online interviews, and online surveys, they see interactive group activities as less suitable for online Any reports and responses or comments on the engagement. Most respondents plan to continue using online formats to complement, but not to replace, physical meetings in future article can be found at the end of the article. research. All long-term effects remain to be seen, but given the Page 1 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 postponement of many stakeholder involvement activities, many projects may face problems at later stages of their realisation. These findings suggest that the pandemic may have catalysed a rapid introduction of specific online formats in academic stakeholder interaction processes. Keywords COVID-19, coronavirus, stakeholder engagement, transdisciplinarity, energy research, Horizon 2020, EU This article is included in the Excellent Science gateway. This article is included in the Research Culture collection. Corresponding authors: Diana Süsser (diana.suesser@iass-potsdam.de), Andrzej Ceglarz (andrzej@renewables-grid.eu) Author roles: Süsser D: Conceptualization, Formal Analysis, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing; Ceglarz A: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing; Stavrakas V: Conceptualization, Formal Analysis, Methodology, Writing – Review & Editing; Lilliestam J: Conceptualization, Writing – Review & Editing Competing interests: No competing interests were disclosed. Grant information: This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No [837089]), (project SENTINEL) and (grant agreement No [837089]), (project TRIPOD). Copyright: © 2021 Süsser D et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: Süsser D, Ceglarz A, Stavrakas V and Lilliestam J. COVID-19 vs. stakeholder engagement: the impact of coronavirus containment measures on stakeholder involvement in European energy research projects [version 3; peer review: 2 approved] Open Research Europe 2021, 1:57 https://doi.org/10.12688/openreseurope.13683.3 First published: 25 May 2021, 1:57 https://doi.org/10.12688/openreseurope.13683.1 Page 2 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Background: stakeholder engagement in research  REVISED           Amendments from Version 2 projects and COVID-19 In recent years, the importance of stakeholder involvement Dear Readers, and transdisciplinarity in sustainability research has grown We have updated the preprint version 1 of this article in response to the comments of two reviewers. These changes quickly, including energy research (Fazey et al., 2018; Lutz mainly relate to the methodological details of our study, & Bergmann, 2018; Mielke et al., 2016). Where values are particularly to clarify our approach better, and to the discussion contested (Funtowicz & Ravetz, 2006), transformations are section, in which we now go into more detail about the conflicting ( Renn, 2019) and decisions are urgent, transdiscipli- implications of our study for stakeholder-based research after narity is an answer for developing societally relevant solutions the pandemic. The full details of the changes are found in the Responses to Reviewers comments. We have made a few to complex, real-world problems (Lang et al., 2012; Lutz & final changes from version 2 to the final article regarding the Bergmann, 2018; Scholz & Steiner, 2015). Furthermore, stake- formatting. holder involvement can increase the relevance of research, bring Kind regards, higher acceptability and accountability of the problem, and the authors increase legitimacy and societal ownership of the research. Given these potential benefits of engaging stakeholders, funding bodies Any further responses from the reviewers can be found at  the end of the article also now encourage, and oftentimes require, the involvement of stakeholders in research (e.g., European Commission, 2020b). Stakeholder involvement is today much more than a social- Introduction scientific add-on: these engagement activities shape the The coronavirus 2019 (COVID-19) pandemic has strongly affected projects themselves, often including co-creation of both societies and economies across the world, including the scien- research questions and project aims, and the projects often tific community. The social distancing and lockdown measures seek to influence the societal processes with which they engage applied in most countries have potentially influenced one par - (Bracken et al., 2015; Klenk et al., 2015). Stakeholders can ticular aspect prominent in many research projects: stakeholder be involved in research to different degrees, encompass- involvement. With the growing importance of stakeholder ing information, consultation, cooperation, collaboration and interaction in research, the tie between science and practice has empowerment (Schneider & Buser, 2018; Stauffacher et al., improved, but science has also become vulnerable to the avail- 2008). Although the degrees of engagement depend on research- ability and readiness of stakeholders to interact with researchers. project phases (Bruhn et al., 2019) and involvement formats Based on our own experiences, we expected that many (Mielke et al., 2017), many of them are based on the physical researchers have faced specific challenges to interact with stake - presence of stakeholders in one location – and these have been holders online, and they had to “experiment” with different particularly strongly affected by the COVID-19 crisis. online formats and tools. Hence, it is important to understand the impacts of the pandemic on stakeholder involvement in Academia responded to the coronavirus pandemic and its research and the satisfaction of coping measures to draw containment measures in various ways: normatively, by encour- implications on what this new mode of online stakeholder aging the promotion of a culture of care and the redefini - involvement means for future choices of physical and digital tion of excellence in teaching or research, e.g., by focusing stakeholder activities. more on inequalities in academic institutional environments (Corbera et al., 2020), but also pragmatically, by quickly adapt- In this paper, we investigate the effects of the COVID-19 ing to the distancing measures and moving academic interactions, crisis on the stakeholder work in European energy research like lectures, seminars and conferences, online (Leal Filho et al., (focusing mainly, but not exclusively, on Horizon 2020 projects), 2020; Schwarz et al., 2020). Similarly, funding bodies reacted during the first wave of the coronavirus pandemic in spring and to the coronavirus outbreak too. For example, the European summer 2020. We address three research questions: (i) how Commission announced that the “ force majeure” clause can be much of a problem are the coronavirus containment meas- invoked in Horizon 2020 projects, if the grant beneficiaries are ures for stakeholder engagement in European energy research not able to fulfil their obligations due to coronavirus restrictions projects?, (ii) how have researchers responsible for stakeholder (European Commission, 2020a), including the stakeholder engagement coped with the new situation?, and (iii) how do engagement activities. This is very important, because in more researchers evaluate the coping measures (if undertaken)? We stakeholder-dependent projects, the COVID-19 crisis certainly report and discuss the findings of a survey distributed to all has the potential to make entire projects unfeasible. Recent running European Union (EU) funded energy projects research (Corbera et al., 2020; Leal Filho et al., 2020) provides with stakeholder components, carried out in June-August important findings on how academia and sustainability research - 2020, investigating the effects of coronavirus containment meas- ers have been impacted and have dealt with the crisis, but did ures on stakeholder involvement in European energy research. not address the impact on stakeholder engagement in research This study is not only relevant for the scientific community projects. We contribute to the closing of this gap by investigat- to gain a better understanding of applied coping strategies for ing the impacts on stakeholder engagement in energy research stakeholder engagement in the times of the COVID-19 pan- and providing insights into how the research community has demic, but also for funding bodies, who have to make decisions coped with the restrictions, as well as what has worked best about how to support research projects under the new conditions. in the first months after the coronavirus pandemic started. Page 3 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 email to projects identified in the CORDIS research database, Methods and existing networks (e.g. partner projects). In addition, the To identify the impact of the COVID-19 crisis and contain- European Commission Directorate-General of Research & ment measures on stakeholder involvement in energy research Innovation helped with the distribution of the survey. We also projects, we carried out an online survey study among peo- promoted the survey via social media channels, such as Research- ple responsible for stakeholder engagement who work in Gate, Twitter and LinkedIn, specifying that the survey should energy research projects across Europe. The survey was done in be completed only by project coordinators or partners respon- summer 2020, around four months after the coronavirus sible for stakeholder engagement activities. The survey was outbreak in Europe. During spring and early summer 2020, all online for twelve weeks during the period June-August 2020. European countries had COVID-19-related containment meas- We allowed also for non-Horizon2020 to participate in our ures in place. These included lockdowns in most countries survey, to expand the sample and broaden the respondent – leading to a closure of many academic and research entities, base to capture experiences from a higher project diversity. cultural institutions and other public spaces as well as strict Additionally, we allowed for responses from multiple research- restrictions on non-essential travels, and introduction of social ers from one project, because different stakeholder activities distancing rules on meeting other people. are often performed in different temporal and geographi- cal contexts, each with a different containment situation. We The survey was designed as a collaboration between research- treated multiple responses from single projects as individual ers for the projects SENTINEL (Horizon2020; energy), TRIPOD responses. (European Research Council; energy), and PANDORA (Horizon 2020; fisheries) as an explorative, semi-quantitative, self- We analysed the statistical survey in three steps: first, we com - completion online questionnaire (cf. Bryman, 2012), using the piled and compared the quantitative responses; the resulting online tool “LimeSurvey” (LimeSurvey, 2020). Survey questions descriptive analysis is the core of the results below (Creswell & were structured around five blocks: Creswell, 2018). Second, we complemented the results based on the written replies. Third, we applied a logistic regression - A. general questions concerning the stakeholder analysis using two different discrete choice models to derive pre- engagement activities in the projects; liminary insights on the correlation of the i) “very negatively” and ii) “not at all” responses in terms of the impact of COVID- - B. COVID-19 impacts on stakeholder engagement; 19 on stakeholder engagement with relevant explanatory vari- ables from other categories of the survey. We performed a logis- - C. coping strategies and alternative formats implemented in tic regression analysis to further shed light on potential factors response to coronavirus restrictions; that could explain the very negative impacts of COVID-19 on the stakeholder engagement activities of the energy research - D. evaluation of the implemented alternative formats; and projects, but also on factors that could explain the zero effect of the pandemic on stakeholders’ involvement. Our goal was not - E. demographic data. to perform a complete econometric analysis (e.g., best-fitting model information criteria, evaluation of parameter estimates The survey contained independent questions as well as ques- using quasi standard errors, etc.), but to identify meaning- tions that built on previous answers. We used different ques- ful correlations between the variables under study to com- tion formats, from Likert-like scales to multiple choice and plement the explanatory analysis of the descriptive statistics free text boxes, depending on the variables to be addressed. of the survey data. We pre-tested the survey with our project partners and adapted it in response. The questionnaire is available as extended data Methodological details on the logistic regression (Süsser et al., 2021). analysis As dependent variables, we selected the two marginal cases For data collection, we identified 195 Horizon 2020 energy “COVID-19 pandemic influence on stakeholder activity/ research projects relevant to our study. The CORDIS data- engagement– Not at all” and “COVID-19 pandemic influence base brought 365 hits of projects using the search keywords on stakeholder activity/engagement– Very negatively” to ‘energy’ and ‘stakeholder’, which started no later than January help us further derive some meaningful explanations. The 2020 and ran at least until the end of 2020. We contacted only latter is also attributed to the fact that, given the format that 195 out of the 365 projects, as the rest of the projects did not focus the online survey took place, our sample did not allow for the on energy questions, were rather technological/industry-focused development of one theoretical prediction model that could without a clear stakeholder component, and/or did not pro- include all the different Likert-scale responses. We converted vide any contact details. We distributed the survey widely via both cases to two separate dummy variables (“1 = YES to the question”, “0 = NO to the question”). For each one of the two dependent variables, we formulated a separate discrete choice Based on the basic questionnaire, each project adapted the questionnaire to model and selected an initial set of explanatory variables based its specific context and distributed it in the relevant communities. Here, we report only on the energy research survey, whereas the fisheries survey is part on its relevance to each one of them. We then converted again of another publication (Köpsel et al., 2021) the categorical explanatory variables to dummy variables. Page 4 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 We made the final selection of the explanatory variables to be for the data collection and processing in the context of this included in each discrete choice model through a trial-and-error study, under the guidance of the Institute for Advanced approach: Different sets of relevant explanatory variables Sustainability Studies (IASS) data protection service. This were tested, examining in parallel the correlation matrix to has been supported by bilateral data protection agreements. test for collinearity issues among the covariates, until no sig- The respondents agreed to our data protection standards via a nificant collinearities were observed, to ensure that each final GDPR disclaimer, and by participating in the survey. set of explanatory variables is able to adequately predict the respective dependent variable. Results Stakeholder engagement in energy research projects For both discrete choice models, the probability that the stake- Stakeholder engagement is a crucial or important com- holder engagement activities of each project i are very negatively ponent of by far most projects in our sample (Figure 2a). affected (or are not affected at all) is modelled: The researchers mainly engage with stakeholders periodi- cally in specific phases of the project ( Figure 2b). Most of the respondents engage with EU stakeholders; about one third P(y =1) = Λ(β⋅ x ) i i of the projects also work with non-European stakeholders, for example in the US, China or Indonesia. where: ○ y is the dependent variable describing if the stakeholder According to the original project plans, 2020 was supposed activities of a project i are very negatively affected, to be a major year for stakeholder engagement for almost all or are not affected at all; respondents; hence almost every project was affected in some way by the coronavirus containment measures in Europe. ○ x is the vector of independent/explanatory variables for the A variety of physical and online activities were planned in th i project; 2020 – mainly workshops, information events and confer- ences. Almost half of the respondents had planned online ○ β is the parameter vector to be estimated; and interaction formats, such as webinars. ○ Λ is the logistic distribution (Spyridaki et al., 2020). The respondents have different motives for engaging with The logistic cumulative distribution function is defined as: stakeholders. As shown in Figure 3, the linear research mode is dominant, in which stakeholders are either viewed as the β⋅x target audience for results (“dissemination”), or as research P(y = 1|x ) = Λ(β⋅ x ) = i i i β⋅x i subjects (“access stakeholders”). However, more transdisci- 1+e plinary and co-creative motives are also high on the agenda, including research question identification and implementation where P is the probability of y occurring. The maximum of findings/technologies. likelihood (ML) estimation method is used to estimate the parameter vector β. The impacts of the coronavirus containment measures on stakeholder engagement activities and outcomes Sample description The first wave of the coronavirus and its containment meas - We received 84 complete responses from 72 different energy ures affected stakeholder activities in energy research projects projects: 62 different EU Horizon 2020-funded projects mainly negatively: almost nine of ten respondents perceive (31% of the stakeholder-engaging EU-funded energy somewhat or very negative effects (Figure 4). Furthermore, research projects running at the time), and 10 projects with projects planning face-to-face workshops in 2020 are more other funding sources. For most projects, we received only negatively affected by the crisis, which is not surprising, as one response; for eight projects we received two responses , such events were de facto banned in most countries by social and five responses from one project. Most projects started distancing measures. The same applies to projects where in 2018/19 and will end in 2021/22; practically all are three- stakeholder involvement has a higher priority: our regres- year projects. The important demographic sample data of sion analysis shows a positive correlation between the projects surveyed respondents is summarised in Figure 1. that stated a very negative influence and responses that stated that stakeholder engagement is crucial for the successful Ethics requirements implementation of the project (Table 1). In addition, projects The research has been conducted under the ethics require- that focused on engaging with policymakers were more ments and guidelines of the SENTINEL project (Deliverables affected: we find a negative correlation between no influ - 11.1 and 11.2), which follows the guidelines of the European ence (‘not at all’) and projects engaging with policymakers, Commission. We have applied an ethically-robust methodology but not for those engaging with other stakeholder groups, a negative correlation between negative effects and projects engaging with energy industry representatives, and positive We received two responses from the SENTINEL project, which were not from the authors of this paper. correlation between no influence and projects engaging Page 5 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Figure  1.  Demographics of surveyed respondents: a) What is your gender?, b) What is your main field of research?, c) How long have you worked in this field?, d) How long have you been engaging with stakeholders in European Union projects?, n = 84. Figure  2.  Importance of stakeholder engagement and engaged stakeholder groups: a) How important is stakeholder engagement for the success of your project?, b) What stakeholder groups are being engaged in your project?, n = 84. Page 6 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Figure  3.  Motives  for  stakeholder  engagement:  What is the purpose of your stakeholder engagement? (multiple responses possible), n = 84. (Figure 4). For these projects, as expected, we find that if the inter - action with stakeholders is ‘not very important’ for the project, the effect of the containment measures is smaller (cf. Table 2). Additionally, a small percentage assessed the influence of the pandemic to be positive (Figure 4), which is possibly related to the better response of stakeholders to online formats (Figure 5), enabling more frequent exchange with stakeholders or access to stakeholders living further away. In contrast, for many respondents it became harder to reach stake- holders (Figure 5). Respondents underlined that stakeholders’ life was negatively affected by the crisis: stakeholders “experi- enced an increase in stress and workload”, had “difficult[ies] to perform the work foreseen due to the closure or reduction of activities”, experienced dropped incomes, were “unable to work”, or may even have become unemployed. It is not surprising that these impacts have led to a shifting of stakehold- ers’ priorities away from the projects (Figure 5). This result is also supported by our regression analysis, which indicated a positive correlation between negative effects and responses that stated that priority of engagement decreased very much since the beginning of the pandemic, and a positive correlation between no influence and responses that stated that priority Figure  4.  Impact  of  the  coronavirus  disease  2019   of engagement was not affected at all. In contrast, the priorities (COVID-19)  pandemic  on  stakeholder  activities:  Does the COVID-19 pandemic influence your stakeholder activity/ for stakeholder engagement of most researchers did not engagement negatively or positively?, n = 84. change, but for some they decreased or increased. Respondents that were personally more affected are especially more likely to report that their stakeholder engagement priority has changed ‘very much’ (cf. Table 1). This indicates that people’s private life with civil society organisations. This could be due to the situation has an impact on their work life: with offices moving fact that, typically, engagement activities that involve poli- into homes, work and private life became linked more closely. cymakers take place as physical workshops and meetings, while activities that involve energy industry and civil society representatives took place in various formats, including More than half of the researchers expect a negative influence online channels also before the COVID-19 pandemic. Table 2 on the outcomes of the overall stakeholder engagement proc- shows the detailed results of the regression analysis. ess (Figure 6), not only in terms of engagement frequency and similar quantitative aspects, but also in the quality of inter- Only 10% of the respondents reported that the COVID-19 actions and stakeholder-based input for the projects. One measures had not effect on the stakeholder engagement responded explained: Page 7 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Table 1.  Logistic regression model I.  Dependent variable: “Impact of COVID-19 on stakeholder engagement of energy research projects – Very negatively”. Explanatory variables Prediction  model Category Name Coefficient Stakeholder groups engaged Policymakers -3.224 (2.333) Energy industry -3.726 (2.076)* Geographical/spatial focus EU -11.974 (5.945)** COVID-19 cases 7.116 (3.194)** Frequency of engagement Weekly -5.075 (4.310) Importance of stakeholder engagement for the success of the project Crucial 9.043 (4.083)** Year of stakeholder engagement according to project plan/proposal 2021 -9.139 (4.033)** 2022 5.058 (2.893)* Engagement activities according to the project plan/proposal (year  Face-to-face workshops 6.999 (3.579)** 2020) Information events for stakeholders -6.269 (3.009)** Face-to-face interviews 1.920 (1.745) Purpose of stakeholder engagement Disseminate research results 0.917 (1.895) Priority of stakeholder engagement changed due to COVID-19 Priority decreased 6.768 (3.736)* Change in relationship to stakeholders Stakeholders priority has shifted -3.719 (2.497) away from the project It is harder to reach stakeholders 4.886 (3.708) Impact of changes in your stakeholder engagement activities on  Delays in the flow of data to other 5.366 (2.683)** proceedings and results of your overall project work packages The project duration will need to be 6.037 (3.045)** extended Deliverables’ submission has been/ 3.548 (2.288)* will be delayed Constant -2.336 (6.257) Notes: - Standard errors are reported in parentheses. - Superscripts***, **and *indicate statistical significance of 1%, 5% and 10% level, respectively. Table 2.  Logistic regression model II.  Dependent variable: “Impact of COVID-19 on stakeholder engagement of energy research projects – Not at all”. Explanatory variables Prediction  model Category Name Coefficient Stakeholder groups engaged Policymakers -11.824 (6.787)* Civil Society Organizations -9.133 (4.800)* Geographical/spatial focus Non-EU -8.448 (5.098)* Importance of stakeholder engagement for the success of the  Not very important 25.546 (13.188)* project Page 8 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Explanatory variables Prediction  model Category Name Coefficient Year of stakeholder engagement according to project plan/ 2021 3.084 (2.814) proposal Engagement activities according to the project plan/proposal (year  Online survey 4.834 (4.365) 2020) Purpose of stakeholder engagement Access to data and information to -3.660 (2.223)* understand a research problem Disseminate research results 10.459 (6.826) Priority of stakeholder engagement changed due to COVID-19  Priority unchanged 4.116 (2.231)* Change in relationship to stakeholders No change -8.377 (5.058)* Impact of changes in your stakeholder engagement activities on  Overall workflow is not impacted 4.882 (2.910)* proceedings and results of your overall project No negative impact 4.328 (3.172) The project will be carried out as 6.973 (3.686)* planned, with the envisioned results Constant -17.752 (9.453)* Notes: - Standard errors are reported in parentheses. - Superscripts***, **and *indicate statistical significance of 1%, 5% and 10% level, respectively. Figure  5.  Perceived  changes  of  relationship  with  stakeholders  (multiple  responses  possible):  Please tick the boxes if you agree with the following statements, n = 84. I think the COVID-19 restrictions on in-person community This quote underlines that the more co-creative processes engagement will limit the value of stakeholder feedback. We may also suffer from reduced possibilities for co-designing have just completed the Comprehensive Plan update, and the research questions, co-owning the results and co-agreeing on four community engagement workshops created a bonding its implications – the very aim of transdisciplinary research. among the community that had great value. The participants felt ownership to the results and support the implementation The impact of the COVID-19 crisis on project workflows actions. This attribute will be even more critical in the Cli- and outcomes mate Action Plan, as some of the actions are a bit more contro- The coronavirus restrictions have negative impacts on the versial. Lacking the interactive discussions and bonding over workflow of most projects, mainly leading to delays in the shared outcome is a weakness of the COVID-imposed process. flow of data between work packages, as shown in Figure 7a. Page 9 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 We find a strong correlation between negative effects and the responses “Delays in the flow of data to other work packages”, “Deliverables’ submission has been/will be delayed” (Table 1). This implies that delays have led to a stronger feeling by researchers’ that their stakeholder activities are negatively affected by the pan- demic. This might be related to the fact that delayed input stem- ming from the stakeholder engagement process can be potentially considered in the project differently as initially planned. Most respondents expected that changes in stakeholder engage- ment activities will affect the proceedings and results of the overall project (Figure 7b). Although all respondents believe that the overall project objectives can be still completed, the majority of the projects will not be carried out as planned, and, thus, results will be different than expected. Furthermore, more than one third of the respondents expect that they will need to extend the project duration, which leads to their percep- tion of stakeholder activities being ‘very negatively’ affected by the crisis (cf. Table 1). This is not only related to the challenge of involving stakeholders, but also because researchers had to “adapt […] to this format[s] and approach[es that] require[d] Figure  6.  Impact  of  the  coronavirus  disease  2019  (COVID-19)  a learning curve for [their] teams”. This capacity build- pandemic  on  outcomes  of  the  stakeholder  engagement  ing for dealing with online communication tools has been an process:  Do you think that the COVID-19 situation will important step for most of the respondents as few activities influence the outcomes of your stakeholder engagement process?, n = 84. took place as physical, socially distanced events (Table 3). Figure  7.  Impacts of the coronavirus disease 2019 (COVID-19) crisis on the projects: a) Do you think that the COVID-19 situation will have a negative impact on the workflow within your project?; b) How do you think the changes in your stakeholder engagement activities will affect the proceedings and results of your overall project? (multiple choices possible), n = 84. Page 10 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Table 3.  Overview of engagement activities and impacts of the coronavirus disease 2019 (COVID-19) crisis on the  implementation (numbers), multiple choices possible, n = 84.                               Type of activity                                 Alternative activities: Implemented  Socially  Format  Planned Delayed Cancelled Other as planned distanced changed Originally planned: Information events for stakeholders 46 0 7 24 11 27 2 Face-to-face workshops 64 0 9 29 10 45 7 Conferences 39 1 2 20 10 28 4 Focus groups 24 1 5 17 5 19 1 Face-to-face interviews 28 1 1 14 6 16 2 Online interviews 14 8 - 8 0 3 1 Face-to-face survey 9 0 1 1 6 4 0 Online survey 24 18 - 7 1 0 0 Webinars 29 21 - 5 1 3 3 % of strategy applied: 18% 9% 45% 18% 52% 7% conferences, because as one wrote “online concentration Coping strategies of researchers to deal with span and endurance of people is limited”. However, shorter containment measures events may lead “sometimes to very superficial results Researchers adapted their involvement activities to the because [there is] no time to deepen certain aspects”, as one restrictions: only one of six stakeholder engagement activi- researcher reported. In addition, respondents suggested the ties were implemented as planned – almost all of which were splitting up of participants in “smaller targeted online events already planned to be online – whereas two thirds were either (workshop, focus group, interviews), where not too many cancelled or delayed. The most common coping strategies were people are present”, as well as “break-out groups coupled with the postponement of concrete stakeholder involvement activi- interactive polling tools, appeared to increase stakeholder ties, presumably hoping for looser restrictions in the future, retention and participation over the course of a small 1-day and changes in formats – and often a combination of the workshop.” two measures (Table 3). For some projects it became easier to engage stakehold- Among the alternative engagement formats (if formats were ers online, especially “ policymakers appear to be easier to changed), online workshops and webinars were the most com- engage in short online meetings than longer physical meet- mon (Table 4). Conferences, face-to-face interviews and ings”. In contrast, other stakeholders left the projects, as focus groups were often directly replaced by the respective consequence to the social distancing measures, as one online format. Information events, as well as face-to-face respondent explained: workshops, were mainly replaced by webinars and online workshops. Interestingly, respondents often performed more Not all stakeholders […] wanted to continue meeting than one alternative engagement activity, suggesting that online. Many stakeholders found the planned in-person meet- the online formats are not seen as perfect complements to ing of their peers in another city as a motivation to join the physical meetings. project in the first place. Assessment of alternative stakeholder engagement In addition, one quarter of the respondents stated that they formats could not reach stakeholders via digital tools, which con- Our results show that many alternative online formats cerned mainly citizens, local authorities, and locally-based – although not the researchers’ first choice − have been use - businesses. One respondent expressed their concern: “Many ful for projects: in particular, webinars, online interviews and of our target stakeholders are elderly, and many have limited online surveys are widely seen as suitable online engagement computer access.” formats (Figure 8). In contrast, experiences with online focus groups, online conferences and workshops were rather mixed. Interactive workshops and networking formats seem to be Nevertheless, the majority (55 out of 66 respondents, free text challenging, and respondents recommended rather short online reply) plans to continue online engagement activities after Page 11 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Table 4.  Overview of alternative online engagement activities performed, if ‘format changed’ (numbers), multiple choices  possible, n = 84. Type of activity Webinar Online  Online  Online focus  Online  Online  Other workshop conference groups interviews survey Information  15 14 6 5 2 3 1 events Online content, e.g., videos Face-to-face workshops 17 32 5 7 6 3 5 Mailed survey, online group, not decided Conference 10 7 18 1 1 2 4 Blogs, not decided Focus groups 2 6 1 11 6 4 2 Not decided Face-to-face interviews 1 1 0 1 14 1 2 Online meetings, telephone survey Face-to-face survey 1 1 0 0 2 2 0 Online interviews 3 1 0 1 0 0 0 Webinar(s) 2 0 0 0 0 0 1 Not decided Figure  8.  Assessment  of  alternative,  online  engagement  methods:  Considering the goals of the stakeholder activity you wanted to perform originally, how suitable were the following formats as a replacement? Page 12 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 the contact restrictions are lifted; some were unsure about it in the future for one-to-one and unidirectional interactions yet, while a few expressed their scepticism – “hopefully not”. with stakeholders, simply because they provide good results. One respondent stated: “The experience with the online work- shop was good, so we will likely consider doing it again. Perspectives for stakeholder engagement after COVID-19 However, this will not really replace face-to-face events. This The COVID-19 crisis may prove to be a window of opportu- is rather a complement to face-to-face workshops/activities”. nity for digitalisation in stakeholder-involving research. We In their free-text responses, many respondents gave similar find that social distancing measures enforced an unforeseen statements, agreeing that online formats cannot replace face-to- shift to online engagement activities in five of six cases. This face meetings, as they are “essential to engage with stakeholders verifies findings by Schwarz et al. (2020) for the academic to allow for clearer communication and networking”. context: researchers are willing to also use digital tools for science-stakeholder interactions. Researchers quickly built up Discussion new capacities to involve stakeholders online and showed a Impacts of the COVID-19 pandemic on stakeholder high openness in trying new engagement formats instead of engagement activities resigning. This is also true for stakeholders: although some The COVID-19 crisis has affected stakeholder engagement stakeholder’s engagement in research projects decreased, most in energy research mainly negatively, but it did not stop it. of them showed openness for new online engagement activi- Adding to Leal Filho et al. (2020), we find that the pandemic ties to varying degrees, but generally at least satisfactorily, the has rearranged the work environment and the private life not online formats worked too. This offers a positive perspective only of energy researchers but also of stakeholders. Thus, it is for stakeholder engagement while containment measures are not surprising that planning of, and participation in, engage- still in place, but also shows that digital formats are likely to ment activities has moved down on professional agendas. be used also after the pandemic, simply because they have Researchers reported a diversity of motives for engaging with proven their usefulness for particular aims. Furthermore, it can stakeholders: while some mainly communicate results, others also contribute to the further improvement and development co-produce and co-design research and practical projects of online engagement formats, including the software and the together. Projects that rely more on the involvement of specific approaches developed by researchers. stakeholders were more affected by the impacts of the pandemic. Nevertheless, adding to previous insights for the academic Researchers coped with the pandemic most commonly by post- context (Schwarz et al., 2020), we find that online engage - poning activities and/or changing formats. Almost half of the ment activities seem unsuitable to replace physical interactions projects decided to shift stakeholder engagement activities with stakeholders completely. This is because a relatively short in time, likely in the hope that in-person communication will duration of online events reduces the depth of interaction and become possible soon. While this may initially be a useful strat- the quality of the results. This suggests that some projects egy, to benefit from a deeper in-person engagement later on, might miss important information via online events, and the delays may accumulate within a project. If tasks and work- outcomes of the online stakeholder engagement must thus be flows are interrupted, projects might fail to take stakeholder critically evaluated. perspectives into account and/ or to finish projects on time. Furthermore, almost all engagement activities that took place In addition, we find that online events may not be able to have been moved online, which appears to be the most replace physical events, because of different levels of stakehold- natural solution with social distancing measures in place. ers’ commitment to contribute to research projects: participat- However, this also raises the question of how suitable the ing in physical events requires more time and other resources various digital formats are for engaging with stakeholders. than switching on the computer. Hence, there is a risk of reduced commitment to online-only engagement processes. This implies that researchers who choose online formats need to rethink Researchers have had good experiences with some digital offline formats when applying them online to ensure that formats, but not with all of them. Webinars and online inter- stakeholder are encouraged to join and are activated in online views, which are based either on one-to-one interaction or meetings. A complete shift from offline to online for all unidirectional communication, are found to work well online purposes does not appear possible or useful. and provide satisfying results. Offering such formats online has clear advantages: it makes them easily accessible to a potentially larger audience of stakeholders, saves travel Last, we agree with Beaunoyer et al. (2020) that not all stake- times and emissions. Moreover, they allow, for example, the holders are used to online technologies, which may lead to recording of such activities, which makes evaluation work technology-related inequalities and digital exclusion of par- easier after the event. Contrary to this, formats requiring ticular stakeholder groups. Hence, researchers must be aware group activities, a higher level of stakeholder involvement of the stakeholder groups that cannot be easily reached and multidirectional, interactive communication (cf. Späth & digitally and adapt their approaches accordingly. For example, Scolobig, 2017) are evaluated much less favourably by policymakers could be generally better reached via dig- respondents. Consequently, researchers must carefully ital tools. However, on the other hand, projects that involved think about the objectives of the engagement and assess the policymakers were more affected by the containment meas- suitability of online formats carefully: not all interaction types ures, probably because policymakers had to shift their priorities are equally suited for online formats. It also suggests that due to coronavirus-related issues. Nevertheless, in the future, researchers could specifically continue using online formats we expect a shift towards a combination of online and Page 13 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 COVID-19 on stakeholder engagement activities in energy offline activities at different times as well as hybrid formats research projects. Scientists could use our publicly available that combine both forms. dataset to perform more robust econometric analyses (incl. best-fitting model information criteria, evaluation of param - Implications on the future funding of research projects eter estimates using quasi standard errors, etc.), also account- Whereas stakeholder involvement is a crucial component of ing for limitations due to the small sample size and number many research projects funded today, the COVID-19 crisis of responses in some variables that could cause issues of mul- revealed that this requirement has made these projects vulner- ticollinearity between the independent variables and affect able to changes in stakeholder availability and accessibility. Thus, the predictive power of the statistical models. it raises questions concerning the resilience of transdiscipli- nary research: access to stakeholders can only partially be influ - Finally, only energy research projects were within the scope enced by the researchers themselves, and to some extent, they of this study, with a focus on Horizon 2020 projects. Thus, our are simply exposed to the risk of failing engagement activi- results are only generalisable beyond energy, where compa- ties due to external factors. Consequently, the COVID-19 rable results have been derived from similar research in other crisis can also be seen as a resilience test for the participa- disciplines. It would be relevant to see similar studies also in tory aspirations of the research funding bodies. While the other specific fields, to potentially monitor and compare how European Commission’s response has been generally sympathetic different research areas respond and adapt their approaches, in that context, not all researchers were met with open ears to enable cross-disciplinary learning and an exchange of when requesting for a project extension due to coronavirus experiences. restrictions. This issue may need to be addressed by funders, especially because coronavirus restrictions seem to continue for a longer time. Researchers, stakeholders, and funding Conclusions institutions need to recognise the current situation and must The coronavirus social distancing and lockdown measures stay flexible with their approaches. have had a mainly negative effect on stakeholder engage- ment in energy research projects, especially by interrupting the exchange between researchers and stakeholders, caus- Limitations and prospects for future research ing delays in the project workflow, and changing outcomes of Measures against the COVID-19 pandemic are constantly stakeholder engagement processes. Given this difficult changing, and so are the framework conditions for stake- situation, researchers have generally been able to quickly adapt holder interactions. Situations that respondents experienced by finding new ways of engaging with stakeholders, switch - in spring 2020 during the first wave of the pandemic may not ing especially to online workshops and webinars. This shows necessarily be the same in spring 2021 or 2022. Moreover, as that digital engagement has become a new norm during coronavirus restrictions also vary between countries, there COVID-19. Researchers have had positive experiences with are different possibilities for stakeholder interactions in several online engagement formats, in particular webinars, different places. In this research, we could account neither for online surveys and interviews – but are much less content with specific phases of the pandemic, nor for situations in specific workshops and conferences. This suggests that online formats country-based contexts; as a result, future research could work particularly well for one-to-one and unidirectional investigate these changes and differences over space and time. formats, whereas interactive and group activities are less suited for digital formats. While various digital participa- Since many activities were ongoing at the time of the survey, tion formats offer new opportunities for involving stakeholders we could only identify initial lessons on the coping strate- more frequently by reaching a larger audience and being less gies and the success of implemented measures. Although we time-consuming, an important disadvantage is that some social received insights on the expected impacts of coping strategies groups are excluded from the process and derived results on the project outcomes, since the stakeholder engagement might be more superficial. Thus, online engagement will likely processes are ongoing, the actual impacts will likely materi- continue after the crisis, but only to complement and not to alise towards the end of the projects. To this end, an updated replace physical meetings in energy research. The long-term version of the same survey in 2021 or 2022 would be inter- effects on energy research remain to be seen. However, given esting and relevant. Furthermore, in this context, it would be the large amount of hope put into postponing events, there valuable to learn from transdisciplinary research and projects is a clear risk that projects will not be able to finish on time where stakeholder engagement plays a vital role in co-creation or with the intended contents, depending on the duration of processes how the shift to the online formats has influenced the coronavirus-related restrictions. the outcomes. Most probably, more interactions in the digital world have not only serious implications on the general Although the COVID-19 crisis is a monumental challenge for social reality, but specifically for approaches to complex, all, including for us ourselves as researchers, positive changes real-world problems related to sustainability. can be triggered. When forced to adapt, researchers and stake- holders quickly started experimenting with new formats – and In addition, the regression analysis performed was meant developed several solutions that were found useful and to supplement the explanatory analysis of the descriptive attractive. Quite possibly, after the pandemic ends, we will statistics and provide preliminary indications on the potential find that more of our work has moved into the online causalities that could explain the negative impacts of Page 14 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 space – and we will know why personal contacts are (Version 1). http://doi.org/10.5281/zenodo.4765630 (Süsser et al., irreplaceable, after all. 2021) This project contains the following extended data: Data availability Underlying data • questionnaire_impact covid-19 on stakeholder The survey data underlying this study are not openly avail- engagment_SENTINEL_2020.pdf able to protect the anonymity of participating individu- Data are available under the terms of the Creative Commons als and projects. Given that we asked for many details of the Attribution 4.0 International license (CC-BY 4.0). projects, like funding source, duration etc., we cannot guar- antee anonymity. For further COVID-19 research and the replicability of the research in another study context, access to semi-anonymised data can be granted. For Acknowledgments further information on the data please contact Diana Süsser We thank Vera Köpsel (University Hamburg, project PANDORA (diana.suesser@iass-potsdam.de). (grant agreement No. 773713)) for designing together with us the survey underlying the results presented in this paper. We thank Extended data Jeremias Herberg and Stephen Gary Williams (IASS) for their ZENODO: Questionnaire related to on the impact of COVID-19 feedback on the survey design. We also thank all our participants on stakeholder engagement in European energy research of the survey for taking the time for participating in our research. References Beaunoyer E, Dupéré S, Guitton MJ: COVID-19 and digital inequalities:  Leal Filho W, Azul AM, Wall T, et al.: COVID-19: the impact of a global crisis on  Reciprocal impacts and mitigation strategies. Comput Human Behav. 2020; sustainable development research. Sustain Sci. 2020; 16: 1–15. 111: 106424. PubMed Abstract   | Publisher Full Text   | Free Full Text   PubMed Abstract   | Publisher Full Text   | Free Full Text   LimeSurvey: An Open Source survey tool /LimeSurvey GmbH . 2020. Bracken LJ, Bulkeley HA, Whitman G: Transdisciplinary research:  Lutz LM, Bergmann M: Transdisziplinarität: Forschungsansatz für die  understanding the stakeholder perspective. J Environ Plan Manag. 2015; Energiewende. In: Holstenkamp, L., Radtke, J. (Eds.), Handbuch Energiewende 58(7): 1291–1308. Und Partizipation. Springer VS, Wiesbaden, 2018; 43–56. Publisher Full Text   Publisher Full Text   Bruhn T, Herberg J, Molinengo G, et al.: Grounded action design  Mielke J, Vermaßen H, Ellenbeck S: Ideals, practices, and future prospects of  – Transdisciplinary co-creation for better transformative processes.  stakeholder involvement in sustainability science. Proc Natl Acad Sci USA. Frameworks for transdisciplinary research #9. Gaia - Ecological Perspectives 2017; 114(50): E10648–E10657. for Science and Society. 2019; 28(4): 336. PubMed Abstract   | Publisher Full Text   | Free Full Text   Publisher Full Text   Mielke J, Vermaßen H, Ellenbeck S, et al.: Stakeholder involvement in  Bryman A: Social Research Methods . Fourth edi. ed. Oxford: Oxford Univ. Pr. sustainability science—A critical view. Energy Res Soc Sci. 2016; 17: 71–81. Publisher Full Text   Reference Sourc e Renn O: The function(s) of transdisciplinary research in conflicting Corbera E, Anguelovski I, Honey-Rosés J, et al.: Academia in the Time of  transformations. Gaia. 2019; 28(1): 44–51. COVID-19: Towards an Ethics of Care. Plan Theory Pract. 2020; 21(2): 191–199. Publisher Full Text   Publisher Full Text   Schneider F, Buser T: Promising degrees of stakeholder interaction in  Creswell JW, Creswell JD: Research Design: Qualitative, Quantitative, and  research for sustainable development. Sustain Sci. 2018; 13(1): 129–142. Mixed Methods . fifth edit. ed. SAGE Publications, Los Angeles. 2018. PubMed Abstract   | Publisher Full Text   | Free Full Text   Reference Sourc e Scholz RW, Steiner G: The real type and ideal type of transdisciplinary  European Commission: Public Engagement in Responsible Research and  processes: part I—theoretical foundations. Sustain Sci. 2015; 10: 527–544. Innovation. [WWW Document]. 2020a. Publisher Full Text   Reference Sourc e Schwarz M, Scherrer A, Hohmann C, et al.: COVID-19 and the academy: It is  European Commission: Can the clause on “force majeure” in the Horizon  time for going digital. Energy Res Soc Sci. 2020; 68: 101684. 2020 grant agreement be invoked in relation to the COVID-19 outbreak? PubMed Abstract   | Publisher Full Text   | Free Full Text   [WWW Document]. 2020b. Reference Sourc e Späth L, Scolobig A: Stakeholder empowerment through participatory  Fazey I, Schäpke N, Caniglia G, et al.: Ten essentials for action-oriented and  planning practices: The case of electricity transmission lines in France and  Norway. Energy Res Soc Sci. 2017; 23: 189–198. second order energy transitions, transformations and climate change  Publisher Full Text   research. Energy Res Soc Sci. 2018; 40: 54–70. Publisher Full Text   Spyridaki NA, Stavrakas V, Dendramis Y, et al.: Understanding technology  Funtowicz S, Ravetz J: Values and Uncertainties . In: Handbook of ownership to reveal adoption trends for energy efficiency measures in the Greek residential sector. Energy Policy. 2020; 140: 111413. Transdisciplinary Research. Springer Netherlands, Dordrecht, 2006; 361–368. Publisher Full Text   Publisher Full Text   Klenk NL, Meehan K, Pinel SL, et al.: GLOBAL CHANGE SCIENCE. Stakeholders    Stagl S: Opportunities of post COVID-19 European recovery funds in  in climate science: Beyond lip service? Science. 2015; 350(6262): 743–4. transitioning towards a circular and climate neutral economy.  2020. PubMed Abstract   | Publisher Full Text   Reference Sourc e Köpsel V, de Moura Kiipper G, Peck MA: Stakeholder engagement vs. social  Staua ff cher M, Flüeler T, Krütli P, et al.: Analytic  and  dynamic  approach  to  distancing—how does the Covid-19 pandemic affect participatory research collaboration:  A  transdisciplinary  case  study  on  sustainable  landscape  in EU marine science projects? Marit Stud. 2021; 20: 189–205. development  in  a  swiss  prealpine  region. Syst Pract Action Res. 2008; 21: 409–422. Publisher Full Text   Publisher Full Text   Lang DJ, Wiek A, Bergmann M, et al.: Transdisciplinary research in  Süsser D, Ceglarz A, Stavrakas V, et al.: Questionnaire related to Süsser et al.  sustainability science: Practice, principles, and challenges. Sustain Sci. 2012; (2021) on the impact of COVID-19 on stakeholder engagement in European  7: 25–43. energy research.  (Version 1). Zenodo. 2021. Publisher Full Text   http://www.doi.org/10.5281/zenodo.4765630 Page 15 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Open Peer Review Current Peer Review Status: Version 2 Reviewer Report 07 October 2021 https://doi.org/10.21956/openreseurope.15198.r27706 © 2021 Fell M. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Michael J. Fell UCL Energy Institute, University College London, London, UK I can confirm I’m happy with the author responses to the review.” Competing Interests: No competing interests were disclosed. I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Reviewer Report 30 September 2021 https://doi.org/10.21956/openreseurope.15198.r27707 © 2021 Leal Filho W et al. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Walter Leal Filho European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany Amanda Lange Salvia European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany The authors have addressed our comments. Competing Interests: No competing interests were disclosed. Page 16 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Version 1 Reviewer Report 25 June 2021 https://doi.org/10.21956/openreseurope.14757.r26979 © 2021 Fell M. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Michael J. Fell UCL Energy Institute, University College London, London, UK This paper uses a survey of EU energy research projects to describe the effect that Covid-19 response measures have had on stakeholder engagement. We find the approach to be quite clearly described and appropriate to the research questions. The findings are clearly described, although some clarification is needed around the regression analysis. More consideration of the implications of the findings would be valuable. We recommend the piece should be indexed, subject to a number of revisions. It would be useful if the scope of the work could be made a bit more precise. In the introduction it states “we investigate the effects of the COVID-19 crisis on the stakeholder work in European energy research”. However, participant recruitment seems to have been mainly aimed at Horizon 2020 projects, and the response rate reported is a % of EU-funded energy research project. The final sample was mainly Horizon projects (albeit a fairly small proportion of those eligible), plus some others. More clarity here, and consideration of implications of this for generalisability of the conclusions, is necessary. We are not fully clear on how exactly the regression analysis was conducted. Firstly, we were unable to identify exactly which survey question was used to determine the dependent variable. Please could this be clarified? The nearest one we could find was “How does the COVID-19 pandemic influence the stakeholder/engagement in your project? Likert scale: very positively <-> very negatively”. If the response options for the dependent variable were in the form of a response scale, how were responses which were not “Not at all” or “Very negatively” treated? It would also be useful if the authors could expand on why they decided to use logistic regression, and why it had to be run twice. It wasn’t quite clear how discrete choice modelling, and mentioned at the end of the methods section, was employed. Ideally more would also be said on how particular variables were selected for inclusion (i.e. one over the other), during the process of “examining in parallel the correlation matrix to test for collinearity issues among the covariates, until no significant collinearities are observed”. The regression results show a large number of variables, but the article only engages superficially Page 17 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 with these findings. It would be helpful if the regression results could be moved to the results section, and some of the key effects pulled out and described in the text. It would then be useful to see some discussion of the regression results – both in terms of limitations due to sample size, but also on the more interesting findings. For example (while significance level is low), there was a large negative effect size for policymakers and civil society organisations in the “Not at all” analysis – could be implications of this be considered? Generally, the discussion is quite light on what the implications of the findings could be – both for the projects which data was captured on and going forward. Are/will important viewpoints more likely to be missed, e.g. those less able to access digital avenues for engagement? Some more minor points: ○ It could be useful to provide an indicative timeline of some of the main Covid-related restrictions in Europe. This will obviously vary by country, but some general info could be helpful in interpreting this work in future ○ “Researchers agreed that online formats cannot replace face-to-face meetings” – is this based on a quantitative result from the survey and, if so, could this be included? Or is it just inferred from the quote? Is the work clearly and accurately presented and does it engage with the current literature? Yes Is the study design appropriate and is the work technically sound? Partly Are sufficient details of methods and analysis provided to allow replication by others? Partly Are all the source data and materials underlying the results available? Partly If applicable, is the statistical analysis and its interpretation appropriate? Partly Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Quantitative and qualitative social science research in energy. I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Author Response 30 Aug 2021 Page 18 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Diana Süsser, Institute for Advanced Sustainability Studies (IASS), Potsdam, Germany Response to Reviewer 2 R2.0 This paper uses a survey of EU energy research projects to describe the effect that Covid-19 response measures have had on stakeholder engagement. We find the approach to be quite clearly described and appropriate to the research questions. The findings are clearly described, although some clarification is needed around the regression analysis. More consideration of the implications of the findings would be valuable. We recommend the piece should be indexed, subject to a number of revisions. We are very glad to read that the reviewer appreciates the descriptions of our approach, research question, and findings. We have expanded the discussion to add further aspects about the implications of our findings, and we have added details about the regression analysis (also in response to comment 1.1 above). R2.1 It would be useful if the scope of the work could be made a bit more precise. In the introduction it states “we investigate the effects of the COVID-19 crisis on the stakeholder work in European energy research”. However, participant recruitment seems to have been mainly aimed at Horizon 2020 projects, and the response rate reported is a % of EU-funded energy research project. The final sample was mainly Horizon projects (albeit a fairly small proportion of those eligible), plus some others. More clarity here, and consideration of implications of this for generalisability of the conclusions, is necessary. Thank you for pointing this out. In the scope of the study were both Horizon 2020 and non- Horizon 2020 projects, but we only sent direct emails to the former. We allowed other projects to participate in our survey and we asked the projects if they were Horizon projects or not. We revised the text to make this clearer in the introduction and method section. We believe that many of our findings are generalisable beyond energy research – and specifically in cases where similar results have been found in other research and comment on this in the revised Discussion. R2.2 We are not fully clear on how exactly the regression analysis was conducted. Firstly, we were unable to identify exactly which survey question was used to determine the dependent variable. Please could this be clarified? The nearest one we could find was “How does the COVID-19 pandemic influence the stakeholder/engagement in your project? Likert scale: very positively <-> very negatively”. If the response options for the dependent variable were in the form of a response scale, how were responses which were not “Not at all” or “Very negatively” treated? It would also be useful if the authors could expand on why they decided to use logistic regression, and why it had to be run twice. It wasn’t quite clear how discrete choice modelling, and mentioned at the end of the methods section, was employed. Ideally more would also be said on how particular variables were selected for inclusion (i.e. one over the other), during the process of “examining in parallel the correlation matrix to test for collinearity issues among the covariates, until no significant collinearities are observed”. Thank you very much for these insightful comments/ suggestions. We made sure to elaborate more in the text on how the regression analysis was performed, so that it is more comprehensible to the reader. The main changes in response to this comment are found in Page 19 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 the Methods section. R2.3 The regression results show a large number of variables, but the article only engages superficially with these findings. It would be helpful if the regression results could be moved to the results section, and some of the key effects pulled out and described in the text. It would then be useful to see some discussion of the regression results – both in terms of limitations due to sample size, but also on the more interesting findings. For example (while significance level is low), there was a large negative effect size for policymakers and civil society organisations in the “Not at all” analysis – could be implications of this be considered? We thank the reviewer for pointing out this very important issue. We have added more results to the results section. The Tables have been moved to the Results section. We added limitations of our approach to the limitations section of the Discussion and suggested further implications for research. 2.4 Generally, the discussion is quite light on what the implications of the findings could be – both for the projects which data was captured on and going forward. Are/will important viewpoints more likely to be missed, e.g. those less able to access digital avenues for engagement? We agree with the reviewers and hence, we expanded our Discussion and now emphasise the implications of our research better. R2.5 Some more minor points: It could be useful to provide an indicative timeline of some of the main Covid-related restrictions in Europe. This will obviously vary by country, but some general info could be helpful in interpreting this work in future. We think that because of the difference timeline in the countries and given out study context, a detailed timeline is not necessary – or possible to generate with a reasonable time effort. We agree with the spirit of the comment, however, and added additional information about the containment measures in place in general during the time our survey was conducted (see Method section). R2.6 “Researchers agreed that online formats cannot replace face-to-face meetings” – is this based on a quantitative result from the survey and, if so, could this be included? Or is it just inferred from the quote? Thank you for this question. We have adapted the text in the Results section (‘Assessment of alternative stakeholder engagement formats’) to make this clearer. We did not specifically ask this question, but we got this statement as answers to open questions we asked. We asked the participants the open questions: “Do you plan to continue using any of the new formats after restrictions are lifted?  Please briefly outline which ones and why” (100% response) and “Please explain briefly in what sense COVID-19 will influence the results of your stakeholder engagement process” (79% response). Based on the answers given, we find that the majority plans to continue using online formats, but mostly in combination with physical formats, for example, “[We will continue using] online workshop, but they cannot fully replace physical meetings”; “The experience with the online workshop was good, so we will likely consider doing it again. However, this will not really replace face-to-face events. This is rather a complement to face-to-face workshops/activities”; “An online workshop CANNOT replace a face to face meeting, as not all senses are used.” Page 20 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Competing Interests: No competing interests were disclosed. Reviewer Report 07 June 2021 https://doi.org/10.21956/openreseurope.14757.r26978 © 2021 Leal Filho W et al. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Walter Leal Filho European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany Amanda Lange Salvia European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany The paper aims at investigating the effects of the pandemic on energy research, particularly in European projects with planned stakeholder engagement. The rationale for the study is well described and so are the discussions and implications. However, some improvements are needed as follows: ○ The results/discussions could have been explored in a dedicated subsection or in more detail the contrasting results of the regression analysis, especially given the detailed methodology and the volume of information provided in the first tables. ○ I recommend adjusting the title or adding a note to Table 4, to make it clearer for the reader e.g. explaining that the lines refer to the original/planned activities and that the columns refer to the alternative online ones. ○ There are some light grammar/syntax errors on the text and it needs to be carefully edited.  ○ Please provide/adjust the conclusions section in a way that you may summarise the main lessons from the paper, and outline future prospects. ○ Each reference should be checked, to make sure they are all complete, with publisher, place of publication. Please check them all. Also, please cross check them as to make sure they are all on the text and in the references list.  Is the work clearly and accurately presented and does it engage with the current literature? Yes Is the study design appropriate and is the work technically sound? Page 21 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Yes Are sufficient details of methods and analysis provided to allow replication by others? Yes Are all the source data and materials underlying the results available? Yes If applicable, is the statistical analysis and its interpretation appropriate? Yes Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Sustainability and climate change We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above. Author Response 30 Aug 2021 Diana Süsser, Institute for Advanced Sustainability Studies (IASS), Potsdam, Germany Response to Reviewer 1 R1.0 The paper aims at investigating the effects of the pandemic on energy research, particularly in European projects with planned stakeholder engagement. The rationale for the study is well described and so are the discussions and implications. We are glad to read that the reviewer values our study description, discussions, and implications. However, some improvements are needed as follows: R1.1 The results/discussions could have been explored in a dedicated subsection or in more detail the contrasting results of the regression analysis, especially given the detailed methodology and the volume of information provided in the first tables. Thank you very much for this suggestion. We have enriched the results section with some more regression analysis-relevant explanations. R1.2 I recommend adjusting the title or adding a note to Table 4, to make it clearer for the reader e.g. explaining that the lines refer to the original/planned activities and that the columns refer to the alternative online ones. We have adapted the line/column headings in Table 4 to make this clear. R1.3 There are some light grammar/syntax errors on the text and it needs to be carefully edited.  We have reviewed and proof-read the text again and corrected errors. Page 22 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 R1.4 Please provide/adjust the conclusions section in a way that you may summarise the main lessons from the paper, and outline future prospects. We have adapted the conclusion section. We now do not only summarise the main findings, but also provide some key implications. Furthermore, we outline prospects for future research to assess the more long-term impacts of the pandemic on stakeholder engagement and to identify best practices for stakeholder engagement beyond COVID-19. R1.5 Each reference should be checked, to make sure they are all complete, with publisher, place of publication. Please check them all. Also, please cross check them as to make sure they are all on the text and in the references list.  Thank you very much for this important note. We have cross-checked all the references and adapted the list accordingly. Competing Interests: No competing interests were disclosed. Page 23 of 23 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Open Research Europe Unpaywall

COVID-19 vs. stakeholder engagement: the impact of coronavirus containment measures on stakeholder involvement in European energy research projects

Open Research EuropeOct 14, 2021

Loading next page...
 
/lp/unpaywall/covid-19-vs-stakeholder-engagement-the-impact-of-coronavirus-qYimjZH0iI

References (54)

Publisher
Unpaywall
ISSN
2732-5121
DOI
10.12688/openreseurope.13683.3
Publisher site
See Article on Publisher Site

Abstract

version 3 The coronavirus (COVID-19) pandemic has affected societies and economies around the world, and the scientific community is no (revision) exception. Whereas the importance of stakeholder engagement in 14 Oct 2021 research has grown quickly the consequences of the pandemic on this version 2 has so far not been empirically studied. In this paper, we investigate the effects of the COVID-19 crisis on European energy research, in (revision) report report particular the stakeholder work, during the first wave of the 27 Sep 2021 coronavirus in spring and summer 2020. We pose the research questions: (i) How much of a problem are the coronavirus version 1 containment measures for stakeholder engagement? (ii) How have report report 25 May 2021 researchers coped with the situation, and (iii) How do they evaluate alternative stakeholder activities implemented? We conducted an 1. Walter Leal Filho, Hamburg University of online survey among European energy research projects with stakeholder engagement between June and August 2020. We found Applied Sciences, Hamburg, Germany that only one of six engagement activities could be implemented as Amanda Lange Salvia, Hamburg University planned, whereas almost half were cancelled or delayed. The most of Applied Sciences, Hamburg, Germany common coping strategies were changing involvement formats – mainly to webinars or online workshops – or postponement. Whereas 2. Michael J. Fell, University College London, respondents are largely satisfied with one-to-one and unidirectional London, UK online formats, such as webinars, online interviews, and online surveys, they see interactive group activities as less suitable for online Any reports and responses or comments on the engagement. Most respondents plan to continue using online formats to complement, but not to replace, physical meetings in future article can be found at the end of the article. research. All long-term effects remain to be seen, but given the Page 1 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 postponement of many stakeholder involvement activities, many projects may face problems at later stages of their realisation. These findings suggest that the pandemic may have catalysed a rapid introduction of specific online formats in academic stakeholder interaction processes. Keywords COVID-19, coronavirus, stakeholder engagement, transdisciplinarity, energy research, Horizon 2020, EU This article is included in the Excellent Science gateway. This article is included in the Research Culture collection. Corresponding authors: Diana Süsser (diana.suesser@iass-potsdam.de), Andrzej Ceglarz (andrzej@renewables-grid.eu) Author roles: Süsser D: Conceptualization, Formal Analysis, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing; Ceglarz A: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing; Stavrakas V: Conceptualization, Formal Analysis, Methodology, Writing – Review & Editing; Lilliestam J: Conceptualization, Writing – Review & Editing Competing interests: No competing interests were disclosed. Grant information: This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No [837089]), (project SENTINEL) and (grant agreement No [837089]), (project TRIPOD). Copyright: © 2021 Süsser D et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: Süsser D, Ceglarz A, Stavrakas V and Lilliestam J. COVID-19 vs. stakeholder engagement: the impact of coronavirus containment measures on stakeholder involvement in European energy research projects [version 3; peer review: 2 approved] Open Research Europe 2021, 1:57 https://doi.org/10.12688/openreseurope.13683.3 First published: 25 May 2021, 1:57 https://doi.org/10.12688/openreseurope.13683.1 Page 2 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Background: stakeholder engagement in research  REVISED           Amendments from Version 2 projects and COVID-19 In recent years, the importance of stakeholder involvement Dear Readers, and transdisciplinarity in sustainability research has grown We have updated the preprint version 1 of this article in response to the comments of two reviewers. These changes quickly, including energy research (Fazey et al., 2018; Lutz mainly relate to the methodological details of our study, & Bergmann, 2018; Mielke et al., 2016). Where values are particularly to clarify our approach better, and to the discussion contested (Funtowicz & Ravetz, 2006), transformations are section, in which we now go into more detail about the conflicting ( Renn, 2019) and decisions are urgent, transdiscipli- implications of our study for stakeholder-based research after narity is an answer for developing societally relevant solutions the pandemic. The full details of the changes are found in the Responses to Reviewers comments. We have made a few to complex, real-world problems (Lang et al., 2012; Lutz & final changes from version 2 to the final article regarding the Bergmann, 2018; Scholz & Steiner, 2015). Furthermore, stake- formatting. holder involvement can increase the relevance of research, bring Kind regards, higher acceptability and accountability of the problem, and the authors increase legitimacy and societal ownership of the research. Given these potential benefits of engaging stakeholders, funding bodies Any further responses from the reviewers can be found at  the end of the article also now encourage, and oftentimes require, the involvement of stakeholders in research (e.g., European Commission, 2020b). Stakeholder involvement is today much more than a social- Introduction scientific add-on: these engagement activities shape the The coronavirus 2019 (COVID-19) pandemic has strongly affected projects themselves, often including co-creation of both societies and economies across the world, including the scien- research questions and project aims, and the projects often tific community. The social distancing and lockdown measures seek to influence the societal processes with which they engage applied in most countries have potentially influenced one par - (Bracken et al., 2015; Klenk et al., 2015). Stakeholders can ticular aspect prominent in many research projects: stakeholder be involved in research to different degrees, encompass- involvement. With the growing importance of stakeholder ing information, consultation, cooperation, collaboration and interaction in research, the tie between science and practice has empowerment (Schneider & Buser, 2018; Stauffacher et al., improved, but science has also become vulnerable to the avail- 2008). Although the degrees of engagement depend on research- ability and readiness of stakeholders to interact with researchers. project phases (Bruhn et al., 2019) and involvement formats Based on our own experiences, we expected that many (Mielke et al., 2017), many of them are based on the physical researchers have faced specific challenges to interact with stake - presence of stakeholders in one location – and these have been holders online, and they had to “experiment” with different particularly strongly affected by the COVID-19 crisis. online formats and tools. Hence, it is important to understand the impacts of the pandemic on stakeholder involvement in Academia responded to the coronavirus pandemic and its research and the satisfaction of coping measures to draw containment measures in various ways: normatively, by encour- implications on what this new mode of online stakeholder aging the promotion of a culture of care and the redefini - involvement means for future choices of physical and digital tion of excellence in teaching or research, e.g., by focusing stakeholder activities. more on inequalities in academic institutional environments (Corbera et al., 2020), but also pragmatically, by quickly adapt- In this paper, we investigate the effects of the COVID-19 ing to the distancing measures and moving academic interactions, crisis on the stakeholder work in European energy research like lectures, seminars and conferences, online (Leal Filho et al., (focusing mainly, but not exclusively, on Horizon 2020 projects), 2020; Schwarz et al., 2020). Similarly, funding bodies reacted during the first wave of the coronavirus pandemic in spring and to the coronavirus outbreak too. For example, the European summer 2020. We address three research questions: (i) how Commission announced that the “ force majeure” clause can be much of a problem are the coronavirus containment meas- invoked in Horizon 2020 projects, if the grant beneficiaries are ures for stakeholder engagement in European energy research not able to fulfil their obligations due to coronavirus restrictions projects?, (ii) how have researchers responsible for stakeholder (European Commission, 2020a), including the stakeholder engagement coped with the new situation?, and (iii) how do engagement activities. This is very important, because in more researchers evaluate the coping measures (if undertaken)? We stakeholder-dependent projects, the COVID-19 crisis certainly report and discuss the findings of a survey distributed to all has the potential to make entire projects unfeasible. Recent running European Union (EU) funded energy projects research (Corbera et al., 2020; Leal Filho et al., 2020) provides with stakeholder components, carried out in June-August important findings on how academia and sustainability research - 2020, investigating the effects of coronavirus containment meas- ers have been impacted and have dealt with the crisis, but did ures on stakeholder involvement in European energy research. not address the impact on stakeholder engagement in research This study is not only relevant for the scientific community projects. We contribute to the closing of this gap by investigat- to gain a better understanding of applied coping strategies for ing the impacts on stakeholder engagement in energy research stakeholder engagement in the times of the COVID-19 pan- and providing insights into how the research community has demic, but also for funding bodies, who have to make decisions coped with the restrictions, as well as what has worked best about how to support research projects under the new conditions. in the first months after the coronavirus pandemic started. Page 3 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 email to projects identified in the CORDIS research database, Methods and existing networks (e.g. partner projects). In addition, the To identify the impact of the COVID-19 crisis and contain- European Commission Directorate-General of Research & ment measures on stakeholder involvement in energy research Innovation helped with the distribution of the survey. We also projects, we carried out an online survey study among peo- promoted the survey via social media channels, such as Research- ple responsible for stakeholder engagement who work in Gate, Twitter and LinkedIn, specifying that the survey should energy research projects across Europe. The survey was done in be completed only by project coordinators or partners respon- summer 2020, around four months after the coronavirus sible for stakeholder engagement activities. The survey was outbreak in Europe. During spring and early summer 2020, all online for twelve weeks during the period June-August 2020. European countries had COVID-19-related containment meas- We allowed also for non-Horizon2020 to participate in our ures in place. These included lockdowns in most countries survey, to expand the sample and broaden the respondent – leading to a closure of many academic and research entities, base to capture experiences from a higher project diversity. cultural institutions and other public spaces as well as strict Additionally, we allowed for responses from multiple research- restrictions on non-essential travels, and introduction of social ers from one project, because different stakeholder activities distancing rules on meeting other people. are often performed in different temporal and geographi- cal contexts, each with a different containment situation. We The survey was designed as a collaboration between research- treated multiple responses from single projects as individual ers for the projects SENTINEL (Horizon2020; energy), TRIPOD responses. (European Research Council; energy), and PANDORA (Horizon 2020; fisheries) as an explorative, semi-quantitative, self- We analysed the statistical survey in three steps: first, we com - completion online questionnaire (cf. Bryman, 2012), using the piled and compared the quantitative responses; the resulting online tool “LimeSurvey” (LimeSurvey, 2020). Survey questions descriptive analysis is the core of the results below (Creswell & were structured around five blocks: Creswell, 2018). Second, we complemented the results based on the written replies. Third, we applied a logistic regression - A. general questions concerning the stakeholder analysis using two different discrete choice models to derive pre- engagement activities in the projects; liminary insights on the correlation of the i) “very negatively” and ii) “not at all” responses in terms of the impact of COVID- - B. COVID-19 impacts on stakeholder engagement; 19 on stakeholder engagement with relevant explanatory vari- ables from other categories of the survey. We performed a logis- - C. coping strategies and alternative formats implemented in tic regression analysis to further shed light on potential factors response to coronavirus restrictions; that could explain the very negative impacts of COVID-19 on the stakeholder engagement activities of the energy research - D. evaluation of the implemented alternative formats; and projects, but also on factors that could explain the zero effect of the pandemic on stakeholders’ involvement. Our goal was not - E. demographic data. to perform a complete econometric analysis (e.g., best-fitting model information criteria, evaluation of parameter estimates The survey contained independent questions as well as ques- using quasi standard errors, etc.), but to identify meaning- tions that built on previous answers. We used different ques- ful correlations between the variables under study to com- tion formats, from Likert-like scales to multiple choice and plement the explanatory analysis of the descriptive statistics free text boxes, depending on the variables to be addressed. of the survey data. We pre-tested the survey with our project partners and adapted it in response. The questionnaire is available as extended data Methodological details on the logistic regression (Süsser et al., 2021). analysis As dependent variables, we selected the two marginal cases For data collection, we identified 195 Horizon 2020 energy “COVID-19 pandemic influence on stakeholder activity/ research projects relevant to our study. The CORDIS data- engagement– Not at all” and “COVID-19 pandemic influence base brought 365 hits of projects using the search keywords on stakeholder activity/engagement– Very negatively” to ‘energy’ and ‘stakeholder’, which started no later than January help us further derive some meaningful explanations. The 2020 and ran at least until the end of 2020. We contacted only latter is also attributed to the fact that, given the format that 195 out of the 365 projects, as the rest of the projects did not focus the online survey took place, our sample did not allow for the on energy questions, were rather technological/industry-focused development of one theoretical prediction model that could without a clear stakeholder component, and/or did not pro- include all the different Likert-scale responses. We converted vide any contact details. We distributed the survey widely via both cases to two separate dummy variables (“1 = YES to the question”, “0 = NO to the question”). For each one of the two dependent variables, we formulated a separate discrete choice Based on the basic questionnaire, each project adapted the questionnaire to model and selected an initial set of explanatory variables based its specific context and distributed it in the relevant communities. Here, we report only on the energy research survey, whereas the fisheries survey is part on its relevance to each one of them. We then converted again of another publication (Köpsel et al., 2021) the categorical explanatory variables to dummy variables. Page 4 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 We made the final selection of the explanatory variables to be for the data collection and processing in the context of this included in each discrete choice model through a trial-and-error study, under the guidance of the Institute for Advanced approach: Different sets of relevant explanatory variables Sustainability Studies (IASS) data protection service. This were tested, examining in parallel the correlation matrix to has been supported by bilateral data protection agreements. test for collinearity issues among the covariates, until no sig- The respondents agreed to our data protection standards via a nificant collinearities were observed, to ensure that each final GDPR disclaimer, and by participating in the survey. set of explanatory variables is able to adequately predict the respective dependent variable. Results Stakeholder engagement in energy research projects For both discrete choice models, the probability that the stake- Stakeholder engagement is a crucial or important com- holder engagement activities of each project i are very negatively ponent of by far most projects in our sample (Figure 2a). affected (or are not affected at all) is modelled: The researchers mainly engage with stakeholders periodi- cally in specific phases of the project ( Figure 2b). Most of the respondents engage with EU stakeholders; about one third P(y =1) = Λ(β⋅ x ) i i of the projects also work with non-European stakeholders, for example in the US, China or Indonesia. where: ○ y is the dependent variable describing if the stakeholder According to the original project plans, 2020 was supposed activities of a project i are very negatively affected, to be a major year for stakeholder engagement for almost all or are not affected at all; respondents; hence almost every project was affected in some way by the coronavirus containment measures in Europe. ○ x is the vector of independent/explanatory variables for the A variety of physical and online activities were planned in th i project; 2020 – mainly workshops, information events and confer- ences. Almost half of the respondents had planned online ○ β is the parameter vector to be estimated; and interaction formats, such as webinars. ○ Λ is the logistic distribution (Spyridaki et al., 2020). The respondents have different motives for engaging with The logistic cumulative distribution function is defined as: stakeholders. As shown in Figure 3, the linear research mode is dominant, in which stakeholders are either viewed as the β⋅x target audience for results (“dissemination”), or as research P(y = 1|x ) = Λ(β⋅ x ) = i i i β⋅x i subjects (“access stakeholders”). However, more transdisci- 1+e plinary and co-creative motives are also high on the agenda, including research question identification and implementation where P is the probability of y occurring. The maximum of findings/technologies. likelihood (ML) estimation method is used to estimate the parameter vector β. The impacts of the coronavirus containment measures on stakeholder engagement activities and outcomes Sample description The first wave of the coronavirus and its containment meas - We received 84 complete responses from 72 different energy ures affected stakeholder activities in energy research projects projects: 62 different EU Horizon 2020-funded projects mainly negatively: almost nine of ten respondents perceive (31% of the stakeholder-engaging EU-funded energy somewhat or very negative effects (Figure 4). Furthermore, research projects running at the time), and 10 projects with projects planning face-to-face workshops in 2020 are more other funding sources. For most projects, we received only negatively affected by the crisis, which is not surprising, as one response; for eight projects we received two responses , such events were de facto banned in most countries by social and five responses from one project. Most projects started distancing measures. The same applies to projects where in 2018/19 and will end in 2021/22; practically all are three- stakeholder involvement has a higher priority: our regres- year projects. The important demographic sample data of sion analysis shows a positive correlation between the projects surveyed respondents is summarised in Figure 1. that stated a very negative influence and responses that stated that stakeholder engagement is crucial for the successful Ethics requirements implementation of the project (Table 1). In addition, projects The research has been conducted under the ethics require- that focused on engaging with policymakers were more ments and guidelines of the SENTINEL project (Deliverables affected: we find a negative correlation between no influ - 11.1 and 11.2), which follows the guidelines of the European ence (‘not at all’) and projects engaging with policymakers, Commission. We have applied an ethically-robust methodology but not for those engaging with other stakeholder groups, a negative correlation between negative effects and projects engaging with energy industry representatives, and positive We received two responses from the SENTINEL project, which were not from the authors of this paper. correlation between no influence and projects engaging Page 5 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Figure  1.  Demographics of surveyed respondents: a) What is your gender?, b) What is your main field of research?, c) How long have you worked in this field?, d) How long have you been engaging with stakeholders in European Union projects?, n = 84. Figure  2.  Importance of stakeholder engagement and engaged stakeholder groups: a) How important is stakeholder engagement for the success of your project?, b) What stakeholder groups are being engaged in your project?, n = 84. Page 6 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Figure  3.  Motives  for  stakeholder  engagement:  What is the purpose of your stakeholder engagement? (multiple responses possible), n = 84. (Figure 4). For these projects, as expected, we find that if the inter - action with stakeholders is ‘not very important’ for the project, the effect of the containment measures is smaller (cf. Table 2). Additionally, a small percentage assessed the influence of the pandemic to be positive (Figure 4), which is possibly related to the better response of stakeholders to online formats (Figure 5), enabling more frequent exchange with stakeholders or access to stakeholders living further away. In contrast, for many respondents it became harder to reach stake- holders (Figure 5). Respondents underlined that stakeholders’ life was negatively affected by the crisis: stakeholders “experi- enced an increase in stress and workload”, had “difficult[ies] to perform the work foreseen due to the closure or reduction of activities”, experienced dropped incomes, were “unable to work”, or may even have become unemployed. It is not surprising that these impacts have led to a shifting of stakehold- ers’ priorities away from the projects (Figure 5). This result is also supported by our regression analysis, which indicated a positive correlation between negative effects and responses that stated that priority of engagement decreased very much since the beginning of the pandemic, and a positive correlation between no influence and responses that stated that priority Figure  4.  Impact  of  the  coronavirus  disease  2019   of engagement was not affected at all. In contrast, the priorities (COVID-19)  pandemic  on  stakeholder  activities:  Does the COVID-19 pandemic influence your stakeholder activity/ for stakeholder engagement of most researchers did not engagement negatively or positively?, n = 84. change, but for some they decreased or increased. Respondents that were personally more affected are especially more likely to report that their stakeholder engagement priority has changed ‘very much’ (cf. Table 1). This indicates that people’s private life with civil society organisations. This could be due to the situation has an impact on their work life: with offices moving fact that, typically, engagement activities that involve poli- into homes, work and private life became linked more closely. cymakers take place as physical workshops and meetings, while activities that involve energy industry and civil society representatives took place in various formats, including More than half of the researchers expect a negative influence online channels also before the COVID-19 pandemic. Table 2 on the outcomes of the overall stakeholder engagement proc- shows the detailed results of the regression analysis. ess (Figure 6), not only in terms of engagement frequency and similar quantitative aspects, but also in the quality of inter- Only 10% of the respondents reported that the COVID-19 actions and stakeholder-based input for the projects. One measures had not effect on the stakeholder engagement responded explained: Page 7 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Table 1.  Logistic regression model I.  Dependent variable: “Impact of COVID-19 on stakeholder engagement of energy research projects – Very negatively”. Explanatory variables Prediction  model Category Name Coefficient Stakeholder groups engaged Policymakers -3.224 (2.333) Energy industry -3.726 (2.076)* Geographical/spatial focus EU -11.974 (5.945)** COVID-19 cases 7.116 (3.194)** Frequency of engagement Weekly -5.075 (4.310) Importance of stakeholder engagement for the success of the project Crucial 9.043 (4.083)** Year of stakeholder engagement according to project plan/proposal 2021 -9.139 (4.033)** 2022 5.058 (2.893)* Engagement activities according to the project plan/proposal (year  Face-to-face workshops 6.999 (3.579)** 2020) Information events for stakeholders -6.269 (3.009)** Face-to-face interviews 1.920 (1.745) Purpose of stakeholder engagement Disseminate research results 0.917 (1.895) Priority of stakeholder engagement changed due to COVID-19 Priority decreased 6.768 (3.736)* Change in relationship to stakeholders Stakeholders priority has shifted -3.719 (2.497) away from the project It is harder to reach stakeholders 4.886 (3.708) Impact of changes in your stakeholder engagement activities on  Delays in the flow of data to other 5.366 (2.683)** proceedings and results of your overall project work packages The project duration will need to be 6.037 (3.045)** extended Deliverables’ submission has been/ 3.548 (2.288)* will be delayed Constant -2.336 (6.257) Notes: - Standard errors are reported in parentheses. - Superscripts***, **and *indicate statistical significance of 1%, 5% and 10% level, respectively. Table 2.  Logistic regression model II.  Dependent variable: “Impact of COVID-19 on stakeholder engagement of energy research projects – Not at all”. Explanatory variables Prediction  model Category Name Coefficient Stakeholder groups engaged Policymakers -11.824 (6.787)* Civil Society Organizations -9.133 (4.800)* Geographical/spatial focus Non-EU -8.448 (5.098)* Importance of stakeholder engagement for the success of the  Not very important 25.546 (13.188)* project Page 8 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Explanatory variables Prediction  model Category Name Coefficient Year of stakeholder engagement according to project plan/ 2021 3.084 (2.814) proposal Engagement activities according to the project plan/proposal (year  Online survey 4.834 (4.365) 2020) Purpose of stakeholder engagement Access to data and information to -3.660 (2.223)* understand a research problem Disseminate research results 10.459 (6.826) Priority of stakeholder engagement changed due to COVID-19  Priority unchanged 4.116 (2.231)* Change in relationship to stakeholders No change -8.377 (5.058)* Impact of changes in your stakeholder engagement activities on  Overall workflow is not impacted 4.882 (2.910)* proceedings and results of your overall project No negative impact 4.328 (3.172) The project will be carried out as 6.973 (3.686)* planned, with the envisioned results Constant -17.752 (9.453)* Notes: - Standard errors are reported in parentheses. - Superscripts***, **and *indicate statistical significance of 1%, 5% and 10% level, respectively. Figure  5.  Perceived  changes  of  relationship  with  stakeholders  (multiple  responses  possible):  Please tick the boxes if you agree with the following statements, n = 84. I think the COVID-19 restrictions on in-person community This quote underlines that the more co-creative processes engagement will limit the value of stakeholder feedback. We may also suffer from reduced possibilities for co-designing have just completed the Comprehensive Plan update, and the research questions, co-owning the results and co-agreeing on four community engagement workshops created a bonding its implications – the very aim of transdisciplinary research. among the community that had great value. The participants felt ownership to the results and support the implementation The impact of the COVID-19 crisis on project workflows actions. This attribute will be even more critical in the Cli- and outcomes mate Action Plan, as some of the actions are a bit more contro- The coronavirus restrictions have negative impacts on the versial. Lacking the interactive discussions and bonding over workflow of most projects, mainly leading to delays in the shared outcome is a weakness of the COVID-imposed process. flow of data between work packages, as shown in Figure 7a. Page 9 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 We find a strong correlation between negative effects and the responses “Delays in the flow of data to other work packages”, “Deliverables’ submission has been/will be delayed” (Table 1). This implies that delays have led to a stronger feeling by researchers’ that their stakeholder activities are negatively affected by the pan- demic. This might be related to the fact that delayed input stem- ming from the stakeholder engagement process can be potentially considered in the project differently as initially planned. Most respondents expected that changes in stakeholder engage- ment activities will affect the proceedings and results of the overall project (Figure 7b). Although all respondents believe that the overall project objectives can be still completed, the majority of the projects will not be carried out as planned, and, thus, results will be different than expected. Furthermore, more than one third of the respondents expect that they will need to extend the project duration, which leads to their percep- tion of stakeholder activities being ‘very negatively’ affected by the crisis (cf. Table 1). This is not only related to the challenge of involving stakeholders, but also because researchers had to “adapt […] to this format[s] and approach[es that] require[d] Figure  6.  Impact  of  the  coronavirus  disease  2019  (COVID-19)  a learning curve for [their] teams”. This capacity build- pandemic  on  outcomes  of  the  stakeholder  engagement  ing for dealing with online communication tools has been an process:  Do you think that the COVID-19 situation will important step for most of the respondents as few activities influence the outcomes of your stakeholder engagement process?, n = 84. took place as physical, socially distanced events (Table 3). Figure  7.  Impacts of the coronavirus disease 2019 (COVID-19) crisis on the projects: a) Do you think that the COVID-19 situation will have a negative impact on the workflow within your project?; b) How do you think the changes in your stakeholder engagement activities will affect the proceedings and results of your overall project? (multiple choices possible), n = 84. Page 10 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Table 3.  Overview of engagement activities and impacts of the coronavirus disease 2019 (COVID-19) crisis on the  implementation (numbers), multiple choices possible, n = 84.                               Type of activity                                 Alternative activities: Implemented  Socially  Format  Planned Delayed Cancelled Other as planned distanced changed Originally planned: Information events for stakeholders 46 0 7 24 11 27 2 Face-to-face workshops 64 0 9 29 10 45 7 Conferences 39 1 2 20 10 28 4 Focus groups 24 1 5 17 5 19 1 Face-to-face interviews 28 1 1 14 6 16 2 Online interviews 14 8 - 8 0 3 1 Face-to-face survey 9 0 1 1 6 4 0 Online survey 24 18 - 7 1 0 0 Webinars 29 21 - 5 1 3 3 % of strategy applied: 18% 9% 45% 18% 52% 7% conferences, because as one wrote “online concentration Coping strategies of researchers to deal with span and endurance of people is limited”. However, shorter containment measures events may lead “sometimes to very superficial results Researchers adapted their involvement activities to the because [there is] no time to deepen certain aspects”, as one restrictions: only one of six stakeholder engagement activi- researcher reported. In addition, respondents suggested the ties were implemented as planned – almost all of which were splitting up of participants in “smaller targeted online events already planned to be online – whereas two thirds were either (workshop, focus group, interviews), where not too many cancelled or delayed. The most common coping strategies were people are present”, as well as “break-out groups coupled with the postponement of concrete stakeholder involvement activi- interactive polling tools, appeared to increase stakeholder ties, presumably hoping for looser restrictions in the future, retention and participation over the course of a small 1-day and changes in formats – and often a combination of the workshop.” two measures (Table 3). For some projects it became easier to engage stakehold- Among the alternative engagement formats (if formats were ers online, especially “ policymakers appear to be easier to changed), online workshops and webinars were the most com- engage in short online meetings than longer physical meet- mon (Table 4). Conferences, face-to-face interviews and ings”. In contrast, other stakeholders left the projects, as focus groups were often directly replaced by the respective consequence to the social distancing measures, as one online format. Information events, as well as face-to-face respondent explained: workshops, were mainly replaced by webinars and online workshops. Interestingly, respondents often performed more Not all stakeholders […] wanted to continue meeting than one alternative engagement activity, suggesting that online. Many stakeholders found the planned in-person meet- the online formats are not seen as perfect complements to ing of their peers in another city as a motivation to join the physical meetings. project in the first place. Assessment of alternative stakeholder engagement In addition, one quarter of the respondents stated that they formats could not reach stakeholders via digital tools, which con- Our results show that many alternative online formats cerned mainly citizens, local authorities, and locally-based – although not the researchers’ first choice − have been use - businesses. One respondent expressed their concern: “Many ful for projects: in particular, webinars, online interviews and of our target stakeholders are elderly, and many have limited online surveys are widely seen as suitable online engagement computer access.” formats (Figure 8). In contrast, experiences with online focus groups, online conferences and workshops were rather mixed. Interactive workshops and networking formats seem to be Nevertheless, the majority (55 out of 66 respondents, free text challenging, and respondents recommended rather short online reply) plans to continue online engagement activities after Page 11 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Table 4.  Overview of alternative online engagement activities performed, if ‘format changed’ (numbers), multiple choices  possible, n = 84. Type of activity Webinar Online  Online  Online focus  Online  Online  Other workshop conference groups interviews survey Information  15 14 6 5 2 3 1 events Online content, e.g., videos Face-to-face workshops 17 32 5 7 6 3 5 Mailed survey, online group, not decided Conference 10 7 18 1 1 2 4 Blogs, not decided Focus groups 2 6 1 11 6 4 2 Not decided Face-to-face interviews 1 1 0 1 14 1 2 Online meetings, telephone survey Face-to-face survey 1 1 0 0 2 2 0 Online interviews 3 1 0 1 0 0 0 Webinar(s) 2 0 0 0 0 0 1 Not decided Figure  8.  Assessment  of  alternative,  online  engagement  methods:  Considering the goals of the stakeholder activity you wanted to perform originally, how suitable were the following formats as a replacement? Page 12 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 the contact restrictions are lifted; some were unsure about it in the future for one-to-one and unidirectional interactions yet, while a few expressed their scepticism – “hopefully not”. with stakeholders, simply because they provide good results. One respondent stated: “The experience with the online work- shop was good, so we will likely consider doing it again. Perspectives for stakeholder engagement after COVID-19 However, this will not really replace face-to-face events. This The COVID-19 crisis may prove to be a window of opportu- is rather a complement to face-to-face workshops/activities”. nity for digitalisation in stakeholder-involving research. We In their free-text responses, many respondents gave similar find that social distancing measures enforced an unforeseen statements, agreeing that online formats cannot replace face-to- shift to online engagement activities in five of six cases. This face meetings, as they are “essential to engage with stakeholders verifies findings by Schwarz et al. (2020) for the academic to allow for clearer communication and networking”. context: researchers are willing to also use digital tools for science-stakeholder interactions. Researchers quickly built up Discussion new capacities to involve stakeholders online and showed a Impacts of the COVID-19 pandemic on stakeholder high openness in trying new engagement formats instead of engagement activities resigning. This is also true for stakeholders: although some The COVID-19 crisis has affected stakeholder engagement stakeholder’s engagement in research projects decreased, most in energy research mainly negatively, but it did not stop it. of them showed openness for new online engagement activi- Adding to Leal Filho et al. (2020), we find that the pandemic ties to varying degrees, but generally at least satisfactorily, the has rearranged the work environment and the private life not online formats worked too. This offers a positive perspective only of energy researchers but also of stakeholders. Thus, it is for stakeholder engagement while containment measures are not surprising that planning of, and participation in, engage- still in place, but also shows that digital formats are likely to ment activities has moved down on professional agendas. be used also after the pandemic, simply because they have Researchers reported a diversity of motives for engaging with proven their usefulness for particular aims. Furthermore, it can stakeholders: while some mainly communicate results, others also contribute to the further improvement and development co-produce and co-design research and practical projects of online engagement formats, including the software and the together. Projects that rely more on the involvement of specific approaches developed by researchers. stakeholders were more affected by the impacts of the pandemic. Nevertheless, adding to previous insights for the academic Researchers coped with the pandemic most commonly by post- context (Schwarz et al., 2020), we find that online engage - poning activities and/or changing formats. Almost half of the ment activities seem unsuitable to replace physical interactions projects decided to shift stakeholder engagement activities with stakeholders completely. This is because a relatively short in time, likely in the hope that in-person communication will duration of online events reduces the depth of interaction and become possible soon. While this may initially be a useful strat- the quality of the results. This suggests that some projects egy, to benefit from a deeper in-person engagement later on, might miss important information via online events, and the delays may accumulate within a project. If tasks and work- outcomes of the online stakeholder engagement must thus be flows are interrupted, projects might fail to take stakeholder critically evaluated. perspectives into account and/ or to finish projects on time. Furthermore, almost all engagement activities that took place In addition, we find that online events may not be able to have been moved online, which appears to be the most replace physical events, because of different levels of stakehold- natural solution with social distancing measures in place. ers’ commitment to contribute to research projects: participat- However, this also raises the question of how suitable the ing in physical events requires more time and other resources various digital formats are for engaging with stakeholders. than switching on the computer. Hence, there is a risk of reduced commitment to online-only engagement processes. This implies that researchers who choose online formats need to rethink Researchers have had good experiences with some digital offline formats when applying them online to ensure that formats, but not with all of them. Webinars and online inter- stakeholder are encouraged to join and are activated in online views, which are based either on one-to-one interaction or meetings. A complete shift from offline to online for all unidirectional communication, are found to work well online purposes does not appear possible or useful. and provide satisfying results. Offering such formats online has clear advantages: it makes them easily accessible to a potentially larger audience of stakeholders, saves travel Last, we agree with Beaunoyer et al. (2020) that not all stake- times and emissions. Moreover, they allow, for example, the holders are used to online technologies, which may lead to recording of such activities, which makes evaluation work technology-related inequalities and digital exclusion of par- easier after the event. Contrary to this, formats requiring ticular stakeholder groups. Hence, researchers must be aware group activities, a higher level of stakeholder involvement of the stakeholder groups that cannot be easily reached and multidirectional, interactive communication (cf. Späth & digitally and adapt their approaches accordingly. For example, Scolobig, 2017) are evaluated much less favourably by policymakers could be generally better reached via dig- respondents. Consequently, researchers must carefully ital tools. However, on the other hand, projects that involved think about the objectives of the engagement and assess the policymakers were more affected by the containment meas- suitability of online formats carefully: not all interaction types ures, probably because policymakers had to shift their priorities are equally suited for online formats. It also suggests that due to coronavirus-related issues. Nevertheless, in the future, researchers could specifically continue using online formats we expect a shift towards a combination of online and Page 13 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 COVID-19 on stakeholder engagement activities in energy offline activities at different times as well as hybrid formats research projects. Scientists could use our publicly available that combine both forms. dataset to perform more robust econometric analyses (incl. best-fitting model information criteria, evaluation of param - Implications on the future funding of research projects eter estimates using quasi standard errors, etc.), also account- Whereas stakeholder involvement is a crucial component of ing for limitations due to the small sample size and number many research projects funded today, the COVID-19 crisis of responses in some variables that could cause issues of mul- revealed that this requirement has made these projects vulner- ticollinearity between the independent variables and affect able to changes in stakeholder availability and accessibility. Thus, the predictive power of the statistical models. it raises questions concerning the resilience of transdiscipli- nary research: access to stakeholders can only partially be influ - Finally, only energy research projects were within the scope enced by the researchers themselves, and to some extent, they of this study, with a focus on Horizon 2020 projects. Thus, our are simply exposed to the risk of failing engagement activi- results are only generalisable beyond energy, where compa- ties due to external factors. Consequently, the COVID-19 rable results have been derived from similar research in other crisis can also be seen as a resilience test for the participa- disciplines. It would be relevant to see similar studies also in tory aspirations of the research funding bodies. While the other specific fields, to potentially monitor and compare how European Commission’s response has been generally sympathetic different research areas respond and adapt their approaches, in that context, not all researchers were met with open ears to enable cross-disciplinary learning and an exchange of when requesting for a project extension due to coronavirus experiences. restrictions. This issue may need to be addressed by funders, especially because coronavirus restrictions seem to continue for a longer time. Researchers, stakeholders, and funding Conclusions institutions need to recognise the current situation and must The coronavirus social distancing and lockdown measures stay flexible with their approaches. have had a mainly negative effect on stakeholder engage- ment in energy research projects, especially by interrupting the exchange between researchers and stakeholders, caus- Limitations and prospects for future research ing delays in the project workflow, and changing outcomes of Measures against the COVID-19 pandemic are constantly stakeholder engagement processes. Given this difficult changing, and so are the framework conditions for stake- situation, researchers have generally been able to quickly adapt holder interactions. Situations that respondents experienced by finding new ways of engaging with stakeholders, switch - in spring 2020 during the first wave of the pandemic may not ing especially to online workshops and webinars. This shows necessarily be the same in spring 2021 or 2022. Moreover, as that digital engagement has become a new norm during coronavirus restrictions also vary between countries, there COVID-19. Researchers have had positive experiences with are different possibilities for stakeholder interactions in several online engagement formats, in particular webinars, different places. In this research, we could account neither for online surveys and interviews – but are much less content with specific phases of the pandemic, nor for situations in specific workshops and conferences. This suggests that online formats country-based contexts; as a result, future research could work particularly well for one-to-one and unidirectional investigate these changes and differences over space and time. formats, whereas interactive and group activities are less suited for digital formats. While various digital participa- Since many activities were ongoing at the time of the survey, tion formats offer new opportunities for involving stakeholders we could only identify initial lessons on the coping strate- more frequently by reaching a larger audience and being less gies and the success of implemented measures. Although we time-consuming, an important disadvantage is that some social received insights on the expected impacts of coping strategies groups are excluded from the process and derived results on the project outcomes, since the stakeholder engagement might be more superficial. Thus, online engagement will likely processes are ongoing, the actual impacts will likely materi- continue after the crisis, but only to complement and not to alise towards the end of the projects. To this end, an updated replace physical meetings in energy research. The long-term version of the same survey in 2021 or 2022 would be inter- effects on energy research remain to be seen. However, given esting and relevant. Furthermore, in this context, it would be the large amount of hope put into postponing events, there valuable to learn from transdisciplinary research and projects is a clear risk that projects will not be able to finish on time where stakeholder engagement plays a vital role in co-creation or with the intended contents, depending on the duration of processes how the shift to the online formats has influenced the coronavirus-related restrictions. the outcomes. Most probably, more interactions in the digital world have not only serious implications on the general Although the COVID-19 crisis is a monumental challenge for social reality, but specifically for approaches to complex, all, including for us ourselves as researchers, positive changes real-world problems related to sustainability. can be triggered. When forced to adapt, researchers and stake- holders quickly started experimenting with new formats – and In addition, the regression analysis performed was meant developed several solutions that were found useful and to supplement the explanatory analysis of the descriptive attractive. Quite possibly, after the pandemic ends, we will statistics and provide preliminary indications on the potential find that more of our work has moved into the online causalities that could explain the negative impacts of Page 14 of 23 Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 space – and we will know why personal contacts are (Version 1). http://doi.org/10.5281/zenodo.4765630 (Süsser et al., irreplaceable, after all. 2021) This project contains the following extended data: Data availability Underlying data • questionnaire_impact covid-19 on stakeholder The survey data underlying this study are not openly avail- engagment_SENTINEL_2020.pdf able to protect the anonymity of participating individu- Data are available under the terms of the Creative Commons als and projects. Given that we asked for many details of the Attribution 4.0 International license (CC-BY 4.0). projects, like funding source, duration etc., we cannot guar- antee anonymity. For further COVID-19 research and the replicability of the research in another study context, access to semi-anonymised data can be granted. For Acknowledgments further information on the data please contact Diana Süsser We thank Vera Köpsel (University Hamburg, project PANDORA (diana.suesser@iass-potsdam.de). (grant agreement No. 773713)) for designing together with us the survey underlying the results presented in this paper. We thank Extended data Jeremias Herberg and Stephen Gary Williams (IASS) for their ZENODO: Questionnaire related to on the impact of COVID-19 feedback on the survey design. We also thank all our participants on stakeholder engagement in European energy research of the survey for taking the time for participating in our research. References Beaunoyer E, Dupéré S, Guitton MJ: COVID-19 and digital inequalities:  Leal Filho W, Azul AM, Wall T, et al.: COVID-19: the impact of a global crisis on  Reciprocal impacts and mitigation strategies. Comput Human Behav. 2020; sustainable development research. Sustain Sci. 2020; 16: 1–15. 111: 106424. PubMed Abstract   | Publisher Full Text   | Free Full Text   PubMed Abstract   | Publisher Full Text   | Free Full Text   LimeSurvey: An Open Source survey tool /LimeSurvey GmbH . 2020. Bracken LJ, Bulkeley HA, Whitman G: Transdisciplinary research:  Lutz LM, Bergmann M: Transdisziplinarität: Forschungsansatz für die  understanding the stakeholder perspective. J Environ Plan Manag. 2015; Energiewende. In: Holstenkamp, L., Radtke, J. (Eds.), Handbuch Energiewende 58(7): 1291–1308. Und Partizipation. Springer VS, Wiesbaden, 2018; 43–56. Publisher Full Text   Publisher Full Text   Bruhn T, Herberg J, Molinengo G, et al.: Grounded action design  Mielke J, Vermaßen H, Ellenbeck S: Ideals, practices, and future prospects of  – Transdisciplinary co-creation for better transformative processes.  stakeholder involvement in sustainability science. Proc Natl Acad Sci USA. Frameworks for transdisciplinary research #9. Gaia - Ecological Perspectives 2017; 114(50): E10648–E10657. for Science and Society. 2019; 28(4): 336. PubMed Abstract   | Publisher Full Text   | Free Full Text   Publisher Full Text   Mielke J, Vermaßen H, Ellenbeck S, et al.: Stakeholder involvement in  Bryman A: Social Research Methods . Fourth edi. ed. Oxford: Oxford Univ. Pr. sustainability science—A critical view. Energy Res Soc Sci. 2016; 17: 71–81. Publisher Full Text   Reference Sourc e Renn O: The function(s) of transdisciplinary research in conflicting Corbera E, Anguelovski I, Honey-Rosés J, et al.: Academia in the Time of  transformations. Gaia. 2019; 28(1): 44–51. COVID-19: Towards an Ethics of Care. Plan Theory Pract. 2020; 21(2): 191–199. Publisher Full Text   Publisher Full Text   Schneider F, Buser T: Promising degrees of stakeholder interaction in  Creswell JW, Creswell JD: Research Design: Qualitative, Quantitative, and  research for sustainable development. Sustain Sci. 2018; 13(1): 129–142. Mixed Methods . fifth edit. ed. SAGE Publications, Los Angeles. 2018. PubMed Abstract   | Publisher Full Text   | Free Full Text   Reference Sourc e Scholz RW, Steiner G: The real type and ideal type of transdisciplinary  European Commission: Public Engagement in Responsible Research and  processes: part I—theoretical foundations. Sustain Sci. 2015; 10: 527–544. Innovation. [WWW Document]. 2020a. Publisher Full Text   Reference Sourc e Schwarz M, Scherrer A, Hohmann C, et al.: COVID-19 and the academy: It is  European Commission: Can the clause on “force majeure” in the Horizon  time for going digital. Energy Res Soc Sci. 2020; 68: 101684. 2020 grant agreement be invoked in relation to the COVID-19 outbreak? PubMed Abstract   | Publisher Full Text   | Free Full Text   [WWW Document]. 2020b. Reference Sourc e Späth L, Scolobig A: Stakeholder empowerment through participatory  Fazey I, Schäpke N, Caniglia G, et al.: Ten essentials for action-oriented and  planning practices: The case of electricity transmission lines in France and  Norway. Energy Res Soc Sci. 2017; 23: 189–198. second order energy transitions, transformations and climate change  Publisher Full Text   research. Energy Res Soc Sci. 2018; 40: 54–70. Publisher Full Text   Spyridaki NA, Stavrakas V, Dendramis Y, et al.: Understanding technology  Funtowicz S, Ravetz J: Values and Uncertainties . In: Handbook of ownership to reveal adoption trends for energy efficiency measures in the Greek residential sector. Energy Policy. 2020; 140: 111413. Transdisciplinary Research. Springer Netherlands, Dordrecht, 2006; 361–368. Publisher Full Text   Publisher Full Text   Klenk NL, Meehan K, Pinel SL, et al.: GLOBAL CHANGE SCIENCE. Stakeholders    Stagl S: Opportunities of post COVID-19 European recovery funds in  in climate science: Beyond lip service? Science. 2015; 350(6262): 743–4. transitioning towards a circular and climate neutral economy.  2020. PubMed Abstract   | Publisher Full Text   Reference Sourc e Köpsel V, de Moura Kiipper G, Peck MA: Stakeholder engagement vs. social  Staua ff cher M, Flüeler T, Krütli P, et al.: Analytic  and  dynamic  approach  to  distancing—how does the Covid-19 pandemic affect participatory research collaboration:  A  transdisciplinary  case  study  on  sustainable  landscape  in EU marine science projects? Marit Stud. 2021; 20: 189–205. development  in  a  swiss  prealpine  region. Syst Pract Action Res. 2008; 21: 409–422. Publisher Full Text   Publisher Full Text   Lang DJ, Wiek A, Bergmann M, et al.: Transdisciplinary research in  Süsser D, Ceglarz A, Stavrakas V, et al.: Questionnaire related to Süsser et al.  sustainability science: Practice, principles, and challenges. Sustain Sci. 2012; (2021) on the impact of COVID-19 on stakeholder engagement in European  7: 25–43. energy research.  (Version 1). Zenodo. 2021. Publisher Full Text   http://www.doi.org/10.5281/zenodo.4765630 Page 15 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Open Peer Review Current Peer Review Status: Version 2 Reviewer Report 07 October 2021 https://doi.org/10.21956/openreseurope.15198.r27706 © 2021 Fell M. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Michael J. Fell UCL Energy Institute, University College London, London, UK I can confirm I’m happy with the author responses to the review.” Competing Interests: No competing interests were disclosed. I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Reviewer Report 30 September 2021 https://doi.org/10.21956/openreseurope.15198.r27707 © 2021 Leal Filho W et al. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Walter Leal Filho European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany Amanda Lange Salvia European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany The authors have addressed our comments. Competing Interests: No competing interests were disclosed. Page 16 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Version 1 Reviewer Report 25 June 2021 https://doi.org/10.21956/openreseurope.14757.r26979 © 2021 Fell M. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Michael J. Fell UCL Energy Institute, University College London, London, UK This paper uses a survey of EU energy research projects to describe the effect that Covid-19 response measures have had on stakeholder engagement. We find the approach to be quite clearly described and appropriate to the research questions. The findings are clearly described, although some clarification is needed around the regression analysis. More consideration of the implications of the findings would be valuable. We recommend the piece should be indexed, subject to a number of revisions. It would be useful if the scope of the work could be made a bit more precise. In the introduction it states “we investigate the effects of the COVID-19 crisis on the stakeholder work in European energy research”. However, participant recruitment seems to have been mainly aimed at Horizon 2020 projects, and the response rate reported is a % of EU-funded energy research project. The final sample was mainly Horizon projects (albeit a fairly small proportion of those eligible), plus some others. More clarity here, and consideration of implications of this for generalisability of the conclusions, is necessary. We are not fully clear on how exactly the regression analysis was conducted. Firstly, we were unable to identify exactly which survey question was used to determine the dependent variable. Please could this be clarified? The nearest one we could find was “How does the COVID-19 pandemic influence the stakeholder/engagement in your project? Likert scale: very positively <-> very negatively”. If the response options for the dependent variable were in the form of a response scale, how were responses which were not “Not at all” or “Very negatively” treated? It would also be useful if the authors could expand on why they decided to use logistic regression, and why it had to be run twice. It wasn’t quite clear how discrete choice modelling, and mentioned at the end of the methods section, was employed. Ideally more would also be said on how particular variables were selected for inclusion (i.e. one over the other), during the process of “examining in parallel the correlation matrix to test for collinearity issues among the covariates, until no significant collinearities are observed”. The regression results show a large number of variables, but the article only engages superficially Page 17 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 with these findings. It would be helpful if the regression results could be moved to the results section, and some of the key effects pulled out and described in the text. It would then be useful to see some discussion of the regression results – both in terms of limitations due to sample size, but also on the more interesting findings. For example (while significance level is low), there was a large negative effect size for policymakers and civil society organisations in the “Not at all” analysis – could be implications of this be considered? Generally, the discussion is quite light on what the implications of the findings could be – both for the projects which data was captured on and going forward. Are/will important viewpoints more likely to be missed, e.g. those less able to access digital avenues for engagement? Some more minor points: ○ It could be useful to provide an indicative timeline of some of the main Covid-related restrictions in Europe. This will obviously vary by country, but some general info could be helpful in interpreting this work in future ○ “Researchers agreed that online formats cannot replace face-to-face meetings” – is this based on a quantitative result from the survey and, if so, could this be included? Or is it just inferred from the quote? Is the work clearly and accurately presented and does it engage with the current literature? Yes Is the study design appropriate and is the work technically sound? Partly Are sufficient details of methods and analysis provided to allow replication by others? Partly Are all the source data and materials underlying the results available? Partly If applicable, is the statistical analysis and its interpretation appropriate? Partly Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Quantitative and qualitative social science research in energy. I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Author Response 30 Aug 2021 Page 18 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Diana Süsser, Institute for Advanced Sustainability Studies (IASS), Potsdam, Germany Response to Reviewer 2 R2.0 This paper uses a survey of EU energy research projects to describe the effect that Covid-19 response measures have had on stakeholder engagement. We find the approach to be quite clearly described and appropriate to the research questions. The findings are clearly described, although some clarification is needed around the regression analysis. More consideration of the implications of the findings would be valuable. We recommend the piece should be indexed, subject to a number of revisions. We are very glad to read that the reviewer appreciates the descriptions of our approach, research question, and findings. We have expanded the discussion to add further aspects about the implications of our findings, and we have added details about the regression analysis (also in response to comment 1.1 above). R2.1 It would be useful if the scope of the work could be made a bit more precise. In the introduction it states “we investigate the effects of the COVID-19 crisis on the stakeholder work in European energy research”. However, participant recruitment seems to have been mainly aimed at Horizon 2020 projects, and the response rate reported is a % of EU-funded energy research project. The final sample was mainly Horizon projects (albeit a fairly small proportion of those eligible), plus some others. More clarity here, and consideration of implications of this for generalisability of the conclusions, is necessary. Thank you for pointing this out. In the scope of the study were both Horizon 2020 and non- Horizon 2020 projects, but we only sent direct emails to the former. We allowed other projects to participate in our survey and we asked the projects if they were Horizon projects or not. We revised the text to make this clearer in the introduction and method section. We believe that many of our findings are generalisable beyond energy research – and specifically in cases where similar results have been found in other research and comment on this in the revised Discussion. R2.2 We are not fully clear on how exactly the regression analysis was conducted. Firstly, we were unable to identify exactly which survey question was used to determine the dependent variable. Please could this be clarified? The nearest one we could find was “How does the COVID-19 pandemic influence the stakeholder/engagement in your project? Likert scale: very positively <-> very negatively”. If the response options for the dependent variable were in the form of a response scale, how were responses which were not “Not at all” or “Very negatively” treated? It would also be useful if the authors could expand on why they decided to use logistic regression, and why it had to be run twice. It wasn’t quite clear how discrete choice modelling, and mentioned at the end of the methods section, was employed. Ideally more would also be said on how particular variables were selected for inclusion (i.e. one over the other), during the process of “examining in parallel the correlation matrix to test for collinearity issues among the covariates, until no significant collinearities are observed”. Thank you very much for these insightful comments/ suggestions. We made sure to elaborate more in the text on how the regression analysis was performed, so that it is more comprehensible to the reader. The main changes in response to this comment are found in Page 19 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 the Methods section. R2.3 The regression results show a large number of variables, but the article only engages superficially with these findings. It would be helpful if the regression results could be moved to the results section, and some of the key effects pulled out and described in the text. It would then be useful to see some discussion of the regression results – both in terms of limitations due to sample size, but also on the more interesting findings. For example (while significance level is low), there was a large negative effect size for policymakers and civil society organisations in the “Not at all” analysis – could be implications of this be considered? We thank the reviewer for pointing out this very important issue. We have added more results to the results section. The Tables have been moved to the Results section. We added limitations of our approach to the limitations section of the Discussion and suggested further implications for research. 2.4 Generally, the discussion is quite light on what the implications of the findings could be – both for the projects which data was captured on and going forward. Are/will important viewpoints more likely to be missed, e.g. those less able to access digital avenues for engagement? We agree with the reviewers and hence, we expanded our Discussion and now emphasise the implications of our research better. R2.5 Some more minor points: It could be useful to provide an indicative timeline of some of the main Covid-related restrictions in Europe. This will obviously vary by country, but some general info could be helpful in interpreting this work in future. We think that because of the difference timeline in the countries and given out study context, a detailed timeline is not necessary – or possible to generate with a reasonable time effort. We agree with the spirit of the comment, however, and added additional information about the containment measures in place in general during the time our survey was conducted (see Method section). R2.6 “Researchers agreed that online formats cannot replace face-to-face meetings” – is this based on a quantitative result from the survey and, if so, could this be included? Or is it just inferred from the quote? Thank you for this question. We have adapted the text in the Results section (‘Assessment of alternative stakeholder engagement formats’) to make this clearer. We did not specifically ask this question, but we got this statement as answers to open questions we asked. We asked the participants the open questions: “Do you plan to continue using any of the new formats after restrictions are lifted?  Please briefly outline which ones and why” (100% response) and “Please explain briefly in what sense COVID-19 will influence the results of your stakeholder engagement process” (79% response). Based on the answers given, we find that the majority plans to continue using online formats, but mostly in combination with physical formats, for example, “[We will continue using] online workshop, but they cannot fully replace physical meetings”; “The experience with the online workshop was good, so we will likely consider doing it again. However, this will not really replace face-to-face events. This is rather a complement to face-to-face workshops/activities”; “An online workshop CANNOT replace a face to face meeting, as not all senses are used.” Page 20 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Competing Interests: No competing interests were disclosed. Reviewer Report 07 June 2021 https://doi.org/10.21956/openreseurope.14757.r26978 © 2021 Leal Filho W et al. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Walter Leal Filho European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany Amanda Lange Salvia European School of Sustainability Science and Research, Hamburg University of Applied Sciences, Hamburg, Germany The paper aims at investigating the effects of the pandemic on energy research, particularly in European projects with planned stakeholder engagement. The rationale for the study is well described and so are the discussions and implications. However, some improvements are needed as follows: ○ The results/discussions could have been explored in a dedicated subsection or in more detail the contrasting results of the regression analysis, especially given the detailed methodology and the volume of information provided in the first tables. ○ I recommend adjusting the title or adding a note to Table 4, to make it clearer for the reader e.g. explaining that the lines refer to the original/planned activities and that the columns refer to the alternative online ones. ○ There are some light grammar/syntax errors on the text and it needs to be carefully edited.  ○ Please provide/adjust the conclusions section in a way that you may summarise the main lessons from the paper, and outline future prospects. ○ Each reference should be checked, to make sure they are all complete, with publisher, place of publication. Please check them all. Also, please cross check them as to make sure they are all on the text and in the references list.  Is the work clearly and accurately presented and does it engage with the current literature? Yes Is the study design appropriate and is the work technically sound? Page 21 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 Yes Are sufficient details of methods and analysis provided to allow replication by others? Yes Are all the source data and materials underlying the results available? Yes If applicable, is the statistical analysis and its interpretation appropriate? Yes Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Sustainability and climate change We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above. Author Response 30 Aug 2021 Diana Süsser, Institute for Advanced Sustainability Studies (IASS), Potsdam, Germany Response to Reviewer 1 R1.0 The paper aims at investigating the effects of the pandemic on energy research, particularly in European projects with planned stakeholder engagement. The rationale for the study is well described and so are the discussions and implications. We are glad to read that the reviewer values our study description, discussions, and implications. However, some improvements are needed as follows: R1.1 The results/discussions could have been explored in a dedicated subsection or in more detail the contrasting results of the regression analysis, especially given the detailed methodology and the volume of information provided in the first tables. Thank you very much for this suggestion. We have enriched the results section with some more regression analysis-relevant explanations. R1.2 I recommend adjusting the title or adding a note to Table 4, to make it clearer for the reader e.g. explaining that the lines refer to the original/planned activities and that the columns refer to the alternative online ones. We have adapted the line/column headings in Table 4 to make this clear. R1.3 There are some light grammar/syntax errors on the text and it needs to be carefully edited.  We have reviewed and proof-read the text again and corrected errors. Page 22 of 23 Open Research Europe Open Research Europe 2021, 1:57 Last updated: 14 OCT 2021 R1.4 Please provide/adjust the conclusions section in a way that you may summarise the main lessons from the paper, and outline future prospects. We have adapted the conclusion section. We now do not only summarise the main findings, but also provide some key implications. Furthermore, we outline prospects for future research to assess the more long-term impacts of the pandemic on stakeholder engagement and to identify best practices for stakeholder engagement beyond COVID-19. R1.5 Each reference should be checked, to make sure they are all complete, with publisher, place of publication. Please check them all. Also, please cross check them as to make sure they are all on the text and in the references list.  Thank you very much for this important note. We have cross-checked all the references and adapted the list accordingly. Competing Interests: No competing interests were disclosed. Page 23 of 23

Journal

Open Research EuropeUnpaywall

Published: Oct 14, 2021

There are no references for this article.