Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/taylor-francis/news-recommender-systems-a-programmatic-research-review-31uht9R4nQ
References
References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.
- Publisher
- Taylor & Francis
- Copyright
- © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
- ISSN
- 2380-8977
- eISSN
- 2380-8985
- DOI
- 10.1080/23808985.2022.2142149
- Publisher site
- See Article on Publisher Site
Abstract
ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION https://doi.org/10.1080/23808985.2022.2142149 a a b c c Eliza Mitova , Sina Blassnig , Edina Strikovic , Aleksandra Urman , Aniko Hannak , b a Claes H. de Vreese and Frank Esser a b Department of Communication and Media Research, University of Zurich, Zurich, Switzerland; Amsterdam School of Communication Research (ASCoR), University of Amsterdam, Amsterdam, The Netherlands; Department of Informatics, University of Zurich, Zurich, Switzerland ABSTRACT KEYWORDS News recommender systems; News recommender systems (NRS) are becoming a ubiquitous part of the algorithms; digital digital media landscape. Particularly in the realm of political news, the journalism; news adoption of NRS can significantly impact journalistic distribution, in turn personalisation affecting journalistic work practices and news consumption. Thus, NRS touch both the supply and demand of political news. In recent years, there has been a strong increase in research on NRS. Yet, the field remains dispersed across supply and demand research perspectives. Therefore, the contribution of this programmatic research review is threefold. First, we conduct a scoping study to review scholarly work on the journalistic supply and user demand sides. Second, we identify underexplored areas. Finally, we advance five recommendations for future research from a political communication perspective. In today’s high-choice digital media environment, the supply of information is almost infinite (Van Aelst et al., 2017). At the same time, media organisations are competing for readers’ attention and engagement, while ad-revenues are declining and the percentage of people paying for online news remains low (Newman, 2022). One potentially promising remedy for these predicaments is offered by news recommender systems (NRS): algorithmic solutions that filter, suggest, and prioritise content based on previous or similar users’ behaviour, explicitly stated user preferences, popularity metrics, and other content-specific features (Karimi et al., 2018). By considering audience members’ preferences, news organisations can help users navigate the abundance of available information, thereby fostering stronger and deeper connections with audiences over time (Vrijenhoek et al., 2021). Although the immediate benefits for user experience might be apparent, NRS carry more pro- found implications for the journalistic supply and audience demand sides of political news. On the one hand, the introduction of algorithmic solutions such as NRS into news work can alter the supply of news, in particular, journalistic distribution practices (Bastian et al., 2020; Diakopoulos, 2019). Furthermore, if the visibility of news content is primarily driven by algorithms based on user preferences and popularity metrics rather than on human judgement, this can ultimately also affect journalistic selection and creation practices (Carlson, 2018; Møller, 2022b; Napoli, 2014). On the other hand, how news is delivered and curated can also influence the demand for news, i.e. citizens’ information behaviour and, more broadly, the formation of an informed democratic citi- zenry (Moeller et al., 2016). Particularly the rise of social networking sites like Facebook and news aggregators like Google and Apple News that employ sophisticated personalisation algorithms CONTACT Eliza Mitova eliza.mitova@uzh.ch Department of Communication and Media Research, University of Zurich, Andreasstrasse 15, Zurich 8050, Switzerland © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 2 E. MITOVA ET AL. based on inferred as well as explicitly stated user preferences (Haim et al., 2018) has fuelled discus- sions about algorithms’ potential to drive selective exposure and, ultimately, the establishment of filter bubbles (Pariser, 2011) and political polarisation (Nelson & Webster, 2017; Slechten et al., 2021). These reflections show that NRS may have important democratic implications on the supply and demand sides of political news, which makes an investigation of these systems through a political communication lens crucial. A political communication perspective on NRS can illuminate the role of processes related to both the journalistic production and audiences’ acquisition of political information in shaping democratic society. The growing relevance of NRS as an object of study is reflected in the increased scholarly interest in NRS on the journalistic supply and the audience demand sides. However, scholarly work on these two sides often follows distinct theoretical and methodological approaches, whose findings have yet to be synthesised. At the same time, similar to research on other technological innovations, there is a risk of overestimating the novelty, significance, and change related to NRS (Steensen & Westlund, 2021, pp. 82–85), making the collection and synthesis of existing research even more important. Therefore, our main contribution is the provision of a programmatic review of the field to date from a political communication perspective. Such a review is necessary to lay the foundation for the integration of theoretical, methodological, and empirical insights from the journalistic supply and user demand sides. A political communication lens allows us to analyse two aspects: a) how NRS affect processes related to the journalistic representation of political reality and its dissemination in news stories, and b) how NRS affect processes involved in the acquisition of political information by the audience (see also Van Aelst et al., 2017). As NRS touch both the supply and demand of pol- itical news and thus carry important democratic implications, a political communication perspective is particularly relevant to the research field of NRS. To assess the breadth and depth of previous research addressing NRS, our review follows the methodology of the so-called ‘scoping study’ by Arksey and O’Malley (2005) (see also Levac et al., 2010). A scoping study is a specific approach to reviewing scientific evidence in which the main inter- est is to systematise literature that has developed in a short time around a new and complex phenomenon, and thus map an evolving field of research. We first clarify the concept of NRS and identify the specific research questions that guide our review. Second, we describe the procedure we followed when searching and selecting relevant studies for inclusion in this review and how we classified these studies according to larger research perspectives. Third, we describe – by way of a thematic synthesis – in more detail the extent, range, and nature of research on NRS. This allows us, in a fourth step, to delineate areas of emphasis in existing scholarship and, more impor- tantly, to identify research gaps and development opportunities. The logical fifth and final step is to make recommendations for future research. We conclude by suggesting five promising directions for news recommender research: (1) more research integrating the demand and supply sides, (2) more theory-explicated work, (3) more com- parative research, (4) more interdisciplinary work, and (5) more industry-academia collaboration. News recommender systems: an emerging research field News recommender systems (NRS) rely on algorithms that provide users with personalised rec- ommendations based on past data about interactions with news content, for example, past user or similar users’ behaviour, explicitly stated user preferences, overall popularity metrics, or other content-specific features related to articles’ actual content such as their topic, length, or presence of multimedia elements (Feng et al., 2020; Karimi et al., 2018). The most common filtering mechan- isms include content-based filtering, collaborative filtering, and hybrid approaches which combine content- and collaborative-based criteria. Content-based filtering considers characteristics of the news piece itself and its relevance to one’s interest (as determined by, e.g., a user’s consumption history), while collaborative filtering also considers what users with similar news preferences consume (Feng et al., 2020). ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 3 Recommender systems in the specific field of political news have greater social relevance than those in other areas (Karimi et al., 2018) as NRS touch both sides of the political information environ- ment (Van Aelst et al., 2017): the supply of political news by journalistic media and the demand for political information by audiences (see also Guzman, 2019). On the supply side, the introduction of algorithmic solutions such as NRS, which rely on auto- mation and machine learning (Feng et al., 2020), can challenge traditional journalistic practices (Belair-Gagnon & Holton, 2018). Regarding NRS, such challenges primarily concern the distribution of news, as NRS can supplement or even replace traditional news curation by human editors with algorithmic selection (Diakopoulos, 2019). Yet, the implementation of NRS may ultimately also affect journalistic selection and production practices. Since NRS can give some preference to certain content over other, owing, for instance, to its popularity or similarity to previously consumed stories, rather than based on human judgement, ‘the question of “what deserves attention?” at the heart of the professional judgment of journalism shifts to a different, personalized query of “what does this person want?”‘ (Carlson, 2018, p. 1765). This shift, further accelerated by the availability of metrics-oriented insight into user demand, ‘can change newsroom cultures and affect how jour- nalists understand good journalism’ (Møller, 2022b, p. 7; Carlson, 2018). As a result, technologies such as NRS may influence the amount, shape, and quality of political news provided by the media (Davis, 2013; Napoli, 2014). On the demand side, NRS may affect the amount and quality of political news people consume (Beam, 2014). If NRS rely heavily on preference matching, employing personalisation for political news could, in theory, facilitate exposure to primarily attitude-consistent content and thus restrict citizens’ opportunities to encounter counter-attitudinal information (Dylko, 2016; Moeller et al., 2016). Such insistent preference matching by recommender systems can work against the ideal of a broadly and diversely informed public in which each citizen weighs conflicting alternatives and can undermine the chances for deliberative forms of democracy (Helberger, 2019). The use of NRS is, therefore, a prototypical example of how supply and demand sides of political com- munication are intertwined. On the one hand, the supply of algorithmically curated news affects how users interact with and make sense of political content. On the other hand, news organisations ‘tailor [NRS] to a particular view of their audience and how these technologies may impact audience members’ engagement in society’ (Guzman, 2019, p. 1186). This anticipation of audience preferences and its potential impact on the supply of political news are testament to the growing interconnected- ness of supply and demand sides in high-choice media environments (see also Van Aelst et al., 2017,p.6). Despite increasing scholarly interest in NRS, there is a lack of reflection on the need to bring the two perspectives together and a lack of awareness of the additional potential for insight of such a step. To date, the related research on the supply and demand sides often follows different theoretical and methodological approaches, and the empirical findings await synthesis. Thus, to lay the foun- dation for a more integrated scholarly approach to NRS, we review and evaluate studies on both the supply and the demand sides. The research questions that guide this scoping study thus read: RQ1) What scientific evidence exists regarding the use and impact of NRS with respect to news organisations (i.e. the supply side)? RQ2) What scientific evidence exists regarding the use and impact of NRS with respect to news audiences (i.e. the demand side)? By mapping the literature around NRS, we identify underexplored areas, thereby creating an assessment that serves us as a starting point for advancing recommendations for future research on NRS which place particular emphasis on a hitherto neglected political communication perspective. Conducting the scoping study: literature search To identify relevant peer-reviewed studies for our scoping study, we first conducted manual searches of the following databases: Web of Science and EBSCO Communication & Mass Media Complete. The main keywords guiding the search were ‘news recommender systems’, ‘news recommendation’, and ‘news personalisation’. We limited our search to peer-reviewed English-language studies published 4 E. MITOVA ET AL. between 2011 and mid-2022. We deemed a 12-year time span appropriate for tracing the develop- ment of this nascent field. Another exclusion criterion was the discipline of the studies. On Web of Science, we excluded disciplines such as medicine, engineering, and natural sciences from our search. We then analysed the title and abstract of each study to determine if it relates to political news. Therefore, we included papers which investigated recommender systems on news aggregators like Google News or Apple News and social networking sites like Twitter, YouTube, and Facebook, if their title or abstract made a connection to political news. We excluded articles that were not related to the political news domain (e.g. articles dealing with Spotify or Netflix) and/or primarily dealt with the technological implementation of recommender systems. Following this relevance check, we removed duplicates. These first steps yielded 41 relevant studies, of which 34 investigated NRS on news websites, and 7 examined content recommender systems on social media or news aggregators. In a second step, following a snowball method, we examined the initial set of 41 articles in greater detail to identify additional literature (49), of which 29 explicitly dealt with NRS on news websites and 20 with the recommendation of political content on news aggregators or social media. The search query method, followed by snowballing, produced a total of 90 English-language studies. Of these 90 studies, 63 explicitly addressed NRS on news websites. The remaining 27 focused on news personalisation on intermediaries such as Google News, Apple News, Twitter, and Facebook. Of the 63 studies on NRS, 21 addressed the supply side, 24 the demand side, and 18 pursued other perspectives which are not discussed in detail in the following because they are not empirical. Attesting to the rapidly increasing interest in the topic of NRS is the fact that more than two thirds (43) of the 63 studies were published between 2019 and mid-2022. A full list of articles con- sidered in this review can be found in the Appendix. Figure 1 displays our data selection procedure in a flowchart. Part of the second procedural step of our scoping study was developing a data-charting form to classify all studies. We created an Excel matrix using common dimensions to compare relevant study characteristics, such as the methodological and theoretical approaches used, the sample, and main findings. Following Levac et al. (2010)’s guidelines for scoping studies, two members of the research team responsible for the classification of the studies independently extracted relevant data from the sample to agree on the inclusion criteria and parameters of the data-charting form. They met several times to determine whether their approach to data extraction corresponded to the purpose of the review. The research team repeatedly discussed the interpretation of the studies to ensure the val- idity and reliability of the findings. Following this, the first author adapted and completed the data- charting form. A consolidated data-charting form is an important prerequisite for reporting the outcome of a scoping review, namely collating, and summarising the studies in the form of a the- matic synthesis and identifying previous research foci and remaining research gaps. We take these steps first for supply research and then for demand research on NRS. Research evidence on the use and impact of NRS with respect to news organisations (supply side) In addressing the first research question (RQ1), we find that supply-side research can be categorised into three main areas: 1) current state of NRS implementation, 2) journalistic attitudes towards NRS and their effects on news practice and role perceptions, and 3) organisational dynamics surrounding NRS. Current state of NRS implementation Several studies have analysed the extent to which NRS are currently employed (e.g. Bastian et al., 2020; Bodó, 2019; Kunert & Thurman, 2019; Loebbecke et al., 2021; Møller, 2022a; Thurman, 2011; Thurman & Schifferes, 2012; Van den Bulck & Moe, 2018). These studies point to the increasing ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 5 Figure 1. Flowchart detailing the study selection process. use of NRS. Overall, however, current research indicates that newspapers’ use of personalisation remains limited (Loebbecke et al., 2021; Møller, 2022a). News organisations that have NRS in place employ different approaches that range from more basic solutions based on content popularity to more advanced ones based on user profiles and past behaviour (Bodó, 2019; Kunert & Thurman, 2019; Møller, 2022a). Current NRS consider implicit 6 E. MITOVA ET AL. feedback, including geolocation and reading history, as well as explicit signals, such as users’ stated topic preferences (Bastian et al., 2020; Bodó, 2019; Kunert & Thurman, 2019). Several studies docu- ment the use of explicit personalisation, wherein users can customise how and which content is dis- played to them through e.g. subscribing to specific topics (Kunert & Thurman, 2019; Loebbecke et al., 2021). Recommendations are oftentimes visible either on the front page, below articles, in sidebars, in ‘Recommended for you’ sections, or grouped together with advertising content (Kunert & Thurman, 2019; Loebbecke et al., 2021; Møller, 2022a). Research has also shown that news organis- ations do not always develop their NRS solutions in-house, but sometimes rely on third-party soft- ware solutions that combine editorial with promotional content based on users’ previous consumption (Loebbecke et al., 2021; Møller, 2022a). However, to date, studies documenting current NRS usage pertain to a small set of countries such as Germany (e.g. Loebbecke et al., 2021), the UK, (e.g. Bodó, 2019), and the US (e.g. Kunert & Thurman, 2019). Individual studies indicate that NRS strategies can vary across organisations depending on their respective business models, target audiences, and technological affinities. For example, news outlets interested in fostering loyalty and encouraging subscriptions employ algorithms that aim to recommend less popular and niche content such as local, cultural, or sports news (Bodó, 2019). Organisations that are generally more apprehensive of technology may be more reluctant to use NRS (Van den Bulck & Moe, 2018). Particularly public service media that aim to address a broad audience can be wary of advanced NRS applications (Bodó, 2019;Hildén, 2021;Sørensen, 2013, 2020; Van den Bulck & Moe, 2018). Conversely, the increased usage of targeted and person- alised advertising can drive experimentations with NRS (Couldry & Turow, 2014;Lewis &Westlund, 2015;Malcorps, 2019). News outlets’ target audiences can also affect how NRS are conceptualised (Bastian et al., 2021;Sørensen, 2013;VandenBulck &Moe, 2018). At thesametime, arecent study by Bastian et al. (2020) finds no substantial differences in front-end news personalisation between quality and tabloid media (see also Loebbecke et al., 2021;Smets et al., 2022 for similar results). However, as studies detailing the current state of NRS solutions have often focused on individual organisations (but see e.g. Bodó, 2019; Kunert & Thurman, 2019), it is difficult to draw conclusions about which factors best explain NRS adoption across different organisation types. This notwithstanding, current research indicates that organisational characteristics such as the editorial mission and business model may influence whether and how NRS are implemented. Sim- ultaneously, recent findings also imply that there might be a certain homogenisation and suggest that the close observation and imitation of other news organisations’ NRS solutions potentially play an important role (e.g. Bastian et al., 2020; Loebbecke et al., 2021). Underexplored Areas. As a comparative approach has not been at the forefront of studies yet, a set of comparable factors driving NRS usage across different organisational and national contexts has yet to be identified. Future research may, for example, simultaneously examine how macro-level factors related to the political and media system or the level of technological development and meso-level factors like media organisations’ financial models affect NRS adoption (see also Rec- ommendation 3 below). As of now, however, a) the application of NRS across different types of news organisations and contexts as well as b) the reasons underlying NRS usage or differences therein remain underexplored. Journalistic attitudes towards NRS and implications for news work This area investigates journalists’ attitudes towards NRS and how the integration of such solutions relates to prevalent editorial practices, journalistic role perceptions, and ideals. Overall, despite initial scepticism (Groot Kormelink & Costera Meijer, 2014; Linden, 2017), research on AI and data-driven technologies indicates that journalists have become more optimistic about algorithmic technologies (e.g. Schapals & Porlezza, 2020). One reason for this is the belief that such solutions can take over manual tasks and free up resources for more creative, contextual, and investigative work (Schapals & Porlezza, 2020). ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 7 Similar positive perceptions can also be observed regarding the adoption of NRS within news organisations (e.g. Bodó, 2019; Møller, 2022a). A perceived benefit of NRS lies in their potential to increase customer satisfaction and boost subscriptions (Bodó, 2019; Malcorps, 2019; Møller, 2022a; Smets et al., 2022). Thus, NRS are sometimes seen as a vehicle to alleviate potential budgetary con- straints (Bodó, 2019; Malcorps, 2019). According to news professionals, NRS can also be used to rec- ommend relevant content that users might have otherwise missed due to the rapidly changing makeup of news websites’ homepages and the high volume of new articles published every day (Bodó, 2019; Møller, 2022a). Similarly, some journalists believe that NRS can be employed to promote niche and less popular content such as local, sports, and cultural stories, thereby better addressing the interests of previously underserved audiences (Bodó, 2019; Møller, 2022a; Thurman, 2011). Research also indicates that news workers are conscious of problems relating to NRS. Such con- cerns can pertain to technological limitations and the complexity of current NRS solutions (Bastian et al., 2021; Bodó, 2019; Sørensen, 2020). Typical challenges for NRS include, for example, the short lifespan of news stories (Gulla et al., 2022; Schjøtt Hansen & Hartley, 2021), a limited content pool of articles (Bodó, 2019), data and privacy protection (Bastian et al., 2021; Sørensen, 2020), the difficulty of measuring the long-term success of such systems beyond user clicks (Bodó, 2019), or explaining NRS applications in an understandable way to users who are not necessarily technologically well- versed (Bastian et al., 2021). News professionals are also aware that news organisations might lack sufficient user and cookie data as well as metadata to deliver content recommendations that match audiences’ preferences (Gulla et al., 2022; Sørensen, 2020). Additionally, studies have shown that news workers express doubts about user demand for far- reaching news personalisation and are conscious of concerns their users might have, such as filter bubbles, lack of a user-friendly design, or privacy protection (Bodó, 2019; Møller, 2022a; Thurman, 2011; Van den Bulck & Moe, 2018). Similarly, Bastian et al. (2021) and Møller (2022a)’s findings reveal that news workers are sceptical of the extent to which core values such as journalistic auton- omy, public service, or the provision of information on important issues can be achieved through the usage of NRS. Therefore, they voice concerns over a potential dominance of commercial imperatives over journalism’s democratic mission (see also Gulla et al., 2022; Schjøtt Hansen & Hartley, 2021; Sørensen, 2020 for similar findings). Attitudes towards NRS may also vary depending on the individual characteristics of news workers. Recent research on NRS indicates that professional background and position within the organisation account for differences in expectations as to which goals NRS should pursue and the traditional jour- nalistic values to which they should adhere (e.g. Bastian et al., 2021; Smets et al., 2022). Moreover, Bastian et al. (2021) discovers that perceptions towards how NRS affect a news organisation’s repu- tation vary across news workers with different job profiles, such as editors and data scientists. To date, however, most studies on journalistic attitudes towards NRS have been qualitative and based on case-specific, non-representative samples (e.g. Bastian et al., 2021; Bodó, 2019; Møller, 2022a), which does not allow to draw generalised conclusions about what factors explain attitudinal differences at an individual level. The integration of algorithmic technologies into news work can also affect journalistic role percep- tions. For example, relying on algorithmic solutions such as NRS may lead to a re-examination of jour- nalists’ role as information gatekeepers, as they must now share their editorial autonomy with new human and non-human actors (Bastian et al., 2021; Cools et al., 2021; Milosavljević & Vobič, 2019). Nevertheless, studies have also shown that classical ideals and norms (e.g. objectivity, autonomy, and reporting on what is considered relevant) remain crucial to journalists (e.g. Bastian et al., 2021; Milosavljević & Vobič, 2019; Møller, 2022a; Sørensen, 2020). Closely connected to changes related to the fulfilment of traditional norms and values are impli- cations of NRS for news work more generally. To date, studies have primarily examined which strat- egies news organisations employ to counteract potential shortcomings of algorithmic technologies that stem from their non-journalistic origin. These involve stronger human oversight over algorithms 8 E. MITOVA ET AL. (Bastian et al., 2021; Bodó, 2019; Svensson, 2021), editorial control (Gulla et al., 2022; Møller, 2022a; Schjøtt Hansen & Hartley, 2021; Sørensen, 2020), the technological implementation of features such as transparency and diversity (Bastian et al., 2021; Bodó, 2019; Hildén, 2021; Møller, 2022a), or for- going the use of NRS for news that journalists think everyone should read (Møller, 2022a; Van den Bulck & Moe, 2018). However, except for studies on coping strategies, research has yet to examine to what extent NRS can alter work practices related to the everyday journalistic production of political news, such as story selection, framing, newswriting, and content creation. Underexplored Areas. At the time of writing, the following research gaps become apparent: only few studies have identified d) a comparable set of factors that explain journalistic attitudes towards NRS, or studied e) the impact of NRS on work practices beyond coping strategies as well as relatedly f) how the adoption of NRS affects the representation of political reality on the production side. Organisational dynamics surrounding NRS Current research on algorithmic and data-driven technologies indicates that the integration of data- driven technologies into news work can alter organisational dynamics and create new sites of struggle e.g. Belair-Gagnon et al., 2020; Hagar & Diakopoulos, 2019). Similarly, studies on NRS find that the introduction of NRS can affect decision-making processes and create tensions between members of different organisational divisions (e.g. Smets et al., 2022; Svensson, 2021). Recent research suggests that the implementation of NRS can shift power dynamics, as new tech- nological actors, such as data scientists, who are responsible for the maintenance and design of NRS, are integrated into the news organisation (Schjøtt Hansen & Hartley, 2021; Svensson, 2021). As a result, technological actors become increasingly involved in decision-making and strategy develop- ment within news organisations (Smets et al., 2022; Svensson, 2021). Consequently, editorial and managerial teams also need to consider ‘choices regarding the technical design and parameteriza- tion of recommender algorithms’ (see also Bastian et al., 2021; Bodó, 2019, p. 1063). Findings also indicate that news personalisation strategies can be shaped by marketing depart- ments (Malcorps, 2019) and user-experience researchers (Bastian et al., 2021), as they are traditionally responsible for the collection and analysis of user-related data for commercial purposes. Svensson (2021)’s recent interview study reveals tensions between editorial staff and tech actors stemming from different goals and perceptions towards the usefulness of algorithms. Similarly, Smets et al. (2022) shows that the successful implementation of NRS necessitates negotiations between different stakeholders such as editors, technologists, and managerial staff, concerning, for example, a shared understanding of what is important to users or what is technically implementable. Underexplored Areas. More than the other two research areas on the supply side, systematic research has yet to be conducted on how the adoption of NRS affects and structures organisational dynamics. Thus, two additional research gaps become evident: a closer examination of the impact of NRS on g) organisational processes and decision-making, as well as their effect on h) the relation- ships and power dynamics between members of different organisational divisions involved in the implementation of NRS. Interim conclusion Research on the supply side has investigated how and to what extent NRS are implemented in media organisations, and to a lesser extent, how NRS relate to journalistic attitudes and practices as well as the organisational dynamics and decision-making processes surrounding the adoption and mainten- ance of NRS projects. Supply-side scholarship has employed different methods such as content analyses (e.g. Bastian et al., 2020) and analyses based on browsing data (e.g. Loebbecke et al., 2021). To date, studies inves- tigating journalistic reactions and attitudes toward NRS have mostly relied on qualitative in-depth ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 9 interviews with news professionals (e.g. Bastian et al., 2021; Bodó, 2019; Møller, 2022a). However, few studies have looked at the role of technologists and managers involved in NRS projects (but see e.g. Bastian et al., 2021; Smets et al., 2022; Svensson, 2021). Regarding theoretical frameworks, several studies on the supply side have consulted classical journalism theories such as gatekeeping (e.g. Møller, 2022a). Similarly, authors like Bastian et al. (2021) and Milosavljević and Vobič (2019) have drawn on Deuze’s(2005) notion of journalism as an ideology to examine how traditional journalistic values and ideals are challenged by the adop- tion of NRS. Individual studies have also employed an institutionalist approach (e.g. Smets et al., 2022; Svensson, 2021). In some cases, NRS research builds on previous empirical knowledge, or follows a descriptive or case-study approach (Bodó, 2019; Smets et al., 2022) rather than formulating theory-driven assumptions (see also e.g. Kunert & Thurman, 2019; Loebbecke et al., 2021; Van den Bulck & Moe, 2018). Most research focuses on Western countries such as the US (e.g. Kunert & Thurman, 2019), the UK (e.g. Bodó, 2019), Germany (e.g. Loebbecke et al., 2021), the Netherlands (e.g. Bastian et al., 2021), and Belgium (e.g. Malcorps, 2019; Møller, 2022a) (but see Bastian et al., 2020; Makhortykh & Bastian, 2020 for non-Western countries). Only a handful of studies follow a comparative approach across countries (e.g. Hildén, 2021; Milosavljević & Vobič, 2019; Møller, 2022a; Van den Bulck & Moe, 2018), types of media organisations (e.g. Bodó, 2019; Gulla et al., 2022; Loebbecke et al., 2021), or time (e.g. Kunert & Thurman, 2019). More than other algorithmic technologies that primarily affect journalistic production processes, NRS are at the intersection between the supply and demand of news. As Guzman (2019) argued, research on the supply side of NRS ‘necessarily involves, or at least evokes, the audience’ (p. 1186). Therefore, research also needs to model how NRS both affect and are affected by news audiences (see also Van Aelst et al., 2017). Thus, we turn to the crucial demand side. Research evidence on the use and impact of NRS with respect to news audiences (demand side) In addressing the second research question (RQ2) of our scoping study, we find that demand-side research can be categorised into three main areas: 1) user attitudes towards algorithmic technol- ogies, 2) effects of NRS usage on political news consumption and information reception, and 3) pol- itical implications of NRS usage. User attitudes towards algorithmic technologies and NRS This research area frequently draws a connection to the concept of the ‘machine heuristic’ (Sundar & Kim, 2019). The underlying theoretical model argues that positive perceptions towards algorithmic technologies stem from implicit, albeit possibly faulty, beliefs that some of these technologies’ characteristics (e.g. their impartiality) make them fairer and less biased than humans (Sundar & Kim, 2019). Accordingly, the term algorithmic appreciation has been coined to summarise favourable attitudes towards algorithms in decision-making processes (Logg et al., 2019), whereas algorithmic aversion subsumes negative attitudes and general distrust of such technologies (Dietvorst et al., 2015). These two opposing assessments have also been used to describe user attitudes towards algorithmic decision-making in the news sector. To investigate attitudes towards algorithmic technologies in the realm of news, several studies have juxtaposed human editorial curation with algorithmic selection (e.g. Araujo et al., 2020; Thurman et al., 2019). On the one hand, these studies show that NRS are perceived to be as fair and useful as human editors (Araujo et al., 2020). On the other hand, survey results also indicate that users think NRS which recommend content based on their previous consumption are a better way to get news than either NRS that consider similar users’ behaviour, what is currently popular (Joris et al., 2021; Thurman et al., 2019), or editorial selection (Thurman et al., 2019). 10 E. MITOVA ET AL. These findings allude to users’ preference for recommendations based on content-based filtering rather than based on popularity-driven and collaborative filtering. However, as some findings reveal that users prefer content-based filtering over editorial selection (e.g. Thurman et al., 2019), whereas others suggest users perceive them to be equally useful (e.g. Araujo et al., 2020), it is, as of now, unclear whether algorithmic selection takes precedence over manual curation. Diverging user attitudes towards NRS have been traced to a multiplicity of individual character- istics such as privacy concerns (Araujo et al., 2020; Joris et al., 2021; Li & Unger, 2012; Thurman et al., 2019; but see Bodó et al., 2019 for nonsignificant effects), level of educational attainment (Gran et al., 2021; Joris et al., 2021; Thurman et al., 2019; Wieland et al., 2021), attitudes towards a shared public sphere (Bodó et al., 2019), duty to keep informed (Wieland et al., 2021), overall knowledge about algorithms (Araujo et al., 2020; Gran et al., 2021), confidence in one’s ability to judge the relevance of information (van der Velden & Loecherbach, 2021), technology optimism (Joris et al., 2021; Lim & Zhang, 2022), and, similarly, trust in algorithmic systems (Shin, 2020) and the media (Lee & Suh, 2022; Thurman et al., 2019; van der Velden & Loecherbach, 2021). General attitudes towards the news outlet can also translate to perceptions towards the outputs of its recommender system. For instance, users might assess the trustworthiness of personalised newsfeeds based on the degree of overall trust they have in the news brand (Monzer et al., 2020). However, as existing studies have each focused on a subset of these factors, it is difficult to assess their relative importance and potential interactions. Research has also analysed the extent to which users are aware of personalisation algorithms on intermediaries such as social media (e.g. Powers, 2017; Proferes, 2017; Swart, 2021), and, to a lesser extent, on news websites (e.g. Gran et al., 2021; Zarouali et al., 2021). These studies paint a mixed picture, with some pointing to low awareness of the existence of algorithms and limited understand- ing of how they filter information (Eslami et al., 2015; Gran et al., 2021; Powers, 2017; Swart, 2021; Zarouali et al., 2021), while others detect a high awareness thereof (Proferes, 2017; Rader & Gray, 2015). Additionally, research on social media has found that user knowledge of how filtering algor- ithms work and affect news feeds is often based on unproven assumptions (so-called ‘folk theories’) about algorithms’ inner workings (Bucher, 2017; DeVito et al., 2018; Swart, 2021) as well as miscon- ceptions and erroneous beliefs about algorithms’ prowess (Zarouali et al., 2021). Several studies have further investigated users’ perceived benefits (e.g. Harambam et al., 2019; Lim & Zhang, 2022; Wieland et al., 2021) and concerns (e.g. Bodó et al., 2019; Joris et al., 2021; Thurman et al., 2019; Wieland et al., 2021), as well as improvement wishes regarding NRS (e.g. Har- ambam et al., 2019; Monzer et al., 2020). A main benefit of NRS is seen in the perception that they enable users to receive only news that matches their interests, saving them time spent looking for relevant content (Harambam et al., 2019; Lim & Zhang, 2022; Shin, 2020). At the same time, users are cognisant of risks such as manipulation or stereotyping (Harambam et al., 2019; Monzer et al., 2020), lack of privacy protection (Joris et al., 2021; Thurman et al., 2019; Wieland et al., 2021), and limited exposure to counter-attitudinal or important information (Harambam et al., 2019; Li & Unger, 2012; Monzer et al., 2020; Thurman et al., 2019). Trust towards NRS is further influenced by the degree of transparency about the process of personalisation itself and the people in charge of the underlying algorithms (Monzer et al., 2020; Shin, 2020), as well as the level of exercisable user control (Groot Kormelink & Costera Meijer, 2014; Harambam et al., 2019; Lee & Suh, 2022; Monzer et al., 2020). However, a desire for more control over algorithmic recommendations does not necessarily mean that users will actively exercise their agency (Groot Kormelink & Costera Meijer, 2014; Monzer et al., 2020; see also ter Hoeve et al., 2017 for news aggregators). As of now, few studies have examined users’ perceived benefits and concerns simultaneously, making it difficult to assess whether negative or positive perceptions currently take precedence (but see Harambam et al., 2019; Monzer et al., 2020; Wieland et al., 2021). Additionally, a clear com- parison between studies examining benefits and concerns is hampered by diverging study designs, as some employ a qualitative interview approach (e.g. Harambam et al., 2019; Monzer et al., 2020), ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 11 whereas the remaining ones make use of quantitative surveys (e.g. Bodó et al., 2019; Thurman et al., 2019; Wieland et al., 2021). Underexplored Areas. Studies in this research area have analysed various aspects and influencing factors on users’ attitudes towards news personalisation, making it difficult to assess these factors’ relative importance and potential interactions between them. Moreover, few studies have examined algorithmic awareness, specifically regarding NRS on news websites. In light thereof, the following research gaps materialise at the time of writing: j) a more systematic comparison of individual factors influencing user attitudes, k) the juxtaposition of perceived benefits and concerns related to NRS, l) a closer examination of algorithmic awareness relating to news websites. Effects of NRS on political news consumption Studies have analysed how news personalisation relates to news consumption more generally. Anec- dotal findings indicate that recommender systems can drive engagement with news content (Beam & Kosicki, 2014), influence which stories users click on (Beam, 2014; Yang, 2016), and increase usage time and user satisfaction (Bryanov et al., 2020; Lee & Lee, 2022; Wieland et al., 2021). However, more important from a democratic perspective is the assumption that personalised news exposure can affect political information reception and, more generally, news consumption patterns (Moeller et al., 2016). Oftentimes, studies in this research area have aimed to investigate whether recommender systems facilitate the establishment of information cocoons (so-called ‘filter bubbles’) due to a stronger exposure to similar and attitude-consistent content (Pariser, 2011). These endeavours frequently consult the selective exposure paradigm, which has its origin in cognitive dissonance theory (Festinger, 1962). Selective exposure refers to biases in news consump- tion that stem from citizens’ tendency to select political content that is congenial to their point of view (Guess et al., 2018). This ingrained predisposition can further intensify and self-reinforce due to algorithmic filtering, whereby certain information is hidden from users because it does not match their preferences or is deemed unpopular. This, in turn, can propagate the eventual establish- ment of filter bubbles (Dylko, 2016; Fletcher & Nielsen, 2018) and undermine the establishment of a shared common core of political issues, which is vital for democracy (Moeller et al., 2016; Zuiderveen Borgesius et al., 2016). To examine the empirical validity of this theoretical assumption, studies have often examined the filtering algorithms of news aggregators and social media and their effect on political news output (e.g. Bandy & Diakopoulos, 2021; Evans et al., 2022; Haim et al., 2018). Overall, research indicates that the risk of getting stuck in such a bubble on intermediaries such as Google News (e.g. Evans et al., 2022; Haim et al., 2018; Nechushtai & Lewis, 2019), Apple News (e.g. Bandy & Diakopoulos, 2020), Facebook (e.g. Bakshy et al., 2015; Beam et al., 2018; Bechmann & Nielbo, 2018; Moeller et al., 2016; Papa & Photiadis, 2021; but see Levy, 2021 for divergent findings), or Twitter (e.g. Bandy & Dia- kopoulos, 2021; Chen et al., 2021; but see Jürgens & Stark, 2022 for divergent findings) is low and often exaggerated (see also Bruns, 2019; Fletcher et al., 2021; Möller, 2021; Zuiderveen Borgesius et al., 2016). Though relatively few studies have examined effects of recommender systems on news websites on the breadth and depth of political news consumption, their findings also largely disprove fears about filter bubbles (Beam, 2014; Heitz et al., 2022; Lunardi et al., 2020; Yang, 2016; for simulated user behaviour see Bountouridis et al., 2019; Möller et al., 2018; but also Dylko et al., 2017, 2018 for divergent findings). Scholars have, however, highlighted the importance of compar- ing different platforms and filtering mechanisms (e.g. Bountouridis et al., 2019; Haim et al., 2018; Jürgens & Stark, 2022; Möller et al., 2018). One possible explanation for the lack of empirical evidence for filter bubbles is that users them- selves do not necessarily strive to avoid attitude-incongruent information. Various studies have shown that individual choice can counteract involuntary selective exposure due to algorithmic filtering (e.g. Bakshy et al., 2015; Beam, 2014; Bechmann & Nielbo, 2018; Lee & Suh, 2022; Levy, 2021; Papa & Photiadis, 2021). Non-experimental surveys indicate that users’ news diets in digital 12 E. MITOVA ET AL. settings are relatively diverse, disproving claims about users’ ingrained tendency for selective exposure (e.g. Dubois & Blank, 2018; Fletcher & Nielsen, 2018; Guess et al., 2018). Effects on news reception can further be moderated by various technological and nontechnological factors, such as recommendations’ placement on the webpage (Yang, 2016), the level of exercisable user control (Lee & Suh, 2022), the ability to customise recommendation preferences (Beam, 2014; Dylko et al., 2018), or users’ desire to seek out balanced political information due to its higher infor- mation utility (Dylko et al., 2018; Heitz et al., 2022). To date, however, only few studies have looked specifically at the impact of recommender systems on news websites on users’ political news consumption choices (but see Beam, 2014; Dylko et al., 2017, 2018; Heitz et al., 2022; Lee & Suh, 2022; Lunardi et al., 2020; Yang, 2016). One recent exception constitutes a longitudinal experimental study by Heitz et al. (2022), which con- cludes that recommender systems on news websites can effectively nudge news consumers to engage with opposing political viewpoints if they are programmed to promote exposure to both counter-and pro-attitudinal information (see also Möller et al., 2018 for similar findings using simu- lated user behaviour). Furthermore, NRS scholarship until now has mostly modelled effects on political news reception with the selective exposure paradigm (e.g. Dylko et al., 2017; Yang, 2016). Few studies have thoroughly examined the impact of NRS on the breadth and depth of political information acqui- sition through the lens of other theories (but see e.g. Beam (2014)’s consideration of the elabor- ation-likelihood-model). Underexplored Areas. Even though the implications of recommender systems for audiences’ pol- itical news consumption have been studied to a moderate degree, they have often been investi- gated in the context of social media and news aggregators. Therefore, the following research gaps emerge: The specificeffects of NRS on m) news consumption, particularly related to news web- sites, have yet to be thoroughly explored empirically and in view of other theoretical frameworks besides the selective exposure paradigm. Moreover, studies have not fully investigated n) the longer-term impacts of NRS on the breadth and depth, for example the diversity as well as the quality, of users’ political information acquisition. Political implications of NRS usage Scholars have argued that NRS might impair political knowledge due to stronger exposure to one- sided information (Beam, 2014; Bruns, 2019; Eskens et al., 2017). However, empirical findings paint a more nuanced picture. For example, Beam (2014) finds that user-driven news customisation leads to a higher elaboration, i.e. more thoughtful processing of the news message and higher attention to its content, which, positively affects political knowledge. As users are presented with information that matches their interests and is, therefore, more familiar, they are likely to engage with the content more systematically, leading to knowledge gain (Beam, 2014). However, more recently, Heitz et al. (2022) shows that the repeated usage of a news platform that is optimised to promote exposure to both counter-and pro-attitudinal political viewpoints does not enhance the users’ perceived pol- itical knowledgeability. Several studies have also analysed the effects of algorithmically recommended content on politi- cal opinion polarisation. Dylko and colleagues’ (2018) experimental study of a custom-built news page and Cho and colleagues’ (2020) laboratory study of YouTube discover that system-driven per- sonalisation, wherein users do not explicitly customise their feeds and are mostly exposed to atti- tude-consistent information, increases attitudinal and affective polarisation (see also Levy, 2021; Ohme, 2021 for similar effects on attitude reinforcement). Conversely, Beam et al.’s panel study (2018) finds no attitude polarisation effects from Facebook use for political news but modest depolarisation effects due to increased exposure to counter-attitudinal news sources. Similarly, Heitz et al.’s experimental study (2022) shows that diversity-optimising recommender systems, ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 13 which promote exposure to both counter- and pro-attitudinal political information, can enhance tol- erance for opposing views. The effects of the reliance on algorithmically driven news platforms on political participation have rarely been studied. For example, Feezell et al. (2021) find that exposure to algorithmically curated news on Google and Facebook is associated with higher online political participation. Ohme’s(2021) panel study uncovers similar effects of exposure to political information through Facebook’s algor- ithmically curated newsfeed on political participation. In contrast, Heitz et al. (2022)’s findings suggest that the diversity-optimising news recommenders that foster the exposure to opposing pol- itical viewpoints do not enhance political participation. Overall, to date, few studies have specifically examined the effects of recommender systems on news websites on political attitudes and behaviour (but see Beam, 2014; Dylko et al., 2018; Heitz et al., 2022). Underexplored Areas. As empirical examinations of NRS and their implications for political behav- iour specifically are scarce, two additional research gaps on the demand side emerge: To date, there has only been sparse examination of the effects of NRS on o) political knowledge and p) political attitudes and behaviour. Interim conclusion Research on the demand side has mostly investigated user attitudes towards and awareness of algo- rithmic decision-making. Studies have also addressed the extent to which news personalisation affects the establishment of filter bubbles, the diversity of readers’ political news consumption, and to a lesser extent political behaviour, knowledge, and opinion polarisation. However, research has often focused on news aggregators such as Google News and social media such as Facebook rather than on recommender systems on news websites. There has been little systematic research on individual-level differences that affect attitudes towards NRS specifically and how the reliance on NRS steers subsequent news consumption, information reception, and political behaviour more generally. Existing investigations have mostly measured cross-sectional news exposure with the help of quantitative surveys (e.g. Beam & Kosicki, 2014;Bodóetal., 2019; Joris et al., 2021) or experiments (e.g. Beam, 2014;Dylko et al., 2018). In contrast to studies on the supply side which are often qualitative in nature, such an approach is seldomly employed in research on the demand side (but see e.g. Harambam et al., 2019;Monzeretal., 2020). Individual studies on NRS have employed longitudinal designs (e.g. Heitz et al., 2022; see also e.g. Beam et al., 2018 and Jürgens & Stark, 2022 for intermediaries). Recent studies have also made use of simulations and agent-based approaches, particularly in the context of news aggregators or social media (e.g. Bandy & Diakopoulos, 2020, 2021; but see Bountouridis et al., 2019;Haim, 2020;Haim et al., 2018;Möller etal., 2018 for NRS on news websites), as well as real browsing and tracking data (e.g. Levy, 2021). However, thus far, studies that aim to incorporate communication and computer science perspectives, for example, by programming NRS that are responsive to real users’ or computer simulated agents’ interaction with them, remain scarce (but see Heitz et al., 2022;Lee &Suh, 2022;Lunardi et al., 2020;Shin, 2020; for a simulation approach Bountouridis et al., 2019; Möller et al., 2018). Demand-side studies have thus far focused on a handful of Western countries, including the Neth- erlands (e.g. Bodó et al., 2019), Germany (e.g. Monzer et al., 2020; Wieland et al., 2021) and the US (e.g. Beam, 2014; Dylko et al., 2017) (but see e.g. Lee & Suh, 2022; Thurman et al., 2019; Yang, 2016 for non-Western samples). To date, demand-side studies have rarely employed a comparative approach across countries (but see e.g. Thurman et al., 2019; van der Velden & Loecherbach, 2021), time (but see e.g. Feezell et al., 2021; Jürgens & Stark, 2022; and Heitz et al., 2022 for NRS), types of platforms (but see e.g. Jürgens & Stark, 2022), or filtering mechanisms (but see e.g. Bountouridis et al., 2019; Möller et al., 2018). 14 E. MITOVA ET AL. Lastly, studies that explore the effects of NRS on news consumption, political polarisation, and knowledge have largely employed the selective exposure paradigm. Several studies have drawn on previously established empirical work rather than formulated theory-explicated assumptions (e.g. Monzer et al., 2020; Wieland et al., 2021). Other theories that could potentially model and explain effects on political news use and behaviour have seldomly been considered (but see e.g. Beam, 2014). Conclusions: recommendations for news recommender research Having summarised the literature on NRS, we are now able to draw conclusions about (1) the extent and range of scholarly literature on NRS; (2) the issues NRS-related research has focused on so far on the supply and demand sides; and (3) the diversity of NRS research in terms of methodology, theory, interdisciplinarity, and geographic scope. Our first conclusion is that while a significant and steadily growing amount of scholarly work on NRS has been assembled in the past years, the theoretical, methodological, and empirical insights from supply and demand perspectives have yet to be synthesised. Such an encapsulated view on NRS can contribute to a fragmented research field. Thus, to help researchers see relevant connec- tions in this emerging research field, we pursued an integrative approach. Drawing on political com- munication as an overarching framework and considering the potential significance of NRS for an informed democratic public, we aimed to lay the groundwork for a more holistic, systematic approach to NRS which considers the implications of NRS for the production, dissemination, and consumption of political information. Our second conclusion about previously well-studied topics allows us to answer our two research questions. RQ1 asked, ‘What scientific evidence exists regarding the use and impact of NRS with respect to news organisations?’ Here, our review revealed that research can be categorised into three broader areas: 1) the current state of NRS implementation in news organisations; 2) journalistic attitudes towards NRS and their effects on news practices and role perceptions; and 3) organisational dynamics and hierarchies surrounding NRS. However, there are still significant research gaps in all three areas. Although research on the actual implementation of NRS has recently increased, we still know little about how NRS adoption varies across different types of news organisations or political and media systems. On a micro level, research, as it is often case-specific and relies on qualitative data, has yet to systematically examine what factors influence journalistic attitudes towards NRS. Additionally, research has yet to thoroughly analyse whether the use of NRS has longer-term impacts on what political stories are selected and curated on news pages or how the adoption of NRS affects the rep- resentation of political reality on the production side. On a meso level, still little is known about how the inclusion of technological actors who are responsible for the maintenance of NRS shape organ- isational dynamics. Even though supply-side studies do not always examine the adoption of NRS in the political news domain in detail, our review indicates that such systems may have important impli- cations for news work, organisational practices, and the realisation of values such as public service and autonomy which, in turn, can carry over to the representation of political reality and its disse- mination in news content. RQ2 asked, ‘What scientific evidence exists regarding the use and impact of NRS with respect to news audiences?’ Here, our review revealed that research can be categorised into three areas: 1) user attitudes towards NRS; 2) the effects of NRS usage on political news consumption; and 3) the political implications of NRS usage. Again, there are still significant research gaps in all three areas. Although research has explored user attitudes toward NRS, the examination of a multiplicity of relevant factors using different study designs has hampered the clear identification of individual factors influencing user attitudes toward NRS. Furthermore, the relative importance and juxtaposi- tion of perceived benefits and concerns as well as algorithmic awareness, specifically regarding NRS on news websites, still need to be analysed more systematically. The effects of personalisation ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 15 on political news consumption and information reception – particularly regarding the potential emergence of filter bubbles – and implications for political behaviour have often been analysed in the context of social media and news aggregators. Thus, there is a lack of research specifically investigating NRS on news websites, their effects on the breadth and depth of political information reception, and their democratic implications for political outcomes, such as knowledge, attitudes, and participation. Our third conclusion relates to the diversity of NRS research in terms of methodology, theory, interdisciplinarity, and geographic scope. Our review showed that NRS-related research has employed a wide range of quantitative and qualitative methods. Overall, research on the supply side has most often used a qualitative approach (e.g. Bastian et al., 2021; Bodó, 2019; Møller, 2022a), whereas studies on the demand side have, for the most part, quantitively examined user atti- tudes and effects on news consumption (e.g. Bodó et al., 2019; Heitz et al., 2022; Joris et al., 2021). Regarding the extent of theory-explicated work, NRS research on the supply side has often drawn on classical journalism or institutionalist theories. On the demand side, studies have predominantly employed the selective exposure paradigm. However, we also found studies that have drawn on pre- viously established empirical work rather than on theory-explicated assumptions. In terms of geographic scope, research on both the supply and demand sides is often case- specific and limited to a small number of Western countries, such as the US, Germany, or the Netherlands. Taken together, our evaluation of the existing literature reveals that NRS are an increasingly popular research topic that can be studied through various methodological and theoretical lenses. Yet, despite, or perhaps because of, the rapidly increasing scholarly interest, the field remains divided along the fault lines of supply and demand-sides perspectives. Our review uncov- ered important research gaps that affirm that more research on NRS specifically is needed. Addition- ally, to aid future research in closing the identified political news-related research gaps, we would like to conclude our scoping study with five recommendations covering five aspects: the integration of the supply and demand sides, theory-explicated work, comparative research, interdisciplinarity, and industry-academia collaboration. Recommendation 1: more research integrating the supply and demand sides In today’s high-choice information environment, the demand and supply sides of news are inextric- ably linked (Van Aelst et al., 2017). This is particularly true for NRS as their outputs both affect and are necessarily affected by user behaviour (Guzman, 2019; Abdollahpouri et al., 2020). As audiences’ engagement with such systems steer their output, they can, in turn, also affect journalistic distri- bution practices and, ultimately, possibly decisions regarding future story selection and creation (Bastian et al., 2021; Carlson, 2018; Møller, 2022b). Therefore, similarly to political communication research in general (Van Aelst et al., 2017), future NRS scholarship should aim to consider the inter- connectedness between the supply and demand sides. One growing research strain that aims to do so theoretically analyses how NRS should be pro- grammed to satisfy normative and democratically relevant journalistic criteria (e.g. Harambam et al., 2019; Helberger, 2019; Helberger et al., 2018; Vrijenhoek et al., 2021). Such theoretical scholarly endeavours offer suggestions for how the supply of NRS should ideally be configured by taking into consideration users’ demands as well as democratically deduced journalistic ideals. Specifically, scho- lars have argued that NRS should expose users to diverse information regarding topics, sources, and viewpoints to ensure an informed democratic citizenry (Harambam et al., 2018; Helberger, 2019; Möller et al., 2018). What is expected of an algorithm, however, differs according to the normative perspective and democratic model to which one adheres (Helberger et al., 2018; Vermeulen, 2022). Thus, the practical implementation of concepts such as diversity must be defined with a view of the requirements each democratic paradigm imposes on its media and citizens (Helberger, 2019). 16 E. MITOVA ET AL. Transparency, accountability and explicability of the employed algorithms have also mostly been discussed theoretically (e.g. Bastian et al., 2019; Descampe et al., 2022; Diakopoulos & Koliska, 2017; Helberger et al., 2018; Leerssen, 2020; Milano et al., 2020; Møller, 2022b). Studies havefurther pointed to the importance of enabling user agency through features such as customisability and the ability to turn off algorithmic recommendations (Monzer et al., 2020; Vermeulen, 2022). Additionally, individ- ual studies have developed news recommender systems which can be calibrated with different democratically relevant features in mind (e.g. Descampe et al., 2022; Loecherbach & Trilling, 2020; Vrijenhoek et al., 2021). However, research has yet to empirically investigate the extent to which these considerations are present in current NRS design and how the realisation of concepts such as diversity, transparency, and user control (or, conversely, the lack thereof) affects user demand (but see e.g. Heitz et al., 2022; Lee & Suh, 2022). Such an empirical approach is crucial to study the interdependence between the supply and demand sides, as well as possible discrepancies in user expectations and actual NRS design. Additionally, studies should consider the reciprocal feedback loops that result from user signals and the outputs generated by NRS (Bodó et al., 2019; Guzman, 2019). Simulations, specifically agent- based approaches, may prove especially fruitful to capture these feedback loops for several reasons (Gilbert & Troitzsch, 2005). First, by emulating human behaviour, simulations can capture the dynamic effects of algorithmic recommender systems on recommendation outcomes and sub- sequent user engagement with the recommended content (Haim, 2020; Möller et al., 2018). Relat- edly, they also allow researchers to predict how such systems evolve in the long term, even if empirical data on their filtering mechanisms or user engagement with such systems is not available yet (Bountouridis et al., 2019; see also Bokányi & Hannák, 2020). Thus, in the case of NRS, simulations can help model how specific filtering mechanisms and algorithmic interventions affect both the diversity of the recommended news content as well as the diversity of users’ subsequent news reading behaviour (see also Recommendation 4 below). Recommendation 2: more theory-explicated work Our review revealed that research on NRS has consulted different theoretical frameworks which focus on various micro and, to a lesser extent, meso-level or macro-level factors. However, our engagement with the literature also indicated that scholarship on the supply side has, in some cases,relied on a descriptive or a case-study approach. On the demand side, several studies follow an inductive, empirical rather than a deductive, theory-driven approach (see also Ahva & Steensen, 2020; Steensen & Westlund, 2021 for a similar interpretation of existing digital communication research). To model the effects on journalistic work, news consumption, as well as political attitudes and subsequent behaviour, future studies can profit from a stronger consideration of theories of journalism, media effects, and political psychology. On the supply side, besides classical journalism theories such as journalistic role theory (Hanitzsch, 2007), Deuze’s(2005) work on journalism as an ideology (for an empirical investigation, see e.g. Bastian et al., 2021; Milosavljević & Vobič, 2019), and gatekeeping theory (Shoemaker & Vos, 2009; for an empirical examination see e.g. Møller, 2022a), additional theoretical frameworks can also adequately model how the introduction of NRS affects communication flows and journalistic prac- tices. As Soffer (2021) argues, classical media theories, such as the two-step flow of mass communi- cation (Katz & Lazarsfeld, 2017), can also be adapted to model the role that personalisation algorithms play in digital environments. Alluding to algorithms’ contribution in personalising users’ news experience, Soffer (2021) delineates how algorithms, like opinion leaders who tradition- ally disseminated mass media content, ‘can compensate for the lack of interpersonal conversation in the mass media era’ (p. 304). Another potentially fruitful framework might be institutional theory (Napoli, 2014;Scott et al., 1994) or ,more prominently, new institutionalism (Powell & DiMaggio, 2012;Ryfe, 2006), as it can ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 17 shed light on the processes of legitimisation, imitation, and homogenisation of NRS usage within and across organisations (Caplan & Boyd, 2018). Studies have already discovered imitation pro- cesses in the implementation of data-driven solutions, such as audience analytics across different types of media organisations (e.g. Christin, 2017;Zamithetal., 2020; for NRS see Smets et al., 2022). Other branches of institutional theory that focus on endogenous factors rather than on environmental pressures, such as the institutional logics perspective (Thornton et al., 2012; Greenwood et al., 2017), can also uncover emerging tensions between organisational div- isions that adhere to different commercial and societal values but are jointly involved in the implementation and maintenance of NRS (for an empirical investigation see e.g. Svensson, 2021). To account for the multiplicity of actors who take part in the design of NRS, scholars have also argued in favour of a multistakeholder approach, which considers the interplay between the various stakeholders on the demand and supply sides who affect or are affected by the delivery of recommendations to users (Abdollahpouri et al., 2020;Smets et al., 2022). To capture in a comprehensive way the cognitive thought and deliberation processes underlying the usage and effects of NRS on the demand side, i.e. for audiences, scholars could consult additional theories besides selective exposure. Media-effects theories, such as uses and gratifications theory (UGT) (Sundar & Limperos, 2013), can be used to model both user expectations towards NRS and perceived gratifications of personalised news consumption (for an empirical investigation see e.g. Lee & Suh, 2022; van der Velden & Loecherbach, 2021). Broader psychology theories, such as the elaboration likelihood model (Petty & Cacioppo, 1986) or the theory of motivated reasoning (Taber & Lodge, 2006), might also be employedto capture the effects of NRS on phenomena such as the breadth and depth of political information reception or political decision-making (Dylko, 2016). For example, Beam (2014)’s experimental study draws on the elaboration-likelihood-model to explain why readers are more likely to process news content that isrecommended to them based on their political preferences more deeply than attitude-inconsistent messages. Similarly, the theory of motivated reasoning can be consulted to examine the motivational influences guiding political information consumption in personalised news environments, owing, for instance, to individuals’ inclination to denigrate arguments that challenge their beliefs or to their political sophistication and knowledge (see also Dylko, 2016; p. 397; Heitz et al., 2022; Strickland et al., 2011, p. 937). These theoretical frameworks can further be enriched with approaches that model attitudes towards algorithmic technologies, such as the Modality-Agency-Interactivity-Navigability (MAIN) model (Sundar, 2008; Sundar & Kim, 2019), which examines the role of different mental shortcuts such as the machine heuristic in decisions about the credibility of machine-driven communication (for an empirical investigation see e.g. Tandoc et al., 2020). Other potentially fruitful theories include the technology acceptance model (Venkatesh & Davis, 2000), which predicts acceptance or rejection of algorithmic technologies based on factors such as their perceived ease of use (for an empirical investigation see e.g. Lim & Zhang, 2022), or the newly developed algorithmic persuasion framework, which models the persuasive effects of algorithm- mediated communication on beliefs, attitudes, and behaviour (Zarouali et al., 2021). Recommendation 3: more comparative research Our review indicates that research on NRS has so far focused mainly on a few Western countries, with most empirical investigations being single-country studies – adeficit often found in political com- munication research (Van Aelst et al., 2017). However, contextual factors of different political infor- mation environments, such as the media and political systems, legal frameworks, or overall technological development, may influence the implementation of NRS and its effects on news use and user behaviour. Therefore, future research on NRS should more often pursue a comparative approach. 18 E. MITOVA ET AL. Especially in political and media systems that are less stable or non-democratic, NRS could poten- tially have important repercussions for news consumption and political knowledge if they are biased towards certain political opinions or if news portals are not transparent about their data practices (Bastian et al., 2020; Makhortykh & Wijermars, 2021). This highlights the need for more de-Wester- nized work on digital technologies such as NRS (Makhortykh & Bastian, 2020). In addition, few studies on the supply side have investigated the implementation of NRS across different types of news organisations (but see e.g. Bodó, 2019; Kunert & Thurman, 2019). Comparing public service and private mediaor quality and tabloid media, might uncover important differences in NRS usage owing to divergent financial resources, target audiences, and overall journalistic missions. Recommendation 4: more interdisciplinary work The era of big data has undeniably changed the way we study social science phenomena (Bonenfant & Meurs, 2020). Consequently, communication and computer science now share more research interests than ever before. This has inspired various calls for stronger interdisciplinary collaboration between them (Bonenfant & Meurs, 2020). The study of NRS is a perfect example of a research field that is of great interest to both disciplines and can thus particularly benefit from a stronger interdis- ciplinary approach. To date, however, there have been few attempts at an integration of the different perspectives (but see e.g. Helberger et al., 2018; Loecherbach & Trilling, 2020; Vrijenhoek et al., 2021). Future NRS research should continue to focus on the technological advancement of algorithmic solutions (e.g. Descampe et al., 2022; Raza & Ding, 2021). However, it should also be mindful of the special characteristics and normative dimensions of political news media and what these imply for the technical implementation of NRS. To this end, computer science can inform communication researchers about innovative and tech- nologically advanced methodological approaches to NRS research. Simulations can help model aggregate effects that are difficult to capture in the field with real user data (Haim, 2020). Algorithmic audits can uncover important information about the filtering mechanisms employed by current NRS, to which communication scholars would not otherwise be privy (Bandy & Diakopoulos, 2020). Additionally, computer scientists can shed light on the technical limitations and impact of current implementation efforts towards transparency, privacy, usability, or user-friendly platform interfaces and offer suggestions on how best to overcome them (e.g. Descampe et al., 2022). Vice versa, communication scholars can enrich computer science research on NRS with insights from media effects and journalism theories on the role and impact of NRS for journalism and audi- ences. Moreover, political communication scientists can point to democratically and normatively rel- evant criteria, such as transparency and diversity, which need to be implemented in NRS design to ensure an informed democratic citizenry. From a democratic perspective, the consideration of nor- mative and democratically relevant NRS features is particularly important (Helberger, 2019). In addition, interdisciplinary collaboration with law scholars can be fruitful, as examples have proven (e.g. Eskens, 2019; Helberger et al., 2018; Krebs et al., 2019; Vermeulen, 2022). Such collabor- ations, on the one hand, allow scholars to account for the implications of legal frameworks for the implementation and use of NRS. On the other hand, it seems crucial that insights from social science and computer science inform the regulation and governance of NRS. Recommendation 5: more industry-academia collaboration Many researchers have noted the importance and advantages of industry-academia collaboration in media and communication research (Freeman, 2016; Grubenmann, 2016; Jensen, 2020). NRS research can particularly benefit from such partnerships. To understand NRS’ effect on news workers and audiences, systematic knowledge of the underlying algorithms is crucial. To date, however, most research on news personalisation has relied on incomplete information, agent- ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 19 based testing, and anecdotal evidence about the inner workings of recommender systems. This is partly due to research focusing on intermediaries owned by international tech companies, whose personalisation algorithms constitute trade secrets (DeVito, 2017; Thorson et al., 2021). Through collaborations with news organisations, extensive testing of existing NRS can be con- ducted using, amongst other approaches, simulations based on real audience data (e.g. Möller et al., 2018). This can inform researchers about the real-life, possibly long-term implications of NRS for news consumption, user satisfaction, and political information behaviour. Moreover, insights from the newsroom can inform researchers about the current state of NRS implementation as well as news organisations’ hopes and reservations related to the future of news personalisation. This knowledge can be used to inspire scholarly investigations of important, hitherto unexplored phenomena. Findings can subsequently be relayed to news organisations in such a way that supports them in future NRS endeavours. Thus, collaborations between academia and the news industry can prove equally beneficial to both sides and create pathways to industry impacts with democratic ramifications. Afterthought NRS can carry important implications for journalism, audiences, and democracy. The societal rel- evance of such algorithmic systems in the domain of political news is further reflected in the increas- ing amount of scholarly work on the topic. As the number of studies analysing NRS grows, however, so does the risk of the research field becoming fragmented across demand and supply perspectives. As Van Aelst et al. (2017)argue, ‘a full understanding of political information environments needstotakenot only the supplysidebut alsothedemandside into account’ (p. 4). This is par- ticularly true for NRS on news websites because their outputs affect and are affected by anticipated as well as actual user behaviour (Guzman, 2019). Thus, if scholarship is to fully understand the implications of algorithmically driven technologies that can play an incremental role in the disse- mination and acquisition of political information, future research endeavours need to be mindful of phenomena both on the supply and demand sides. To lay the groundwork for such a holistic approach to NRS, we used our scoping study to present a programmatic research review on the topic of news recommender systems. Our goal was threefold: In a first step, we reviewed the lit- erature on NRS on the supply and demand sides that has accumulated in the past twelve years, paying particular attention to the political news domain. In doing so, we were able to synthesise the key focus areas to date as well as identify conflicting empirical evidence and underexplored areas on both sides. Following this evaluation, we advanced five recommendations on how future NRS research can continue to evolve from a political communication perspective. We believe that our review can provide an important impetus for the maturing of NRS research and hope that this contribution will help scholars find inspiration and orientation in this exciting, rapidly growing field of research. Notes 1. In political communication research, the demand side traditionally ‘encompasses how various segments within a society make use of political news and information’ (Van Aelst et al., 2017, p. 4). Demand for and subsequently use of specific content or digital services can in turn affect audience behaviour. For example, in their article, Van Aelst et al. (2017) jointly examine ‘the demand for and the effects of polarized news’ (p. 13, emphasis in original). Therefore, in the following, we also subsume effects of NRS on political attitudes and political information behaviour under the demand side. 2. Although news personalisation does not necessarily equate with NRS, as recommendations constitute only one possible form of personalisation, some studies use these terms interchangeably (e.g. Bastian et al., 2020). 3. Additionally, a search of the relevant databases for studies prior to 2011 revealed only a limited number of studies that matched our eligibility criteria. Upon closer inspection, these studies examined less technologically 20 E. MITOVA ET AL. advanced NRS solutions (e.g. Knobloch-Westerwick et al., 2005), which we believe cannot be treated as equiva- lents to more sophisticated NRS as examined in research published between 2011 and 2022. 4. Of the 18 ‘other’ studies, 15 had a theoretical or normative focus, and three (3) were literary reviews with a com- puter science focus. It is further noteworthy that of the 24 demand side studies, six (6) measured or simulated users’ interaction with self-programmed NRS, thereby at least partially accounting for the connection between the supply and demand sides. Disclosure statement No potential conflict of interest was reported by the author(s). Funding This work was supported by Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung: [Grant Number 407740_197523]. ORCID Eliza Mitova https://orcid.org/0000-0002-4463-0042 Sina Blassnig http://orcid.org/0000-0002-7815-0186 Claes H. de Vreese http://orcid.org/0000-0002-4962-1698 Frank Esser http://orcid.org/0000-0002-1627-1521 References Abdollahpouri, H., Adomavicius, G., Burke, R., Guy, I., Jannach, D., Kamishima, T., Krasnodebski, J., & Pizzato, L. (2020). Multistakeholder recommendation: Survey and research directions. User Modeling and User-Adapted Interaction, 30 (1), 127–158. https://doi.org/10.1007/s11257-019-09256-1 Ahva, L., & Steensen, S. (2020). Theory in journalism studies. In K. Wahl-Jorgensen, & T. Hanitzsch (Eds.), The handbook of journalism studies (pp. 71–103). Routledge. Araujo, T., Helberger, N., Kruikemeier, S., & Vreese, C. H. (2020). In AI we trust? Perceptions about automated decision- making by artificial intelligence. AI & SOCIETY, 35(6), 611–623. https://doi.org/10.1007/s00146-019-00931-w Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616 Bakshy, E., Messing, S., & Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130–1132. https://doi.org/10.1126/science.aaa1160 Bandy, J., & Diakopoulos, N. (2020). Auditing news curation systems: A case study examining algorithmic and editorial logic in apple news. Proceedings of the International AAAI Conference on Web and Social Media, 14, 36–47. Bandy, J., & Diakopoulos, N. (2021). More accounts, fewer links: How algorithmic curation impacts media exposure in Twitter timelines. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1–28. https://doi.org/10. 1145/3449152 Bastian, M., Helberger, N., & Makhortykh, M. (2021). Safeguarding the journalistic DNA: Attitudes towards the role of professional values in algorithmic news recommender designs. Digital Journalism, 9(6), 1–29. https://doi.org/10. 1080/21670811.2021.1912622 Bastian, M., Makhortykh, M., & Dobber, T. (2019). News personalization for peace: How algorithmic recommendations can impact conflict coverage. International Journal of Conflict Management, 309–328. https://doi.org/10.1108/ IJCMA-02-2019-0032 Bastian, M., Makhortykh, M., Harambam, J., & van Drunen, M. (2020). Explanations of news personalisation across countries and media types. Internet Policy Review, 9(4), 1–34. https://doi.org/10.14763/2020.4.1504 Beam, M. A. (2014). Automating the news. Communication Research, 41(8), 1019–1041. https://doi.org/10.1177/ Beam, M. A., Hutchens, M. J., & Hmielowski, J. D. (2018). Facebook news and (de)polarization: Reinforcing spirals in the 2016 US election. Information, Communication & Society, 21(7), 940–958. https://doi.org/10.1080/1369118X.2018.1444783 Beam, M. A., & Kosicki, G. M. (2014). Personalized news portals. Journalism & Mass Communication Quarterly, 91(1), 59–77. https://doi.org/10.1177/1077699013514411 Bechmann, A., & Nielbo, K. L. (2018). Are we exposed to the same “news” in the news feed? Digital Journalism, 6(8), 990– 1002. https://doi.org/10.1080/21670811.2018.1510741 ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 21 Belair-Gagnon, V., & Holton, A. E. (2018). Strangers to the game? Interlopers, intralopers, and shifting news production. Media and Communication, 6(4), 70–78. https://doi.org/10.17645/mac.v6i4.1490 Belair-Gagnon, V., Zamith, R., & Holton, A. E. (2020). Role orientations and audience metrics in newsrooms: An examin- ation of journalistic perceptions and their drivers. Digital Journalism, 8(3), 347–366. https://doi.org/10.1080/ 21670811.2019.1709521 Bodó, B. (2019). Selling news to audiences – A qualitative inquiry into the emerging logics of algorithmic news perso- nalization in European quality news media. Digital Journalism, 7(8), 1054–1075. https://doi.org/10.1080/21670811. 2019.1624185 Bodó, B., Helberger, N., Eskens, S., & Möller, J. (2019). Interested in diversity. Digital Journalism, 7(2), 206–229. https://doi. org/10.1080/21670811.2018.1521292 Bokányi, E., & Hannák, A. (2020). Understanding inequalities in ride-hailing services through simulations. Scientific Reports, 10(1), 1–11. https://doi.org/10.1038/s41598-020-63171-9 Bonenfant, M., & Meurs, M.-J. (2020). Collaboration between social sciences and computer science: Toward a cross-dis- ciplinary methodology for studying big social data from online communities. In J. Hunsinger, M. M. Allen, & L. Klastrup (Eds.), Second international handbook of internet research (pp. 47–63). Springer. Bountouridis, D., Harambam, J., Makhortykh, M., Marrero, M., Tintarev, N., & Hauff,C. (2019). Siren: A simulation frame- work for understanding the effects of recommender systems in online news environments. Proceedings of the confer- ence on fairness, accountability, and transparency, 150–159. Bruns, A. (2019). Filter bubble. Internet Policy Review, 8(4), 1–14. https://doi.org/10.14763/2019.4.1426 Bryanov, K., Watson, B. K., Pingree, R. J., & Santia, M. (2020). Effects of partisan personalization in a news portal exper- iment. Public Opinion Quarterly, 84(S1), 216–235. https://doi.org/10.1093/poq/nfaa011 Bucher, T. (2017). The algorithmic imaginary: Exploring the ordinary affects of Facebook algorithms. Information, Communication & Society, 20(1), 30–44. https://doi.org/10.1080/1369118X.2016.1154086 Caplan, R., & Boyd, D. (2018). Isomorphism through algorithms: Institutional dependencies in the case of Facebook. Big Data & Society, 5(1), 205395171875725–12. https://doi.org/10.1177/2053951718757253 Carlson, M. (2018). Automating judgment? Algorithmic judgment, news knowledge, and journalistic professionalism. New Media & Society, 20(5), 1755–1772. http://dx.doi.org/10.1177/1461444817706684 Chen, W., Pacheco, D., Yang, K.-C., & Menczer, F. (2021). Anomalous collapses of Nares Strait ice arches leads to enhanced export of Arctic sea ice. Nature Communications, 12(1), 1–10. https://doi.org/10.1038/s41467-020-20314-w Cho, J., Ahmed, S., Hilbert, M., Liu, B., & Luu, J. (2020). Do search algorithms endanger democracy? An experimental investigation of algorithm effects on political polarization. Journal of Broadcasting & Electronic Media, 64(2), 150– 172. https://doi.org/10.1080/08838151.2020.1757365 Christin, A. (2017). Algorithms in practice: Comparing web journalism and criminal justice. Big Data & Society, 4(2), 205395171771885–14. https://doi.org/10.1177/2053951717718855 Cools, H., Van Gorp, B., & Opgenhaffen, M. (2021). When algorithms recommend what’s new(s): New dynamics of decision-making and autonomy in newsgathering. Media and Communication, 9(4), 198–207. https://doi.org/10. 17645/mac.v9i4.4173 Couldry, N., & Turow, J. (2014). Advertising, big data, and the clearance of the public realm. Marketers’ New Approaches to the Content Subsidy, 8, 1710–1726. Davis, A. (2013). The impact of market forces, new technologies and political PR on UK journalism. In R. Kuhn, & R. K. Nielsen (Eds.), Political journalism in transition. Western Europe in a comparative perspective (pp. 111–128). I.B. Tauris. Descampe, A., Massart, C., Poelman, S., Standaert, F.-X., & Standaert, O. (2022). Automated news recommendation in front of adversarial examples and the technical limits of transparency in algorithmic accountability. AI & SOCIETY, 37(1), 67–80. https://doi.org/10.1007/s00146-021-01159-3 Deuze, M. (2005). What is journalism? Journalism, 6(4), 442–464. https://doi.org/10.1177/1464884905056815 DeVito, M. A. (2017). From editors to algorithms. Digital Journalism, 5(6), 753–773. https://doi.org/10.1080/21670811. 2016.1178592 DeVito, M. A., Birnholtz, J., Hancock, J. T., French, M., & Liu, S. (2018). How people form folk theories of social media feeds and what it means for how we study self-presentation. Proceedings of the 2018 CHI conference on human factors in computing systems, 1–12. Diakopoulos, N. (2019). Automating the news. Harvard University Press. Diakopoulos, N., & Koliska, M. (2017). Algorithmic transparency in the news media. Digital Journalism, 5(7), 809–828. https://doi.org/10.1080/21670811.2016.1208053 Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. https://doi.org/10.1037/xge0000033 Dubois, E., & Blank, G. (2018). The echo chamber is overstated: The moderating effect of political interest and diverse media. Information, Communication & Society, 21(5), 729–745. https://doi.org/10.1080/1369118X.2018.1428656 Dylko, I., Dolgov, I., Hoffman, W., Eckhart, N., Molina, M., & Aaziz, O. (2017). The dark side of technology: An experimental investigation of the influence of customizability technology on online political selective exposure. Computers in Human Behavior, 73, 181–190. https://doi.org/10.1016/j.chb.2017.03.031 22 E. MITOVA ET AL. Dylko, I., Dolgov, I., Hoffman, W., Eckhart, N., Molina, M., & Aaziz, O. (2018). Impact of customizability technology on pol- itical polarization. Journal of Information Technology & Politics, 15(1), 19–33. https://doi.org/10.1080/19331681.2017. Dylko, I. B. (2016). How technology encourages political selective exposure. Communication Theory, 26(4), 389–409. https://doi.org/10.1111/comt.12089 Eskens, S. (2019). A right to reset your user profile and more: GDPR-rights for personalized news consumers. International Data Privacy Law, 9(3), 153–172. https://doi.org/10.1093/idpl/ipz007 Eskens, S., Helberger, N., & Moeller, J. (2017). Challenged by news personalisation: Five perspectives on the right to receive information. Journal of Media Law, 9(2), 259–284. https://doi.org/10.1080/17577632.2017.1387353 Eslami, M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K., Hamilton, K., & Sandvig, C. (2015). I always assumed that I wasn’t really that close to [her]” Reasoning about Invisible Algorithms in News Feeds. Proceedings of the 33rd annual ACM conference on human factors in computing systems, 153–162. Evans, R., Jackson, D., & Murphy, J. (2022). Google News and machine gatekeepers: Algorithmic personalisation and news diversity in online news search. Digital Journalism,1–19. https://doi.org/10.1080/21670811.2022.2055596 Feezell, J. T., Wagner, J. K., & Conroy, M. (2021). Exploring the effects of algorithm-driven news sources on political behavior and polarization. Computers in Human Behavior, 116, 106626–11. https://doi.org/10.1016/j.chb.2020.106626 Feng, C., Khan, M., Rahman, A. U., & Ahmad, A. (2020). News recommendation systems - accomplishments, challenges & future directions. IEEE Access, 8, 16702–16725. https://doi.org/10.1109/ACCESS.2020.2967792 Festinger, L. (1962). Cognitive dissonance. Scientific American, 207(4), 93–106. https://doi.org/10.1038/ scientificamerican1062-93 Fletcher, R., Kalogeropoulos, A., & Nielsen, R. K. (2021). More diverse, more politically varied: How social media, search engines and aggregators shape news repertoires in the United Kingdom. New Media & Society,1–22. https://doi.org/ 10.1177/146144482110273 Fletcher, R., & Nielsen, R. K. (2018). Automated serendipity. Digital Journalism, 6(8), 976–989. https://doi.org/10.1080/ 21670811.2018.1502045 Freeman, M. (2016). Industrial approaches to media: A methodological gateway to industry studies. Springer. Gilbert, N., & Troitzsch, K. (2005). Simulation for the social scientist. McGraw-Hill Education (UK). Gran, A.-B., Booth, P., & Bucher, T. (2021). To be or not to be algorithm aware: A question of a new digital divide? Information, Communication & Society, 24(12), 1779–1796. https://doi.org/10.1080/1369118X.2020.1736124 Greenwood, R., Oliver, C., Lawrence, T. B., & Meyer, R. E. (2017). The Sage handbook of organizational institutionalism. Sage. Groot Kormelink, T., & Costera Meijer, I. (2014). Tailor-Made news. Journalism Studies, 15(5), 632–641. https://doi.org/10. 1080/1461670X.2014.894367 Grubenmann, S. (2016). Action research. Digital Journalism, 4(1), 160–176. https://doi.org/10.1080/21670811.2015. Guess, A., Nyhan, B., Lyons, B., & Reifler, J. (2018). Avoiding the echo chamber about echo chambers. Knight Foundation, 2,1–25. Gulla, J., Svendsen, R., Zhang, L., Stenbom, A., & Frøland, J. (2022). Recommending news in traditional media companies. AI Magazine, 42(3), 55–69. https://doi.org/10.1609/aimag.v42i3.18146 Guzman, A. L. (2019). Prioritizing the audience’s view of automation in journalism. Digital Journalism, 7(8), 1185–1190. https://doi.org/10.1080/21670811.2019.1681902 Hagar, N., & Diakopoulos, N. (2019). Optimizing content with A/B headline testing: Changing newsroom practices. Media and Communication, 7(1), 117–127. https://doi.org/10.17645/mac.v7i1.1801 Haim, M. (2020). Agent-based testing: An automated approach toward artificial reactions to human behavior. Journalism Studies, 21(7), 895–911. https://doi.org/10.1080/1461670X.2019.1702892 Haim, M., Graefe, A., & Brosius, H.-B. (2018). Burst of the filter bubble? Digital Journalism, 6(3), 330–343. https://doi.org/ 10.1080/21670811.2017.1338145 Hanitzsch, T. (2007). Deconstructing journalism culture: Toward a universal theory. Communication Theory, 17(4), 367– 385. https://doi.org/10.1111/j.1468-2885.2007.00303.x Harambam, J., Bountouridis, D., Makhortykh, M., & van Hoboken, J. (2019). Designing for the better by taking users into account. Proceedings of the 13th ACM conference on recommender systems, 69–77. Harambam, J., Helberger, N., & van Hoboken, J. (2018). Democratizing algorithmic news recommenders: How to materi- alize voice in a technologically saturated media ecosystem. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 376(2133), 1–21. http://dx.doi.org/10.1098/rsta.2018.0088 Heitz, L., Lischka, J. A., Birrer, A., Paudel, B., Tolmeijer, S., Laugwitz, L., & Bernstein, A. (2022). Benefits of diverse news recommendations for democracy: A user study. Digital Journalism,1–21. https://doi.org/10.1080/21670811.2021. Helberger, N. (2019). On the democratic role of news recommenders. Digital Journalism, 7(8), 993–1012. https://doi.org/ 10.1080/21670811.2019.1623700 Helberger, N., Karppinen, K., & D’Acunto, L. (2018). Exposure diversity as a design principle for recommender systems. Information, Communication & Society, 21(2), 191–207. https://doi.org/10.1080/1369118X.2016.1271900 ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 23 Hildén, J. (2021). The public service approach to recommender systems: Filtering to cultivate. Television & New Media, 23 (7), 1–20. https://doi.org/10.1177/152747642110201 Jensen, K. B. (2020). A handbook of media and communication research: Qualitative and quantitative methodologies. Routledge. Joris, G., Grove, F. D., Van Damme, K., & De Marez, L. (2021). Appreciating news algorithms: Examining audiences’ per- ceptions to different news selection mechanisms. Digital Journalism, 9(5), 1–30. https://doi.org/10.1080/21670811. 2021.1912626 Jürgens, P., & Stark, B. (2022). Mapping exposure diversity: The divergent effects of algorithmic curation on news con- sumption. Journal of Communication, 72(3), 322–344. https://doi.org/10.1093/joc/jqac009 Karimi, M., Jannach, D., & Jugovac, M. (2018). News recommender systems – Survey and roads ahead. Information Processing & Management, 54(6), 1203–1227. https://doi.org/10.1016/j.ipm.2018.04.008 Katz, E., & Lazarsfeld, P. F. (2017). Personal influence: The part played by people in the flow of mass communications. Routledge. Knobloch-Westerwick, S., Sharma, N., Hansen, D. L., & Alter, S. (2005). Impact of popularity indications on readers’ selec- tive exposure to online news. Journal of Broadcasting & Electronic Media, 49(3), 296–313. https://doi.org/10.1207/ s15506878jobem4903_3 Krebs, L. M., Alvarado Rodriguez, O. L., Dewitte, P., Ausloos, J., Geerts, D., Naudts, L., & Verbert, K. (2019). Tell me what you know: GDPR implications on designing transparency and accountability for news recommender systems. Extended abstracts of the 2019 CHI conference on human factors in computing systems, 1–6. Kunert, J., & Thurman, N. (2019). The form of content personalisation at mainstream, transatlantic news outlets: 2010– 2016. Journalism Practice, 13(7), 759–780. https://doi.org/10.1080/17512786.2019.1567271 Lee, S., & Suh, K.-S. (2022). Deliberate news consumption through the quantified self and the self-regulatory process. Information & Management, 59(2), 1–12. https://doi.org/10.1016/j.im.2021.103585 Lee, S. Y., & Lee, S. W. (2022). Normative or effective? The role of news diversity and trust in news recommendation ser- vices. International Journal of Human–Computer Interaction,1–14. https://doi.org/10.1080/10447318.2022.2057116 Leerssen, P. (2020). The soap box as a black box: Regulating transparency in social media recommender systems. European Journal of Law and Technology, 11(2), 1–51. http://dx.doi.org/10.2139/ssrn.3544009 Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5 (1), 1–9. https://doi.org/10.1186/1748-5908-5-69 Levy, R. (2021). Social media, news consumption, and polarization: Evidence from a field experiment. American Economic Review, 111(3), 831–870. https://doi.org/10.1257/aer.20191777 Lewis, S. C., & Westlund, O. (2015). Big data and journalism. Digital Journalism, 3(3), 447–466. https://doi.org/10.1080/ 21670811.2014.976418 Li, T., & Unger, T. (2012). Willing to pay for quality personalization? Trade-off between quality and privacy. European Journal of Information Systems, 21(6), 621–642. https://doi.org/10.1057/ejis.2012.13 Lim, J. S., & Zhang, J. (2022). Adoption of AI-driven personalization in digital news platforms: An integrative model of technology acceptance and perceived contingency. Technology in Society, 69(2022), 1–10. https://doi.org/10.1016/j. techsoc.2022.101965 Linden, C.-G. (2017). Decades of automation in the newsroom. Digital Journalism, 5(2), 123–140. https://doi.org/10.1080/ 21670811.2016.1160791 Loebbecke, C., Oberschulte, F., & Boboschko, I. (2021). Mass media deploying digital personalization: An empirical inves- tigation. International Journal on Media Management, 23(3–4), 176–203. https://doi.org/10.1080/14241277.2022. Loecherbach, F., & Trilling, D. (2020). 3bij3 – Developing a framework for researching recommender systems and their effects. Computational Communication Research, 2(1), 53–79. https://doi.org/10.5117/CCR2020.1.003.LOEC Logg, J. M., Minson, J. A., & Moore, D. A. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151,90–103. https://doi.org/10.1016/j.obhdp.2018.12.005 Lunardi, G. M., Machado, G. M., Maran, V., & de Oliveira, J. P. M. (2020). A metric for filter bubble measurement in recom- mender algorithms considering the news domain. Applied Soft Computing, 97(A), 1–12. https://doi.org/10.1016/j. asoc.2020.106771 Makhortykh, M., & Bastian, M. (2020). Personalizing the war: Perspectives for the adoption of news recommendation algorithms in the media coverage of the conflict in Eastern Ukraine. Media, War & Conflict, 15(1-21), 25–45. https://doi.org/10.1177/1750635220906254 Makhortykh, M., & Wijermars, M. (2021). Can filter bubbles protect information freedom? Discussions of algorithmic news recommenders in Eastern Europe. Digital Journalism,1–25. https://doi.org/10.1080/21670811.2021. Malcorps, S. (2019). News website personalisation: The co-creation of content, audiences and services by online journal- ists and marketers. Journal of Media Business Studies, 16(3), 230–247. https://doi.org/10.1080/16522354.2019.1689766 Milano, S., Taddeo, M., & Floridi, L. (2020). Recommender systems and their ethical challenges. AI & SOCIETY, 35(4), 957– 967. https://doi.org/10.1007/s00146-020-00950-y 24 E. MITOVA ET AL. Milosavljević, M., & Vobič,I.(2019). Human still in the loop. Digital Journalism, 7(8), 1098–1116. https://doi.org/10.1080/ 21670811.2019.1601576 Moeller, J., Trilling, D., Helberger, N., Irion, K., & Vreese, C. (2016). Shrinking core? Exploring the differential agenda setting power of traditional and personalized news media. Info, 18(6), 26–41. https://doi.org/10.1108/info-05- 2016-0020 Möller, J. (2021). Filter bubbles and digital echo chambers. In H. Tumber, & S. Waisboard (Eds.), The routledge companion to media disinformation and populism (pp. 92–100). Routledge. Möller, J., Trilling, D., Helberger, N., & van Es, B. (2018). Do not blame it on the algorithm: An empirical assessment of multiple recommender systems and their impact on content diversity. Information, Communication & Society, 21(7), 959–977. https://doi.org/10.1080/1369118X.2018.1444076 Møller, L. A. (2022a). Recommended for you: How newspapers normalise algorithmic news recommendation to fit their gatekeeping role. Journalism Studies, 23(7), 800–817. https://doi.org/10.1080/1461670X.2022.2034522 Møller, L. A. (2022b). Between personal and public interest: How algorithmic news recommendation reconciles with journalism as an ideology. Digital Journalism,1–19. https://doi.org/10.1080/21670811.2022.2032782 Monzer, C., Moeller, J., Helberger, N., & Eskens, S. (2020). User perspectives on the news personalisation process: Agency, trust and utility as building blocks. Digital Journalism, 8(9), 1142–1162. https://doi.org/10.1080/21670811.2020. Napoli, P. M. (2014). Automated media: An institutional theory perspective on algorithmic media production and con- sumption. Communication Theory, 24(3), 340–360. https://doi.org/10.1111/comt.12039 Nechushtai, E., & Lewis, S. C. (2019). What kind of news gatekeepers do we want machines to be? Filter bubbles, frag- mentation, and the normative dimensions of algorithmic recommendations. Computers in Human Behavior, 90, 298– 307. https://doi.org/10.1016/j.chb.2018.07.043 Nelson, J. L., & Webster, J. G. (2017). The myth of partisan selective exposure: A portrait of the online political news audi- ence. Social Media+ Society, 3(3), 1–13. https://doi.org/10.1177/2056305117729314. Newman, N. (2022, June 15). Overview and key findings of the 2022 Digital News Report. Reuters Institute for the Study of Journalism. https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2022/dnr-executive-summary. Ohme, J. (2021). Algorithmic social media use and its relationship to attitude reinforcement and issue-specific political participation – The case of the 2015 European immigration movements. Journal of Information Technology & Politics, 18(1), 36–54. https://doi.org/10.1080/19331681.2020.1805085 Papa, V., & Photiadis, T. (2021). Algorithmic curation and users’ civic attitudes: A study on Facebook news feed results. Information, 12(522), 1–13. https://doi.org/10.3390/info12120522 Pariser, E. (2011). The filter bubble: How the new personalized web is changing what we read and how we think. Penguin. Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. In R. E. Petty & J. T. Cacioppo (Eds.), Communication and persuasion (pp. 1–24). NY: Springer. Powell, W. W., & DiMaggio, P. J. (2012). The new institutionalism in organizational analysis. University of Chicago press. Powers, E. (2017). My news feed is filtered? Digital Journalism, 5(10), 1315–1335. https://doi.org/10.1080/21670811.2017. Proferes, N. (2017). Information flow solipsism in an exploratory study of beliefs about Twitter. Social Media+ Society, 3 (1), 1–17. https://doi.org/10.1177/2056305117698493 Rader, E., & Gray, R. (2015). Understanding user beliefs about algorithmic curation in the Facebook news feed. Proceedings of the 33rd annual ACM conference on human factors in computing systems, 173–182. Raza, S., & Ding, C. (2021). News Recommender System: A review of recent progress, challenges, and opportunities. Artificial Intelligence Review, 55, 749–800. https://doi.org/10.1007/s10462-021-10043-x Ryfe, D. M. (2006). Guest editor’s introduction: New institutionalism and the news. Political Communication, 23(2), 135– 144. https://doi.org/10.1080/10584600600728109 Schapals, A. K., & Porlezza, C. (2020). Assistance or resistance? Evaluating the intersection of automated journalism and journalistic role conceptions. Media and Communication, 8(3), 16–26. http://dx.doi.org/10.17645/mac.v8i3.3054 Schjøtt Hansen, A., & Hartley, J. M. (2021). Designing what’s news: An ethnography of a personalization algorithm and the data-driven (Re)Assembling of the news. Digital Journalism,1–19. https://doi.org/10.1080/21670811.2021. Scott, R. W., Scott, W. R., & Meyer, J. W. (1994). Institutional environments and organizations: Structural complexity and individualism. Sage. Shin, D. (2020). How do users interact with algorithm recommender systems? The interaction of users, algorithms, and performance. Computers in Human Behavior, 109, 106344. https://doi.org/10.1016/j.chb.2020.106344 Shoemaker, P. J., & Vos, T. (2009). Gatekeeping theory. Routledge. Slechten, L., Courtois, C., Coenen, L., & Zaman, B. (2021). Adapting the selective exposure perspective to algorithmically governed platforms: The case of Google Search. Communication Research, 0(0), 1–27. https://doi.org/10.1177/ Smets, A., Hendrickx, J., & Ballon, P. (2022). We’re in this together: A multi-stakeholder approach for news recommen- ders. Digital Journalism,1–19. https://doi.org/10.1080/21670811.2021.2024079 ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 25 Soffer, O. (2021). Algorithmic personalization and the two-step flow of communication. Communication Theory, 31(3), 297–315. https://doi.org/10.1093/ct/qtz008 Sørensen, J. K. (2013). PSB goes personal: The failure of personalised PSB web pages. MedieKultur: Journal of Media and Communication Research, 29(55), 43–71. https://doi.org/10.7146/mediekultur.v29i55.7993 Sørensen, J. K. (2020). The datafication of public service media. MedieKultur: Journal of Media and Communication Research, 36(69), 090–115. https://doi.org/10.7146/mediekultur.v36i69.121180 Steensen, S., & Westlund, O. (2021). What is digital journalism studies? Taylor & Francis. Strickland, A. A., Taber, C. S., & Lodge, M. (2011). Motivated reasoning and public opinion. Journal of Health Politics, Policy and Law, 36(6), 935–944. https://doi.org/10.1215/03616878-1460524 Sundar, S. S. (2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. MacArthur Foundation Digital Media and Learning Initiative Cambridge, MA. Sundar, S. S., & Kim, J. (2019). Machine heuristic: When we trust computers more than humans with our personal infor- mation. Proceedings of the 2019 CHI conference on human factors in computing systems, 1–9. Sundar, S. S., & Limperos, A. M. (2013). Uses and grats 2.0: New gratifications for new media. Journal of Broadcasting & Electronic Media, 57(4), 504–525. https://doi.org/10.1080/08838151.2013.845827 Svensson, J. (2021). Logics, tensions and negotiations in the everyday life of a news-ranking algorithm. Journalism, 0(0), 1–18. https://doi.org/10.1177/14648849211063373 Swart, J. (2021). Experiencing algorithms: How young people understand, feel about, and engage with algorithmic news selection on social media. Social Media+ Society, 7(2), 1–11. https://doi.org/10.1177/20563051211008828 Taber, C. S., & Lodge, M. (2006). Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science, 50(3), 755–769. https://doi.org/10.1111/j.1540-5907.2006.00214.x Tandoc, E. C., Yao, L. J., & Wu, S. (2020). Man vs. machine? The impact of algorithm authorship on news credibility. Digital Journalism, 8(4), 548–562. https://doi.org/10.1080/21670811.2020.1762102 ter Hoeve, M., Heruer, M., Odijk, D., Schuth, A., & de Rijke, M. (2017). Do news consumers want explanations for person- alized news rankings. Fatrec workshop on responsible recommendation proceedings. Thornton, P. H., Ocasio, W., & Lounsbury, M. (2012). The institutional logics perspective: A new approach to culture, struc- ture and process. OUP Oxford. Thorson, K., Cotter, K., Medeiros, M., & Pak, C. (2021). Algorithmic inference, political interest, and exposure to news and politics on Facebook. Information, Communication & Society, 24(2), 183–200. https://doi.org/10.1080/1369118X.2019. Thurman, N. (2011). Making ‘The Daily Me’: Technology, economics and habit in the mainstream assimilation of person- alized news. Journalism, 12(4), 395–415. https://doi.org/10.1177/1464884910388228 Thurman, N., Moeller, J., Helberger, N., & Trilling, D. (2019). My friends, editors, algorithms, and I. Digital Journalism, 7(4), 447–469. https://doi.org/10.1080/21670811.2018.1493936 Thurman, N., & Schifferes, S. (2012). The future of personalization at news websites: Lessons from a longitudinal study. Journalism Studies, 13(5–6), 775–790. https://doi.org/10.1080/1461670X.2012.664341 Van Aelst, P., Strömbäck, J., Aalberg, T., Esser, F., de Vreese, C. H., Matthes, J., Hopmann, D. N., Salgado, S., Hubé, N., Stępińska, A., Papathanassopoulos, S., Berganza, R., Legnante, G., Reinemann, C., Sheafer, T., & Stanyer, J. (2017). Political communication in a high-choice media environment: a challenge for democracy? Annals of the International Communication Association, 41(1), 3–27. https://doi.org/10.1080/23808985.2017.1288551 Van den Bulck, H., & Moe, H. (2018). Public service media, universality and personalisation through algorithms: Mapping strategies and exploring dilemmas. Media, Culture & Society, 40(6), 875–892. https://doi.org/10.1177/ van der Velden, M., & Loecherbach, F. (2021). Epistemic overconfidence in algorithmic news selection. Media and Communication, 9(4), 182–197. https://doi.org/10.17645/mac.v9i4.4167 Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926 Vermeulen, J. (2022). To nudge or not to nudge: News recommendation as a tool to achieve online media pluralism. Digital Journalism,1–20. https://doi.org/10.1080/21670811.2022.2026796 Vrijenhoek, S., Kaya, M., Metoui, N., Möller, J., Odijk, D., & Helberger, N. (2021). Recommenders with a mission: Assessing diversity in news recommendations. Proceedings of the 2021 conference on human information interaction and retrie- val, 173–183. Wieland, M., von Nordheim, G., & Kleinen-von Königslöw, K. (2021). One recommender fits All? An exploration of user satisfaction with text-based news recommender systems. Media and Communication, 9(4), 208–221. https://doi.org/ 10.17645/mac.v9i4.4241 Yang, J. (2016). Effects of popularity-based news recommendations (“most-viewed”) on users’ exposure to online news. Media Psychology, 19(2), 243–271. https://doi.org/10.1080/15213269.2015.1006333 Zamith, R., Belair-Gagnon, V., & Lewis, S. C. (2020). Constructing audience quantification: Social influences and the devel- opment of norms about audience analytics and metrics. New Media & Society, 22(10), 1763–1784. https://doi.org/10. 1177/1461444819881735 26 E. MITOVA ET AL. Zarouali, B., Helberger, N., & de Vreese, C. H. (2021). Investigating algorithmic misconceptions in a media context: Source of a new digital divide? Media and Communication, 9(4), 134–144. https://doi.org/10.17645/mac.v9i4.4090 Zuiderveen Borgesius, F. J., Trilling, D., Möller, J., Bodó, B., Vreese, C. H., Helberger, N., & Internet Policy Review. (2016). Should we worry about filter bubbles? Internet Policy Review, 5(1), 1–16. https://doi.org/10.14763/2016.1.401 Appendix List of NRS studies included in the synthesis The included studies are categorised into supply-side studies; demand-side studies; studies examin- ing the interaction between supply and demand; normative/theoretical papers; reviews with a com- puter science focus; and studies on political news personalisation on intermediaries Supply-side studies 1. Bastian, M., Helberger, N., & Makhortykh and Wijermars (2021). Safeguarding the Journalistic DNA: Attitudes towards the Role of Professional Values in Algorithmic News Recommender Designs. Digital Journalism,1–29. 2. Bastian, M., Makhortykh, M., Harambam, J., & van Drunen, M. (2020). Explanations of news per- sonalisation across countries and media types. Internet Policy Review, 9(4), 1–34. 3. Bodó, B. (2019). Selling News to Audiences – A Qualitative Inquiry into the Emerging Logics of Algorithmic News Personalization in European Quality News Media. Digital Journalism, 7(8), 1054–1075. 4. Cools, H., Van Gorp, B., & Opgenhaffen, M. (2021). When Algorithms Recommend What’s New (s): New Dynamics of Decision-Making and Autonomy in Newsgathering. Media and Communi- cation, 9(4), 198–207. 5. Couldry, N., & Turow, J. (2014). Advertising, Big Data, and the Clearance of the Public Realm. Mar- keters’ New Approaches to the Content Subsidy, 8, 1710–1726. 6. Diakopoulos, N., & Koliska, M. (2017). Algorithmic Transparency in the News Media. Digital Jour- nalism, 5(7), 809–828. 7. Gulla, J., Svendsen, R., Zhang, L., Stenbom, A., & Frøland, J. (2021). Recommending news in tra- ditional media companies. AI Magazine, 42(3), 55–69. 8. Hildén, J. (2021). The Public Service Approach to Recommender Systems: Filtering to Cultivate. Television & New Media,1–20. 9. Kunert, J., & Thurman, N. (2019). The form of content personalisation at mainstream, transatlantic news outlets: 2010–2016. Journalism Practice, 13(7), 759–780. 10. Loebbecke, C., Oberschulte, F., & Boboschko, I. (2021). Mass Media Deploying Digital Personali- zation: An Empirical Investigation. International Journal on Media Management, 23(3–4), 176– 11. Malcorps, S. (2019). News website personalisation: The co-creation of content, audiences and services by online journalists and marketers. Journal of Media Business Studies, 16(3), 230–247. 12. Milosavljević, M., & Vobič, I. (2019). Human Still in the Loop. Digital Journalism, 7(8), 1098–1116. 13. Møller, L. A. (2022a). Recommended for You: How Newspapers Normalise Algorithmic News Rec- ommendation to Fit Their Gatekeeping Role. Journalism Studies,1–18. 14. Schjøtt Hansen, A., & Hartley, J. M. (2021). Designing What’s News: An Ethnography of a Perso- nalization Algorithm and the Data-Driven (Re) Assembling of the News. Digital Journalism,1–19. 15. Smets, A., Hendrickx, J., & Ballon, P. (2022). We’re in This Together: A Multi-Stakeholder Approach for News Recommenders. Digital Journalism,1–19. 16. Sørensen, J. K. (2013). PSB goes personal: The failure of personalised PSB web pages. MedieKultur: Journal of Media and Communication Research, 29(55), 43–71. 17. Sørensen, J. K. (2020). The datafication of public service media dreams, dilemmas and practical problems: A case study of the implementation of personalized recommendations at the Danish ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 27 public service media ‘DR.’ MedieKultur: Journal of Media and Communication Research, 36(69), 90–115. 18. Svensson, J. (2021). Logics, tensions and negotiations in the everyday life of a news-ranking algorithm. Journalism,1–18. 19. Thurman, N. (2011). Making ‘The Daily Me’: Technology, economics and habit in the mainstream assimilation of personalized news. Journalism, 12(4), 395–415. 20. Thurman, N., & Schifferes, S. (2012). The future of personalization at news websites: Lessons from a longitudinal study. Journalism Studies, 13(5–6), 775–790. 21. Van den Bulck, H., & Moe, H. (2018). Public service media, universality and personalisation through algorithms: Mapping strategies and exploring dilemmas. Media, Culture & Society, 40 (6), 875–892. Demand-side studies 1. Araujo, T., Helberger, N., Kruikemeier, S., & Vreese, C. H. (2020). In AI we trust? Perceptions about automated decision-making by artificial intelligence. AI & SOCIETY, 21(6), 611–623. 2. Beam, M. A. (2014). Automating the News: How Personalized News Recommender System Design Choices Impact News Reception. Communication Research, 41(8), 1019–1041. 3. Beam, M. A., & Kosicki, G. M. (2014). Personalized news portals: Filtering systems and increased news exposure. Journalism & Mass Communication Quarterly, 91(1), 59–77. 4. Bodó, B., Helberger, N., Eskens, S., & Möller, J. (2019). Interested in Diversity. Digital Journalism, 7 (2), 206–229. 5. Dylko, I., Dolgov, I., Hoffman, W., Eckhart, N., Molina, M., & Aaziz, O. (2017). The dark side of tech- nology: An experimental investigation of the influence of customizability technology on online political selective exposure. Computers in Human Behavior, 73, 181–190. 6. Dylko, I., Dolgov, I., Hoffman, W., Eckhart, N., Molina, M., & Aaziz, O. (2018). Impact of customiz- ability technology on political polarization. Journal of Information Technology & Politics, 15(1), 19–33. 7. Gran, A.-B., Booth, P., & Bucher, T. (2021). To be or not to be algorithm aware: A question of a new digital divide? Information, Communication & Society, 24(12), 1779–1796. 8. Groot Kormelink, T., & Costera Meijer, I. (2014). Tailor-Made News: Meeting the demands of news users on mobile and social media. Journalism Studies, 15(5), 632–641. 9. Harambam, J., Bountouridis, D., Makhortykh, M., & van Hoboken, J. (2019). Designing for the better by taking users into account. Proceedings of the 13th ACM Conference on Recommender Systems,69–77. 10. Joris, G., Grove, F. D., Van Damme, K., & De Marez, L. (2021). Appreciating News Algorithms: Examining Audiences’ Perceptions to Different News Selection Mechanisms. Digital Journalism, 1–30. 11. Li, T., & Unger, T. (2012). Willing to pay for quality personalization? Trade-off between quality and privacy. European Journal of Information Systems, 21(6), 621–642. 12. Lim, J. S., & Zhang, J. (2022). Adoption of AI-driven personalization in digital news platforms: An integrative model of technology acceptance and perceived contingency. Technology in Society, 69,1–10. 13. Monzer, C., Moeller, J., Helberger, N., & Eskens, S. (2020). User Perspectives on the News Perso- nalisation Process: Agency, Trust and Utility as Building Blocks. Digital Journalism, 8(9), 1142– 14. Thurman, N., Moeller, J., Helberger, N., & Trilling, D. (2019). My Friends, Editors, Algorithms, and I. Digital Journalism, 7(4), 447–469. 15. van der Velden, M., & Loecherbach, F. (2021). Epistemic overconfidence in algorithmic news selection. Media and Communication, 9(4), 182–197. 28 E. MITOVA ET AL. 16. Wieland, M., von Nordheim, G., & Kleinen-von Königslöw, K. (2021). One Recommender Fits All? An Exploration of User Satisfaction With Text-Based News Recommender Systems. Media and Communication, 9(4), 208–221. 17. Yang, J. (2016). Effects of popularity-based news recommendations (‘most-viewed’) on users’ exposure to online news. Media Psychology, 19(2), 243–271. 18. Zarouali, B., Helberger, N., & de Vreese, C. H. (2021). Investigating Algorithmic Misconceptions in a Media Context: Source of a New Digital Divide? Media and Communication, 9(4), 134–144. Studies investigating the interaction between supply and demand sides 1. Bountouridis, D., Harambam, J., Makhortykh, M., Marrero, M., Tintarev, N., & Hauff, C. (2019). Siren: A simulation framework for understanding the effects of recommender systems in online news environments. Proceedings of the Conference on Fairness, Accountability, and Transparency, 150– 2. Heitz, L., Lischka, J. A., Birrer, A., Paudel, B., Tolmeijer, S., Laugwitz, L., & Bernstein, A. (2022). Benefits of Diverse News Recommendations for Democracy: A User Study. Digital Journalism, 1–21. 3. Lee, S., & Suh, K.-S. (2022). Deliberate news consumption through the quantified self and the self- regulatory process. Information & Management, 59(2), 1–12. 4. Lunardi, G. M., Machado, G. M., Maran, V., & de Oliveira, J. P. M. (2020). A metric for Filter Bubble measurement in recommender algorithms considering the news domain. Applied Soft Computing, 97(A), 1–12. 5. Möller, J., Trilling, D., Helberger, N., & van Es, B. (2018). Do not blame it on the algorithm: An empirical assessment of multiple recommender systems and their impact on content diversity. Information, Communication & Society, 21(7), 959–977. 6. Shin, D. (2020). How do users interact with algorithm recommender systems? The interaction of users, algorithms, and performance. Computers in Human Behavior, 109, 106344. Theoretical papers 1. Bastian, M., Makhortykh, M., & Dobber, T. (2019). News personalization for peace: How algorith- mic recommendations can impact conflict coverage. International Journal of Conflict Manage- ment, 30(3), 309-328. 2. Descampe, A., Massart, C., Poelman, S., Standaert, F.-X., & Standaert, O. (2022). Automated news recommendation in front of adversarial examples and the technical limits of transparency in algorithmic accountability. AI & SOCIETY, 37(1), 67–80. 3. Dylko, I. B. (2016). How technology encourages political selective exposure. Communication Theory, 26(4), 389–409. 4. Harambam, J., Helberger, N., & van Hoboken, J. (2018). Democratizing algorithmic news recom- menders: How to materialize voice in a technologically saturated media ecosystem. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 376(2133). 5. Helberger, N. (2019). On the Democratic Role of News Recommenders. Digital Journalism, 7(8), 993–1012. 6. Helberger, N., Karppinen, K., & D’Acunto, L. (2018). Exposure diversity as a design principle for recommender systems. Information, Communication & Society, 21(2), 191–207. 7. Krebs, L. M., Alvarado Rodriguez, O. L., Dewitte, P., Ausloos, J., Geerts, D., Naudts, L., & Verbert, K. (2019). Tell me what you know: GDPR implications on designing transparency and ANNALS OF THE INTERNATIONAL COMMUNICATION ASSOCIATION 29 accountability for news recommender systems. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems,1–6. 8. Loecherbach, F., & Trilling, D. (2020). 3bij3 - Developing a framework for researching recommen- der systems and their effects. Computational Communication Research, 2(1), 53–79. 9. Makhortykh, M., & Bastian, M. (2020). Personalizing the war: Perspectives for the adoption of news recommendation algorithms in the media coverage of the conflict in Eastern Ukraine. Media, War & Conflict, 15(1), 1–21. 10. Makhortykh, M., & Wijermars, M. (2021). Can Filter Bubbles Protect Information Freedom? Discus- sions of Algorithmic News Recommenders in Eastern Europe. Digital Journalism,1–25. 11. Milano, S., Taddeo, M., & Floridi, L. (2020). Recommender systems and their ethical challenges. AI & SOCIETY, 35(4), 957–967. 12. Møller, L. A. (2022b). Between Personal and Public Interest: How Algorithmic News Recommen- dation Reconciles with Journalism as an Ideology. Digital Journalism,1–19. 13. Soffer, O. (2021). Algorithmic personalization and the two-step flow of communication. Com- munication Theory, 31(3), 297–315. 14. Vermeulen, J. (2022). To Nudge or Not to Nudge: News Recommendation as a Tool to Achieve Online Media Pluralism. Digital Journalism,1–20. 15. Vrijenhoek, S., Kaya, M., Metoui, N., Möller, J., Odijk, D., & Helberger, N. (2021). Recommenders with a mission: Assessing diversity in news recommendations. Proceedings of the 2021 Confer- ence on Human Information Interaction and Retrieval, 173–183. Literary reviews with a computer science focus 1. Feng, C., Khan, M., Rahman, A. U., & Ahmad, A. (2020). News recommendation systems-accom- plishments, challenges & future directions. IEEE Access, 8, 16702–16725. 2. Karimi, M., Jannach, D., & Jugovac, M. (2018). News recommender systems – Survey and roads ahead. Information Processing & Management, 54(6), 1203–1227. 3. Raza, S., & Ding, C. (2021). News Recommender System: A review of recent progress, challenges, and opportunities. Artificial Intelligence Review 55, 749–800. . Studies investigating political news personalisation on intermediaries 1. Bakshy, E., Messing, S., & Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130–1132. 2. Bandy, J., & Diakopoulos, N. (2020). Auditing News Curation Systems: A Case Study Examining Algorithmic and Editorial Logic in Apple News. Proceedings of the International AAAI Conference on Web and Social Media, 14,36–47. 3. Bandy, J., & Diakopoulos, N. (2021). More Accounts, Fewer Links: How Algorithmic Curation Impacts Media Exposure in Twitter Timelines. Proceedings of the ACM on Human-Computer Inter- action, 5(CSCW1), 1–28. 4. Beam, M. A., Hutchens, M. J., & Hmielowski, J. D. (2018). Facebook news and (de) polarization: Reinforcing spirals in the 2016 US election. Information, Communication & Society, 21(7), 940– 5. Bechmann, A., & Nielbo, K. L. (2018). Are we exposed to the same ‘news’ in the news feed? An empirical analysis of filter bubbles as information similarity for Danish Facebook users. Digital Journalism, 6(8), 990–1002. 6. Bryanov, K., Watson, B. K., Pingree, R. J., & Santia, M. (2020). Effects of Partisan Personalization in a News Portal Experiment. Public Opinion Quarterly, 84(S1), 216–235. 30 E. MITOVA ET AL. 7. Bucher, T. (2017). The algorithmic imaginary: Exploring the ordinary affects of Facebook algor- ithms. Information, Communication & Society, 20(1), 30–44. 8. Chen, W., Pacheco, D., Yang, K.-C., & Menczer, F. (2021). Neutral bots probe political bias on social media. Nature Communications, 12(1), 1–10. 9. Cho, J., Ahmed, S., Hilbert, M., Liu, B., & Luu, J. (2020). Do search algorithms endanger democ- racy? An experimental investigation of algorithm effects on political polarization. Journal of Broadcasting & Electronic Media, 64(2), 150–172. 10. DeVito, M. A., Birnholtz, J., Hancock, J. T., French, M., & Liu, S. (2018). How people form folk the- ories of social media feeds and what it means for how we study self-presentation. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems,1–12. 11. Eslami, M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K., Hamilton, K., & Sandvig, C. (2015). ‘ I always assumed that I wasn’t really that close to [her]’ Reasoning about Invisible Algorithms in News Feeds. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 153–162. 12. Evans, R., Jackson, D., & Murphy, J. (2022). Google News and machine gatekeepers: Algorithmic personalisation and news diversity in online news search. Digital Journalism,1–19. 13. Feezell, J. T., Wagner, J. K., & Conroy, M. (2021). Exploring the effects of algorithm-driven news sources on political behavior and polarization. Computers in Human Behavior, 116,1–11. 14. Fletcher, R., Kalogeropoulos, A., & Nielsen, R. K. (2021). More diverse, more politically varied: How social media, search engines and aggregators shape news repertoires in the United Kingdom. New Media & Society,1–22. 15. Fletcher, R., & Nielsen, R. K. (2018). Automated serendipity: The effect of using search engines on news repertoire balance and diversity. Digital Journalism, 6(8), 976–989. 16. Haim, M., Graefe, A., & Brosius, H.-B. (2018). Burst of the filter bubble? Effects of personalization on the diversity of Google News. Digital Journalism, 6(3), 330–343. 17. Jürgens, P., & Stark, B. (2022). Mapping Exposure Diversity: The Divergent Effects of Algorithmic Curation on News Consumption. Journal of Communication, 72(3), 322–344. 18. Lee, S. Y., & Lee, S. W. (2022). Normative or Effective? The Role of News Diversity and Trust in News Recommendation Services. International Journal of Human–Computer Interaction,1–14. 19. Levy, R. (2021). Social media, news consumption, and polarization: Evidence from a field exper- iment. American Economic Review, 111(3), 831–870. 20. Moeller, J., Trilling, D., Helberger, N., Irion, K., & Vreese, C. (2016). Shrinking core? Exploring the differential agenda setting power of traditional and personalized news media. Info, 18(6), 26–41. 21. Nechushtai, E., & Lewis, S. C. (2019). What kind of news gatekeepers do we want machines to be? Filter bubbles, fragmentation, and the normative dimensions of algorithmic recommendations. Computers in Human Behavior, 90, 298–307. 22. Ohme, J. (2021). Algorithmic social media use and its relationship to attitude reinforcement and issue-specific political participation–The case of the 2015 European immigration movements. Journal of Information Technology & Politics, 18(1), 36–54. 23. Papa, V., & Photiadis, T. (2021). Algorithmic Curation and Users’ Civic Attitudes: A Study on Face- book News Feed Results. Information , 12, 1 –13. 24. Powers, E. (2017). My news feed is filtered? Awareness of news personalization among college students. Digital Journalism, 5(10), 1315–1335. 25. Rader, E., & Gray, R. (2015). Understanding user beliefs about algorithmic curation in the Face- book news feed. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 173–182. 26. Swart, J. (2021). Experiencing algorithms: How young people understand, feel about, and engage with algorithmic news selection on social media. Social Media + Society, 7(2), 1–11. 27. ter Hoeve, M., Heruer, M., Odijk, D., Schuth, A., & de Rijke, M. (2017). Do news consumers want explanations for personalized news rankings. FATREC Workshop on Responsible Recommendation Proceedings.
Journal
Annals of the International Communication Association – Taylor & Francis
Published: Jan 2, 2023
Keywords: News recommender systems; algorithms; digital journalism; news personalisation