Get 20M+ Full-Text Papers For Less Than $1.50/day. Subscribe now for You or Your Team.

Learn More →

Methods for the thematic synthesis of qualitative research in systematic reviews

Methods for the thematic synthesis of qualitative research in systematic reviews Background: There is a growing recognition of the value of synthesising qualitative research in the evidence base in order to facilitate effective and appropriate health care. In response to this, methods for undertaking these syntheses are currently being developed. Thematic analysis is a method that is often used to analyse data in primary qualitative research. This paper reports on the use of this type of analysis in systematic reviews to bring together and integrate the findings of multiple qualitative studies. Methods: We describe thematic synthesis, outline several steps for its conduct and illustrate the process and outcome of this approach using a completed review of health promotion research. Thematic synthesis has three stages: the coding of text 'line-by-line'; the development of 'descriptive themes'; and the generation of 'analytical themes'. While the development of descriptive themes remains 'close' to the primary studies, the analytical themes represent a stage of interpretation whereby the reviewers 'go beyond' the primary studies and generate new interpretive constructs, explanations or hypotheses. The use of computer software can facilitate this method of synthesis; detailed guidance is given on how this can be achieved. Results: We used thematic synthesis to combine the studies of children's views and identified key themes to explore in the intervention studies. Most interventions were based in school and often combined learning about health benefits with 'hands-on' experience. The studies of children's views suggested that fruit and vegetables should be treated in different ways, and that messages should not focus on health warnings. Interventions that were in line with these suggestions tended to be more effective. Thematic synthesis enabled us to stay 'close' to the results of the primary studies, synthesising them in a transparent way, and facilitating the explicit production of new concepts and hypotheses. Conclusion: We compare thematic synthesis to other methods for the synthesis of qualitative research, discussing issues of context and rigour. Thematic synthesis is presented as a tried and tested method that preserves an explicit and transparent link between conclusions and the text of primary studies; as such it preserves principles that have traditionally been important to systematic reviewing. Page 1 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 synthesis described in this paper have so far been used Background The systematic review is an important technology for the explicitly in three systematic reviews [16-18]. evidence-informed policy and practice movement, which aims to bring research closer to decision-making [1,2]. The review used as an example in this paper This type of review uses rigorous and explicit methods to To illustrate the steps involved in a thematic synthesis we bring together the results of primary research in order to draw on a review of the barriers to, and facilitators of, provide reliable answers to particular questions [3-6]. The healthy eating amongst children aged four to 10 years old picture that is presented aims to be distorted neither by [17]. The review was commissioned by the Department of biases in the review process nor by biases in the primary Health, England to inform policy about how to encourage research which the review contains [7-10]. Systematic children to eat healthily in the light of recent surveys high- review methods are well-developed for certain types of lighting that British children are eating less than half the research, such as randomised controlled trials (RCTs). recommended five portions of fruit and vegetables per Methods for reviewing qualitative research in a systematic day. While we focus on the aspects of the review that relate way are still emerging, and there is much ongoing devel- to qualitative studies, the review was broader than this opment and debate [11-14]. and combined answering traditional questions of effec- tiveness, through reviewing controlled trials, with ques- In this paper we present one approach to the synthesis of tions relating to children's views of healthy eating, which findings of qualitative research, which we have called 'the- were answered using qualitative studies. The qualitative matic synthesis'. We have developed and applied these studies were synthesised using 'thematic synthesis' – the methods within several systematic reviews that address subject of this paper. We compared the effectiveness of questions about people's perspectives and experiences interventions which appeared to be in line with recom- [15-18]. The context for this methodological develop- mendations from the thematic synthesis with those that ment is a programme of work in health promotion and did not. This enabled us to see whether the understand- public health (HP & PH), mostly funded by the English ings we had gained from the children's views helped us to Department of Health, at the EPPI-Centre, in the Social explain differences in the effectiveness of different inter- Science Research Unit at the Institute of Education, Uni- ventions: the thematic synthesis had enabled us to gener- versity of London in the UK. Early systematic reviews at ate hypotheses which could be tested against the findings the EPPI-Centre addressed the question 'what works?' and of the quantitative studies – hypotheses that we could not contained research testing the effects of interventions. have generated without the thematic synthesis. The meth- However, policy makers and other review users also posed ods of this part of the review are published in Thomas et questions about intervention need, appropriateness and al. [27] and are discussed further in Harden and Thomas acceptability, and factors influencing intervention imple- [21]. mentation. To address these questions, our reviews began to include a wider range of research, including research Qualitative research and systematic reviews often described as 'qualitative'. We began to focus, in par- The act of seeking to synthesise qualitative research means ticular, on research that aimed to understand the health stepping into more complex and contested territory than issue in question from the experiences and point of view is the case when only RCTs are included in a review. First, of the groups of people targeted by HP&PH interventions methods are much less developed in this area, with fewer (We use the term 'qualitative' research cautiously because completed reviews available from which to learn, and sec- it encompasses a multitude of research methods at the ond, the whole enterprise of synthesising qualitative same time as an assumed range of epistemological posi- research is itself hotly debated. Qualitative research, it is tions. In practice it is often difficult to classify research as often proposed, is not generalisable and is specific to a being either 'qualitative' or 'quantitative' as much research particular context, time and group of participants. Thus, contains aspects of both [19-22]. Because the term is in in bringing such research together, reviewers are open to common use, however, we will employ it in this paper). the charge that they de-contextualise findings and wrongly assume that these are commensurable [11,13]. When we started the work for our first series of reviews These are serious concerns which it is not the purpose of which included qualitative research in 1999 [23-26], there this paper to contest. We note, however, that a strong case was very little published material that described methods has been made for qualitative research to be valued for the for synthesising this type of research. We therefore experi- potential it has to inform policy and practice [11,28-30]. mented with a variety of techniques borrowed from stand- In our experience, users of reviews are interested in the ard systematic review methods and methods for analysing answers that only qualitative research can provide, but are primary qualitative research [15]. In later reviews, we were not able to handle the deluge of data that would result if able to refine these methods and began to apply thematic they tried to locate, read and interpret all the relevant analysis in a more explicit way. The methods for thematic research themselves. Thus, if we acknowledge the unique Page 2 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 importance of qualitative research, we need also to recog- as 'thematic analysis' in order to formalise the identifica- nise that methods are required to bring its findings tion and development of themes. together for a wide audience – at the same time as preserv- ing and respecting its essential context and complexity. We now move to a description of the methods we used in our example systematic review. While this paper has the The earliest published work that we know of that deals traditional structure for reporting the results of a research with methods for synthesising qualitative research was project, the detailed methods (e.g. precise terms we used written in 1988 by Noblit and Hare [31]. This book for searching) and results are available online. This paper describes the way that ethnographic research might be identifies the particular issues that relate especially to synthesised, but the method has been shown to be appli- reviewing qualitative research systematically and then to cable to qualitative research beyond ethnography [32,11]. describing the activity of thematic synthesis in detail. As well as meta-ethnography, other methods have been developed more recently, including 'meta-study' [33], Methods 'critical interpretive synthesis' [34] and 'metasynthesis' Searching When searching for studies for inclusion in a 'traditional' [13]. statistical meta-analysis, the aim of searching is to locate Many of the newer methods being developed have much all relevant studies. Failing to do this can undermine the in common with meta-ethnography, as originally statistical models that underpin the analysis and bias the described by Noblit and Hare, and often state explicitly results. However, Doyle [[39], p326] states that, "like that they are drawing on this work. In essence, this meta-analysis, meta-ethnography utilizes multiple empirical method involves identifying key concepts from studies studies but, unlike meta-analysis, the sample is purposive rather and translating them into one another. The term 'translat- than exhaustive because the purpose is interpretive explanation ing' in this context refers to the process of taking concepts and not prediction". This suggests that it may not be neces- from one study and recognising the same concepts in sary to locate every available study because, for example, another study, though they may not be expressed using the results of a conceptual synthesis will not change if ten identical words. Explanations or theories associated with rather than five studies contain the same concept, but will these concepts are also extracted and a 'line of argument' depend on the range of concepts found in the studies, may be developed, pulling corroborating concepts their context, and whether they are in agreement or not. together and, crucially, going beyond the content of the Thus, principles such as aiming for 'conceptual saturation' original studies (though 'refutational' concepts might not might be more appropriate when planning a search strat- be amenable to this process). Some have claimed that this egy for qualitative research, although it is not yet clear notion of 'going beyond' the primary studies is a critical how these principles can be applied in practice. Similarly, component of synthesis, and is what distinguishes it from other principles from primary qualitative research meth- the types of summaries of findings that typify traditional ods may also be 'borrowed' such as deliberately seeking literature reviews [e.g. [32], p209]. In the words of Marga- studies which might act as negative cases, aiming for max- rete Sandelowski, "metasyntheses are integrations that are imum variability and, in essence, designing the resulting more than the sum of parts, in that they offer novel interpreta- set of studies to be heterogeneous, in some ways, instead tions of findings. These interpretations will not be found in any of achieving the homogeneity that is often the aim in sta- one research report but, rather, are inferences derived from tak- tistical meta-analyses. ing all of the reports in a sample as a whole" [[14], p1358]. However you look, qualitative research is difficult to find Thematic analysis has been identified as one of a range of [40-42]. In our review, it was not possible to rely on sim- potential methods for research synthesis alongside meta- ple electronic searches of databases. We needed to search ethnography and 'metasynthesis', though precisely what extensively in 'grey' literature, ask authors of relevant the method involves is unclear, and there are few exam- papers if they knew of more studies, and look especially ples of it being used for synthesising research [35]. We for book chapters, and we spent a lot of effort screening have adopted the term 'thematic synthesis', as we trans- titles and abstracts by hand and looking through journals lated methods for the analysis of primary research – often manually. In this sense, while we were not driven by the termed 'thematic' – for use in systematic reviews [36-38]. statistical imperative of locating every relevant study, As Boyatzis [[36], p4] has observed, thematic analysis is when it actually came down to searching, we found that "not another qualitative method but a process that can be used there was very little difference in the methods we had to with most, if not all, qualitative methods...". Our approach use to find qualitative studies compared to the methods concurs with this conceptualisation of thematic analysis, we use when searching for studies for inclusion in a meta- since the method we employed draws on other estab- analysis. lished methods but uses techniques commonly described Page 3 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 Quality assessment tation of data as findings (as for example when data are Assessing the quality of qualitative research has attracted used to 'let participants speak for themselves'). Sand- much debate and there is little consensus regarding how elowski and Barroso [53] have argued that the findings of quality should be assessed, who should assess quality, qualitative (and, indeed, all empirical) research are dis- and, indeed, whether quality can or should be assessed in tinct from the data upon which they are based, the meth- relation to 'qualitative' research at all [43,22,44,45]. We ods used to derive them, externally sourced data, and take the view that the quality of qualitative research researchers' conclusions and implications. should be assessed to avoid drawing unreliable conclu- sions. However, since there is little empirical evidence on In our example review, while it was relatively easy to iden- which to base decisions for excluding studies based on tify 'data' in the studies – usually in the form of quotations quality assessment, we took the approach in this review to from the children themselves – it was often difficult to use 'sensitivity analyses' (described below) to assess the identify key concepts or succinct summaries of findings, possible impact of study quality on the review's findings. especially for studies that had undertaken relatively sim- ple analyses and had not gone much further than describ- In our example review we assessed our studies according ing and summarising what the children had said. To to 12 criteria, which were derived from existing sets of cri- resolve this problem we took study findings to be all of teria proposed for assessing the quality of qualitative the text labelled as 'results' or 'findings' in study reports – research [46-49], principles of good practice for conduct- though we also found 'findings' in the abstracts which ing social research with children [50], and whether studies were not always reported in the same way in the text. employed appropriate methods for addressing our review Study reports ranged in size from a few pages to full final questions. The 12 criteria covered three main quality project reports. We entered all the results of the studies issues. Five related to the quality of the reporting of a verbatim into QSR's NVivo software for qualitative data study's aims, context, rationale, methods and findings analysis. Where we had the documents in electronic form (e.g. was there an adequate description of the sample used this process was straightforward even for large amounts of and the methods for how the sample was selected and text. When electronic versions were not available, the recruited?). A further four criteria related to the sufficiency results sections were either re-typed or scanned in using a of the strategies employed to establish the reliability and flat-bed or pen scanner. (We have since adapted our own validity of data collection tools and methods of analysis, reviewing system, 'EPPI-Reviewer' [54], to handle this and hence the validity of the findings. The final three cri- type of synthesis and the screenshots below show this teria related to the assessment of the appropriateness of the software.) study methods for ensuring that findings about the barri- ers to, and facilitators of, healthy eating were rooted in Detailed methods for thematic synthesis children's own perspectives (e.g. were data collection The synthesis took the form of three stages which over- methods appropriate for helping children to express their lapped to some degree: the free line-by-line coding of the views?). findings of primary studies; the organisation of these 'free codes' into related areas to construct 'descriptive' themes; Extracting data from studies and the development of 'analytical' themes. One issue which is difficult to deal with when synthesis- ing 'qualitative' studies is 'what counts as data' or 'find- Stages one and two: coding text and developing descriptive themes ings'? This problem is easily addressed when a statistical In our children and healthy eating review, we originally meta-analysis is being conducted: the numeric results of planned to extract and synthesise study findings according RCTs – for example, the mean difference in outcome to our review questions regarding the barriers to, and facil- between the intervention and control – are taken from itators of, healthy eating amongst children. It soon published reports and are entered into the software pack- became apparent, however, that few study findings age being used to calculate the pooled effect size [3,51]. addressed these questions directly and it appeared that we were in danger of ending up with an empty synthesis. We Deciding what to abstract from the published report of a were also concerned about imposing the a priori frame- 'qualitative' study is much more difficult. Campbell et al. work implied by our review questions onto study findings [11] extracted what they called the 'key concepts' from the without allowing for the possibility that a different or qualitative studies they found about patients' experiences modified framework may be a better fit. We therefore tem- of diabetes and diabetes care. However, finding the key porarily put our review questions to one side and started concepts in 'qualitative' research is not always straightfor- from the study findings themselves to conduct an the- ward either. As Sandelowski and Barroso [52] discovered, matic analysis. identifying the findings in qualitative research can be complicated by varied reporting styles or the misrepresen- Page 4 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 There were eight relevant qualitative studies examining thesis of qualitative research: the translation of concepts children's views of healthy eating. We entered the verba- from one study to another [32,55]. However, this process tim findings of these studies into our database. Three may not be regarded as a simple one of translation. As we reviewers then independently coded each line of text coded each new study we added to our 'bank' of codes and according to its meaning and content. Figure 1 illustrates developed new ones when necessary. As well as translat- this line-by-line coding using our specialist reviewing soft- ing concepts between studies, we had already begun the ware, EPPI-Reviewer, which includes a component process of synthesis (For another account of this process, designed to support thematic synthesis. The text which see Doyle [[39], p331]). Every sentence had at least one was taken from the report of the primary study is on the code applied, and most were categorised using several left and codes were created inductively to capture the codes (e.g. 'children prefer fruit to vegetables' or 'why eat meaning and content of each sentence. Codes could be healthily?'). Before completing this stage of the synthesis, structured, either in a tree form (as shown in the figure) or we also examined all the text which had a given code as 'free' codes – without a hierarchical structure. applied to check consistency of interpretation and to see whether additional levels of coding were needed. (In The use of line-by-line coding enabled us to undertake grounded theory this is termed 'axial' coding; see Fisher what has been described as one of the key tasks in the syn- [55] for further discussion of the application of axial cod- li Figure 1 ne-by-line coding in EPPI-Reviewer line-by-line coding in EPPI-Reviewer. Page 5 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 ing in research synthesis.) This process created a total of cal tree structure. New codes were created to capture the 36 initial codes. For example, some of the text we coded meaning of groups of initial codes. This process resulted as "bad food = nice, good food = awful" from one study in a tree structure with several layers to organize a total of [56] were: 12 descriptive themes (Figure 2). For example, the first layer divided the 12 themes into whether they were con- 'All the things that are bad for you are nice and all the cerned with children's understandings of healthy eating or things that are good for you are awful.' (Boys, year 6) influences on children's food choice. The above example, [[56], p74] about children's preferences for food, was placed in both areas, since the findings related both to children's reac- 'All adverts for healthy stuff go on about healthy things. The tions to the foods they were given, and to how they adverts for unhealthy things tell you how nice they taste.' behaved when given the choice over what foods they [[56], p75] might eat. A draft summary of the findings across the stud- ies organized by the 12 descriptive themes was then writ- Some children reported throwing away foods they knew had ten by one of the review authors. Two other review been put in because they were 'good for you' and only ate authors commented on this draft and a final version was the crisps and chocolate. [[56], p75] agreed. Reviewers looked for similarities and differences between the codes in order to start grouping them into a hierarchi- re Figure 2 lationships between descriptive themes relationships between descriptive themes. Page 6 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 Stage three: generating analytical themes effective in increasing consumption. As one child noted Up until this point, we had produced a synthesis which astutely, 'All adverts for healthy stuff go on about healthy kept very close to the original findings of the included things. The adverts for unhealthy things tell you how nice they studies. The findings of each study had been combined taste.' [[56], p75]. We captured this line of argument in the into a whole via a listing of themes which described chil- analytical theme entitled 'Children do not see it as their dren's perspectives on healthy eating. However, we did role to be interested in health'. Altogether, this process not yet have a synthesis product that addressed directly resulted in the generation of six analytical themes which the concerns of our review – regarding how to promote were associated with ten recommendations for interven- healthy eating, in particular fruit and vegetable intake, tions. amongst children. Neither had we 'gone beyond' the find- ings of the primary studies and generated additional con- Results cepts, understandings or hypotheses. As noted earlier, the Six main issues emerged from the studies of children's idea or step of 'going beyond' the content of the original views: (1) children do not see it as their role to be inter- studies has been identified by some as the defining char- ested in health; (2) children do not see messages about acteristic of synthesis [32,14]. future health as personally relevant or credible; (3) fruit, vegetables and confectionery have very different mean- This stage of a qualitative synthesis is the most difficult to ings for children; (4) children actively seek ways to exer- describe and is, potentially, the most controversial, since cise their own choices with regard to food; (5) children it is dependent on the judgement and insights of the value eating as a social occasion; and (6) children see the reviewers. The equivalent stage in meta-ethnography is contradiction between what is promoted in theory and the development of 'third order interpretations' which go what adults provide in practice. The review found that beyond the content of original studies [32,11]. In our most interventions were based in school (though fre- example, the step of 'going beyond' the content of the quently with parental involvement) and often combined original studies was achieved by using the descriptive learning about the health benefits of fruit and vegetables themes that emerged from our inductive analysis of study with 'hands-on' experience in the form of food prepara- findings to answer the review questions we had temporar- tion and taste-testing. Interventions targeted at people ily put to one side. Reviewers inferred barriers and facilita- with particular risk factors worked better than others, and tors from the views children were expressing about multi-component interventions that combined the pro- healthy eating or food in general, captured by the descrip- motion of physical activity with healthy eating did not tive themes, and then considered the implications of chil- work as well as those that only concentrated on healthy dren's views for intervention development. Each reviewer eating. The studies of children's views suggested that fruit first did this independently and then as a group. Through and vegetables should be treated in different ways in inter- this discussion more abstract or analytical themes began ventions, and that messages should not focus on health to emerge. The barriers and facilitators and implications warnings. Interventions that were in line with these sug- for intervention development were examined again in gestions tended to be more effective than those which light of these themes and changes made as necessary. This were not. cyclical process was repeated until the new themes were sufficiently abstract to describe and/or explain all of our Discussion Context and rigour in thematic synthesis initial descriptive themes, our inferred barriers and facili- tators and implications for intervention development. The process of translation, through the development of descriptive and analytical themes, can be carried out in a For example, five of the 12 descriptive themes concerned rigorous way that facilitates transparency of reporting. the influences on children's choice of foods (food prefer- Since we aim to produce a synthesis that both generates ences, perceptions of health benefits, knowledge behav- 'abstract and formal theories' that are nevertheless 'empiri- iour gap, roles and responsibilities, non-influencing cally faithful to the cases from which they were developed' factors). From these, reviewers inferred several barriers [[53], p1371], we see the explicit recording of the develop- and implications for intervention development. Children ment of themes as being central to the method. The use of identified readily that taste was the major concern for software as described can facilitate this by allowing them when selecting food and that health was either a sec- reviewers to examine the contribution made to their find- ondary factor or, in some cases, a reason for rejecting ings by individual studies, groups of studies, or sub-pop- food. Children also felt that buying healthy food was not ulations within studies. a legitimate use of their pocket money, which they would use to buy sweets that could be enjoyed with friends. Some may argue against the synthesis of qualitative These perspectives indicated to us that branding fruit and research on the grounds that the findings of individual vegetables as a 'tasty' rather than 'healthy' might be more studies are de-contextualised and that concepts identified Page 7 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 in one setting are not applicable to others [32]. However, purpose of which is to exclude studies that do not provide the act of synthesis could be viewed as similar to the role a reliable answer to the review question. However, given of a research user when reading a piece of qualitative that there were no accepted – or empirically tested – meth- research and deciding how useful it is to their own situa- ods for excluding qualitative studies from syntheses on tion. In the case of synthesis, reviewers translate themes the basis of their quality [57,12,58], we included all stud- and concepts from one situation to another and can ies regardless of their quality. always be checking that each transfer is valid and whether there are any reasons that understandings gained in one Nevertheless, our studies did differ according to the qual- context might not be transferred to another. We attempted ity criteria they were assessed against and it was important to preserve context by providing structured summaries of that we considered this in some way. In systematic reviews each study detailing aims, methods and methodological of trials, 'sensitivity analyses' – analyses which test the quality, and setting and sample. This meant that readers of effect on the synthesis of including and excluding findings our review were able to judge for themselves whether or from studies of differing quality – are often carried out. not the contexts of the studies the review contained were Dixon-Woods et al. [12] suggest that assessing the feasibil- similar to their own. In the synthesis we also checked ity and worth of conducting sensitivity analyses within whether the emerging findings really were transferable syntheses of qualitative research should be an important across different study contexts. For example, we tried focus of synthesis methods work. After our thematic syn- throughout the synthesis to distinguish between partici- thesis was complete, we examined the relative contribu- pants (e.g. boys and girls) where the primary research had tions of studies to our final analytic themes and made an appropriate distinction. We then looked to see recommendations for interventions. We found that the whether some of our synthesis findings could be attrib- poorer quality studies contributed comparatively little to uted to a particular group of children or setting. In the the synthesis and did not contain many unique themes; event, we did not find any themes that belonged to a spe- the better studies, on the other hand, appeared to have cific group, but another outcome of this process was a more developed analyses and contributed most to the realisation that the contextual information given in the synthesis. reports of studies was very restricted indeed. It was there- fore difficult to make the best use of context in our synthe- Conclusion sis. This paper has discussed the rationale for reviewing and synthesising qualitative research in a systematic way and In checking that we were not translating concepts into sit- has outlined one specific approach for doing this: the- uations where they did not belong, we were following a matic synthesis. While it is not the only method which principle that others have followed when using synthesis might be used – and we have discussed some of the other methods to build grounded formal theory: that of ground- options available – we present it here as a tested technique ing a text in the context in which it was constructed. As that has worked in the systematic reviews in which it has Margaret Kearney has noted "the conditions under which been employed. data were collected, analysis was done, findings were found, and products were written for each contributing report should We have observed that one of the key tasks in the synthesis be taken into consideration in developing a more generalized of qualitative research is the translation of concepts and abstract model" [[14], p1353]. Britten et al. [32] suggest between studies. While the activity of translating concepts that it may be important to make a deliberate attempt to is usually undertaken in the few syntheses of qualitative include studies conducted across diverse settings to research that exist, there are few examples that specify the achieve the higher level of abstraction that is aimed for in detail of how this translation is actually carried out. The a meta-ethnography. example above shows how we achieved the translation of concepts across studies through the use of line-by-line Study quality and sensitivity analyses coding, the organisation of these codes into descriptive We assessed the 'quality' of our studies with regard to the themes, and the generation of analytical themes through degree to which they represented the views of their partic- the application of a higher level theoretical framework. ipants. In doing this, we were locating the concept of This paper therefore also demonstrates how the methods 'quality' within the context of the purpose of our review – and process of a thematic synthesis can be written up in a children's views – and not necessarily the context of the transparent way. primary studies themselves. Our 'hierarchy of evidence', therefore, did not prioritise the research design of studies This paper goes some way to addressing concerns regard- but emphasised the ability of the studies to answer our ing the use of thematic analysis in research synthesis review question. A traditional systematic review of con- raised by Dixon-Woods and colleagues who argue that the trolled trials would contain a quality assessment stage, the approach can lack transparency due to a failure to distin- Page 8 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 guish between 'data-driven' or 'theory-driven' approaches. This paper is a contribution to the current developmental Moreover they suggest that, "if thematic analysis is limited to work taking place in understanding how best to bring summarising themes reported in primary studies, it offers little together the findings of qualitative research to inform pol- by way of theoretical structure within which to develop higher icy and practice. It is by no means the only method on order thematic categories..." [[35], p47]. Part of the prob- offer but, by drawing on methods and principles from lem, they observe, is that the precise methods of thematic qualitative primary research, it benefits from the years of synthesis are unclear. Our approach contains a clear sepa- methodological development that underpins the research ration between the 'data-driven' descriptive themes and it seeks to synthesise. the 'theory-driven' analytical themes and demonstrates how the review questions provided a theoretical structure Competing interests within which it became possible to develop higher order The authors declare that they have no competing interests. thematic categories. Authors' contributions The theme of 'going beyond' the content of the primary Both authors contributed equally to the paper and read studies was discussed earlier. Citing Strike and Posner and approved the final manuscript. [59], Campbell et al. [[11], p672] also suggest that synthe- sis "involves some degree of conceptual innovation, or employ- Acknowledgements The authors would like to thank Elaine Barnett-Page for her assistance in ment of concepts not found in the characterisation of the parts producing the draft paper, and David Gough, Ann Oakley and Sandy Oliver and a means of creating the whole". This was certainly true for their helpful comments. The review used an example in this paper was of the example given in this paper. We used a series of funded by the Department of Health (England). The methodological devel- questions, derived from the main topic of our review, to opment was supported by Department of Health (England) and the ESRC focus an examination of our descriptive themes and we do through the Methods for Research Synthesis Node of the National Centre not find our recommendations for interventions con- for Research Methods. In addition, Angela Harden held a senior research tained in the findings of the primary studies: these were fellowship funded by the Department of Health (England) December 2003 new propositions generated by the reviewers in the light – November 2007. The views expressed in this paper are those of the authors and are not necessarily those of the funding bodies. of the synthesis. The method also demonstrates that it is possible to synthesise without conceptual innovation. References The initial synthesis, involving the translation of concepts 1. Chalmers I: Trying to do more good than harm in policy and between studies, was necessary in order for conceptual practice: the role of rigorous, transparent and up-to-date innovation to begin. One could argue that the conceptual evaluations. Ann Am Acad Pol Soc Sci 2003, 589:22-40. 2. Oakley A: Social science and evidence-based everything: the innovation, in this case, was only necessary because the case of education. Educ Rev 2002, 54:277-286. primary studies did not address our review question 3. Cooper H, Hedges L: The Handbook of Research Synthesis New York: directly. In situations in which the primary studies are Russell Sage Foundation; 1994. 4. EPPI-Centre: EPPI-Centre Methods for Conducting Systematic Reviews concerned directly with the review question, it may not be 2006 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=89]. London: EPPI- necessary to go beyond the contents of the original studies Centre, Social Science Research Unit, Institute of Education, Univer- sity of London in order to produce a satisfactory synthesis (see, for exam- 5. Higgins J, Green S, (Eds): Cochrane Handbook for Systematic Reviews of ple, Marston and King, [60]). Conceptually, our analytical Interventions 4.2.6 2006 [http://www.cochrane.org/resources/hand themes are similar to the ultimate product of meta-ethnog- book/]. Updated September 2006. Accessed 24th January 2007 6. Petticrew M, Roberts H: Systematic Reviews in the Social Sciences: A raphies: third order interpretations [11], since both are practical guide Oxford: Blackwell Publishing; 2006. explicit mechanisms for going beyond the content of the 7. Chalmers I, Hedges L, Cooper H: A brief history of research syn- primary studies and presenting this in a transparent way. thesis. Eval Health Prof 2002, 25:12-37. 8. Juni P, Altman D, Egger M: Assessing the quality of controlled The main difference between them lies in their purposes. clinical trials. BMJ 2001, 323:42-46. Third order interpretations bring together the implications 9. Mulrow C: Systematic reviews: rationale for systematic reviews. BMJ 1994, 309:597-599. of translating studies into one another in their own terms, 10. White H: Scientific communication and literature retrieval. whereas analytical themes are the result of interrogating a In The Handbook of Research Synthesis Edited by: Cooper H, Hedges L. descriptive synthesis by placing it within an external theo- New York: Russell Sage Foundation; 1994. 11. Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan retical framework (our review question and sub-ques- J: Evaluating meta-ethnography: a synthesis of qualitative tions). It may be, therefore, that analytical themes are more research on lay experiences of diabetes and diabetes care. Soc Sci Med 2003, 56:671-684. appropriate when a specific review question is being 12. Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, addressed (as often occurs when informing policy and Shaw RL, Smith JA, Young B: How can systematic reviews incor- practice), and third order interpretations should be used porate qualitative research? A critical perspective. Qual Res 2006, 6:27-44. when a body of literature is being explored in and of itself, 13. Sandelowski M, Barroso J: Handbook for Synthesising Qualitative with broader, or emergent, review questions. Research New York: Springer; 2007. 14. Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M: Qualita- tive meta-synthesis: reflections on methodological orienta- tion and ideological agenda. Qual Health Res 2004, 14:1342-1365. Page 9 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 15. Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley 37. Braun V, Clarke V: Using thematic analysis in psychology. Qual A: Applying systematic review methods to studies of people's Res Psychol 2006, 3:77-101 [http://science.uwe.ac.uk/psychology/ views: an example from public health. J Epidemiol Community drvictoriaclarke_files/thematicanalysis%20.pdf]. Health 2004, 58:794-800. 38. Silverman D, Ed: Qualitative Research: Theory, method and practice Lon- 16. Harden A, Brunton G, Fletcher A, Oakley A: Young People, Pregnancy don: Sage; 1997. and Social Exclusion: A systematic synthesis of research evidence to identify 39. Doyle LH: Synthesis through meta-ethnography: paradoxes, effective, appropriate and promising approaches for prevention and support enhancements, and possibilities. Qual Res 2003, 3:321-344. 2006 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=674]. London: 40. Barroso J, Gollop C, Sandelowski M, Meynell J, Pearce PF, Collins LJ: EPPI-Centre, Social Science Research Unit, Institute of Education, The challenges of searching for and retrieving qualitative University of London studies. Western J Nurs Res 2003, 25:153-178. 17. Thomas J, Sutcliffe K, Harden A, Oakley A, Oliver S, Rees R, Brunton 41. Walters LA, Wilczynski NL, Haynes RB, Hedges Team: Developing G, Kavanagh J: Children and Healthy Eating: A systematic review of barriers optimal search strategies for retrieving clinically relevant and facilitators 2003 [http://eppi.ioe.ac.uk/cms/ qualitative studies in EMBASE. Qual Health Res 2006, 16:162-8. Default.aspx?tabid=246]. London: EPPI-Centre, Social Science 42. Wong SSL, Wilczynski NL, Haynes RB: Developing optimal Research Unit, Institute of Education, University of London accessed search strategies for detecting clinically relevant qualitative th 4 July 2008 studies in Medline. Medinfo 2004, 11:311-314. 18. Thomas J, Kavanagh J, Tucker H, Burchett H, Tripney J, Oakley A: 43. Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P: Qualita- Accidental Injury, Risk-Taking Behaviour and the Social Circumstances in tive research methods in health technology assessment: a which Young People Live: A systematic review 2007 [http://eppi.ioe.ac.uk/ review of the literature. Health Technol Assess 1998, 2(16):. cms/Default.aspx?tabid=1910]. London: EPPI-Centre, Social Science 44. Seale C: Quality in qualitative research. Qual Inq 1999, Research Unit, Institute of Education, University of London 5:465-478. 19. Bryman A: Quantity and Quality in Social Research London: Unwin; 45. Spencer L, Ritchie J, Lewis J, Dillon L: Quality in Qualitative Evaluation: 1998. A framework for assessing research evidence London: Cabinet Office; 20. Hammersley M: What's Wrong with Ethnography? London: Routledge; 2003. 1992. 46. Boulton M, Fitzpatrick R, Swinburn C: Qualitative research in 21. Harden A, Thomas J: Methodological issues in combining healthcare II: a structured review and evaluation of studies. diverse study types in systematic reviews. Int J Soc Res Meth J Eval Clin Pract 1996, 2:171-179. 2005, 8:257-271. 47. Cobb A, Hagemaster J: Ten criteria for evaluating qualitative 22. Oakley A: Experiments in Knowing: Gender and methods in the social sci- research proposals. J Nurs Educ 1987, 26:138-143. ences Cambridge: Polity Press; 2000. 48. Mays N, Pope C: Rigour and qualitative research. BMJ 1995, 23. Harden A, Oakley A, Oliver S: Peer-delivered health promotion 311:109-12. for young people: a systematic review of different study 49. Medical Sociology Group: Criteria for the evaluation of qualita- designs. Health Educ J 2001, 60:339-353. tive research papers. Med Sociol News 1996, 22:68-71. 24. Harden A, Rees R, Shepherd J, Brunton G, Oliver S, Oakley A: Young 50. Alderson P: Listening to Children London: Barnardo's; 1995. People and Mental Health: A systematic review of barriers and facilitators 51. Egger M, Davey-Smith G, Altman D: Systematic Reviews in Health Care: 2001 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=256]. London: Meta-analysis in context London: BMJ Publishing; 2001. EPPI-Centre, Social Science Research Unit, Institute of Education, 52. Sandelowski M, Barroso J: Finding the findings in qualitative University of London studies. J Nurs Scholarsh 2002, 34:213-219. 25. Rees R, Harden A, Shepherd J, Brunton G, Oliver S, Oakley A: Young 53. Sandelowski M: Using qualitative research. Qual Health Res 2004, People and Physical Activity: A systematic review of barriers and facilitators 14:1366-1386. 2001 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=260]. London: 54. Thomas J, Brunton J: EPPI-Reviewer 3.0: Analysis and management of EPPI-Centre, Social Science Research Unit, Institute of Education, data for research synthesis. EPPI-Centre software London: EPPI-Centre, University of London Social Science Research Unit, Institute of Education; 2006. 26. Shepherd J, Harden A, Rees R, Brunton G, Oliver S, Oakley A: Young 55. Fisher M, Qureshi H, Hardyman W, Homewood J: Using Qualitative People and Healthy Eating: A systematic review of barriers and facilitators Research in Systematic Reviews: Older people's views of hospital discharge 2001 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=258]. London: London: Social Care Institute for Excellence; 2006. EPPI-Centre, Social Science Research Unit, Institute of Education, 56. Dixey R, Sahota P, Atwal S, Turner A: Children talking about University of London healthy eating: data from focus groups with 300 9–11-year- 27. Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton olds. Nutr Bull 2001, 26:71-79. G, Kavanagh J: Integrating qualitative research with trials in 57. Daly A, Willis K, Small R, Green J, Welch N, Kealy M, Hughes E: Hier- systematic reviews: an example from public health. BMJ 2004, archy of evidence for assessing qualitative health research. J 328:1010-1012. Clin Epidemiol 2007, 60:43-49. 28. Davies P: What is evidence-based education? Br J Educ Stud 58. Popay J: Moving beyond floccinaucinihilipilification: enhancing 1999, 47:108-121. the utility of systematic reviews. J Clin Epidemiol 2005, 29. Newman M, Thompson C, Roberts AP: Helping practitioners 58:1079-80. understand the contribution of qualitative research to evi- 59. Strike K, Posner G: Types of synthesis and their criteria. In dence-based practice. Evid Based Nurs 2006, 9:4-7. Knowledge Structure and Use: Implications for synthesis and interpretation 30. Popay J: Moving Beyond Effectiveness in Evidence Synthesis London: Edited by: Ward S, Reed L. Philadelphia: Temple University Press; National Institute for Health and Clinical Excellence; 2006. 1983. 31. Noblit GW, Hare RD: Meta-Ethnography: Synthesizing qualitative studies 60. Marston C, King E: Factors that shape young people's sexual Newbury Park: Sage; 1988. behaviour: a systematic review. The Lancet 2006, 368:1581-86. 32. Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R: Using meta-ethnography to synthesise qualitative research: a Pre-publication history worked example. J Health Serv Res Policy 2002, 7:209-215. The pre-publication history for this paper can be accessed 33. Paterson B, Thorne S, Canam C, Jillings C: Meta-Study of Qualitative Health Research Thousand Oaks, California: Sage; 2001. here: 34. Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Har- vey J, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretative synthesis of the literature on access http://www.biomedcentral.com/1471-2288/8/45/prepub to healthcare by vulnerable groups. BMC Med Res Methodol 2006, 6:35. 35. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesis- ing qualitative and quantitative evidence: a review of possi- ble methods. J Health Serv Res Policy 2005, 10:45-53. 36. Boyatzis RE: Transforming Qualitative Information Sage: Cleveland; 1998. Page 10 of 10 (page number not for citation purposes) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Medical Research Methodology Springer Journals

Methods for the thematic synthesis of qualitative research in systematic reviews

Loading next page...
 
/lp/springer-journals/methods-for-the-thematic-synthesis-of-qualitative-research-in-fLxvoVMujM

References (74)

Publisher
Springer Journals
Copyright
Copyright © 2008 by Thomas and Harden; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Theory of Medicine/Bioethics; Statistical Theory and Methods; Statistics for Life Sciences, Medicine, Health Sciences
eISSN
1471-2288
DOI
10.1186/1471-2288-8-45
pmid
18616818
Publisher site
See Article on Publisher Site

Abstract

Background: There is a growing recognition of the value of synthesising qualitative research in the evidence base in order to facilitate effective and appropriate health care. In response to this, methods for undertaking these syntheses are currently being developed. Thematic analysis is a method that is often used to analyse data in primary qualitative research. This paper reports on the use of this type of analysis in systematic reviews to bring together and integrate the findings of multiple qualitative studies. Methods: We describe thematic synthesis, outline several steps for its conduct and illustrate the process and outcome of this approach using a completed review of health promotion research. Thematic synthesis has three stages: the coding of text 'line-by-line'; the development of 'descriptive themes'; and the generation of 'analytical themes'. While the development of descriptive themes remains 'close' to the primary studies, the analytical themes represent a stage of interpretation whereby the reviewers 'go beyond' the primary studies and generate new interpretive constructs, explanations or hypotheses. The use of computer software can facilitate this method of synthesis; detailed guidance is given on how this can be achieved. Results: We used thematic synthesis to combine the studies of children's views and identified key themes to explore in the intervention studies. Most interventions were based in school and often combined learning about health benefits with 'hands-on' experience. The studies of children's views suggested that fruit and vegetables should be treated in different ways, and that messages should not focus on health warnings. Interventions that were in line with these suggestions tended to be more effective. Thematic synthesis enabled us to stay 'close' to the results of the primary studies, synthesising them in a transparent way, and facilitating the explicit production of new concepts and hypotheses. Conclusion: We compare thematic synthesis to other methods for the synthesis of qualitative research, discussing issues of context and rigour. Thematic synthesis is presented as a tried and tested method that preserves an explicit and transparent link between conclusions and the text of primary studies; as such it preserves principles that have traditionally been important to systematic reviewing. Page 1 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 synthesis described in this paper have so far been used Background The systematic review is an important technology for the explicitly in three systematic reviews [16-18]. evidence-informed policy and practice movement, which aims to bring research closer to decision-making [1,2]. The review used as an example in this paper This type of review uses rigorous and explicit methods to To illustrate the steps involved in a thematic synthesis we bring together the results of primary research in order to draw on a review of the barriers to, and facilitators of, provide reliable answers to particular questions [3-6]. The healthy eating amongst children aged four to 10 years old picture that is presented aims to be distorted neither by [17]. The review was commissioned by the Department of biases in the review process nor by biases in the primary Health, England to inform policy about how to encourage research which the review contains [7-10]. Systematic children to eat healthily in the light of recent surveys high- review methods are well-developed for certain types of lighting that British children are eating less than half the research, such as randomised controlled trials (RCTs). recommended five portions of fruit and vegetables per Methods for reviewing qualitative research in a systematic day. While we focus on the aspects of the review that relate way are still emerging, and there is much ongoing devel- to qualitative studies, the review was broader than this opment and debate [11-14]. and combined answering traditional questions of effec- tiveness, through reviewing controlled trials, with ques- In this paper we present one approach to the synthesis of tions relating to children's views of healthy eating, which findings of qualitative research, which we have called 'the- were answered using qualitative studies. The qualitative matic synthesis'. We have developed and applied these studies were synthesised using 'thematic synthesis' – the methods within several systematic reviews that address subject of this paper. We compared the effectiveness of questions about people's perspectives and experiences interventions which appeared to be in line with recom- [15-18]. The context for this methodological develop- mendations from the thematic synthesis with those that ment is a programme of work in health promotion and did not. This enabled us to see whether the understand- public health (HP & PH), mostly funded by the English ings we had gained from the children's views helped us to Department of Health, at the EPPI-Centre, in the Social explain differences in the effectiveness of different inter- Science Research Unit at the Institute of Education, Uni- ventions: the thematic synthesis had enabled us to gener- versity of London in the UK. Early systematic reviews at ate hypotheses which could be tested against the findings the EPPI-Centre addressed the question 'what works?' and of the quantitative studies – hypotheses that we could not contained research testing the effects of interventions. have generated without the thematic synthesis. The meth- However, policy makers and other review users also posed ods of this part of the review are published in Thomas et questions about intervention need, appropriateness and al. [27] and are discussed further in Harden and Thomas acceptability, and factors influencing intervention imple- [21]. mentation. To address these questions, our reviews began to include a wider range of research, including research Qualitative research and systematic reviews often described as 'qualitative'. We began to focus, in par- The act of seeking to synthesise qualitative research means ticular, on research that aimed to understand the health stepping into more complex and contested territory than issue in question from the experiences and point of view is the case when only RCTs are included in a review. First, of the groups of people targeted by HP&PH interventions methods are much less developed in this area, with fewer (We use the term 'qualitative' research cautiously because completed reviews available from which to learn, and sec- it encompasses a multitude of research methods at the ond, the whole enterprise of synthesising qualitative same time as an assumed range of epistemological posi- research is itself hotly debated. Qualitative research, it is tions. In practice it is often difficult to classify research as often proposed, is not generalisable and is specific to a being either 'qualitative' or 'quantitative' as much research particular context, time and group of participants. Thus, contains aspects of both [19-22]. Because the term is in in bringing such research together, reviewers are open to common use, however, we will employ it in this paper). the charge that they de-contextualise findings and wrongly assume that these are commensurable [11,13]. When we started the work for our first series of reviews These are serious concerns which it is not the purpose of which included qualitative research in 1999 [23-26], there this paper to contest. We note, however, that a strong case was very little published material that described methods has been made for qualitative research to be valued for the for synthesising this type of research. We therefore experi- potential it has to inform policy and practice [11,28-30]. mented with a variety of techniques borrowed from stand- In our experience, users of reviews are interested in the ard systematic review methods and methods for analysing answers that only qualitative research can provide, but are primary qualitative research [15]. In later reviews, we were not able to handle the deluge of data that would result if able to refine these methods and began to apply thematic they tried to locate, read and interpret all the relevant analysis in a more explicit way. The methods for thematic research themselves. Thus, if we acknowledge the unique Page 2 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 importance of qualitative research, we need also to recog- as 'thematic analysis' in order to formalise the identifica- nise that methods are required to bring its findings tion and development of themes. together for a wide audience – at the same time as preserv- ing and respecting its essential context and complexity. We now move to a description of the methods we used in our example systematic review. While this paper has the The earliest published work that we know of that deals traditional structure for reporting the results of a research with methods for synthesising qualitative research was project, the detailed methods (e.g. precise terms we used written in 1988 by Noblit and Hare [31]. This book for searching) and results are available online. This paper describes the way that ethnographic research might be identifies the particular issues that relate especially to synthesised, but the method has been shown to be appli- reviewing qualitative research systematically and then to cable to qualitative research beyond ethnography [32,11]. describing the activity of thematic synthesis in detail. As well as meta-ethnography, other methods have been developed more recently, including 'meta-study' [33], Methods 'critical interpretive synthesis' [34] and 'metasynthesis' Searching When searching for studies for inclusion in a 'traditional' [13]. statistical meta-analysis, the aim of searching is to locate Many of the newer methods being developed have much all relevant studies. Failing to do this can undermine the in common with meta-ethnography, as originally statistical models that underpin the analysis and bias the described by Noblit and Hare, and often state explicitly results. However, Doyle [[39], p326] states that, "like that they are drawing on this work. In essence, this meta-analysis, meta-ethnography utilizes multiple empirical method involves identifying key concepts from studies studies but, unlike meta-analysis, the sample is purposive rather and translating them into one another. The term 'translat- than exhaustive because the purpose is interpretive explanation ing' in this context refers to the process of taking concepts and not prediction". This suggests that it may not be neces- from one study and recognising the same concepts in sary to locate every available study because, for example, another study, though they may not be expressed using the results of a conceptual synthesis will not change if ten identical words. Explanations or theories associated with rather than five studies contain the same concept, but will these concepts are also extracted and a 'line of argument' depend on the range of concepts found in the studies, may be developed, pulling corroborating concepts their context, and whether they are in agreement or not. together and, crucially, going beyond the content of the Thus, principles such as aiming for 'conceptual saturation' original studies (though 'refutational' concepts might not might be more appropriate when planning a search strat- be amenable to this process). Some have claimed that this egy for qualitative research, although it is not yet clear notion of 'going beyond' the primary studies is a critical how these principles can be applied in practice. Similarly, component of synthesis, and is what distinguishes it from other principles from primary qualitative research meth- the types of summaries of findings that typify traditional ods may also be 'borrowed' such as deliberately seeking literature reviews [e.g. [32], p209]. In the words of Marga- studies which might act as negative cases, aiming for max- rete Sandelowski, "metasyntheses are integrations that are imum variability and, in essence, designing the resulting more than the sum of parts, in that they offer novel interpreta- set of studies to be heterogeneous, in some ways, instead tions of findings. These interpretations will not be found in any of achieving the homogeneity that is often the aim in sta- one research report but, rather, are inferences derived from tak- tistical meta-analyses. ing all of the reports in a sample as a whole" [[14], p1358]. However you look, qualitative research is difficult to find Thematic analysis has been identified as one of a range of [40-42]. In our review, it was not possible to rely on sim- potential methods for research synthesis alongside meta- ple electronic searches of databases. We needed to search ethnography and 'metasynthesis', though precisely what extensively in 'grey' literature, ask authors of relevant the method involves is unclear, and there are few exam- papers if they knew of more studies, and look especially ples of it being used for synthesising research [35]. We for book chapters, and we spent a lot of effort screening have adopted the term 'thematic synthesis', as we trans- titles and abstracts by hand and looking through journals lated methods for the analysis of primary research – often manually. In this sense, while we were not driven by the termed 'thematic' – for use in systematic reviews [36-38]. statistical imperative of locating every relevant study, As Boyatzis [[36], p4] has observed, thematic analysis is when it actually came down to searching, we found that "not another qualitative method but a process that can be used there was very little difference in the methods we had to with most, if not all, qualitative methods...". Our approach use to find qualitative studies compared to the methods concurs with this conceptualisation of thematic analysis, we use when searching for studies for inclusion in a meta- since the method we employed draws on other estab- analysis. lished methods but uses techniques commonly described Page 3 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 Quality assessment tation of data as findings (as for example when data are Assessing the quality of qualitative research has attracted used to 'let participants speak for themselves'). Sand- much debate and there is little consensus regarding how elowski and Barroso [53] have argued that the findings of quality should be assessed, who should assess quality, qualitative (and, indeed, all empirical) research are dis- and, indeed, whether quality can or should be assessed in tinct from the data upon which they are based, the meth- relation to 'qualitative' research at all [43,22,44,45]. We ods used to derive them, externally sourced data, and take the view that the quality of qualitative research researchers' conclusions and implications. should be assessed to avoid drawing unreliable conclu- sions. However, since there is little empirical evidence on In our example review, while it was relatively easy to iden- which to base decisions for excluding studies based on tify 'data' in the studies – usually in the form of quotations quality assessment, we took the approach in this review to from the children themselves – it was often difficult to use 'sensitivity analyses' (described below) to assess the identify key concepts or succinct summaries of findings, possible impact of study quality on the review's findings. especially for studies that had undertaken relatively sim- ple analyses and had not gone much further than describ- In our example review we assessed our studies according ing and summarising what the children had said. To to 12 criteria, which were derived from existing sets of cri- resolve this problem we took study findings to be all of teria proposed for assessing the quality of qualitative the text labelled as 'results' or 'findings' in study reports – research [46-49], principles of good practice for conduct- though we also found 'findings' in the abstracts which ing social research with children [50], and whether studies were not always reported in the same way in the text. employed appropriate methods for addressing our review Study reports ranged in size from a few pages to full final questions. The 12 criteria covered three main quality project reports. We entered all the results of the studies issues. Five related to the quality of the reporting of a verbatim into QSR's NVivo software for qualitative data study's aims, context, rationale, methods and findings analysis. Where we had the documents in electronic form (e.g. was there an adequate description of the sample used this process was straightforward even for large amounts of and the methods for how the sample was selected and text. When electronic versions were not available, the recruited?). A further four criteria related to the sufficiency results sections were either re-typed or scanned in using a of the strategies employed to establish the reliability and flat-bed or pen scanner. (We have since adapted our own validity of data collection tools and methods of analysis, reviewing system, 'EPPI-Reviewer' [54], to handle this and hence the validity of the findings. The final three cri- type of synthesis and the screenshots below show this teria related to the assessment of the appropriateness of the software.) study methods for ensuring that findings about the barri- ers to, and facilitators of, healthy eating were rooted in Detailed methods for thematic synthesis children's own perspectives (e.g. were data collection The synthesis took the form of three stages which over- methods appropriate for helping children to express their lapped to some degree: the free line-by-line coding of the views?). findings of primary studies; the organisation of these 'free codes' into related areas to construct 'descriptive' themes; Extracting data from studies and the development of 'analytical' themes. One issue which is difficult to deal with when synthesis- ing 'qualitative' studies is 'what counts as data' or 'find- Stages one and two: coding text and developing descriptive themes ings'? This problem is easily addressed when a statistical In our children and healthy eating review, we originally meta-analysis is being conducted: the numeric results of planned to extract and synthesise study findings according RCTs – for example, the mean difference in outcome to our review questions regarding the barriers to, and facil- between the intervention and control – are taken from itators of, healthy eating amongst children. It soon published reports and are entered into the software pack- became apparent, however, that few study findings age being used to calculate the pooled effect size [3,51]. addressed these questions directly and it appeared that we were in danger of ending up with an empty synthesis. We Deciding what to abstract from the published report of a were also concerned about imposing the a priori frame- 'qualitative' study is much more difficult. Campbell et al. work implied by our review questions onto study findings [11] extracted what they called the 'key concepts' from the without allowing for the possibility that a different or qualitative studies they found about patients' experiences modified framework may be a better fit. We therefore tem- of diabetes and diabetes care. However, finding the key porarily put our review questions to one side and started concepts in 'qualitative' research is not always straightfor- from the study findings themselves to conduct an the- ward either. As Sandelowski and Barroso [52] discovered, matic analysis. identifying the findings in qualitative research can be complicated by varied reporting styles or the misrepresen- Page 4 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 There were eight relevant qualitative studies examining thesis of qualitative research: the translation of concepts children's views of healthy eating. We entered the verba- from one study to another [32,55]. However, this process tim findings of these studies into our database. Three may not be regarded as a simple one of translation. As we reviewers then independently coded each line of text coded each new study we added to our 'bank' of codes and according to its meaning and content. Figure 1 illustrates developed new ones when necessary. As well as translat- this line-by-line coding using our specialist reviewing soft- ing concepts between studies, we had already begun the ware, EPPI-Reviewer, which includes a component process of synthesis (For another account of this process, designed to support thematic synthesis. The text which see Doyle [[39], p331]). Every sentence had at least one was taken from the report of the primary study is on the code applied, and most were categorised using several left and codes were created inductively to capture the codes (e.g. 'children prefer fruit to vegetables' or 'why eat meaning and content of each sentence. Codes could be healthily?'). Before completing this stage of the synthesis, structured, either in a tree form (as shown in the figure) or we also examined all the text which had a given code as 'free' codes – without a hierarchical structure. applied to check consistency of interpretation and to see whether additional levels of coding were needed. (In The use of line-by-line coding enabled us to undertake grounded theory this is termed 'axial' coding; see Fisher what has been described as one of the key tasks in the syn- [55] for further discussion of the application of axial cod- li Figure 1 ne-by-line coding in EPPI-Reviewer line-by-line coding in EPPI-Reviewer. Page 5 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 ing in research synthesis.) This process created a total of cal tree structure. New codes were created to capture the 36 initial codes. For example, some of the text we coded meaning of groups of initial codes. This process resulted as "bad food = nice, good food = awful" from one study in a tree structure with several layers to organize a total of [56] were: 12 descriptive themes (Figure 2). For example, the first layer divided the 12 themes into whether they were con- 'All the things that are bad for you are nice and all the cerned with children's understandings of healthy eating or things that are good for you are awful.' (Boys, year 6) influences on children's food choice. The above example, [[56], p74] about children's preferences for food, was placed in both areas, since the findings related both to children's reac- 'All adverts for healthy stuff go on about healthy things. The tions to the foods they were given, and to how they adverts for unhealthy things tell you how nice they taste.' behaved when given the choice over what foods they [[56], p75] might eat. A draft summary of the findings across the stud- ies organized by the 12 descriptive themes was then writ- Some children reported throwing away foods they knew had ten by one of the review authors. Two other review been put in because they were 'good for you' and only ate authors commented on this draft and a final version was the crisps and chocolate. [[56], p75] agreed. Reviewers looked for similarities and differences between the codes in order to start grouping them into a hierarchi- re Figure 2 lationships between descriptive themes relationships between descriptive themes. Page 6 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 Stage three: generating analytical themes effective in increasing consumption. As one child noted Up until this point, we had produced a synthesis which astutely, 'All adverts for healthy stuff go on about healthy kept very close to the original findings of the included things. The adverts for unhealthy things tell you how nice they studies. The findings of each study had been combined taste.' [[56], p75]. We captured this line of argument in the into a whole via a listing of themes which described chil- analytical theme entitled 'Children do not see it as their dren's perspectives on healthy eating. However, we did role to be interested in health'. Altogether, this process not yet have a synthesis product that addressed directly resulted in the generation of six analytical themes which the concerns of our review – regarding how to promote were associated with ten recommendations for interven- healthy eating, in particular fruit and vegetable intake, tions. amongst children. Neither had we 'gone beyond' the find- ings of the primary studies and generated additional con- Results cepts, understandings or hypotheses. As noted earlier, the Six main issues emerged from the studies of children's idea or step of 'going beyond' the content of the original views: (1) children do not see it as their role to be inter- studies has been identified by some as the defining char- ested in health; (2) children do not see messages about acteristic of synthesis [32,14]. future health as personally relevant or credible; (3) fruit, vegetables and confectionery have very different mean- This stage of a qualitative synthesis is the most difficult to ings for children; (4) children actively seek ways to exer- describe and is, potentially, the most controversial, since cise their own choices with regard to food; (5) children it is dependent on the judgement and insights of the value eating as a social occasion; and (6) children see the reviewers. The equivalent stage in meta-ethnography is contradiction between what is promoted in theory and the development of 'third order interpretations' which go what adults provide in practice. The review found that beyond the content of original studies [32,11]. In our most interventions were based in school (though fre- example, the step of 'going beyond' the content of the quently with parental involvement) and often combined original studies was achieved by using the descriptive learning about the health benefits of fruit and vegetables themes that emerged from our inductive analysis of study with 'hands-on' experience in the form of food prepara- findings to answer the review questions we had temporar- tion and taste-testing. Interventions targeted at people ily put to one side. Reviewers inferred barriers and facilita- with particular risk factors worked better than others, and tors from the views children were expressing about multi-component interventions that combined the pro- healthy eating or food in general, captured by the descrip- motion of physical activity with healthy eating did not tive themes, and then considered the implications of chil- work as well as those that only concentrated on healthy dren's views for intervention development. Each reviewer eating. The studies of children's views suggested that fruit first did this independently and then as a group. Through and vegetables should be treated in different ways in inter- this discussion more abstract or analytical themes began ventions, and that messages should not focus on health to emerge. The barriers and facilitators and implications warnings. Interventions that were in line with these sug- for intervention development were examined again in gestions tended to be more effective than those which light of these themes and changes made as necessary. This were not. cyclical process was repeated until the new themes were sufficiently abstract to describe and/or explain all of our Discussion Context and rigour in thematic synthesis initial descriptive themes, our inferred barriers and facili- tators and implications for intervention development. The process of translation, through the development of descriptive and analytical themes, can be carried out in a For example, five of the 12 descriptive themes concerned rigorous way that facilitates transparency of reporting. the influences on children's choice of foods (food prefer- Since we aim to produce a synthesis that both generates ences, perceptions of health benefits, knowledge behav- 'abstract and formal theories' that are nevertheless 'empiri- iour gap, roles and responsibilities, non-influencing cally faithful to the cases from which they were developed' factors). From these, reviewers inferred several barriers [[53], p1371], we see the explicit recording of the develop- and implications for intervention development. Children ment of themes as being central to the method. The use of identified readily that taste was the major concern for software as described can facilitate this by allowing them when selecting food and that health was either a sec- reviewers to examine the contribution made to their find- ondary factor or, in some cases, a reason for rejecting ings by individual studies, groups of studies, or sub-pop- food. Children also felt that buying healthy food was not ulations within studies. a legitimate use of their pocket money, which they would use to buy sweets that could be enjoyed with friends. Some may argue against the synthesis of qualitative These perspectives indicated to us that branding fruit and research on the grounds that the findings of individual vegetables as a 'tasty' rather than 'healthy' might be more studies are de-contextualised and that concepts identified Page 7 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 in one setting are not applicable to others [32]. However, purpose of which is to exclude studies that do not provide the act of synthesis could be viewed as similar to the role a reliable answer to the review question. However, given of a research user when reading a piece of qualitative that there were no accepted – or empirically tested – meth- research and deciding how useful it is to their own situa- ods for excluding qualitative studies from syntheses on tion. In the case of synthesis, reviewers translate themes the basis of their quality [57,12,58], we included all stud- and concepts from one situation to another and can ies regardless of their quality. always be checking that each transfer is valid and whether there are any reasons that understandings gained in one Nevertheless, our studies did differ according to the qual- context might not be transferred to another. We attempted ity criteria they were assessed against and it was important to preserve context by providing structured summaries of that we considered this in some way. In systematic reviews each study detailing aims, methods and methodological of trials, 'sensitivity analyses' – analyses which test the quality, and setting and sample. This meant that readers of effect on the synthesis of including and excluding findings our review were able to judge for themselves whether or from studies of differing quality – are often carried out. not the contexts of the studies the review contained were Dixon-Woods et al. [12] suggest that assessing the feasibil- similar to their own. In the synthesis we also checked ity and worth of conducting sensitivity analyses within whether the emerging findings really were transferable syntheses of qualitative research should be an important across different study contexts. For example, we tried focus of synthesis methods work. After our thematic syn- throughout the synthesis to distinguish between partici- thesis was complete, we examined the relative contribu- pants (e.g. boys and girls) where the primary research had tions of studies to our final analytic themes and made an appropriate distinction. We then looked to see recommendations for interventions. We found that the whether some of our synthesis findings could be attrib- poorer quality studies contributed comparatively little to uted to a particular group of children or setting. In the the synthesis and did not contain many unique themes; event, we did not find any themes that belonged to a spe- the better studies, on the other hand, appeared to have cific group, but another outcome of this process was a more developed analyses and contributed most to the realisation that the contextual information given in the synthesis. reports of studies was very restricted indeed. It was there- fore difficult to make the best use of context in our synthe- Conclusion sis. This paper has discussed the rationale for reviewing and synthesising qualitative research in a systematic way and In checking that we were not translating concepts into sit- has outlined one specific approach for doing this: the- uations where they did not belong, we were following a matic synthesis. While it is not the only method which principle that others have followed when using synthesis might be used – and we have discussed some of the other methods to build grounded formal theory: that of ground- options available – we present it here as a tested technique ing a text in the context in which it was constructed. As that has worked in the systematic reviews in which it has Margaret Kearney has noted "the conditions under which been employed. data were collected, analysis was done, findings were found, and products were written for each contributing report should We have observed that one of the key tasks in the synthesis be taken into consideration in developing a more generalized of qualitative research is the translation of concepts and abstract model" [[14], p1353]. Britten et al. [32] suggest between studies. While the activity of translating concepts that it may be important to make a deliberate attempt to is usually undertaken in the few syntheses of qualitative include studies conducted across diverse settings to research that exist, there are few examples that specify the achieve the higher level of abstraction that is aimed for in detail of how this translation is actually carried out. The a meta-ethnography. example above shows how we achieved the translation of concepts across studies through the use of line-by-line Study quality and sensitivity analyses coding, the organisation of these codes into descriptive We assessed the 'quality' of our studies with regard to the themes, and the generation of analytical themes through degree to which they represented the views of their partic- the application of a higher level theoretical framework. ipants. In doing this, we were locating the concept of This paper therefore also demonstrates how the methods 'quality' within the context of the purpose of our review – and process of a thematic synthesis can be written up in a children's views – and not necessarily the context of the transparent way. primary studies themselves. Our 'hierarchy of evidence', therefore, did not prioritise the research design of studies This paper goes some way to addressing concerns regard- but emphasised the ability of the studies to answer our ing the use of thematic analysis in research synthesis review question. A traditional systematic review of con- raised by Dixon-Woods and colleagues who argue that the trolled trials would contain a quality assessment stage, the approach can lack transparency due to a failure to distin- Page 8 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 guish between 'data-driven' or 'theory-driven' approaches. This paper is a contribution to the current developmental Moreover they suggest that, "if thematic analysis is limited to work taking place in understanding how best to bring summarising themes reported in primary studies, it offers little together the findings of qualitative research to inform pol- by way of theoretical structure within which to develop higher icy and practice. It is by no means the only method on order thematic categories..." [[35], p47]. Part of the prob- offer but, by drawing on methods and principles from lem, they observe, is that the precise methods of thematic qualitative primary research, it benefits from the years of synthesis are unclear. Our approach contains a clear sepa- methodological development that underpins the research ration between the 'data-driven' descriptive themes and it seeks to synthesise. the 'theory-driven' analytical themes and demonstrates how the review questions provided a theoretical structure Competing interests within which it became possible to develop higher order The authors declare that they have no competing interests. thematic categories. Authors' contributions The theme of 'going beyond' the content of the primary Both authors contributed equally to the paper and read studies was discussed earlier. Citing Strike and Posner and approved the final manuscript. [59], Campbell et al. [[11], p672] also suggest that synthe- sis "involves some degree of conceptual innovation, or employ- Acknowledgements The authors would like to thank Elaine Barnett-Page for her assistance in ment of concepts not found in the characterisation of the parts producing the draft paper, and David Gough, Ann Oakley and Sandy Oliver and a means of creating the whole". This was certainly true for their helpful comments. The review used an example in this paper was of the example given in this paper. We used a series of funded by the Department of Health (England). The methodological devel- questions, derived from the main topic of our review, to opment was supported by Department of Health (England) and the ESRC focus an examination of our descriptive themes and we do through the Methods for Research Synthesis Node of the National Centre not find our recommendations for interventions con- for Research Methods. In addition, Angela Harden held a senior research tained in the findings of the primary studies: these were fellowship funded by the Department of Health (England) December 2003 new propositions generated by the reviewers in the light – November 2007. The views expressed in this paper are those of the authors and are not necessarily those of the funding bodies. of the synthesis. The method also demonstrates that it is possible to synthesise without conceptual innovation. References The initial synthesis, involving the translation of concepts 1. Chalmers I: Trying to do more good than harm in policy and between studies, was necessary in order for conceptual practice: the role of rigorous, transparent and up-to-date innovation to begin. One could argue that the conceptual evaluations. Ann Am Acad Pol Soc Sci 2003, 589:22-40. 2. Oakley A: Social science and evidence-based everything: the innovation, in this case, was only necessary because the case of education. Educ Rev 2002, 54:277-286. primary studies did not address our review question 3. Cooper H, Hedges L: The Handbook of Research Synthesis New York: directly. In situations in which the primary studies are Russell Sage Foundation; 1994. 4. EPPI-Centre: EPPI-Centre Methods for Conducting Systematic Reviews concerned directly with the review question, it may not be 2006 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=89]. London: EPPI- necessary to go beyond the contents of the original studies Centre, Social Science Research Unit, Institute of Education, Univer- sity of London in order to produce a satisfactory synthesis (see, for exam- 5. Higgins J, Green S, (Eds): Cochrane Handbook for Systematic Reviews of ple, Marston and King, [60]). Conceptually, our analytical Interventions 4.2.6 2006 [http://www.cochrane.org/resources/hand themes are similar to the ultimate product of meta-ethnog- book/]. Updated September 2006. Accessed 24th January 2007 6. Petticrew M, Roberts H: Systematic Reviews in the Social Sciences: A raphies: third order interpretations [11], since both are practical guide Oxford: Blackwell Publishing; 2006. explicit mechanisms for going beyond the content of the 7. Chalmers I, Hedges L, Cooper H: A brief history of research syn- primary studies and presenting this in a transparent way. thesis. Eval Health Prof 2002, 25:12-37. 8. Juni P, Altman D, Egger M: Assessing the quality of controlled The main difference between them lies in their purposes. clinical trials. BMJ 2001, 323:42-46. Third order interpretations bring together the implications 9. Mulrow C: Systematic reviews: rationale for systematic reviews. BMJ 1994, 309:597-599. of translating studies into one another in their own terms, 10. White H: Scientific communication and literature retrieval. whereas analytical themes are the result of interrogating a In The Handbook of Research Synthesis Edited by: Cooper H, Hedges L. descriptive synthesis by placing it within an external theo- New York: Russell Sage Foundation; 1994. 11. Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan retical framework (our review question and sub-ques- J: Evaluating meta-ethnography: a synthesis of qualitative tions). It may be, therefore, that analytical themes are more research on lay experiences of diabetes and diabetes care. Soc Sci Med 2003, 56:671-684. appropriate when a specific review question is being 12. Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, addressed (as often occurs when informing policy and Shaw RL, Smith JA, Young B: How can systematic reviews incor- practice), and third order interpretations should be used porate qualitative research? A critical perspective. Qual Res 2006, 6:27-44. when a body of literature is being explored in and of itself, 13. Sandelowski M, Barroso J: Handbook for Synthesising Qualitative with broader, or emergent, review questions. Research New York: Springer; 2007. 14. Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M: Qualita- tive meta-synthesis: reflections on methodological orienta- tion and ideological agenda. Qual Health Res 2004, 14:1342-1365. Page 9 of 10 (page number not for citation purposes) BMC Medical Research Methodology 2008, 8:45 http://www.biomedcentral.com/1471-2288/8/45 15. Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley 37. Braun V, Clarke V: Using thematic analysis in psychology. Qual A: Applying systematic review methods to studies of people's Res Psychol 2006, 3:77-101 [http://science.uwe.ac.uk/psychology/ views: an example from public health. J Epidemiol Community drvictoriaclarke_files/thematicanalysis%20.pdf]. Health 2004, 58:794-800. 38. Silverman D, Ed: Qualitative Research: Theory, method and practice Lon- 16. Harden A, Brunton G, Fletcher A, Oakley A: Young People, Pregnancy don: Sage; 1997. and Social Exclusion: A systematic synthesis of research evidence to identify 39. Doyle LH: Synthesis through meta-ethnography: paradoxes, effective, appropriate and promising approaches for prevention and support enhancements, and possibilities. Qual Res 2003, 3:321-344. 2006 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=674]. London: 40. Barroso J, Gollop C, Sandelowski M, Meynell J, Pearce PF, Collins LJ: EPPI-Centre, Social Science Research Unit, Institute of Education, The challenges of searching for and retrieving qualitative University of London studies. Western J Nurs Res 2003, 25:153-178. 17. Thomas J, Sutcliffe K, Harden A, Oakley A, Oliver S, Rees R, Brunton 41. Walters LA, Wilczynski NL, Haynes RB, Hedges Team: Developing G, Kavanagh J: Children and Healthy Eating: A systematic review of barriers optimal search strategies for retrieving clinically relevant and facilitators 2003 [http://eppi.ioe.ac.uk/cms/ qualitative studies in EMBASE. Qual Health Res 2006, 16:162-8. Default.aspx?tabid=246]. London: EPPI-Centre, Social Science 42. Wong SSL, Wilczynski NL, Haynes RB: Developing optimal Research Unit, Institute of Education, University of London accessed search strategies for detecting clinically relevant qualitative th 4 July 2008 studies in Medline. Medinfo 2004, 11:311-314. 18. Thomas J, Kavanagh J, Tucker H, Burchett H, Tripney J, Oakley A: 43. Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P: Qualita- Accidental Injury, Risk-Taking Behaviour and the Social Circumstances in tive research methods in health technology assessment: a which Young People Live: A systematic review 2007 [http://eppi.ioe.ac.uk/ review of the literature. Health Technol Assess 1998, 2(16):. cms/Default.aspx?tabid=1910]. London: EPPI-Centre, Social Science 44. Seale C: Quality in qualitative research. Qual Inq 1999, Research Unit, Institute of Education, University of London 5:465-478. 19. Bryman A: Quantity and Quality in Social Research London: Unwin; 45. Spencer L, Ritchie J, Lewis J, Dillon L: Quality in Qualitative Evaluation: 1998. A framework for assessing research evidence London: Cabinet Office; 20. Hammersley M: What's Wrong with Ethnography? London: Routledge; 2003. 1992. 46. Boulton M, Fitzpatrick R, Swinburn C: Qualitative research in 21. Harden A, Thomas J: Methodological issues in combining healthcare II: a structured review and evaluation of studies. diverse study types in systematic reviews. Int J Soc Res Meth J Eval Clin Pract 1996, 2:171-179. 2005, 8:257-271. 47. Cobb A, Hagemaster J: Ten criteria for evaluating qualitative 22. Oakley A: Experiments in Knowing: Gender and methods in the social sci- research proposals. J Nurs Educ 1987, 26:138-143. ences Cambridge: Polity Press; 2000. 48. Mays N, Pope C: Rigour and qualitative research. BMJ 1995, 23. Harden A, Oakley A, Oliver S: Peer-delivered health promotion 311:109-12. for young people: a systematic review of different study 49. Medical Sociology Group: Criteria for the evaluation of qualita- designs. Health Educ J 2001, 60:339-353. tive research papers. Med Sociol News 1996, 22:68-71. 24. Harden A, Rees R, Shepherd J, Brunton G, Oliver S, Oakley A: Young 50. Alderson P: Listening to Children London: Barnardo's; 1995. People and Mental Health: A systematic review of barriers and facilitators 51. Egger M, Davey-Smith G, Altman D: Systematic Reviews in Health Care: 2001 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=256]. London: Meta-analysis in context London: BMJ Publishing; 2001. EPPI-Centre, Social Science Research Unit, Institute of Education, 52. Sandelowski M, Barroso J: Finding the findings in qualitative University of London studies. J Nurs Scholarsh 2002, 34:213-219. 25. Rees R, Harden A, Shepherd J, Brunton G, Oliver S, Oakley A: Young 53. Sandelowski M: Using qualitative research. Qual Health Res 2004, People and Physical Activity: A systematic review of barriers and facilitators 14:1366-1386. 2001 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=260]. London: 54. Thomas J, Brunton J: EPPI-Reviewer 3.0: Analysis and management of EPPI-Centre, Social Science Research Unit, Institute of Education, data for research synthesis. EPPI-Centre software London: EPPI-Centre, University of London Social Science Research Unit, Institute of Education; 2006. 26. Shepherd J, Harden A, Rees R, Brunton G, Oliver S, Oakley A: Young 55. Fisher M, Qureshi H, Hardyman W, Homewood J: Using Qualitative People and Healthy Eating: A systematic review of barriers and facilitators Research in Systematic Reviews: Older people's views of hospital discharge 2001 [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=258]. London: London: Social Care Institute for Excellence; 2006. EPPI-Centre, Social Science Research Unit, Institute of Education, 56. Dixey R, Sahota P, Atwal S, Turner A: Children talking about University of London healthy eating: data from focus groups with 300 9–11-year- 27. Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton olds. Nutr Bull 2001, 26:71-79. G, Kavanagh J: Integrating qualitative research with trials in 57. Daly A, Willis K, Small R, Green J, Welch N, Kealy M, Hughes E: Hier- systematic reviews: an example from public health. BMJ 2004, archy of evidence for assessing qualitative health research. J 328:1010-1012. Clin Epidemiol 2007, 60:43-49. 28. Davies P: What is evidence-based education? Br J Educ Stud 58. Popay J: Moving beyond floccinaucinihilipilification: enhancing 1999, 47:108-121. the utility of systematic reviews. J Clin Epidemiol 2005, 29. Newman M, Thompson C, Roberts AP: Helping practitioners 58:1079-80. understand the contribution of qualitative research to evi- 59. Strike K, Posner G: Types of synthesis and their criteria. In dence-based practice. Evid Based Nurs 2006, 9:4-7. Knowledge Structure and Use: Implications for synthesis and interpretation 30. Popay J: Moving Beyond Effectiveness in Evidence Synthesis London: Edited by: Ward S, Reed L. Philadelphia: Temple University Press; National Institute for Health and Clinical Excellence; 2006. 1983. 31. Noblit GW, Hare RD: Meta-Ethnography: Synthesizing qualitative studies 60. Marston C, King E: Factors that shape young people's sexual Newbury Park: Sage; 1988. behaviour: a systematic review. The Lancet 2006, 368:1581-86. 32. Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R: Using meta-ethnography to synthesise qualitative research: a Pre-publication history worked example. J Health Serv Res Policy 2002, 7:209-215. The pre-publication history for this paper can be accessed 33. Paterson B, Thorne S, Canam C, Jillings C: Meta-Study of Qualitative Health Research Thousand Oaks, California: Sage; 2001. here: 34. Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Har- vey J, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretative synthesis of the literature on access http://www.biomedcentral.com/1471-2288/8/45/prepub to healthcare by vulnerable groups. BMC Med Res Methodol 2006, 6:35. 35. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesis- ing qualitative and quantitative evidence: a review of possi- ble methods. J Health Serv Res Policy 2005, 10:45-53. 36. Boyatzis RE: Transforming Qualitative Information Sage: Cleveland; 1998. Page 10 of 10 (page number not for citation purposes)

Journal

BMC Medical Research MethodologySpringer Journals

Published: Jul 10, 2008

There are no references for this article.