SCMGT577-25B: E-enabled Agile Supply Chains Research Proposal – Individual Assignment

1. Introduction

This assignment aims to provide a first-hand understanding of research activities in e-enabled agile supply chains. The assignment simulates the process of planning, conducting and reporting on a research study, which increases confidence that the research objectives will be achievable. Essentially, you will conduct a comprehensive literature review of the state-of-the-art of SCM and its relationship to other contemporary business initiatives. Although assessed as an individual assignment (contributions subject to peer
assessment), students will work in groups for reasons of mutual support to:

• Coordinate the writing of a research proposal report
• Prepare a presentation based on the report contents
• Create an online learning resource suitable for assisting a novice researcher

2. Theme, Topics and Subtopics

For 2025, the overarching theme is: Transforming Supply Chain Management with AI There are 4 groups. Each group may choose one of the following topic areas, provided it differs from the one selected for their L-Tec presentation. Repeats within the class are not allowed:

1. Data Analytics and AI in SCM
2. Supply Chain Resilience and Risk Mitigation with AI
3. AI in Sourcing and Procurement
4. AI for Collaborative Supply Chains

Each group member may select any subtopic that interests them, provided it falls within the scope of the group’s topic area. Once selected, that subtopic will be the subject of the entire individual report. There is no restriction by industry or country. No repeats are allowed within the group. Some ideas for subtopics are included in Appendix 1.

3. Assessment Summary

This major assignment is worth 32% of the total marks available for the paper, and it has three related deliverables:

1. INDIVIDUAL Research Proposal Report (one submitted by each group)

• This deliverable consists of:
• Part A: An Introduction section authored by the whole group
• Part B: Individual contributions written by each group member 14%

2. GROUP Research Presentation (20 minutes in total) 10%

3. GROUP Online Learning Resource (one submitted by each group) 8%

• This (group) deliverable aims to provide a valuable, engaging, and easy-to-use Online Learning Resource for the class that a novice researcher would find invaluable for quickly getting up to speed. It should incorporate and leverage the contents of the Report.

4. Research Proposal Report

General Requirements – please pay close attention to these details for maximum marks

• Use a formal writing style throughout, i.e. utilise enough subheadings, write in the past tense and avoid using personal pronouns (‘I’, ‘me’, ‘we ‘…)
• Avoid making sweeping statements and sensationalist generalisations. Be sure to back claims made with academic references or class notes
• Cite every source mentioned in your report using the American Psychological Association (APA) style. For details, go to https://apastyle.apa.org/ and select the ‘Style and Grammar’ tab at the top of the page, then select the ‘References’ style guide (Moodle also has details)
• Use of the ‘ResearchRabbit’ AI tool is highly recommended (https://www.researchrabbit.ai/), a citationbased literature search and mapping tool. View a short video here: https://www.youtube.com/watch?v=W1W51rYJA3I&t=47s&ab_channel=ResearchRabbit
• Using ChatGPT or any other AI tool for writing/rewriting/formatting the report is forbidden. The only exception to this is the use of Grammarly TM
• Be sure to comply with the university’s regulations regarding plagiarism and the use of AI.
• All work is submitted to TurnitinTM

Report Guidelines

The body of the report must contain all the (bulleted) sections indicated below and should address every point in sufficient detail. Specifically, the requirements call for a:

Title Page headed: Supply Chain Logistics and Artificial Intelligence (= Overarching Theme)

• The title page must also contain a group identifier, member names and submission date
  – All the items to be centred on the same page

These details are followed (on a new page) by Part A of the report as follows:

Part A: Group Topic (= the Main Title for Part A)

History and Current Status

For Part A (only), the group must summarise the available literature (using journal articles mostly, plus credible articles in the press/online) to:

a. Clearly explain the chosen topic. What are the factors driving the need for global development of the Group Topic within the broader context of the Overarching Theme? What are the commercial and political pressures, legal requirements, technical drivers, and social and cultural issues?

b. What are the main challenges to further development?

c. Each (named) group member should then:

• Name their chosen Subtopic
• Briefly explain the relevance/importance of their Subtopic to the Group Topic.
• Describe significant trends that justify their Subtopic as a crucial research focus.

Finally, please state the total word count for Part A –
Allow 300 words for parts a and b (combined) and 150 words per student for part c.

Requirements for Part B of the Report (the individual student contributions)
Begin a new page:

Part B: Individual Subtopics (= the Main Title for Part B)

For consistency, every group member should format their individually chosen subtopic as follows:

Begin a new page:

Subtopic 1: Title of the first group member’s subtopic

Submitted by: First person’s full name (Student ID) (Here, the first group member listed in Section A describes the state of knowledge and research concerning Subtopic 1)

a) Introduction – History and Current Status

i. Provide a clear description of Subtopic 1, including definitions of terms.
ii. What factors are driving global developments, such as commercial and political pressures, and legal, technical, social, and cultural issues?
iii. What are the main implications for practitioners?

This Introduction (around 300 words) is followed by the main section (Around 2,500 words) entitled:

b) Systematic Literature Review of: Subtopic 1

Preparing for the Review – Recommended Procedure

N.B. It is essential to have a keyword search strategy in place before searching for relevant literature. A comprehensive description of a recommended systematic search process is given in Appendix 2.

The contents to include in this part of the report are: (please use these subheadings)

i. Significant Works

• Place the most significant research works you have identified into a table, i.e., modify the contents of Table 3 in the Otago University guide.
  – DO NOT copy/paste these from an existing publication, as this would be plagiarism

ii. Literature Critique

N.B. When reviewing the literature for your subtopic, it is vital to report on and make sense of the available research findings and NOT merely explain the subtopic (a clear explanation should already be in your introduction). An excellent approach for reviewing academic literature, recommended by Otago University, can be accessed here: https://www.otago.ac.nz/hedc/otago615355.pdf

Hint: When previewing the literature you find, skim through it first to see whether the source is relevant before you read it in detail. Table 2 in the Otago University guide indicates the key areas to check.

VITAL: Review the research findings and do not merely explain and summarise them. It would be best to use the procedure in the guide: https://guides.library.ucsc.edu/write-a-literature-review.  There is a fuller description at the end of this document. Also, you are highly recommended to use the AI tool ‘ResearchRabbit’ (https://www.researchrabbit.ai). In short, write a review of the academic research literature on Subtopic 1, primarily citing refereed academic publications.

Using the recommended approach, critique the research literature, including the research methods and main findings (summary statements are sufficient), as follows:

• Synthesis and Critique
  Synthesise and critique the research literature on Subtopic 1 (provide summary statements)
• Importance of the Research
  Why is it essential to research Subtopic 1?
• Most Significant Research to Date
  Identify and discuss the most significant research on Subtopic 1 (include methods and main findings)
• Major Gaps
  What are the major gaps in the literature concerning Subtopic 1
• Review Summary
  Describe (ideally using a diagram), one academic framework that researchers use to explain the relationship between the main variables impacting/being impacted by Subtopic 1. You may choose from acceptance models /decision-making models/behavioural models, etc. , bu t do no t rep eat any hidde n meta data o r commen ts.

c) Research Proposal

Design a research proposal to address just one of the gaps you identified. This should contain brief statements that address the following points (quality rather than quantity is being assessed):

I. Introductory Section

i. State the primary research question (or hypothesis) for your proposed study, and any secondary questions (hypotheses) that will help to address it (Hint: tiny steps are preferred that extend present knowledge – DO NOT try to solve all the world’s problems in one go!)
ii. Be sure to justify the main research question (hypothesis) using your literature review findings.
iii. Describe how the study findings will be helpful. In what way(s) will they advance knowledge/contribute to the literature?
iv. How will the findings also be helpful to practitioners?

II. Methodology

The proposal must include plans for primary data collection (or simulation). A literature review by itself is insufficient.

i. Justify the choice of overall research methodology (quantitative or qualitative).
ii. What method(s) (interview, experiment, case study, survey, or simulation) have you selected for data collection? Why is this judged to be the most suitable?
iii. What population (persons, departments, organisations, economies) have you selected? Why is this choice appropriate?
iv. What sample have you selected? Why?
v. Procedure: Describe the planned data collection procedure.
vi. Data Analysis: How do you intend to analyse the data to examine each question (hypothesis?
vii. Are the findings likely to be generalisable? (applicable in other contexts and settings)

III. Findings

i. Provide a summary of the expected findings. In the absence of actual results, use shells, i.e. empty tables and graphs, as appropriate to illustrate what the study is hoping to demonstrate

IV Discussion

i. What are the limitations and weaknesses of your proposed study?
ii. What are likely fruitful areas for additional study?
Begin a new page:

Word Count

• 2,800 words for Subtopic 1 with a 10% variation allowed either way)
  o Please state the word count for this subtopic, which excludes its Reference List contents

Reference List

The list is to be alphabetised and separated into different source types – books, journals, conferences, and links (use APA style).

Begin a new page:

Subtopic 2: Title of the second group member’s subtopic
Submitted by: Second group member’s full name (Student ID)

(Here, the second group member listed in Section A describes the state of knowledge and research concerning Subtopic 2):

Introduction
Systematic Literature Review
Research Proposal
Word Count
Reference List

Repeat for the remaining subtopics…

5. Online Learning Resource

• This single deliverable is to be created and submitted by the whole group
• The group should aim to create a valuable, engaging, and user-friendly online learning resource for the class. It should incorporate and leverage the contents of all the individual reports by the group.
• The use of AI tools is encouraged to assist with website creativity and resource development, subject to the conditions described in the ‘Use of AI Tools’ section (on Moodle).
• You can use free drag-and-drop tools, such as those available at www.weebly.com or www.wix.com (or similar), to create and publish your learning resource website.
• While this is an open brief, you should endeavour to make the learning resource helpful, enjoyable and easy to navigate and use (for example, you might link directly to reference sources and other external links, videos, or anything else you judge will bring the contents to life)

6. Recommended Assignment Activities – all parts

a. Everyone undertakes 1-2 hours of individual keyword searching to comprehend the scope of the overarching theme and the available group topic areas.
b. Each group agrees on its chosen topic area (please notify the paper convenor) and achieves a shared understanding of the overarching theme.
c. Every group member undertakes 4-5 hours of hands-on web/library searching to address their subtopics directly. It is vital to source frameworks or models (especially in diagrammatic form) that show whatn researchers believe are the main underlying factors and how they fit together. Also, be alert to opportunities for future research that can appear at the end of academic articles, as these can indicate how frameworks might be extended or reused.
d. Every group member prepares brief notes for Part A of the Research Report ahead of another group meeting to agree the final contents of the final report and the online learning resource.
e. Every group member finalises their individual and shared sections for the report.
f. Every group member prepares brief notes containing their ideas for the online learning resource ahead of the next group meeting to agree on the final format and contents.
g. Every group member undertakes 1-2 hours of hands-on effort to compile the material and develop (and test) the Online Learning Resource.
h. One group member submits the completed Research Proposal Report via Moodle (Own Work Central section)
i. One group member submits the link to the group’s online learning resource via Moodle. Paste the link into a Word document and submit the document via Moodle (Own Work Central section)

Scope and expected effort: approximately 15-20 hours of contribution by each group member.

• Please ensure that the names of every (contributing) group member appear on all work.

Research Proposal Report – Individual Marking Guide

Section A: Introduction (Group) Group:

• Does a Report Summary introduce the overarching Topic and the chosen Subtopic, and why they are both critical areas for study? Was every group member’s (individually determined) Sub-subtopic listed?
• Part A, did the group clearly define its chosen Subtopic and describe what is driving its global need/ development? Commercial pressures, political, legal, technical and social and cultural issues/drivers? Also
• Was each Sub-subtopic briefly introduced, including its relationship to the Subtopic? Are significant trends used to illustrate why this Subsubtopic is a crucial area for study? Future predictions? (total of 100 words/student)
• Word count within 10%? /10%

Section B: Main Body (Individual)

• Referencing: Were the listed references appropriate in terms of APA style?Were the references ordered by the source type?
  Were hyperlinks to reference sources included? /6%
• Word Count: Was the total word count within 10% of the (2,800-word) specification? /4%

Your Research Proposal Report TOTAL: /100%

Research Presentation Assessment

Assessment of presentation performance will be according to the criteria indicated below.

SCMGT577 Research Proposal Presentation – Group Marking Guide

Goal: Participants should aim to report clearly on their analysis and findings in an engaging, informative and professional manner, broadly following the recommended order and content. The time allowed is 20 minutes for the whole presentation.

Guidelines for Preparing the Presentation

This part of the assignment aims to present the research proposal findings to the class, either in person or via online submission.

Each group needs to pool its resources to prepare a presentation based on the contents of the research report.

Please ensure that slides/videos/other media are not too detailed and are engaging and relevant (there is no time for padding or overstating what is blindingly obvious – strict timekeeping is essential).

Content – BY THE WHOLE GROUP (NOT BY EACH INDIVIDUAL)
The presentation should be in 2 parts:
Findings of Part A of the Investigation (5 minutes), followed by
Individual Findings of Part B of the Investigation (balance of the 20-minute presentation)

Recommended Procedure – How to prepare for the presentation

1. Each group:
a. Agrees on who will present each segment of the presentation – the speaker order
b. Agrees who will be the presentation ‘Compiler.’ This person will compile the separate PowerPoint slides from each member into the final presentation and email it, or a link, to the convenor

2. Each group member:
a. Sources the material for their presentation segment – feel free to share relevant information between members
b. Prepares speaker notes – being sure to introduce themselves and signpost what their presentation segment will cover
c. Creates their slides in PowerPoint (or similar) – the first slide should contain the speaker’s details, expressed as: #. First Name Last Name, where # is a reference to the speaker order (the Compiler uses this information when creating the group’s slide deck)
d. Rehearses their segment to ensure that the timing is perfect. It is helpful for the whole team to ‘meet’ for a dry run.

Recommended Procedure – How to submit a video presentation (not required if presenting in class)

Each group member:

a. Records a VIDEO CLIP with the aid of speaker notes. You are recommended to use a recording device(s) that offers the best picture and sound quality. You should also prepare a separate video clip for each slide. The ‘Camera’ app is a basic video editing app included with Windows 10. A more powerful video editor like VideoPad is also worth considering for refining raw video clip(s)

b. Uploads their video clip(s) to YouTube, as this enables YouTube video link(s) to be inserted into the PowerPoint slide(s) rather than video clip(s). This approach reduces the .ppt file size because the videos are stored elsewhere. To do this on a PC, go to YouTube.com/upload; select the video you created on your hard drive; provide any needed details; set Visibility to ‘Unlisted’; and make a note of the video link generated by YouTube at the end (uploading to YouTube is similar for iOS and Android)

c. Incorporates the video link(s) generated by YouTube onto their PowerPoint slide(s). The following graphic illustrates the use of a slide template in PowerPoint suitable for this task. Yellow highlighting indicates how (from the Insert menu) it is possible to insert a ‘two-content’ slide (i.e., a text plus video slide) into your presentation. Orange highlighting indicates the location of video/audio tools if needed. Select Insert | Video| Online Video and enter the
YouTube video link details. The video clip window can be resized and moved somewhere more suitable once done

d. Emails their completed slide(s) to the ‘Compiler’ – the person who agreed to create and submit the final version of the slideshow for the team.

The compiler:

a. Loads all the slides in the correct order into a fresh presentation, adds a title slide and makes any final tweaks to the overall presentation to ensure consistency

b. Emails the finished version directly to the lecturer (eric.deakins@waikato.ac.nz). If a Gmail size limit is encountered, it is best to use a shareable Google Drive link instead. To do this, select ‘Drive’ from the Google apps menu located at the top-right in Gmail; select +New | File upload
(then select the completed PowerPoint file), and the file uploads. While still in Drive, select ‘Recent’ from the menu at the left, then right-click on the file you just uploaded and select ‘Share’ (enter the email address: eric.deakins@waikato.ac.nz and identify your Team in the ‘Add a note’
box below). Press Send. All done!

Beware: Considerable delays and technical issues are not unusual when uploading to YouTuben and Google Drive. Do not leave uploads to the last minute

Addendum

If your group is feeling adventurous and has the time to learn something useful, read on!

OBS Studio (https://obsproject.com/)

OBS is a free download that offers a powerful and relatively user-friendly way to create video clips directly from multiple input sources: e.g., a PC’s slideshow display, video from a camera/USB device, and microphone audio – although there is no limit to the number of sources. Great introduction videos can be found here:

https://www.youtube.com/watch?v=QKmrDUJFRkM
https://www.youtube.com/watch?v=DTk99mHDX_I
https://www.youtube.com/watch?v=uL8BwstqiqE

For teams deciding to go the OBS or similar route, the main difference is that the Compiler downloads/stitches together the separate video clips that were created directly or uploaded to YouTube. The finished video can then be uploaded to YouTube, and the link to it link emailed to the lecturer. Enjoy!

Appendix 1:

Transforming Supply Chain Management with AI Supply chains worldwide are transforming as external pressures, technology trends, and internal evolutions
all prompt companies to reevaluate their network and determine how best to restructure in terms of resilience, capacity and capabilities. AI can help this transformation.

The following topics are available for groups to consider in 2025. The bulleted points are only initial pointers to help you get started – be sure to consider others:

Data Analytics and AI in SCM

• Data Analytics and AI in Supply Chain Decision-making (focus on internal processes)
• Machine Learning Applications in Supply Chain Management
• Leveraging ChatGPT for Text Analysis and Natural Language Understanding in Supply Chains

Supply Chain Resilience and Risk Mitigation with AI
• Identifying Supply Chain Risks and Vulnerabilities with AI
• AI-Driven Risk Prediction
• AI-driven Mitigation Strategies

AI in Sourcing and Procurement

• AI-Enabled Supplier Discovery and Relationship Management (SRM)
• Supplier Performance Monitoring and Optimisation with AI
• AI Implementation issues in Sourcing and Supplier Management

AI for Collaborative Supply Chains

• AI-Driven Collaboration Platforms and Supply Chain Integration
• AI for Demand Forecasting and Inventory Management
• AI Enhanced Collaboration in Supply Chain Networks

Appendix 2:

Guidance on Conducting a Systematic Literature Review – Abridged

Journal of Planning Education and Research
2019, Vol. 39(1) 93–112
© The Author(s) 2017
Article reuse guidelines: sagepub.com/journals-permissions DOI: 10.1177/0739456X17723971 journals.sagepub.com/home/jpe
Yu Xiao1 and Maria Watson1

Abstract

Literature reviews establish the foundation of academic inquiries. In this article, through a systematic search of the literature review methodology, we discuss steps in conducting a systematic literature review.

Process of Literature Review

A successful review involves three major stages: planning, conducting, and reporting results. In the planning stage, researchers and others identify the need for a review, specify research questions (or a tight topic area if a
topic review), and develop a review protocol. When conducting the review, they identify and select primary studies, extract, analyse, and synthesise data. When reporting the review, they write the report to disseminate their findings from the literature review.

All reviews can be conducted following eight common steps (Figure 1).

Step 1: Formulate the Problem

Literature reviews are research/investigation inquiries, and all inquiries should ideally be guided by research questions (or a very tightly defined topic). The selection of studies to be included in the review, methodology for data extraction and synthesis, and reporting should all be geared toward answering the research questions (or thoroughly describing a topic).

A common mistake for novicesis selecting a research question or topic that is too broad. This can result in a massive amount of data identified for the review, making the study unmanageable. If this happens, the researchers should, for example, choose a subtopic within the original area for the review.

Figure 1. Process of systematic literature review.

Identifying the appropriate research question (tightly defined topic) can be an iterative process, and one might consider using a pre-review mapping to help identify candidates. After an initial search of the literature on the research question (topic), the researchers can conduct a quick mapping procedure to determine, for instance, the range of subtopics, the number of studies within each subtopic, and the years the studies were carried out.

Pre-review mapping helps researchers decide whether it is feasible to review the bulk of materials or whether they need to narrow it down to a more specific research question (topic).

Step 2: Develop and Validate the Review Protocol

The review protocol is a pre-set plan specifying the methods used to conduct the review. The review protocol is crucial for rigorous systematic reviews to enhance the review quality because itreduces the possibility of researcher
bias in data selection and analysis. It also increases the reliability of the evaluation because others can use the same protocol to repeat the study for cross-checking and verification.

The review protocol should describe all the review elements, including the study’s purpose, research questions, inclusion criteria, search strategies, quality assessment criteria, screening procedures, and data extraction,
synthesis, and reporting strategies. Including a project timetable in the protocol is also helpful for keeping the study on track.

Step 3: Search the Literature

The quality of a literature review is highly dependent on the literature collected for the study—”Garbage-in, garbage-out.” The literature search finds materials for the appraisal; therefore, a systematic review depends on
a systematic literature search.

Channels for literature search. There are three primary sources to find literature: (1) electronic databases; (2) backwards searching; and (3) forward searching. Because no database includes the complete set of published
materials, a systematic literature search should draw from multiple databases. Google Scholar is a robust openaccess database that archives journal articles and “grey literature,” such as conference proceedings, theses, and
reports.

Google and Google Scholar combined can often find more than three-quarters of open-access publications, although the performance of search engines varies across disciplines. If the study requires literature published
before the Internet age, going through the archive at the library is still necessary.

To obtain a complete list of literature, researchers should conduct a backwards search to identify relevant works cited by the articles. The list of references at the end of the article is an excellent way to find these articles. Also, a forward search is needed to see all articles that have since cited the papers reviewed. Search engines such as Google Scholar and the ISI Citation Index allow forward tracking of reports.

One can also perform backwards and forwards searches by author. The researchers can ensure that their relevant studies are included by searching the publications by the key authors who contribute to the body of work.
Searching for CVs, Google Scholar pages, and listed publications on the researcher’s network, such as ResearchGate.net, is a good way to find their other publications. Contacting the authors by email and phone is an
alternative approach.

Also, finding existing systematic reviews as a starting point for the forward and backwardssearches is often helpful.

Keywords used for the search. The keywords for the search should be derived from the research question (topic). Researchers can dissect the research question into concept domains. For example, if the research question is
“What factors affect business continuity after a natural disaster?” -the fields are “business,” “continuity,” and “natural disaster.” A trial search with these keywords could retrieve a few documents roughly and quickly. For instance, a search of “business” + “continuity” + “natural disaster” on Google Scholar yielded lots of articles on business continuity planning, which do not answer the research question. This tells us we need to adjust the keywords.

One can also extend the concepts in the search statement by synonyms, abbreviations, alternative spellings, and related terms. The synonyms of “business” can be “enterprise” and “firm.” Terms related to “continuity” are
“impact,” “recovery,” and “resilience”/”resiliency.” “Natural disaster” can be further broken down into “flood,” “hurricane,” “earthquake,” “drought,” “hail,” “tornado,” etc.

Many search engines allow the use of Boolean operators in the search. Knowing how to construct the search strings using Boolean “AND” and “OR” is essential. “AND” is often used to join the main terms, and “OR” to include
synonyms. Therefore, a possible search string can be— (“business” OR “firm” OR “enterprise”) AND (“continuity” OR “impact” OR “recovery” OR “resilience” OR “resiliency”) AND (“natural disaster”).

One can also search within the retrieved results to further narrow down to a topic. There are a few things to consider when selecting the correct keywords. First, researchers should balance the degree of exhaustiveness and precision. Broader keywords can retrieve more detailed and inclusive results, but

identify more irrelevant articles. In contrast, usingmore precise keywords can improve the accuracy of the search but might result in missing records. At this early stage, being exhaustive is more important than being accurate. Second, cross-country studies researchers should pay attention to the cultural differences in terminology. For instance, “eminent domain” is called “compulsory acquisition”, and “parking lot” i s called “car park” in Australia
and New Zealand. “Urban revitalisation” is typically called “urban regeneration” in the United Kingdom. The search can only be successful if we use the correct vocabulary from the culture of study.

Third is the issue of the evolving vocabulary. For example, in the US, the interstate highway system was initially called “interstate and defence highways” because it was constructed for defence purposes in the Cold War era. The term “defence” was then dropped from the name. Therefore, researchers should be conscious of the vocabulary changes over time. When searching historical literature, one should use the correct vocabulary from
that time.

Fourth, to determine if the keywords are effective, researchers should compare the trial search results with existing lists of primary studies to assess their performance. Finally, it is crucial to document the search date,
the search string, and the procedure. This allows researchers to backtrack the literature search and periodically repeat the search on the same database and sources to identify new materials that might have appeared since
the initial examination.

Sampling strategy. All literature searches are guided by some sampling logic and search strategies adopted by the reviewers. Depending on the purpose of the review, the search can be exhaustive and comprehensive or selective and representative. For example, the purpose of a scoping study is to map the entire domain and requires a thorough and complete search of the literature. Grey literature, such as reports, theses, and conference proceedings, should be included in the investigation. Omitting these sources could result in publication bias.

Other descriptive reviews are not so strict in their sampling strategy, but a good rule of thumb is that the more comprehensive, the better. Testing reviews to produce generalisable findings, such as the meta-analysis or realist
review, requires a thorough search. However, they are more selective in terms of quality. Grey literature might not be employed in such syntheses because they are usually deemed inferior quality compared to peer-reviewed
studies. Reviews to extend the existing body of work can be selective and purposeful. They don’t require identifying all the literature in the domain, but they do require including representative work. Critical reviews are
flexible in their sampling logic. It can highlightthe deficienciesin theexisting body of work,thusbeing very selective and purposive. It can also serve as an evaluation of the entire field, therefore requiring comprehensiveness.
Refining results with additional restrictions. Other practical criteria might include the publication language, date range of publication, and source of financial support. First, reviewers can only read publications in a language
they can understand. Second, the publication date range is often used to limit the search to specific publication periods. We rarely find all the studies published in human history; even if we can, the bulk of the work may be too
much to review. The most recent research may be more relevant to the current situation and can provide more valuable insights. Lastly, in the case of health care research, for example, researchers may only include studies
receiving non-private funds because private funding may be a source of bias in the results. Stopping rule. A rule of thumb is that the search can stop when repeated searchesresult in the same references with no new results. If no further information is obtained from the latest developments, the researchers can conclude the investigation.

Step 4: Screen for Inclusion

After compiling the list of references, researchers should further screen each article to decide whether itshould be included for data extraction and analysis. An efficient approach is to follow a two-stage procedure: first, a coarse
sieve is applied to the articles for inclusion based on the review of abstracts (described in this step), followed by a fine-quality assessment based on a full-text review (described in step 5). This early screening aims to weed out
articles with content inapplicable to the research question(s) or established criteria. At this stage, reviewers should be inclusive. If in doubt, include the articles.

Criteria for inclusion/exclusion. Researchers should establish inclusion and exclusion criteria based on the research question. Exclude studies unrelated to the research question(s). For instance, this article answers the research
question of how to conduct an effective systematic review; therefore, only articles related to the methodology of literature review are included. Exclude literature reviews on specific topics that provide little guidance on the
relevant subject.

Inclusion and exclusion criteria should be practical. The requirements should be capable of classifying research, can be reliably interpreted, and can result in a manageable amount of literature for the review. Pilot the selection criteria before adoption. The inclusion and exclusion criteria can be based on research design and methodology. For instance, studies may be restricted to those carried out in some geographic regions (e.g., developed vs developing countries), of a particular unit of analysis (e.g., individual business vs the aggregate economy; individual household vs the entire community), studying a specific type of policy or event (e.g., Euclidean zoning vs form-based codes; hurricanes vs earthquakes), adopting a particular design of research (e.g., quantitative vs qualitative; cross-sectional vs time-series; computer simulation vs empirical assessment), obtaining data from specific sources (e.g., primary vs secondary data) and of a particular duration (e.g., long-term vs short-term impacts), and utilising a specific sampling methodology (e.g., random sample vs convenience sample) and measurement (e.g., subjective vs objective measures; self-reported vsresearcher-measured) in data collection.

Screening procedure.

The appraisal itself is commonly based on the abstracts of the studies. One could also read the conclusion section if the abstract does not provide enough information. The individual assessment should be inclusive—if in doubt, alwaysinclude the studies. Finally, maintain a list of excluded papers for record-keeping, reproducibility, and crosschecking. This is particularly important for establishing interrater reliability among multiple reviewers.

Step 5: Assess Quality

After screening for inclusion, researchers should obtain full texts of studies for the quality assessment stage. Quality assessment acts as a fine sieve to refine the full-text articles and is the final stage in preparing the pool of studies
for data extraction and synthesis. A quality appraisal is a means for understanding each study before comparing and integrating findings.

Quality standards differ across various types of reviews. For example, quality assessment is not crucial for some types of descriptive reviews and critical reviews: descriptive reviews, such as scoping reviews, are concerned with
discovering the breadth of studies, not the quality, and critical reviews should include analyses of all quality levels to reveal the complete picture. However, quality assessment is essential for reviews aiming for generalisation, such
as testing reviews.

There is no consensus on how reviewers should deal with quality assessment in their review. Some researchers suggest that studies must be sufficiently similar or homogenous in methodological quality to draw meaningful conclusions in review methods such as meta-analyses. In contrast, others think that excluding a large proportion of research on the grounds of poor methodological quality might introduce selection bias and thus diminish the
generalizability of review findings.

Criteria for quality assessment. The term “quality assessment” often refers to checking the “internal validity” of a study. A study is internally valid if it is free from the main methodological biases. Reviewers can judge the quality of the study by analysing the logic from the data collection method to the data analysis, results, and conclusions. Some researchers also include “external validity” or generalizability of the study in the quality assessment stage.

Ranking studies based on a checklist is a common practice for quality assessment. Because of the differences in research design, qualitative and quantitative studies usually require different checklists. For example, Myers (2008) produced a guide to evaluate case studies, ethnographies, and grounded theory. Petticrew and Roberts (2006) provided a collection of checklists for evaluating randomised controlled trials, observational studies, case-control studies, interrupted time series, and cross-sectional surveys. Research institutions, such as the Critical Appraisal Skills Programme (CASP) within the Public Health Resource Unit in the United Kingdom and the Joanna Briggs Institute (JBI), also provide quality checklists that can be adapted to evaluate studies in planning.

The ranking result from the quality assessment can be used in two ways. One way is to “weight” the study qualitatively by placing studiesinto high,medium, and low categories. One should then rely on high-quality studies to construct major arguments and research synthesis before moving on to medium-quality studies. Low-quality studies can be used as supplements but not as foundational literature. The other way to use quality assessment rankings is to “weight” each study quantitatively. For example, in a meta-analysis, one can run the regression analysis using quality scores as “weights”—this way, the higher-quality work gets countedmore heavily than the lower-quality work.

Quality assessment procedure. The full-textreview provides an opportunity for a final check on inclusion/exclusion. Studies not satisfying the inclusion criteria specified in Step 4 should also be excluded from the final literature list. As in step 4, a list of excluded papers should be maintained for record-keeping, reproducibility, and cross-checking.

Step 6: Extracting Data

There are several established methods for synthesising research, which were discussed in the third section. Many of these synthesis methods have been compiled from the medical field, where quality synthesis is paramount to control the influx of new research, as well as qualitative methods papers looking for appropriate ways to find generalisations and overarching themes. The different literature review typologies discussed earlier and the type of literature being synthesised will guide the reviewer to proper synthesis methods. The synthesis methods, in turn, will guide the data extraction process—for example, if one is doing a meta-analysis, data extraction will be centred on what’s needed for a meta-regression. In contrast, a meta-summary will require the extraction of findings.

In general, the process of data extraction will often involve coding, especially for extending reviews. It is essential to establish whether coding will be inductive or deductive (i.e., whether the coding will be based on the data or
pre-existing concepts) (Suri and Clarke 2009). How studies are coded will have a directimpact on the conclusions of the review. For example, in extending reviews such as meta-ethnography and thematic synthesis, conclusions and
generalisations are made based on the themes and concepts that are coded. If this is done incorrectly or inconsistently, the review is less reliable and valid. Additionally, researchers need to review the entire paper, and notsimply rely on the results or the main interpretation. This is the only way to provide contextfor the findings and prevent distortion of the original paper.

Step 7: Analysing and Synthesising

Data Once the data extraction process is complete, the reviewer will organise the data according to the review they have chosen. Often, this will be some combination of charts, tables, and a textual description, though each review
type will have slightly different reporting standards. For example, a meta-analysis will have a results table for the regression analysis, a meta-summary will report effect and intensity sizes, and a framework synthesis will include
a conceptual model.

There are hazards when synthesising different types of literature due to varying epistemological approaches, political and cultural contexts, and political and scientific infrastructure. Combining qualitative literature (such as differences in ontological positions, epistemological positions, paradigms ofinquiry, foundational theories and philosophies, and methodologies) and quantitative literature (such asstudy heterogeneity) poses problems, and some scholars argue it should not be done. In general, however, there are three types of mixed method review designs: segregated design,
integrated design, and contingent design.
A segregated design involves synthesising qualitative and quantitative studies separately according to their respective synthesis traditions and textually combining both results. The analytic themes serve as the framework
for integrating the findings of the quantitative studies into a comprehensive synthesis. An integrated design, by contrast, analyses and synthesises quantitative and qualitative research together. This can be done by transforming one type into the other—quantifying quantitative data or qualitising qualitative data—or by combining them through Bayesian synthesis or critical interpretive synthesis, as described in the third section. Lastly, contingent design is characterised by being a cycle ofresearch synthesis—a group of qualitative or quantitative studiesis used to answer one specific research question (or sub-research question). Then those results will inform the creation of another research question to be analysed by a separate group of studies, and so on. Although groups of studies may end up being exclusively quantitative and qualitative, “the defining feature of contingent designs is the cycle of research synthesis studies conducted to answer questions raised by previous syntheses, not the grouping of studies or methods as qualitative and quantitative”.

Step 8: Report Findings

For literature reviews to be reliable and independently repeatable, the process of systematic literature review must be reported in sufficient detail. This will allow other researchers to follow the same steps described and arrive at
the same results. Particularly, the inclusion and exclusion criteria should be specified in detail, and the rationale or justification of each of the requirements should be explained in the report. Moreover, researchers should report the findings from the literature search, screening, and quality assessment, for instance, in a flow diagram as shown in Figure 2.

The literature review should follow a clear structure that ties the studies together into key themes, characteristics or subgroups. In general,regardless of the rigidity or flexibility of your review methods, ensure the process is transparent and that the data substantiate conclusions. Eachreviewwill have varying degrees ofsubjectivity and going “beyond” the data.

For example, descriptive reviews should be careful to present the data as it is reported, whereas extended reviews will, by nature of the evaluation, move beyond the data. Make sure you are aware of where your review lies on this spectrum and report findings accordingly. In general, all novel findings and unexpected results should be highlighted. The literature review should also point out opportunities and directions for future research. For another explanation and further guidance, visit: https://www.otago.ac.nz/hedc/otago615355.pdf

Figure 2. Literature search and evaluation for inclusion