Chatbot for TESOL Training in Japan

Putting something together for a chatbot for TESOL training in Japan:

https://bit.ly/2WVW5Io

#training #learning #elearning #teaching #tesol #students #education #courses #teachers #Japan #Thailand


0.0 Title Page

Digital Innovation

Digital Innovation – Chatbots for TESOL

David Robertson

0.1 Executive Summary

This report introduces the concept of chatbots being used for TESOL training, in particular retrieval of resources and handouts that will assist students in their studies in real-time. A chatbot will cut down on teacher-engagement time, meaning less hours worked for trainers and assessors, but still allow students to get the answers to many of their questions. The cost effectiveness of a chatbot and ease of hosting through an SAAS service will mean the benefits will ultimately outweigh the minimal costs involved. Success can be judged by how students engage in this chatbot and the efficiency rate on the information they retrieve from the application.

0.2 Table of Contents

0.0 Title page 1

0.1 Executive summary 2

0.2 Table of contents 3

0.3 Introduction 4

1.0 Higher educational institution background 5

1.1 Goals and objectives 5

1.2 Artificial intelligence 5

1.3 Chatbots 5

1.4 How a chatbot can help 6

2.0 Artificial Intelligence maturity 7

2.1 Different use cases for the type of Artificial Intelligence 7

2.2 Current use by higher educational institutions 8

2.3 Most relevant usage cases 8

2.4 Where big impacts can be made 8

3.0 Specific use case 10

3.1 Helping the institution’s objectives 10

3.2 Benefits to stakeholders 10

4.0 Artificial intelligence solution selection 11

4.1 High-level overview of the solution 11

4.3 Cloud-based convenience 11

4.4 SAAS licensing and costings 12

4.5 Current users 12

5.0 Existing technology used 13

5.1 Artificial intelligence integration 13

5.2 Tech enhancements required 13

6.0 The problem solved by artificial intelligence 14

6.1 The expected benefit 14

6.2 The success criteria 14

7.0 Recommendations and conclusion 15

8.0 References 16

0.3 Introduction

Artificial intelligence is being used more and more in education and training, and is now at the point where people interact with chatbots as though they are real people. While in most cases this level of sophistication is not needed, how a training organisation is to think of what is required of their chatbot, how it can be planned to help key stakeholders perform and complete tasks more efficiently, and how it can be classified as a success is something that most training organisations now have to think about in order to remain competitive.

1.0 Higher educational institution background

TESOL Japan operates as a 3rd party affiliate to the Language Training Institute (LTi), an Australian registered training organisation. LTi  offers two ASQA accredited courses to students wishing to study vocational qualifications in teaching English to speakers other than English (TESOL).

1.1 Goals and objectives

As a thirty party affiliate, the goals and objectives of TESOL Japan are in line with those of LTi:

  1. Maintain ASQA accredited TESOL qualifications that can be delivered to students online and in-person.
  2. Maintain high quality standards and services to TESOL students to ensure upon graduation they are ready for vocational TESOL careers.
  3. Ensure students get authentic and reliable information when requested.

1.2 Artificial intelligence

As most students are studying remotely (online), there is no opportunity to physically guide them through the workbooks and study guides. Online lectures and tutorials are held, but these are often missed by students due to timezone and/ or work issues. Students regularly contact LTi and their 3rd party affiliates for information and resources related to their respective course. 

1.3 Chatbots

Chatbots allow for information to be dispensed in real time and programmed to deliver reliable results to students. Using elements of the five skills as set out in Dyer, Gregerson, & Christensen (2009), we can break down why a chatbot may work best for this application.

LTi and 3rd party affiliate trainers have a vast amount of information available in a learning management system to assist students. Students at the same time are asking for information, which may not be easily available to them without an effort to research and search for it. Asking oneself “I need to get information to students quicker, how can I do this?” and“What if we can get the information to students in real-time?” lean us towards thinking a chatbot can work for this real-time information need. Observations have been made in the past. By being able to break down the information into smaller pieces has been done, and was found to help students get through exercises in less time compared with listening to lectures online. Experimentation has been done by using Telegram instant messaging service (2020) by having students contact a custom group for assistance with finding resources and guides. Students receive back a presentation based on that one particular exercise.

1.4 How a chatbot can help

The goal of instantly making available to students specific information they have requested will be of a substantial benefit to both trainers who are overwhelmed with work as it stands now, and students who want information in real-time but do not want to search for it themselves. A chatbot that is pre-loaded with all areas of general information requests will significantly cut down on labor costs as the information will be in a repository, and not require a trainer to be made available for every student request.

2.0 Artificial Intelligence maturity

Rule-based chatbots have a fixed predefined rule framework and are usually handwritten (Adamopoulou & Moussiades 2020). Eliza (1966) was designed to ‘draw out’ patients’ responses. Programming was linked to a keyword, with each keyword being ranked in importance (Jurafsy & Martin 2018). Parry (1972) added fear and anger variables (Garber 2014; Lasar 2011). This chatbot was the first to pass the Turing Test, making the end user believe they are speaking to a human (Heung-Yeung, Xiaodeng & Di Li 2018). A.L.I.C.E (1995) stores knowledge about English language conversations, based around topics and categories (Abushawa & Atwell 2015). The brain of A.L.I.C.E. consists of roughly 41,000 elements called categories. Each category combines a question and answer, or stimulus and response, called the “pattern” and “template” respectively (Epstein, Roberts, & Beber 2008), where the longest pattern match is chosen to answer the enquiry (Wallace 2003). DARPA was developed in the 1990s, designed for spoken dialogue and more complex tasks, such as travel planning (Walker & Hirschman, 2000). While a speech-based system, the design of the system was to make speech to interfaces that include graphics, maps, pointing and gestures (Cover Pages 2000). Unlike A.L.I.C.E and Parry, DARPA was learning-based (DARPA n.d). The objective of the DARPA Communicator program is to support rapid development of multi-modal speech-enabled dialog systems with advanced conversational capabilities (Walker et. al 2002), leading into Siri (2011), the first mass-scale intelligent personal assistant most used today, along with Google Assistant and Amazon Alexa (Koetsier 2020). 

2.1 Different use cases for the type of Artificial Intelligence

Eliza and Parry were designed to reply using a set categories and to match user prompts with scripted responses. This scripting response mechanism kept responses set within the given framework (Konverso 2020). A.L.I.C.E added Artificial Intelligence Mark-up Language (Marietto et. al 2013). A.L.I.C.E added heuristic pattern matching and therefore suited to longer conversations (Shawar & Atwell 2002). A bot like Siri is purely for information requests, working as a virtual assistant, and is frequently used by users over time (Garcia et. al 2018).

2.2 Current use by higher educational institutions

Some universities use chatbots until misunderstandings occur, then the client is transferred to a human (Duckett 2019). The university of Canberra uses chatbots to perform repetitive tasks and answer commonly asked questions (Cameron 2018). The University of Adelaide uses chatbots, such as Facebook Messenger, to assist students in determining ATAR scores (Brookes 2019). The UA example is limited to only certain functions, and is assisted with human administration for more complex information requests. Mumford is used by the University of Tasmania for things such as Fees, Enrolments, Personal Details, Results, Help and Support (University of Tasmania n.d). If questions or requests get too complicated for the Mumford chatbot, clients will be transferred to a human operator. The University of Sydney has set up a chatbot in regards to the Covid-19 pandemic (Microsoft News Centre 2020). This chatbot was designed around two to three most common questions on Covid-19 and university studies. responses are updated each week, usually based around the most common questions (online classes, university is open, stay home orders etc). The complicated nature of university enquiries means that most universities have a built in human feature to connect students to a staff member.

2.3 Most relevant usage cases

Chatbots can be used to engage students in courses, especially those showing signs of dropping out (Yates-Roberts 2019). Students can stay engaged by being able to communicate issues they are having and information can be quickly sent to them that can assist their studies. While finding a course is also a sample of what chatbots can do (Ubisend 2019), finding resources will also add value to students` studies. Due to the kind of information requested by university students and future students, we see that most chatbot applications have a built in human feature to connect to a staff member.

2.4 Where big impacts can be made

As most students are studying remotely (online), there is no opportunity to physically guide them through the workbooks and study guides. Students regularly contact LTi and their 3rd party affiliates for information and documents related to their respective course. With a chatbot built into the current learning management system that holds the presentations and resources, we can expect better engagement with students.

3.0 Specific use case

A supportive environment, in particular with trainers, are factors that contribute to student retention (Harris et. al 2001). The Department of Education and Training mentions one element of student support is access to educators (Australian Government Department of Education and Training 2017). As discussed in Buchanan and Sharma (2009), dropout occurs due to disengagement. A chatbot that can be supportive of students at various stages of their course by allowing students to ask about specific topics at any time will support those who need extra assistance. 

3.1 Helping the institution’s objectives

The objective is to send students relevant and reliable information in real-time by having students enter their queries into a chatbot application (Sandu 2020). This enhances the digital based learning for students. Clients who use chatbots are concerned with the security and safety being at the same level of other online services (Følstad, Bertinussen Nordheim & Bjørkli 2018). As the current resources and handouts forum is publicly available and does not require any personalised information such as log-ons, the chatbot addition does not require any additional logon or privacy concerns.

3.2 Benefits to stakeholders

Chatbots have the ability to enhance the individual support available to students (Winkler & Matthias 2018). Most chatbots are instantly available to users without needed installations (Adamopoulou & Moussiades 2020). Many students use smartphones to check information (Ng et. al 2017), and a chatbot connected to a simple website forum will not require additional downloads. For training staff, it means a reduced workload as a chatbot will take some load off the simple tasks such as sending students information.

4.0 Artificial intelligence solution selection

A rule-based chatbot allows possible user queries and potential answers to be defined in advance. An A.L.I.C.E style of chatbot with categories and rules matching would be the best choice for this kind of project (Abushawar & Atwell 2015). The AIML interpreter tries to match word by word to obtain the longest pattern match. This allows for a simple design as categories can be controlled. 

4.1 High-level overview of the solution

AIML uses categories (atomic, default, recursive) (Satu, Hasnat Parvez & Al Mamun 2015). In general, chat dialogue systems can be categorised into two types: task-oriented systems that are used to assist the user in completing various tasks within a specified domain, and open-domain systems that aim at performing a natural conversation with the user (Higashinaka et. al 2014). A task-based chatbot will work best as it allows us to categorize specific units and topics based on TESOL topics. The task-based framework allows for a simple design compared to a chatbot looking to create natural language based dialogue.

4.2 Compatibility and benefits of the tech

A cloud based chatbot app allows for the application to simply be coded into the header of a website. In modern app development, apps compete and operate over various ecosystems (Draskovic, Markovic & Znidar 2018). This means no special coding or reprogramming of websites is required, saving money on coding costs and any hardware purchases as no new machinery is required. The scalability and single model design means that it can be used over various websites or intranets. A chat-based chatbot will deliver a reliable service to students as each category can be manually reviewed, therefore quality of resources can be checked by trainers before being made available to students.

4.3 Cloud-based convenience

Most chatbots are cloud based now as companies have their internet services hosted (Patil, Karrupiah, Rao & Niranchana 2017). Students regularly use smartphones with limited memory and downloading apps causes processing power issues (Sinhal 2013). The cloud allows for constant updating and storage of new data, and with almost every popular social media platform now being cloud based (Jesus 2019).

4.4 SAAS licensing and costings

As the chatbot is available through the cloud via software vendors, the chatbot will remain in the cloud software licensing architecture. Solutions now exist that do not require coding technology, and can be built by anyone with basic computer literacy skills. Some examples include:

https://landbot.io/pricing / https://manychat.com/  / https://www.janis.ai/pricing 

The average price for each is AUD$50 per month. The cloud model allows for dynamic pricing, which for smaller operations, which can be expected with the introduction of the chatbot, will allow to keep the costs down (Soni & Hasan 2017).

4.5 Current users

The solution of a cloud based chatbot is not a new idea or concept. Clientele of such chatbots includes Domino’s Pizza and Duolingo, where people select an option, and then go through a series of menus to obtain the information they are after. Dominos introduced a Facebook chatbot in 2016 (Business Insider Intelligence 2016). Duolingo introduced their chatbot in the same year (Lardinois 2016), and continues to grow in popularity (Burgan 2020).

5.0 Existing technology used

Currently there is a learning management system without any add-on applications and administered by one person. This LMS itself is cloud based and therefore a software as a service fee is charged.

5.1 Artificial intelligence integration

Moodle based learning management systems have become more complex and technical over the years (Ryan 2012) and an LMS systems manager needs various skills to manage an LMS (Leslie 2003). Adding a chatbot, in particular one that requires no programming knowledge, will face no issues with the current LMS and administrator. Moodle has various chatbot plug-ins available (Cometchat n.d; Pandita & Faradeli Bandeali n.d), so most Moodle administrators will understand the header coding installation required.

5.2 Tech enhancements required

There is no need to adjust or change any technology other than the code into the header (Google n. d; Microsoft 2019). This requires no new hardware or technology other than the programmer to enter in the code. The chatbot is controlled by the cloud-based application.

6.0 The problem solved by artificial intelligence

Real-time Individualized support from lecturers is nearly impossible. An LMS that allows for the well-managed flow of information can lead to improved equality in learning outcomes (Brinton et. al 2015). There needs to be efforts made to stay connected to students and the information they need in order to complete their studies. Most students use social media chat apps (Winkler & Sollner 2018), so a chatbot will be simple for them to use.

6.1 The expected benefit

The speed of getting information will assist students in their studies, and this is the main benefit of introducing a simple text based chatbot. Searching for information can be exhausting for students (Adzharuddin & Ling 2013). As more information is produced, students need access to this information and the internet has made it easier, faster, and more cost-effective to find information. A chatbot can assist in delivering relevant information in real-time.

6.2 The success criteria

Websites are not willing to build a whole new website for the chatbot (Kuligowska 2015). How efficiently the chatbot can work with the current website will be paramount. Enterprises would be advised to list the criteria and functionality they need from their chatbot applications before deciding on which service to use (Artificial Solution 2020). As for students, how quickly and accurately students can receive information based on their topic will help determine the effectiveness of the chatbot for students.

7.0 Recommendations and conclusion

Ths issue with getting information to students as quickly as they need it calls for a technology where instant responses to questions is available, therefore a chatbot linked to content and resources retrieval is required. A chatbot can give prompt and fast assistance. There is no need for a complicated voice personal assistant style AI chatbot like Siri or Google Assistant. For this case, the chatbot should be category based that works off TESOL related topics and keywords. Finding a chatbot SAAS cloud service that allows for simple task-based categorisation should suffice for the instant communication needs. This can assist in getting information to students faster and in a convenient format. The next phase of the project will be to complete a business model review on how the technology will affect students, alleviate some of the issues that students face in receiving information in a convenient and timely manner, and how it can be packaged to students as a valuable study and training tool.

8.0 References

Abushawar, B & Atwell, E 2015, ALICE chatbot: Trials and outputs, Computación y Sistemas, vol. 19, no. 4, viewed 22 November 2020, <https://www.cys.cic.ipn.mx/ojs/index.php/CyS/article/view/2326>. 

Abu Shawar, B & Atwell, E 2002, A Comparison Between Alice and Elizabeth Chatbot Systems, University of Leeds, School of Computing Research Report Series, viewed 23 November 2020, <http://eprints.whiterose.ac.uk/81930/1/AComparisonBetweenAliceElizabeth.pdf>. 

Adamopoulou, E & Moussiades, L 2020, An Overview of Chatbot Technology, IFIP International Conference on Artificial Intelligence Applications and Innovations, viewed 27 November 2020, <https://link.springer.com/chapter/10.1007%2F978-3-030-49186-4_31>. 

Adamopoulou, E & Moussiades, L 2020, An Overview of Chatbot Technology, Artificial Intelligence Applications and Innovations, no. 584, pp. 373–383, viewed 20 November 2020, <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7256567/#CR28>. 

AIML Foundation, AIML Docs, viewed 29 November 2020, <http://www.aiml.foundation/doc.html>. 

Artificial Solutions 2020, Chatbots: The Definitive Guide, viewed 25 November 2020, <https://www.artificial-solutions.com/chatbots>. 

Australian Government Department of Education and Training 2017, Improving retention, completion and success in higher education, Higher Education Standards Panel Discussion Paper, June 2017, viewed 25 November 2020, <https://docs.education.gov.au/system/files/doc/other/final_discussion_paper.pdf>. 

Adzharuddin NA & Ling, LH 2013, Learning Management System (LMS) among University Students: Does It Work?, International Journal of e-Education, e-Business, e-Management and e-Learning, Vol. 3, No. 3, viewed 24 November 2020, <https://pdfs.semanticscholar.org/ec79/bc13b31192a0b7f7a11670568a523d1ef912.pdf>.

Brinton, C, Rill, R, Ha, S, Chang, M, Smith, R & Ju, W 2015, Individualization for Education at Scale: MIIC Design and Preliminary Evaluation, IEEE Transactions on Learning Technologies, vol. 8, no. 1, <https://ieeexplore.ieee.org/document/6955856/>. 

Brookes, J 2019, Why the University of Adelaide’s chatbot actually works, Which-50, viewed 24 November 2020, <https://which-50.com/why-the-university-of-adelaides-chatbot-actually-works/>. 

Buchanan, C & Sharma, R 2009, A Qualitative Study of TAFE Students Exiting From TAFE Programs, Journal of Institutional Research, vol. 14, no. 2, viewed 23 November 2020, <https://files.eric.ed.gov/fulltext/EJ1055594.pdf>. 

Burgan, A 2020, 2020 Roundup: Best Chatbots of the Year (So Far), Chatbot News, viewed 30 November 2020, <https://discover.bot/bot-talk/best-chatbots-2020/>. 

Business Insider Intelligence 2016, You can now order Domino’s pizza through a chatbot on Facebook Messenger, Business Insider, viewed 28 November 2020, <https://www.businessinsider.com/you-can-now-order-dominos-pizza-through-a-chatbot-on-facebook-messenger-2016-9>. 

Cameron, N 2018, Why the University of Canberra is so keen on chatbots, CMO, viewed 25 November 2020, <https://www.cmo.com.au/article/650921/why-university-canberra-key-chatbots/>. 

Cometchat n.d, Moodle Chat, viewed 20 November 2020, <https://www.cometchat.com/chat-plugins/moodle>. 

Cover Pages, 2000, DARPA Communicator Project and XML Log Standard, Technology Reports, viewed 22 November 2020, <http://xml.coverpages.org/darpaCommunicator.html>. 

DARPA, n.d, AI Next Campaign, viewed 23 November 2020, <https://www.darpa.mil/work-with-us/ai-next-campaign>. 

Duckett, C 2019, Open Universities stays smart by keeping chatbots dumb, Disguising chatbots as humans is not a ticket to success for Open Universities Australia, ZDNet, viewed 25 November 2020, <https://www.zdnet.com/article/open-universities-stays-smart-by-keeping-chatbots-dumb/>. 

Dyer, J, Gregerson, H & Christensen, C 2009, The Innovator’s DNA, Harvard Business Review, viewed 27 November 2020, <https://hbr.org/2009/12/the-innovators-dna>. 

Følstad, A, Bertinussen Nordheim, C & Bjørkli, C 2018, What Makes Users Trust a Chatbot for Customer Service? An Exploratory Interview Study, The Fifth International Conference on Internet Science, St. Petersburg, Russia, viewed 23 November 2020, <https://www.researchgate.net/publication/327839749_What_Makes_Users_Trust_a_Chatbot_for_Customer_Service_An_Exploratory_Interview_Study>. 

Fowley, F & Pahl, C, 2018, Cloud Migration Architecture and Pricing – Mapping a Licensing Business Model for Software Vendors to a SaaS Business Model, Advances in Service-Oriented and Cloud Computing, viewed 25 November 2020, <https://www.researchgate.net/publication/322811171_Cloud_Migration_Architecture_and_Pricing_-_Mapping_a_Licensing_Business_Model_for_Software_Vendors_to_a_SaaS_Business_Model>. 

Garber, M 2014, When PARRY Met ELIZA: A Ridiculous Chatbot Conversation From 1972, The Atlantic, viewed 21 November 2020, <https://www.theatlantic.com/technology/archive/2014/06/when-parry-met-eliza-a-ridiculous-chatbot-conversation-from-1972/372428/>. 

Garcia, MP, Lopez, SS, Donis, H 2018, Everybody is talking about Virtual Assistants, but how are users really using them? 32nd Human Computer Interaction Conference, July 2018, viewed 23 November 2020, <https://doi.org/10.14236/ewic/HCI2018.96>. 

Gasperis, G 2012, Building an AIML Chatter Bot Knowledge-Base Starting from a FAQ and a Glossary, Journal of e-Learning and Knowledge Society: Focus on: Semantic Web and e-Learning, vol. 8, no. 2, viewed 23 November 2020, <http://www.je-lks.org/ojs/index.php/Je-LKS_EN/article/view/414>. 

Google, n. d, Dialogflow Messenger, viewed 29 November 2020, <https://cloud.google.com/dialogflow/es/docs/integrations/dialogflow-messenger>. 

Harris, R, Simons, M, Bridge, K, Bone, J, Symons, H, Clayton, B, Pope B, Cummins, G & Blom, K 2001, Factors that contribute to retention and completion rates for apprentices and trainees, Australian National Training Authority, viewed 23 November 2020, <http://vuir.vu.edu.au/1849/1/Factors_that_contribute.pdf>. 

Heung-Yeung S, Xiao-deng H & Di Li, 2018, From Eliza to XiaoIce: challenges and opportunities with social chatbots, Frontiers of Information Technology & Electronic Engineering, vol. 18, viewed 22 November 2020, <https://link.springer.com/article/10.1631/FITEE.1700826>. 

Higashinaka, R, Imamura, K, Meguro, T, Miyazaki, C, Kobayashi, N, Sugiyama, H, Hirano, T, Makino, T & Matsuo, Y 2014, Towards an open-domain conversational system fully based on natural language processing, Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 928–939, Dublin, Ireland, August 23-29 2014, viewed 24 November 2020, <https://www.aclweb.org/anthology/C14-1088.pdf>. 

Jesus, M, 2019, Cloud-based Conversational Agents for User Acquisition and Engagement, In Proceedings of the 9th International Conference on Cloud Computing and Services Science (CLOSER 2019), viewed 25 November 2020, <https://www.researchgate.net/publication/333336045_Cloud-based_Conversational_Agents_for_User_Acquisition_and_Engagement>. 

Jurafsy, D & Martin, J 2018, Dialog Systems and Chatbots, Speech and Language Processing, viewed 21 November 2020, <https://web.stanford.edu/~jurafsky/slp3/24.pdf>. 

Koetsier, J 2020, Alexa, Siri, Google Assistant: How The Top Smart Assistants Stack Up, Consumer Tech, Forbes Magazine, viewed 20 November 2020, <https://www.forbes.com/sites/johnkoetsier/2020/08/08/alexa-siri-google-assistant-how-the-top-smart-assistants-stack-up/?sh=56beb49e748b>.  

Konverso 2020, What are the key differences between chatbots and virtual agents? 6 minute read, viewed 24 November 2020, <https://www.konverso.ai/blog/what-are-the-key-differences-between-chatbots-and-virtual-agents>. 

KR Asia, Microsoft’s Xiaoice chatbot to become its own company in China, viewed 23 November 2020, <https://kr-asia.com/microsofts-xiaoice-chatbot-to-become-its-own-company-in-china>. 

Kuigowska, K 2015, Commercial Chatbot: Performance Evaluation, Usability Metrics and Quality Standards of Embodied Conversational Agents, Professionals Center for Business Research, vol. 2, no. 2, viewed 22 November 2020, <https://www.researchgate.net/publication/282474589_Commercial_Chatbot_Performance_Evaluation_Usability_Metrics_and_Quality_Standards_of_Embodied_Conversational_Agents>. 

Language Training Institute, 2020, Language Training Institute website, viewed 24 November 2020, <http://lti.edu.au>. 

Lardinois F, 2016, Duolingo’s chatbots help you learn a new language, Tech Crunch, viewed 28 November 2020, <https://techcrunch.com/2016/10/06/duolingos-chatbots-help-you-learn-a-new-language/>.  

Lasar, M 2011, ARPANET’s coming out party: when the Internet first took center stage, Ars Technica, viewed 21 November 2020, <https://arstechnica.com/tech-policy/2011/10/arpanets-coming-out-party-when-the-internet-first-took-center-stage/>. 

Leslie, S 2003, Important Characteristics of Course Management Systems: Findings from the Edutools.info project, 2003 CADE Conference, June 8-11, St. John’s Newfoundland, viewed 20 November 2020, <https://www.edtechpost.ca/gems/cms_characteristics.htm>. 

Marietto, M, de Aguilar, RV, Barbosa, G, Tanaka Botelho, W, Pinheiro Pimentel, E, dos Santos Franca, R & de Silva, Vera Lucia 2013, Artificial Intelligence Markup Language: A Brief Tutorial, International Journal of Computer Science & Engineering Survey, vol. 04, no. 03, viewed 20 November 2020, <http://www.airccse.org/journal/ijcses/papers/4313ijcses01.pdf>. 

Microsoft 2019, Web Chat customization, viewed 29 November 2020, <https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-webchat-customization?view=azure-bot-service-4.0>.  

Microsoft News Centre, 2020, University of Sydney builds AI-infused Corona Chatbot to support students with COVID-19 queries, Leverages Microsoft Cognitive Services to spin up smart agent in one week, Microsoft Features, viewed 25 November 2020, <https://news.microsoft.com/en-au/features/university-of-sydney-builds-ai-infused-corona-chatbot-to-support-students-with-covid-19-queries/>.  

Ng, SF, Hassan, NSIC, Mohammad Nor, NH & Abdul Malik, NAA 2017, The Relationship Between Smartphone Use and Academic Performance: A Case of Students in a Malaysian Tertiary Institution, Malaysian Online Journal of Educational Technology, vol. 5, no. 4, viewed 22 November 2020, <https://files.eric.ed.gov/fulltext/EJ1156718.pdf>.  

Pandita, S & Faradali Bandeali, Lizna n.d, Moodle Integration with DialogFlow (chatbots), Spring ML, viewed 25 November 2020, <https://springml.com/blog/moodle-integration-with-dialogflowchatbots>. 

Patil, A, Karrupiah, M, Rao, A Nagaraja & Niranchana, R 2017, Comparative study of cloud platforms to develop a Chatbot, International Journal of Engineering & Technology, vol. 6, no. 3, viewed 25 November 2020, <https://www.sciencepubco.com/index.php/ijet/article/view/7628>. 

Robert Epstein, R, Roberts, G & Beber, G 2008, Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer, Springer, Dordrecht, viewed 23 November 2020, <https://ebookcentral-proquest-com.ezproxy.usq.edu.au/lib/usq/detail.action?docID=338491>. 

Ryan, T 2012, Learning Management System Migration: An Analysis of Stakeholder Perspectives, International Review of Research in Open and Distance Learning, no. 13, vol. 1, viewed 25 November 2020, <https://www.researchgate.net/publication/236594245_Learning_Management_System_Migration_An_Analysis_of_Stakeholder_Perspectives>. 

Sandu, R 2020, Adoption of AI-Chatbots to Enhance Student Learning Experience in Higher Education in India, Project: Learning through technology, viewed 25 November 2020, <https://www.researchgate.net/publication/338868551_Adoption_of_AI-Chatbots_to_Enhance_Student_Learning_Experience_in_Higher_Education_in_India>. 

Satu, S, Parvez, H & Al Mamun, S 2015, Review of integrated applications with AIML based chatbot, 1st International Conference on Computer & Information Engineering, viewed 22 November 2020, <https://ieeexplore.ieee.org/document/7399324/>. 

Sinhal, A, 2013, Cloud Computing on Smartphone, Computer Engineering and Intelligent Systems, vol. 4, no. 10, viewed 23 November 2020, <https://www.researchgate.net/publication/303922831_Cloud_Computing_on_Smartphone>.                                                                                                               

Soni, A & Hasan, M 2017, Pricing schemes in cloud computing: A review, International Journal of Advanced Computer Research, vol. 7, no. 29, viewed 23 November 2020, <https://www.researchgate.net/publication/314253997_Pricing_schemes_in_cloud_computing_A_review>. 

Spencer, G 2018, Much more than a chatbot: China’s Xiaoice mixes AI with emotions and wins over millions of fans, Stories Asia, viewed 24 November 2020, <https://news.microsoft.com/apac/features/much-more-than-a-chatbot-chinas-xiaoice-mixes-ai-with-emotions-and-wins-over-millions-of-fans/> .

Telegram, 2020, Telegram web-based application, Telegram, viewed 25 November 2020, <http://web.telegram.org>. 

TESOL Japan, 2020, TESOL Japan website, TESOL Japan, viewed 26 November 2020, <http://auedtr.com>.

Ubisend 2019, Why Universities are Using Chatbots: The Future of Higher Education, Chatbot Success Stories, viewed 22 November 2020, <https://blog.ubisend.com/discover-chatbots/universities-using-chatbots>.  

University of Tasmania n.d, Ask our chatbot intern – Mumford, viewed 25 November 2020, <https://askus.utas.edu.au/app/chatbot>.  

Wallace, R 2003, The Elements of AIML Style, ALICE A. I. Foundation, Inc, viewed 23 November 2020, <https://files.ifi.uzh.ch/cl/hess/classes/seminare/chatbots/style.pdf>. 

Walker, M & Hirschman, L 2000, Evaluation for DARPA Communicator spoken dialogue systems, AT&T Labs- Research, viewed 23 November 2020, <https://www.researchgate.net/publication/2624378_Evaluation_for_DARPA_Communicator_spoken_dialogue_systems>. 

Walker, M, Rudnicky, A, Prasad, R, Aberdeen, J, Owen Bratt, E, Garofolo, J, Hastie, H, Le, Audrey, Pellom, B, Potamianos, A, Passonneau, R, Roukos, S, Sanders, G, Seneff, S & Stallard, D 2002, DARPA communicator: cross-system results for the 2001 evaluation, Conference: 7th International Conference on Spoken Language Processing, ICSLP2002 – INTERSPEECH 2002, Denver, Colorado, USA, September 16-20, viewed 20 November 2020, <https://users.soe.ucsc.edu/~maw/papers/commeval01-7.pdf>. 

Winkler, R & Sollner, M, Unleashing the Potential of Chatbots in Education: A State-Of-The-Art Analysis, Academy of Management Annual Meeting Proceedings 2018, no. 1, viewed 23 November 2020, <https://journals.aom.org/doi/10.5465/AMBPP.2018.15903abstract>.  

Yamaguchi, H, Mozgovoy, M & Danielewicz-Betz, A 2018, A Chatbot Based On AIML Rules Extracted From Twitter Dialogues, 2018 Federated Conference on Computer Science and Information Systems, viewed 23 November 2020, <https://annals-csis.org/proceedings/2018/drp/297.html>. 

Yates-Robertson, E 2019, Australian university uses Microsoft Teams and AI to reduce dropout rate, The Record, viewed 20 November 2020, <https://www.technologyrecord.com/Article/australian-university-uses-microsoft-teams-and-ai-to-reduce-dropout-rate-86097>. 

Zhou, L, Gao, Jianfeng, Di Li, Heung-Yeung, S 2019, The Design and Implementation of XiaoIce, an Empathetic Social Chatbot, Computational Linguistics, vol. 46, no. 1, viewed 22 November 2020, <https://www.mitpressjournals.org/doi/pdf/10.1162/COLI_a_00368>.