Looking for a Language Study Buddy in ChatGPT-3

So I decided to try ChatGPT! ChatGPT has been making quite the buzz lately. Functionality-wise, it is Aladdin[1] that appears at the controls of your machine. If you have been in the loop of recent developments, ChatGPT has left impressions of future chatbots and personal assistants. So far, people have used ChatGPT to create games, do assignments, do software programming and so much more. In this blog, I shall talk about how ChatGPT could support a language learner, more specifically, as a study buddy.

Image generated using DALL-E

Testing

Looking for resources on search engines can be overwhelming. Instead of having to hop around websites to find quizzes and resources to test on a topic of choice at the appropriate level, ChatGPT provides you with a set of questions in your desired question type, level and topic. Even better, you do not have to check your answers after attempting the questions. ChatGPT will do it for you.

Looking for language resources online vs ChatGPT, a workflow

Here are some questions that I asked. My target language was Mandarin Chinese but you can experiment to see whether ChatGPT can quiz you in your target language. 

HSK is the standardised exam for Mandarin Chinese.

As asked, the questions were HSK 1 level. However, ChatGPT said it could not read my answers because they weren’t in Chinese. However, only my response to question 6 was ‘idk’ (= I don’t know), while the remaining questions were in Chinese. While ChatGPT could read every other sentence and it understood what 'idk' meant on its own, it was not able to associate the 'idk' with me not knowing the answer. Therefore, if you do not know the answer to a question, it might be better to skip the question instead of telling ChatGPT that you do not know the answer. 

Unfortunately, at this time ChatGPT doesn’t seem to be able to check whether your responses to these open-ended questions are grammatically and conceptually correct. You may also receive some non-questions in your question bank, meaning ChatGPT may not be able to respond to your question with full accuracy.

Other than questions at a specific skill level, ChatGPT can also provide questions on a specific topic. You can even ask it to provide you with easy questions first, followed by harder ones. Since grammar questions usually are not open-ended, ChatGPT would correct you if you gave a wrong answer. This is extremely useful since you wouldn’t have to check your answers and can keep going with your grammar drill.

ChatGPT gives you the right answer if you have given a wrong answer.

Other question types ChatGPT can provide are multi-choice questions, open-ended questions, comprehension paragraphs, True or False questions, and fill in the blanks questions.

Think of ChatGPT as a quiz generator, but expect that it might process your request incorrectly sometimes. The questions may also get repetitive after the first couple of times, meaning ChatGPT is not an infinite quiz generator.

Support

ChatGPT is sensible when responding to people’s emotions of joy, tiredness, frustration, anger, feeling of burnout and more. It can also read emotes and chat lingo.

Current limitations to ChatGPT in the context of language learning

ChatGPT gets cut off while performing long text-based tasks. This is because each reply has a character limit. If it has to perform a long task, it cannot use a second textbox to reply as humans would, and instead provides an incomplete response.

When you ask ChatGPT a factual question, it may provide a wrong answer.

Fact: Duolingo does not offer Lithuanian, nor is it in the incubator as of writing.

Fact: Duolingo has offered German since 2016. 

Now, for comparison, here is the conversation about the number of days in a week. 

ChatGPT insists on the correct answer 一 seven even though I said that it was wrong. I would assume this is because, for questions where there is abundant data, ChatGPT is more trained and therefore, more confident about its response. However, for questions that it has relatively lesser information on, like the languages one, ChatGPT is quicker to admit that it has provided wrong information.

In its defence, ChatGPT is just a language model without access to the internet, and Duolingo could choose to introduce Lithuanian or stop offering German any day and ChatGPT would never know unless it was trained on that data. At the same time, I would like to better understand what actually goes on inside a language model, and how it is trained.

Now, the question arises: How does ChatGPT frame its questions on a topic in a certain language, and would it ever become intelligent enough to test students dynamically while keeping learners’ needs and the latest testing trends in mind? Maybe this is not the focus of ChatGPT, since there are a lot of purposes for it to deliver, but a language company might choose to make a more language-focused version of it. 

Going a step further, would Artificial Intelligence (AI) be able to generate unique language learning questions based on test-taker data on how past performance and prescribe customised resource lists?

Currently, ChatGPT is a text-based AI chat tool. Introducing the ability to work with and share images and other media forms could further enhance its capabilities. In the future, ChatGPT (or its language-specific version) might have more capabilities of a language buddy to help with accountability, providing tips and advice to language learners throughout the language learning process. 

A language learning assistant-buddy like ChatGPT would provide the major advantage of being available to the language learner 24/7. Of course, you wouldn’t feel bad about ‘ghosting’ them and can return to language learning anytime. Furthermore, it could also function as a language practice buddy 一 both for written and spoken forms. 

I think it would also be quite useful to have ChatGPT initiate conversations. The current version only responds to humans. However, being able to ask questions, remind and just... chat about languages and life.

I have to do more research to learn how ChatGPT works. The intersection of human languages and technology interests me. At this point, I might be going way beyond what a language learning model like ChatGPT is capable of. However, with so much innovation happening in the Natural Langauge Processing sphere, we might be approaching a reality in which computers become humans’ best friends, and that might happen sooner than we expect.

Until then, I think ChatGPT-3 does an impressive job of making conversations, almost as if it were human.

P.S. Here is the conversation I had with ChatGPT about Twitter’s logo. With the way our conversation went, I was nearly convinced that Twitter has a new logo.

[1] Update (Jan 20, 2023): By saying that ChatGPT is like Aladdin, I wanted to refer to how magical the idea of a super-efficient personal assistant is. That is the magic of technology and code. At the same time, ChatGPT would not be possible without the continuous efforts of hundreds of individuals, to ensure that the magical technology in its users' hands is safe and non-toxic. You can read more about the working environment of data labellers (who label data as safe, or unsafe and into several other categories for the language model training) here:

OpenAI Used Kenyan Workers On Less Than $2 Per Hour To Make ChatGPT Less Toxic - Time

Highly recommended read

[2] Update (Jan 24, 2023): A Reddit post on Using ChatGPT As A Language Learning Partner (posted Jan 24, 2023 - removed from the subreddit on the same day) initiated discussions on ChatGPT making mistakes in semantics and syntax for less widely-spoken languages. Examples can be found in comments in the discussions. Here are my two cents: As long as you are aware of the limitations of a resource while you are using it and you take advantage of its strengths, there should be no problem with diversifying your resources. ChatGPT-3 is just a preview of what future AI-powered chatbots are capable of, and one day, it might be possible to find a language study buddy that is super intelligent, accurate and works with real-time information (ChatGPT-3 is limited only to information until 2021). I just thought of putting an update on here because this is a hot topic. Therefore, different sources would provide multiple perspectives, which are very helpful in learning more about new technology.