I believe that when it comes to learning German (or most other languages for that matter), something interesting will happen in the near future: On the one hand, technology will make the necessity of being able to communicate by yourself in a foreign language a thing of the past. Already lots of people are using apps like 3PO, DeepL, and even ChatGPT to translate in real time, much like Douglas Adams' Babel Fish. So we'll likely soon witness for good the disappearance of the beloved learning-for-an-upcoming-trip type of language student. You'll be missed.
But let's speak about a bigger chunk of the German-learning contingent: the people who are interested in attaining fluency for longer-term practical situations (such as expats) and those who are actually interested in the language itself (people like me - an exceedingly lonely bunch). In short: People who actually want to learn.
For many, the idea of replacing classical language schools, teachers, and textbooks with an always-available, omniscient, and obedient teacher/conversation partner (whose edutainment parameters you can control) seems like an absolute no-brainer. Enter LLMs like ChatGPT, who seem to already have brought this very future to our screens, today.
I have no doubt that, with proper prompting and a much more hallucination-free version of the program, you could have an amazing experience learning German with ChatGPT (or other models), even as an absolute beginner, but the sad truth is that we're just not there yet (as of October 2024, when this article was written). I've put ChatGPT (version 4o) through the test to see how it does in situations that routinely come up with language students learning German.
ChatGPT is good at creating convincingly "human" text with plausible content, especially in English.
ChatGPT is not good at evaluating truth, correctness, or accuracy. I have seen it decide that faeries are real. I have seen it create junk code and junk crochet patterns.
I do not recommend it for language learning. It will 'lie' because it has no concept of 'truth.'
In addition: as a learner, you are not able to assess its language use for accuracy and style. For all you know, it is making mistakes, or saying something in a consistently odd way. It is something entirely different than someone who knows the language using it and assessing it.
spanktruck
1) It can't correctly answer grammar questions
Let's take a really straightforward question that could come up at the end of a typical A1.1 German course: Why is the past tense (Perfekt) sometimes formed with the verb "haben" (to have) and sometimes with "sein" (to be). ChatGPT choses to answer the question like this:
The problem is that very last part. Whether verbs are transitive or not has nothing to do with how the past is formed at all. It's true that most of the "haben" verbs are transitive, but that doesn't help us understand the super simple actual answer: Motion and changes of state are with "sein" and the rest is with "haben". Bam.
Alright, at least it admits it when it's wrong, in true form. Let's move on to some other examples, lest I be accused of nitpicking. Take a look at this:
When we think of intransitive verbs, I wouldn't say "to sing" should be among them, considering that you can definitely sing something (like a song, or a lullaby, or someone's praises). What's disturbing here is how low on the list of words this mistake appears, considering that if we wanted ChatGPT to be our trusted guide through a jungle of thousands of words in order to reach fluency, we'd expect it to be a lot more discerning.
Exactly! Let's move on.
This is a good question for A1.2. What bothers me about this answer is that it keeps saying, essentially, that when movement is involved, dative can't be used. But that's simply not true. Movement or not, what matters is that the location doesn't change. So if you have a ball rolling around on the tennis court, or a man running around in his room, or a hot air balloon that's flying in the sky, or an ant crawling on the wall... you get the idea. Those locations are all in dative. As a teacher, I'd definitely mention that, instead of saying that there's no movement. Otherwise it's very confusing.
Now, for something more serious:
If you're anything like me (and I'm not a grammar slouch), that last line in the table is a bit odd (especially in light of the strange choice of always including three articles in the cells on the left column even if they don't fit to the gender)... Let's see what it says when we bring this up.
Wait, what? I get that if it wants to use "kein" then the adjectives will end with "-en", but what in the world does it mean by "in all cases except for the nominative and accusative cases, where the adjective should have -e". That's simply wrong.
It goes on to spew total nonsense, contradicting its own examples in the nominative and accusative plural.
So, a major hiccup that would leave any student tearing out their hair if this was their (only) teacher. Next:
This is right, but a terrible, terrible example, since it basically looks just like its English counterpart. This is example ONE in a list of "significant differences between English and German". Come on. And then it doubles down:
Yup.
2) It's unreliable with vocabulary
So, I asked ChatGPT to give me a list of masculine nouns, then feminine nouns, then neuter nouns, so we could practice gender. Have a look at the repetition of the words in these relatively short lists (considering, again, that in order to teach you German you'd expect your teacher to be able to properly introduce thousands of vocabulary words over the span of the course:
You're seeing this, right? Freund (23. and 38. in the first column) and Sohn (33. and 47.), then Tasche (4. and 27. in the second), and Fenster (with number 5. and 40. in the third). Imagine how many repetitions there are likely to be once we ask for more expanded lists of vocab we want it to teach us. And YES, repetition is good, of course, for learning - but that's clearly not the intention here when I prompted it to supply me with lists of unique words that represent each gender. Let's move on.
Here I asked it to tell me some "false friends", words that sound the same in English and German but have different meanings:
Now, you only need to know English to see that the word for number 13. (Kaution) doesn't sound like anything at all having to do with "bail". That's just odd. And 15. (Fein) is not really a false friend, since "fine" means two things in English, including exactly what it's German sound-alike means. Another example of a disturbingly prominent bad example of something. Try to scale that idea...
Now, this is usually a fun thing to do with A1 students to teach them "family member" vocabulary. You know, words like "cousin" and "nephew" and "step-father"... ChatGPT took this concept and made something that took me really long to label "technically correct":
This odd depiction notwithstanding, I think labeling your kids "Kind1" and "Kind2" is either a missed opportunity for a language lesson, or an eery commentary on the interchangeability of humans...
That looming uncle also gives me the creeps...
ANYWAY, I would have done it differently: Look how nice I made mine in my amazing Beginner German Express Course (get access now!):
Sorry for the really shameless plug. Let's learn some countries with ChatGPT, shall we?
You'll notice that my prompt was to name 10 countries that sound TOTALLY different in English and German (because, as a teacher of German to English speakers, I would concentrate on teaching those, and pay a bit less attention to the ones that sound very similar).
I think at least 40% of this list is super easy for English speakers to pick out (Türkei, Polen, Spanien, Kroatien). In my list of difficult countries I would have included Frankreich, Lettland, Estland, Litauen, and maybe Weißrussland (Belarus) as quite a bit harder. Russland, Nordmazedonien, Niederlande, and Norwegen (not to mention Vereinigte Staaten and Vereinigtes Königreich) are also less obvious than Croatia. But I will admit, looking at a map just now I do think that this is tough, as most countries are really very similar-sounding.
Let's do something easy:
I don't know. If it were me, I would not put "Flughafen" and "Fenster" on my list there. Unlike the other ones, they don't sound anything like their English counterparts. Why not take "Sofa" or "Bett" instead? We'll never know.
3) It's bad at correcting your mistakes
In this example, I asked it to be my teacher and do some typical first German lesson drills with me, and although it did not dig particularly deep into its teaching toolkit to come up with these questions (asking me to simply repeat what it said), it was going well enough until I decided to make a tiny little mistake, which seemed blow a little circuit in our teacher's head:
Alright. If this were a real person I'd probably ask if they were ok and needed a break at this point.
But ok, everyone can have a brain fart. Let's do some simple math questions (again, a typical A1.1 activity for learning numbers):
Yeah, great job indeed. If 14+7=20 and that's ok with our computer teacher, then yeah, I think I'd rather move on to something (or someone) else.
4) It REALLY doesn't get pronunciation
This is another totally normal thing you'd expect your teacher to be able to help you with, especially when you're a beginner learning a new language: How to pronounce words. Look at the way ChatGPT wants us to pronounce the "ch" part of "brauchen":
Ok. Not how I would have done it, but fine. Note that it likens that "ch" sound to the "h" in "huge", indicating that it really has no idea how to pronounce it but knows that "chen" at the end of words is usually pronounced in that type of way. Now look at how it wants us to say the "ch" in "Kirche":
Aside from the odd likening of the middle sounds in "bit" and "keer", it goes on to say...
Wait, how am I supposed to put my tongue close to the roof of my mouth AND lightly touching the lower front teeth?? It's almost like it picked two completely incompatible sets of instructions and fused them together here!
Oh, so the "ch" in "Kirche" is ALSO like "huge"? Just like with the word "brauchen"? No, no, no...
Enough. Obviously ChatGPT has no idea, and is comparing each syllable in isolation, with its best guess as to what it might sound like based on its available data. If you've ever had AI make you a picture of a tongue (don't!) you'll see that it has no idea that our human tongue can't be both at the roof of the mouth and touching the bottom teeth - or that "keer" and "bit" require completely different arrangements of our vocal instruments.
So. Not great. How about the alphabet? Can you teach me the names of the letters, ChatGPT?
Ah, so "fow" has a long "ee" sound at the end. Interesting, interesting... And why are C and G not together with their buddies B, D, E, P, and T?
So we are to believe that "o" and "q" rhyme with each other? "Oh" and "koo" are the same sounds? Hmm.
5) It doesn't really know what it's doing
That's really the crux of it, isn't it? A real teacher needs to know the big picture, how words in languages are related to each other, how they build up on each other, how learners need different engaging ways of information to be presented, how and when to repeat and re-introduce concepts, how to do so in relevant contexts, and how to gauge (constantly) whether what you're doing is appropriate for the level of your student. Look at this gem:
ChatGPT does not know what A1 means. It includes adjective declension in dative case ("in einer kleinen Stadt") which is generally done at great pain in level A2.2, subordinate clauses ("der ganz allein ist", "den Hund zu behalten") which I believe is introduced at B1, advanced vocab (wedeln, fressen, überlegen, streicheln), as well as "niemandem" and "niemanden" (B1/B2)...
This has gotten really long. I need to work hard on editing and shortening my blog posts - but I definitely want to add in conclusion that I actually love ChatGPT (please don't hurt me!) and use it a lot (but never to write these posts, don't worry). I think it has so much potential, and that we'll be laughing about this article a few years from now.
But it's not there yet. As a learner, I would use ChatGPT to practice conversing once I'm conversational in a language (starting around B1). Before then, I highly recommend sticking to a human-designed self-study course, or a human-run live course. Or how about both?
It works great for chatting. Grammar is actually pretty good and it can generate good text. It's not *correct* in the sense that it really doesn't get facts right, but it is at least grammatically correct in its delusions. As long as you know this, it's helpful for learning.