I’m building little chatbots for a German class at Middlebury. It’s a pretty solid example of a number of complications. They work (technically) and I didn’t use AI to do the programming but there are some other pieces that pop up that I think are worth documenting.
Goal
The goal is to have the chatbot engage with students in German in defined scenarios with specific vocabulary at the A2 language level.
Problems
The first problem is that I don’t speak German. So I can’t evaluate whether the responses coming back are at the appropriate level of sophistication. This isn’t a unique problem for jobs like mine. I often work with people who teach languages I don’t speak or who are doing things I might not understand. So while I can bounce back and forth through a translator and get a rough idea if the conversation is working, I don’t have the internal knowledge necessary to evaluate the nuance of the language and that nuance is the important part.
The second element is that the vocabulary is given to me as screenshots (and is also in German). There are enough words (50-70) that entering them by hand is not something I want to do. Plus there are accent marks. It’s also harder for me to scan for words or have an intuitive sense for what words might be missing given I don’t speak the language.
Actions
Regarding the level of the language, there’s nothing I can do other than get feedback from the instructor and adjust the prompt instructions. This is not optimal in terms of speed.
My current path for getting the vocabulary words is to upload the images to an LLM and ask it to pull out the German words and create an comma separated list. It’s basic OCR work and might be better done with a specific tool but I’d like to eventually unify this process so I don’t have to do this part at all and being able to do it with one model would be very nice.
Today, I uploaded the 5 screenshots to ChatGPT 4.5, ChatGPT 4o, and Claude 3.7 Sonnet. Because I was curious, I threw the list of words they generated into a spreadsheet. To get some clarity, I ended up having to use the following formula to get a proper alphabetical sort and eliminate duplicate words (despite asking for that from the LLM) – =UNIQUE(SORT(Transpose(SPLIT(B2,”, “))))
Turns out that each LLM model gave me a slightly different list of words. Claude returned 95 unique words. GPT 4.5 gave me 85 and 4o gave me 91. I couldn’t tell that at a glance. I needed to take the time and make the effort to think and analyze the data in another technology where I had developed skills . . . which is exactly the opposite of what AI promises.
To really parse out what the differences are would be a pain, probably more of a hassle than hand-entering them. A quick scan showed things like . . . GPT 4.5 gave me the word “Computerwissenschaftler” which means computer scientist. That is not in the list of words from the source images. “Informatiker” is in the list and it also means computer scientist. Only Claude gave the singular and plural form for Gymnasien. Both were in the image. Minor differences but this is also a simple task. It also erodes my general faith in this working.
Implications
This doesn’t matter a ton here but the pattern definitely has larger ramifications. It reinforces the idea that using LLMs when you can’t analyze the output with a pretty decent degree of sophistication may get you answers but you have no idea if they’re really good answers, sort of right, or clearly wrong. That outsourcing of knowledge and expertise with no intention of developing people’s skills in the long-run feels really frightening. It gives speed and confidence to people who probably shouldn’t have either. It creates polish for people whose work should appear unfinished and incomplete. AI has the potential to make all kinds of unqualified people appear to be experts and the lack of personal energy and time invested means that they can do it more often and across more domains. It has the potential to super-charge the already swollen market of consultants selling easy answers.
When I keep hearing that AI will make connections that humans can never have made across unknowable masses of data . . . I can’t help but wonder if we can then verify that these are accurate connections or if we’ll just have to (or opt to) take it on faith. I know that even doing the limited effort it took to verify this word list isn’t something that people will do regularly, especially as people start to trust the AI output more. Part of that path to “trust” is lack of knowledge and understanding about the output (at least according to one study). That cycle of trust/reduction of critical thinking/de-skilling is one that I can’t help but see as likely.
Clearly, I’m using AI. I can see uses for it. I can have a lot of fun with it.1 I’m can do that while struggling with the details and worrying about AI based on how I’ve seen us use technology over the last quarter century.2 I’ve seen us take a bag of gold and use it as bludgeon, I think we can, and probably will, do far worse with AI. People who have no interest in the work they’re doing aren’t going to turn down ways to avoid it. Business won’t turn down lower paid, lower-skilled workers who they hope to eventually replace entirely. AI will be weaponized in all the ways you can imagine and at least 7 that you never considered.