Multilingual Knowledge: an Informal, Initial Study

Report

Code-switching is a common phenomenon of alternating between different languages in the same utterance, thought, or conversation. We posit that humans code-switch because they feel more comfortable talking about certain topics and domains in one language than another. With the rise of knowledge-intensive language models, we ask ourselves the next, natural question: do language models have more knowledge in certain languages than others? We run an experiment to test this hypothesis and find that language models can perform better when performing chain-of-thought reasoning in different languages. We find that language models do indeed know more about certain topics in certain languages than others. This report serves as an initial, preliminary study of this hypothesis.