Decoding the Secrets: How Transformer-Based LLMs Store and Extract Factual Associations

In recent years, the realm of natural language processing (NLP) has experienced significant growth due to the popularity of transformer-based large language models (LLMs) such as BERT, GPT-3, and T5. These models have demonstrated a remarkable ability to understand and generate fluent and coherent text, further raising interest in their internal mechanisms. A recent study delves into the inner workings of these LLMs to examine how they store and extract factual associations.

Although there have been studies proposing various ways to understand the internal mechanisms of LLMs, researchers in this study adopted an information flow approach to probe DALL-E, a decoder-only transformer model. The primary objective was to identify critical computational points and employ a “knock out” strategy to better understand model behavior.

To achieve this, the researchers focused on the information propagation at critical points and examined the preceding representation construction process. Furthermore, they implemented interventions to vocabulary and Multi-Head Self-Attention (MHSA) and Multi-Layer Perceptron (MLP) sublayers and projections.

The results of this analysis shed light on some fascinating patterns in the subject enrichment process that occur in the early layers of the model. It revealed that the relation is passed to the last token, while the attribute extraction from the subject is realized via attention head parameters.

The study’s implications are of great importance not just for the field of NLP, but also for potential applications in reducing biases in machine learning systems. By enhancing our understanding of the factual associations in LLM architectures, researchers can open new avenues to explore knowledge localization and model editing techniques. Moreover, further investigation of decoder-only LLMs may lead to practical applications in sentiment analysis, language translation, and bias mitigation.

Examining the internal mechanisms of transformer-based large language models is paramount to advancing their performance and reducing biases. Unraveling the information flow approach and critical computational points in these models could enhance our understanding of factual associations and unlock potential applications in various NLP fields. As technology continues to evolve and improve, LLMs will likely become even more crucial in the development of advanced AI applications, making these insights invaluable for the future of the field.