Probing ChatGPT & LLMs: Unraveling Strengths and Boundaries in Compositional Problem Solving

Large Language Models (LLMs) such as ChatGPT, PaLM, LLaMA, and BERT have taken the world by storm, transforming how we interact with machines and implement natural language processing. These models, powered by Transformers, are being increasingly deployed for various applications, including customer support, social media content filtering, and automated content generation. A recent research study has put the spotlight on these models’ capabilities in solving compositional problems, presenting intriguing yet vital findings.

Two Competing Hypotheses

Researchers examining Transformers’ capabilities in compositional problem-solving proposed two hypotheses in their study. The first hypothesis suggests that these AI models can transform complex multi-step reasoning into a linearized matching of subgraphs. Essentially, the models learn to match the appropriate language prompts’ patterns through training data, which makes the process of reasoning detectable as a sequence of steps.

The second hypothesis highlights the inherent limitations of Transformers in solving high-complexity compositional tasks. Despite their impressive natural language understanding capabilities, these LLMs might struggle with solving rare or highly complex examples that have not been adequately represented in the training dataset.

Deciphering Compositional Tasks

The researchers employed three distinct types of compositional tasks to evaluate the Transformers’ capacity in handling complex reasoning. These tasks included multi-digit multiplication, logic grid puzzles, and a classic dynamic programming problem. These tasks served as a reliable benchmark to assess the models’ problem-solving abilities under different reasoning structures and challenges.

Computation Graphs: A Key Tool

In order to understand how Transformers solve these compositional tasks, researchers utilized computation graphs. The tasks were decomposed into submodular functional steps, which were then verbalized as input sequences for the models. By following the input-output dynamics in these computation graphs, the researchers could identify the strengths and weaknesses of the Transformers and how they struggle with computational errors.

Predicting Patterns through Information Gain

An information gain method was applied to predict the Transformers’ patterns, which offered insights into their capabilities and limitations. This analytical approach computed the distribution of information from inputs to outputs, effectively shedding light on how successfully the models can generalize their problem-solving abilities.

Findings: Strengths, Limitations, and Errors

The research findings corroborate the first hypothesis, suggesting that Transformers can match linearized subgraph representations to infer reasoning when given appropriate cues. However, the second hypothesis also holds merit, as LLMs like ChatGPT have limitations in solving uncommon, complex examples.

One notable aspect found in the Transformers was the significant amount of early computational and compounding errors. These mistakes resulted from the models incorrectly interpreting or answering some of the initial steps. As a result, these early errors negatively affect the models’ overall performance in solving complex compositional tasks.

As machine learning enthusiasts, developers, and data scientists strive to push the boundaries of AI, insights from this research can alleviate existing limitations and enhance the utility of LLMs in more advanced, problem-solving applications.