Meta researchers have developed a new prompting technique called "semi-formal reasoning" that significantly improves the accuracy of large language models in code review tasks, achieving up to 93% accuracy in some evaluations.

The structured approach addresses a key challenge in AI-assisted software development: performing reliable code analysis without executing programs, which eliminates the need for expensive computational sandboxes. Current methods often rely on unstructured reasoning that can lead to inaccurate assumptions based on superficial code patterns.

Semi-formal reasoning requires AI agents to complete structured templates that force them to explicitly state premises, trace execution paths, and derive formal conclusions based on verifiable evidence. This systematic approach helps agents handle edge cases and avoid making unsupported claims about code behavior.

In testing across three software engineering tasks—patch equivalence verification, fault localization, and code question answering—the researchers used Claude Opus-4.5 and Sonnet-4.5 models. For patch equivalence tasks, accuracy improved from 78% with standard reasoning to 88% with the structured approach. When evaluating real-world patches with test specifications, the Opus-4.5 model achieved 93% verification accuracy, outperforming both unstructured baselines at 86% and traditional text-similarity algorithms at 73%.

The technique demonstrated its value in real-world scenarios, such as correctly identifying that two patches fixing Django's 2-digit year formatting would behave differently due to a custom format() function that shadows Python's built-in function—a nuance that standard reasoning missed.

However, the approach comes with tradeoffs. Semi-formal reasoning requires approximately 2.8 times more computational steps than standard methods, increasing costs and latency. The structured format can also produce highly confident but incorrect answers when investigations are thorough but incomplete, and performance gains may be minimal for models already proficient at specific tasks.

The researchers have made their prompt templates publicly available, allowing developers to implement the technique without additional model training or specialized tools. They suggest the approach could serve as a flexible alternative to traditional static analysis tools by using task-specific reasoning templates that work across programming languages and frameworks.