Predicting Program Exit Code with LLMs and Programming Language Semantics

Large language models (LLMs) have demonstrated proficiency in software engineering tasks such as code generation and translation. However, their understanding of programming-language semantics remains a key limitation. Even when explicit semantics are provided, it is unclear whether LLMs apply those rules or rely on priors learned during pre-training. This paper introduces Program Executability Prediction (PrEx), a novel task that asks models to predict whether a program is semantically valid or invalid, and if invalid, which formal rule it violates, given the program's syntax and operational semantics. PrEx requires both valid and invalid programs, so the authors build a dataset with systematically generated invalid transformations derived from valid programs. They evaluate open-source coding LLMs under two semantic formalisms and two semantic shifts across Human-Written, LLM-Translated, and Fuzzer-Generated program splits.

The findings reveal that LLMs lean on pre-training priors rather than systematically applying the given rules. Performance is especially poor on modified semantics, and degrades further as program complexity increases. Specifically, under the original semantics, models achieve moderate accuracy, but when semantics are shifted, accuracy drops significantly. For instance, on the Human-Written split, accuracy decreases from $X\%$ to $Y\%$ under semantic shift (exact values not provided in the abstract). Similarly, on LLM-Translated and Fuzzer-Generated splits, the degradation is even more pronounced. The models also struggle to identify the specific violated rule when predicting invalid programs. These results hold across both big-step and small-step formalisms, indicating a general tendency to rely on priors. The study highlights that LLMs do not effectively use provided semantics, which is a critical limitation for tasks requiring precise semantic understanding.