OpenAI updates models to improve coding and migration for legacy languages
Article Summary
OpenAI’s latest models demonstrate enhanced capabilities in low-resource programming languages like COBOL and Fortran, which support critical global infrastructure such as banking and weather forecasting. New internal evaluations show significant performance gains, with GPT-6 Astra achieving an 87.4% pass rate on MultiLangBench across 20 languages compared to 26.8% for GPT-5.6 Sol. Additionally, GPT-6.1 Sol reached 86.6% accuracy on a COBOL migration task with reduced reasoning effort, rising to 97.9% at higher effort, while providing substantial improvements in API cost efficiency.
AI Analysis
## Core Assessment
Any veteran architect who has touched banking mainframes knows why COBOL persists: it underpins 95% of ATM transactions and 80% of in-person credit card swipes because breaking subtle arithmetic behavior breaks real money. OpenAI’s evaluation data shows meaningful progress on this chronic headache. On their internal migration benchmark requiring byte-for-byte output parity, GPT-6.1 Sol jumps from GPT-5.6 Sol's dismal 5.4% up to 86.6% accuracy under low reasoning effort, reaching 97.9% at high effort while trimming API costs by 39%. On MultiLangBench—building a functional SQLite engine from scratch—GPT-6 Astra posts an 87.4% average pass rate across 20 languages, lifting Fortran to 83.5% and COBOL to 72.1%.
The numbers reflect genuine gains in low-resource syntax handling, but treating synthetic benchmarks as a green light for automated migration is a trap. The hardest part of legacy systems is never generating matching code for self-contained modules; it is untangling decades of undocumented batch flows, proprietary mainframe file interfaces, and implicit business rules embedded across spaghetti code. Passing an isolated test suite does not solve system-level risk, and the remaining 2.1% error margin at maximum effort is precisely where silent accounting discrepancies hide.
## Impact and Advice
* **Legacy Systems Architect:** Do not run wholesale language conversions through model APIs. Use these models strictly to extract embedded business logic and draft initial translation candidates, then subject every converted routine to shadow-run verification against production data.
* **Engineering Manager:** Budget for extensive test harnesses before writing a single line of replacement code. Your primary bottleneck is not model inference cost—which OpenAI claims dropped roughly 78% on GPT-6 Luna—but the engineering hours needed to audit edge cases like COBOL fixed-point truncation against target language floating-point libraries.
## Bottom Line
Models are finally learning to parse legacy syntax without falling flat on basic syntax, but until AI can unearth decades of unwritten corporate operational rules, nobody is turning off the mainframe.
Any veteran architect who has touched banking mainframes knows why COBOL persists: it underpins 95% of ATM transactions and 80% of in-person credit card swipes because breaking subtle arithmetic behavior breaks real money. OpenAI’s evaluation data shows meaningful progress on this chronic headache. On their internal migration benchmark requiring byte-for-byte output parity, GPT-6.1 Sol jumps from GPT-5.6 Sol's dismal 5.4% up to 86.6% accuracy under low reasoning effort, reaching 97.9% at high effort while trimming API costs by 39%. On MultiLangBench—building a functional SQLite engine from scratch—GPT-6 Astra posts an 87.4% average pass rate across 20 languages, lifting Fortran to 83.5% and COBOL to 72.1%.
The numbers reflect genuine gains in low-resource syntax handling, but treating synthetic benchmarks as a green light for automated migration is a trap. The hardest part of legacy systems is never generating matching code for self-contained modules; it is untangling decades of undocumented batch flows, proprietary mainframe file interfaces, and implicit business rules embedded across spaghetti code. Passing an isolated test suite does not solve system-level risk, and the remaining 2.1% error margin at maximum effort is precisely where silent accounting discrepancies hide.
## Impact and Advice
* **Legacy Systems Architect:** Do not run wholesale language conversions through model APIs. Use these models strictly to extract embedded business logic and draft initial translation candidates, then subject every converted routine to shadow-run verification against production data.
* **Engineering Manager:** Budget for extensive test harnesses before writing a single line of replacement code. Your primary bottleneck is not model inference cost—which OpenAI claims dropped roughly 78% on GPT-6 Luna—but the engineering hours needed to audit edge cases like COBOL fixed-point truncation against target language floating-point libraries.
## Bottom Line
Models are finally learning to parse legacy syntax without falling flat on basic syntax, but until AI can unearth decades of unwritten corporate operational rules, nobody is turning off the mainframe.