rules and faq

Written for the strongest entrant.

Anything a sharp engineer could argue about, we settle here before launch. Where a figure is served by the challenge API, this page shows the live value.

What counts as correct.

The program is compiled with GnuCOBOL 3.2 using the published flags and command. Anyone can run it and get the right answer for any input. For this challenge, the program as GnuCOBOL runs it is the definition of correct. Overflow is modelled as COBOL truncates it; a value too large for its field loses its leading digits.

Your entry’s outputs are compared with the COBOL’s as canonical decimal strings at each field’s scale. Exactly the program’s output keys on every call; an extra or missing key is a difference.

What you submit.

One Python 3.12 file that defines replay_entry(inputs: dict) -> dict. Field names come from the program’s LINKAGE section; values are decimal strings at the field’s scale. At most 200,000 bytes.

Checked before anything runs:

Imports only from the allowlist.
No clock, randomness, environment, files, network or introspection.
The literal rule: every number and string in your code must appear in the COBOL source, or be 0, 1, −1, a power of ten, or a field name.
Deterministic: every input is run more than once, and any difference fails the entry.

A refusal by the static check is shown at once with its findings and does not count against your daily limit.

If the service itself cannot finish checking an entry, you get “no verdict”, and your first two of those a day do not count. A “no verdict” caused by your entry’s own runs on the secret tests (out of time, crashed or out of memory there) always counts.

The time limit.

Time limits: Replay runs every input of a set three times. Twice each input runs alone, in a fresh process, with the inputs shared between 4 sandboxes: each sandbox may take 0.9 seconds for each input it runs. Once every input runs in order in one process: that run may take 0.2 seconds per input. Each run also gets 20 seconds to start. An entry over a limit fails that set; on the hidden set that reads "failed on a hidden input", like any other failure there.

The same limit applies to the public and the hidden set. Over it on the public set, you are told so with the limit; on the hidden set it reads “failed on a hidden input”, like any other failure there.

What wins.

A hunt catch fails the entry outright. It is shown as “caught”, with the input and both answers. A failure on the secret tests is shown as "failed a secret test" and nothing more: no input, no count, no field, until the challenge closes and the secret tests are published.

A pass is not a win. A pass is not a win, because a correct migration passes too. To claim a win you give one input on which your entry and the COBOL program disagree: your proof input. The service runs your input on the COBOL and on your entry. If they disagree, your entry is re-hunted for 10 minutes (the win check); a catch there fails the entry. Otherwise a judge rules in public, within 5 working days, and labels the win “deliberate trick” or “honest miss”. Both count, on one board. The ruling goes in the public log with its reasons, the fields that differed and the hunt’s coverage; your input and your code are not published.

An entry held at “no verdict” only because the input sets have not been shown strong enough can also be claimed. A caught entry can never be ruled a win. One win per entry.

Any test-based tool can be tricked on purpose: a migration that is wrong only on inputs the tool never tries will pass. The rules say so, and a deliberate trick that gets through counts as a win.

Dates, limits and the reveal.

Opens: 12 Nov 2026, 00:00 UTC
Closes: 16 Nov 2026, 18:00 UTC
Claims deadline: 17 Nov 2026, 18:00 UTC
The reveal: After the claims deadline, once every claim has been checked: the hidden inputs and the seed are published, checked against the commitment before anything is shown. Entrants then see their own hidden results.
Entries: 5 a day per person (UTC)
Claims: 3 a day per person; 3 per entry
Appeals: 3 a day per person; answered within 5 working days, in public
Accounts: One GitHub account per person, at least 30 days old
The wait: about 5 minutes from the start of your run to the hidden verdict, fixed
Engine: 0.7.0, frozen at launch. Fixes ship as new versions; a win counts against the version live when the entry was made.

Appeals.

You can appeal an entry that was refused, failed, caught or given no verdict, and a claim that was rejected or ruled not a win. Appeals are filed from My entries. A judge answers within 5 working days; the decision and its response go in the public log. Your own words are never published.

An upheld entry appeal lets the entry be claimed. An upheld claim appeal re-queues the claim; the win check still runs. A caught entry still cannot be ruled a win: if a catch is shown to be wrong, the fix is an engine release, and you resubmit.

The three tracks.

Most people enter track 1. The other two run alongside.

01Be wrong and passSubmit a Python migration that behaves differently from the COBOL program on some input, and still passes Replay. Then prove it with that input. A ruled win, labelled deliberate trick or honest miss. Both count, on one board.
02Break the evidenceMake a signed result say "matched" where the outputs differ, alter one so it still verifies, read the answers from inside the sandbox, or get out of it. Report privately to security@montoralabs.com first. Written up in public once fixed; the top of the board.
03Models against ReplayRun by us. The leading AI models each migrate the program honestly, then are asked to slip a bug past. Every prompt, output and signed result is published. A story whoever turns up. Results published during the run.

Glossary.

The pages use plain names. These are the precise terms behind them.

Public tests the public set: The published inputs with the COBOL's answers. Your entry runs on all of them.
Secret tests the hidden set: Inputs from a secret seed, committed by hash at launch and published at the close. You see a verdict only.
The bug hunter the branch hunt: Replay reads your entry's own branches and builds inputs to reach each one, then runs them on the COBOL. A catch fails the entry.
Your proof input the witness: One input on which your entry and the COBOL disagree. It is how a pass becomes a win claim.
The win check the re-hunt: Before a judge sees a claim, the entry is hunted again for ten minutes with the client budget.
The signed result the envelope: Every outcome, signed with the challenge key, naming the program's hash and your file's.

Questions, answered.

Isn't this just testing?

Yes, in the strict sense: it is differential testing with exact COBOL semantics, on inputs generated from the program and on inputs found by searching your code, with a signed result that anyone can check. We say so. What the challenge measures is how good those inputs are. Customer work adds the thing a public challenge cannot: recorded production behaviour replayed against the migration.

GnuCOBOL isn't IBM.

Correct. For this challenge the program as GnuCOBOL 3.2 runs it, with the published flags, is the definition of correct. That makes every win and every loss checkable by anyone with a compiler, and takes the dialect argument out of the game. Where IBM Enterprise COBOL would differ (some truncation and arithmetic rules), the published flags are chosen to match it, and the rules say which behaviour is modelled.

What if someone wins?

Then the challenge is working. The win is credited by handle, labelled, and published in the log with its reasons. The input becomes a regression test named after the winner, the engine ships a new version, and we write up how it got through. A win counts against the engine version live when the entry was made.

Why isn't a pass a win?

A pass is not a win, because a correct migration passes too. To claim a win you give one input on which your entry and the COBOL program disagree: your proof input. A correct entry can never win, because there is no such input.

Why do I have to wait for the hidden verdict?

Your public result and the bug hunter's are shown as soon as they are ready, usually within a few minutes. The secret tests are then checked, and their verdict is shown at a fixed time after your run started, however long the check took. Until then your next entry waits too. The wait is fixed so that nothing you can see varies with how your entry behaved on the hidden inputs: not the time a verdict appears, not a queue position, not a charge. A failure on the secret tests is shown as "failed a secret test" and nothing more: no input, no count, no field, until the challenge closes and the secret tests are published.

What does the branch hunt do?

For 180 seconds Replay reads your entry’s own branches and tries to build an input that reaches each one, then runs those inputs on the COBOL. If it finds a disagreement, that is a catch: the entry fails, and you see the input and both answers. A catch is objective, since the COBOL itself disagrees. A catch cannot be ruled a win; if you think a catch is wrong, appeal.

Can I use a library?

Only what the allowlist names. If an honest migration needs something else, request it; a judge decides, and additions are logged in public.

Are deliberate tricks allowed?

Yes. Any test-based tool can be tricked on purpose: a migration that is wrong only on inputs the tool never tries will pass. The rules say so, and a deliberate trick that gets through counts as a win. A trick that gets through is labelled “deliberate trick” and sits on the same board as an “honest miss”. The static check still refuses the cheapest tricks (planted literals, clocks, randomness), and the hunt goes looking for the rest.

How do I report a security finding?

Track 2 findings (a forged or altered signed result, reading the answers inside the sandbox, reaching the network, getting out) go privately to security@montoralabs.com first. They are written up in public once fixed.

Terms.

Draft. These are placeholders for the founder, or a lawyer if a contact will read them cheaply, before launch.

placeholder · founder or lawyer

By entering, you let Montora run, keep and publish your entries as tests, credited by your GitHub handle.
Entry is free and judged on skill. No cash prizes. Rewards: credit, a named regression test, the leaderboard, a Friday shoutout.
One account per person. Montora may remove entries or accounts that break the rules or abuse the service.
Rulings and appeal decisions are final once published, save for a manifest error.
Track 2 findings are disclosed responsibly: privately first, in public once fixed.
No endorsement by any AI company is implied by Track 3. Model providers’ terms are checked before outputs are published.
Privacy: GitHub login and numeric id, entries, claims and appeals are stored; nothing else. Jurisdiction: England and Wales.

Judges’ rubric: placeholder for the founder (roadmap 4.9). The external judge is named before launch.