Jieqi

On its second move, Black took one of Red’s face-down pieces with a cannon it had turned over a move earlier. The referee told Black what it had taken: a horse. Red, whose horse it was, was told nothing. By its seventh move Black had taken three of Red’s face-down pieces, and knew Red’s remaining army better than Red did.

The game is Jieqi (揭棋, “uncovering chess”), Chinese chess with every piece but the two generals shuffled and laid face down. Tencent’s xiangqi app, Tiantian, promoted it in 2017 as xiangqi that is hard to cheat at with an engine’s help, since engines by then far outplayed the best grandmasters. Nearly everything hidden in Jieqi is hidden from both players alike. A capture of a face-down piece, as in that game, is the exception.

Black was jqe, our engine for Jieqi, and Red was AB-JChess, which is built on the leading open-source engine for xiangqi, Pikafish. The first move of each face-down piece is a gamble on what it will turn out to be, and an engine has to put a price on every gamble. Until 4 October 2026 jqe’s search priced those gambles badly. Fixing that was the largest single gain in jqe’s development, worth 137 rating points (Elo) in matches against the previous version.A side that scores 64% is about 100 rating points stronger, one that scores 76% about 200. A 95% interval is worked out so that in 95 matches out of 100 it would contain the true margin.

In the formal matches that followed, jqe played 1,000 games against each of six opponents, three of them builds of one Pikafish version, on deals (layouts of the shuffled pieces) that none of them had met before. Against AB-JChess it scored 71.3% of the points, and against the other five between 77.0% and 99.1% (Figure 1).Besides AB-JChess, jqe played three builds of jieqi_old, the older of Pikafish’s two Jieqi versions: PikaJieQi from the website Mistboard, jieqi_old itself and pikajieqi-enhanced. jieqi_old judges positions by a hand-written evaluation. jqe also played musesfish, which is written in Python, and universe, a phone game that samples possible layouts of the hidden pieces and searches each as if it were known. jqe scored 77.1% against PikaJieQi, 78.7% against jieqi_old, 77.0% against pikajieqi-enhanced, 97.5% against musesfish and 99.1% against universe. Against the four engines that can use more than one thread, the clock was 60+0.6 with 8 threads each. musesfish and universe take a time for each move instead of a game clock, and against them it was 2 seconds a move, with jqe on one thread. A written standard, fixed before the matches, said what would count as jqe being the stronger: a margin of at least 100 rating points over each program, with the whole 95% interval above +50. Against every program, even the lower end of the interval lay above +100.The intervals are worked out over the 500 pairs of games against each program, each pair scored 0, ½, 1, 1½ or 2. Treating the 1,000 games against AB-JChess as independent gives +135 to +182 instead of +136 to +181.

A chariot on a soldier’s point

The board is xiangqi’s: nine files and ten ranks of points, with a river across the middle and a palace of nine points at each end. Red moves first. A chariot moves like a rook in chess. A horse moves like a knight, one point along a line and then one diagonally outward, and a piece on that first point blocks it. A cannon moves like a chariot but captures only by jumping exactly one piece of either side. The general plays the part of the king. It steps one point at a time inside its palace, and the two generals may never face each other along an open file. Advisors step diagonally within the palace, elephants move two points diagonally and stay on their own side of the river, and a soldier steps forward, and sideways too once it has crossed. A side with no legal move loses, whether or not it is in check; in chess, stalemate is a draw.

Jieqi changes the start, and with it the rules for two pieces. The generals stand face up on their usual points. Each side’s other fifteen pieces are shuffled and laid face down on the fifteen starting points, so a chariot may start on a soldier’s point and a soldier on a chariot’s. A face-down piece moves as the piece that belongs on its point would move, and is turned face up where it lands, with its own moves from then on. A face-up advisor may then leave the palace and a face-up elephant may cross the river, since either may have started far from home.The referee follows the rules of the Tiantian app where they could be confirmed, and the conventions of Pikafish’s newer Jieqi version elsewhere; the referee has not been checked against the current app. The app’s own rules page agrees with the referee that 40 moves each without a capture is a draw and that only the captor is told what a captured face-down piece was. It differs on repetition. The app forbids a seventh check or chase in a row by the same piece. The referee follows Pikafish: at the third repetition of a position, a side that has checked throughout loses, and if neither side has, a side that has chased one piece throughout, threatening to take it, loses. The 2017 championship in Sanya added further rules (a draw after 60 moves each without a capture, a draw at the fifth repetition, a draw when neither side can attack) that the referee does not use.

Each face-down piece hides an identity, which it shows when it is turned over. At the start the identity is a soldier with probability 5 in 15 and each other kind of piece with probability 2 in 15, and both players know these odds. Every piece turned over is seen by both, and changes the odds for both alike (Figure 2).

Red is not told what it lost

In the matches a separate referee program held the true deal and told each engine only what its side could know.The referee was tested on its own on ten million moves of random games, and jqe’s move generator agrees with it on every position of 20,000 random games. Figure 3 replays the game this article began with, from the second deal of the formal match against AB-JChess. After the capture on Black’s second move, Black knows that only one of Red’s horses remains. Red still counts thirteen identities it has not seen, two horses among them, for only twelve face-down pieces (Red’s view). After Black’s seventh move, Black knows Red’s remaining pieces better than Red does.

A program has to act on what its side has been told and nothing more. jqe is not told what a lost face-down piece was. Like Red with its two horses, it goes on counting that identity among those its face-down pieces may still have, so it counts more identities than it has pieces. It makes no attempt to work out, from how its opponent plays, what the opponent was told.Before any face-down capture, both players count the same unseen identities. After one, the captor knows exactly which identities the owner has left, and the owner no longer does. The Sanya regulations of 2017 tell players to keep the captured pieces they have seen out of the opponent’s sight.

Suspicion falls on the search

Like chess engines, jqe looks ahead. Its alpha-beta search tries moves and replies to some depth, and cuts off any line as soon as it is shown to be no better than one already found. Plain alpha-beta finds the same answer as searching every line, but skips most of them. A small network judges each position at the end of a line. The network uses the NNUE design, which is cheap to update after each move, and it was trained only on positions from jqe’s own games.The training went in rounds, and no network after the fourteenth did measurably better, so the formal matches use the fourteenth. The network’s 1,470 inputs say which kind of piece stands on which point, with a face-down piece as a kind of its own, and how many of each identity are still unseen, set out once from each side’s point of view. They feed a layer of 256 units for each side, then layers of 16 and 32 units, then one output. The first network, trained on games played with a hand-written evaluation, beat that evaluation by 92 rating points. Each later round trained a network on the best network’s own games at 5,000 positions searched per move, and kept it only if it won a match of 400 games (and, from the tenth round, also did better on a fixed set of positions held out from training). Each position’s training target mixes the search’s score for it with the result of the game it came from. The fourteenth was trained on 82 million positions. Wider networks, more data and new training targets were all tried, and none did better.

The first hint of a problem came on the morning of 2 October 2026, from development matches at one second a move. There jqe trailed AB-JChess by 26 rating points, and led jieqi_old, an older Jieqi version of Pikafish, by only 28.Both margins have 95% intervals that include zero: from −75 to +21 against AB-JChess, and from −20 to +77 against jieqi_old. At a tenth of a second a move jqe had led jieqi_old by 100 to 130 points, but at that clock the engines built on Pikafish searched only one to three plies deep. Extra time bought jqe less depth than it bought jieqi_old. A ply is one move by one side, and each tenfold increase in time let jqe search three or four plies deeper, and jieqi_old about eight. jqe also searched about thirty times as many positions as AB-JChess in the same time,Positions searched, or nodes, are a rough measure across engines this different: a small network is quicker to run, and each engine cuts its search short in its own ways. and its network predicted the results of games no worse than AB-JChess’s own. That morning an entry in the project’s log of decisions blamed the gap at longer clocks on the search, chiefly on the moves that turn a piece over, and the work that followed put the search ahead of the network.

Over the next two days, as other changes went into jqe, its margin over AB-JChess on one set of development deals was +76 at one second a move, +47 at two and +30 at eight (Figure 4).The margins of +76 and +47 come from 2 October, and +30 from 3 October. The development matches were played on deals kept apart from the formal ones. The margins of 76 and 30 points have 95% intervals from +25 to +130 and from −15 to +75. The intervals of the first and last overlap, so those matches could not show that jqe’s lead fell with time.

The price of a gamble

Call a move that turns a piece over a reveal. Expected score is the points a side expects from a game, a win counting one and a draw half. In the search a reveal leads to a chance node, where chance decides which identity turns up. The chance node’s value is the average of the expected scores after each identity the piece might have, each weighted by its chance. A line with several reveals branches quickly, and in the opening most moves are reveals.

A check took 72 reveals that jqe had chosen after the opening in development games against AB-JChess. It searched every identity of each reveal again, in a separate search for each. On average the value jqe’s search had put on a reveal was too high by 2.1 percentage points of expected score. The error on a typical reveal, counted whichever way it fell, was 5.3 points.The reference searched each identity with an open window to 11 plies after the reveal, the horizon of the 12-ply search being checked. The typical error is the root mean square. The 72 reveals come from development games at 8 seconds a move and were chosen by the old method itself, which may favour the reveals it overrated. The average error of 2.1 points has a 95% interval of ±1.1. In play, too, jqe kept finding its reveals worse than it had thought. In 112 development games, its estimate of its expected score drifted down after its reveals, by 1.2 percentage points a reveal on average. That came to about 14 percentage points over a game.The 112 games pitted the old version against the fixed one, and the drift is measured after the old version’s reveals.

The cause is the shortcut that makes alpha-beta search fast. Suppose jqe’s face-down piece is equally likely to be a chariot or a soldier. As a chariot it would leave jqe an expected score of 60%, as a soldier 10%, so the move is worth 35%. Suppose too that jqe already has another move worth 40% in hand. Searching the reveal, it needs to know only whether the reveal beats 40%. Once the soldier is shown to be worse, the search stops and reports a bound just under 40%; the soldier’s true 10% is never worked out. The old method averaged what came back, and a chariot at 60% with a soldier at 40% make 50%. The reveal now looks better than the move worth 40%, though it is worth only 35%. Where a player picks the best move, a bound is as good as a value, since the line will not be chosen either way. At a chance node the values are averaged, and a bound throws the average off. Engine writers call the value of the best other move found so far α. In effect the old method counted every identity worse than α as if it were worth α, and the others at about their own value, so the rule could only push a reveal’s value up.A first model of the error, which clipped good identities to the top of the search’s window of scores as well as bad ones to the bottom, predicted an error the other way, 2.5 points too low. But jqe’s search is fail-soft: a line that fails returns the best score it found, and need not stop at the edge of the window. In jqe an identity better than the window comes back with a score well above it, close to its value, while a worse one comes back just under α, so only the bad identities are clipped, and the average is pulled up.

A model of the old method searches each identity of two real reveals from the formal games again, as jqe would, and then applies that rule (Figure 5). On the first, whose error is close to the average, the model adds the hatched 2.3 points, and on the second, picked as one of the largest, about 9. On both, the average also puts the reveal just above the best other move, so there the error did not change jqe’s choice. Against a slightly better other move it would have, as moving α in the figure shows.

In 1983 B. W. Ballard published a search procedure for games of chance, Star1. It keeps alpha-beta’s cuts at a chance node, but makes one only when the result is safe whatever values the identities not yet searched turn out to have. In principle Star1 removes the error. On the same reveals it took 11.5 times the work, and its values were still 0.9 points too high on average.Star1 gives each identity a window worked out from the reveal’s own window and bounds on the identities not yet searched. Its typical error was 2.8 points. Ballard’s procedures treat the position after a chance event as a chance node, and assume a game of chance with nothing concealed, which Jieqi is only until a face-down piece is captured. Alpha-beta programs for Chinese dark chess, a game in which turning a piece over is itself a move, search such moves as chance nodes, some with Star1 and its refinement Star2. jqe’s shallow searches of every identity resemble ChanceProbCut, which cuts a chance node when shallow searches predict that its value lies outside the window, and less closely Star2’s probing, which searches one reply of each outcome for a bound. jqe averages the identities in expected score, the engine’s score passed through the logistic curve it was trained on. On that scale, turning a loss into a draw counts for more than making a large lead larger.

jqe’s new method gives up depth instead. It searches every identity without that cut, but six plies less deep than the full search, so a bad identity comes back with an estimate of its own value instead of a bound just under α. The likeliest identity is the kind of piece with the most copies still unseen, usually the soldier, and jqe searches it at the full depth too, keeping that value in place of the shallow one. On the same 72 reveals the new method cut the average error from 2.1 points to 0.9, the same as Star1’s, at 1.15 times the work.The typical error fell from 5.3 to 3.3. With the new method the estimate moved by only 0.1 points a reveal, and upward. Star1 should in principle be exact, and the check does not show why its values, and the new method’s, are still 0.9 points high. The drift after reveals all but vanished, and in games the new method was worth 137 rating points against the previous version.The gain was measured in 80 pairs of games between the old version and the new at a short clock, and has a 95% interval from 83 to 198. Such gains, between versions of one engine, are not evidence of strength against other programs. The new method at every reveal along a line, with three smaller refinements, added another 34, with an interval from 11 to 56.

The formal match against AB-JChess, played after the change, gave each side 60 seconds a game plus 0.6 seconds a move. That clock, written 60+0.6, came to about two seconds a move over a game of average length. Before the change, in development matches at that clock, jqe’s margin had been +60. In the formal match it was +158, on new deals and with 8 threads each.By 5 October jqe had the change, the refinements that followed it and some speed work. Its margin in development was then 200, with an interval from +152 to +257, on one thread each and on a computer so loaded that the clock was worth about a third of its label. Development figures run high, since they are the ones the work was steered by, and that interval contains +158, so chance alone could account for the gap. At five times that clock jqe still won clearly, by a margin of +177, in a shorter run on the development deals, though even there it left about a fifth of its time unused. That run was too short to show whether the margin grows or shrinks with time.jqe scored 73½ of 100 games. The margin’s interval runs from +106 to +266, and on the same deals at 60+0.6, on one thread each, the margin had been +200. The clock was 300 seconds a game plus 3 a move, with 8 threads each. The run was planned for up to 100 pairs and could be cut short once the answer was clear; it was stopped at 50.

When the colour wins

A shuffle can leave a side’s chariots stranded behind its own soldiers, or hand the other side a cannon on a point where it attacks at once. In the formal matches, whatever luck a shuffle handed Red, each engine got it once. Against each opponent jqe played 500 pairs of games, in which the programs played each deal twice, swapping colours. An engine tuned on a set of deals looks better on those deals than on others, so none of these deals had been used in development.The match program generated the deals in order from a published seed, so that anyone can generate them again. Before the first game it recorded the clocks, threads and other settings, and no match stopped early. The project’s log of decisions has the same settings, entered about 40 minutes into the first run. The seed is the first twelve hex digits of the SHA-256 of a published phrase, and pair i plays the i-th deal generated from it. Each manifest records the SHA-256 of the programs and networks the match used, checked whenever a match resumed. A game was to be replayed once if an engine failed or overran its time by too much; in the formal matches no game was voided, and none was lost on time or by an illegal move.

Against AB-JChess, jqe won both games of 238 deals and lost both of 30. In 209 deals the same colour won both games, whichever program played it: Red in 120, Black in 89 (Figure 6). Each of those deals cost jqe one game, 209 of its 275 losses in all, so most of its losses came on deals it won with the other colour.The project’s report counts a separate finding among jqe’s known weaknesses. In the development games that jqe lost to AB-JChess, its own estimate of its expected score had already fallen to 32% on average by its tenth move. Those games were decided early, and no specific flaw was found in them.

The engine that could not be run

Pikafish’s newer Jieqi version was written for a network, but no network for it was published. Its maintainer says none is available, and the version has since been removed. jqe could play only jieqi_old, which judges positions by a hand-written evaluation, and beat it by a margin of +227. The rating list on Pikafish’s site puts Pikafish’s network about 756 points above its hand-written evaluation, but in xiangqi, not Jieqi. Nor did jqe play strong human players, so whether an engine like it could help a player cheat at Jieqi is untested.Also not tested: the engines inside playing apps such as Tiantian. ZenoJchess and ForgeQi had no usable release, and eight other public programs were not run: none has an interface the match program can drive, and some have defects or no engine at all. AB-JChess’s author has also published two new versions, on 6 and 10 October; jqe played the version of 30 September.

The other limits are smaller. The standard required the agreed clock and threads for each engine, and that condition was met only in part. Other work kept the computer busy, so for long stretches neither engine had its 8 threads free, and the clocks were in effect shorter than their labels by an amount the formal runs did not record.Each engine got 8 threads of a 32-thread computer, and every logical processor was in use 44 minutes into the run against AB-JChess. Where a slowdown could be dated within a match, jqe’s score while it lasted and at full speed differed by no more than chance would explain. At first the standard let each opponent use as many threads as helped it. On 11 October, when five of the six formal matches had finished, we settled on equal threads as the standard, which is also how the rating list on Pikafish’s site compares engines. A planned run giving AB-JChess sixteen threads was never played.The standard’s last condition, source code and a network from which every result can be rebuilt, is met.

In most games AB-JChess ran its clock down until it was playing on the 0.6 seconds it gained each move. PikaJieQi and jieqi_old went lower still, their clocks falling to 0.2 seconds in the median game. jqe kept a median reserve of about 15 seconds, so it may owe part of its margin to the way the engines spent their clocks.

The standard also called for a test of whether jqe could be exploited. A copy of jqe was trained on 100,000 games against it to learn its weak points. Had the copy then won more than 60% of its games against the original, jqe would have failed. It won less than half.The copy won 48.1% and 46.3% of its games at two settings of how many positions it searched a move. The copy had little to learn from an opponent so like itself. It was one small attack of one kind, and passing it is not evidence that jqe is hard to exploit.

Take its face-down pieces

Figure 7 is jqe itself, running in your browser with its 1.5 MB network. It runs on one thread and with only the time you give it, so it will usually play weaker than in the matches. Take one of its face-down pieces and you are told what it was, while jqe goes on counting the piece among those it may still have.

@misc{jieqi,
  author = {Devin O'Keefe},
  title  = {Jieqi},
  year   = {2026},
  url    = {https://devinokeefe.com/jieqi/}
}