Wednesday, April 18, 2018

A final look at season 11

Season 11 is over, Stockfish is the winner. Season 12 is about to start, there is almost no pause between seasons now, and I want to have a last look at season 11 as a whole.

Season 11 changed the competition format and introduced a division system. Engines were split to 4 divisions based on season 10 results, and a fifth division was used for candidate engines that did not participate in season 10. In general I think the new format was a success. The engines' results depended on their performance against other engines of similar strength. In the previous format in early stages there were many games where the result was almost known in advance due to the huge difference in engine rating.

In the lower divisions one of the interesting engines was Laser. In season 10 it finished last and it started season 11 on division 3. Between seasons the engine was upgraded and it was known that it was much stronger, it was not a big surprise that it promoted to division 2. In division 2 Laser still played well and managed to promote again to division 1. The division 1 engines were stronger than Laser, yet it finished the division above the relegation threshold and will start season 12 in division 1. I would like to see other engines improve between seasons, this is the sort of thing that makes TCEC interesting.
 
In the premier division it was expectded that the top-3 engines - Stockfish, Houdini and Komodo - would fight for a place in the superfinal. In season 10 the race was very close, and Stockfish lost by half a point. This season Komodo was able to keep up until the halfway point of the premier division, but then lost sight of the leaders. It finished third despite beating Stockfish once, the only Stockfish defeat in the division. Stockfish finished first, with much better results against lower ranked engines than Houdini.

Based on the 12 games Houdini and Stockfish played in the premier division I assumed the superfinal would be close with many draws. Instead, the superfinal was extremely one-sided, Houdini struggled in many openings and lost 20-2. Stockfish was upgraded between divisions on top of the upgrade between seasons, and Houdini was the same as season 10, and it showed.

Leela to play in season 12

Towards the end of season 10 Google published its Alpha Zero chess project, where it used specialized hardware and its infinite resources to train a neural net based engine using only the rules of chess and self-play. The claim was that this engine could beat Stockfish.There was a lot of controversy surrounding this claim, and Google hasn't responded to any of the arguments, probably it only used chess as a proof of concept and it has bigger plans for the technology. The chess community, on the other hand, did not ignore the claim and a new chess engine was born - Leela Chess Zero. This project tries to imitate Google's ideas using standard hardware, where everyone can participate in the ongoing training procedure. At the end of season 11 Leela got to a level where it could compete (and lose) against the weakest engines in TCEC. It played exhibition matches against Scorpio and against a weak Stockfish, losing both but not without winning a few games. Its rating was estimated to be around 2750. Just before season 12 started Anton announced that Leela will play in division 4 after an engine dropped out. There have been last minute bugfixes and improvements, this is going to be exciting to watch.


site stats
This is a graph of the all time monthly hits for this blog: 



During the season the statistics reports were promoted on the chat and used in chessdom. The last 6 months have seen at least 1000 hits each month, peaking towards the higher divisions. The total number of hits is over 22,000.

Thanks to all the readers, I hope to continue being informative as well as interesting in the next season.

Go TCEC!!


Tuesday, April 17, 2018

Season 11 superfinal, games 91-100

The superfinal of season 11 is over, Stockfish beat Houdini 20-2 with 78 draws. This was one of the most one-sided superfinal matches in TCEC history.

There were a few exchanges at the start of game 91, Houdini gave an early pawn, the center was blocked and there was one open file on the queen side when the engines started to shuffle on move 29. After 70 moves the engines opened up the position, after exchanging queens the evals came down to 0. The game ended in a repetition with only pawns on the board. Evals were close to 0 in game 92. The position was more open and the engines gradually exchanged pieces until reaching a tablebase draw.

Game 93 started with Stockfish a pawn up and eval close to 1. After an early series of exchanges only RRB vs RRB remained on move 23, Stockfish had passers on the king side and Houdini had passers in the center. Both engines queened pawns and the game reached a tablebase draw. In game 94 Stockfish got the pawn back and evals were at 0 from move 20. Stockfish sacrificed a bishop and exposed the white king, Houdini traded down to a drawn rook ending.

Game 95 was a King's Gambit, a second variant of this opening in the superfinal. The white king was moving forward facing a black queen and two pawns. Houdini was a pawn up and had a small eval advantage. It chose to give back the pawn and exchange queens. Stockfish's king was awkward on the 3rd rank, yet it stayed safe. Stockfish exchanged all rooks leaving only BB vs BB, evals came down to 0. The game continued 40 more moves before draw adjudication.
In game 96 Stockfish didn't press on the king side, it patiently developed its pieces instead. Houdini regained the pawn by capturing on b7 with its queen, opening a file for a black rook. Both kings remained in the center, Stockfish's king side pawns looked intimidating.



Evals started to increase, first Houdini traded a rook for a knight and pawns, then Stockfish won a bishop for two more pawns. After exchanging queens only RN vs RRB remained, Houdini was 4 pawns up but playing a rook down against Stockfish is not a great idea...



The white king found safety behind its pawns, but it couldn't come out to help with the defense. Stockfish had an extra piece and it started to capture the white pawns. Its last pawn became a passer and started to march, ensuring a win for Stockfish.

In game 97 Stockfish started with an eval of around 1. The engines locked the center and shuffled for a while. Gradually the position opened up, first Houdini made a hole in the pawn line on the king side and then Stockfish opened two files on the queen side. After another shuffling period Stockfish gave a rook for a bishop and pawns and the engines traded down to a BB vs RB position. Stockfish's eval was above 1, it got a pawn to the 7th rank but Houdini gave the material back to get a drawn opposite color bishops ending. In game 98 Houdini kept a passer on the queen side, the center was locked and Stockfish concentrated forces on the king side. The engines were essentially shuffling from move 30, after 30 moves the engines exchanged a pair of rooks and continued shuffling. After avoiding the 50-move draw the engines traded down to a bishop ending and the game reached a tablebase draw.

Stockfish had an eval advantage starting game 99. The engines castled in opposite positions but did not push pawns forward. On move 31 Stockfish exchanged queens and created an advanced passer in the center, giving Houdini a similar passer though less advanced. Only RRN vs RRB remained and evals stayed under 1, the game seemed to be heading for a draw.



Stockfish had a pawn advantage on the queen side and it created another passer there. Houdini supported its passer with all pieces available, matching the white attacking pieces. Then Houdini allowed the white d pawn to reach d7 forcing a rook to move, Stockfish saw a winning line and its eval jumped over 3.



The win was not clear at first, the engines eliminated the passers in the center and exchanged minor pieces. Stockfish then took over the 7th rank with its rooks, captured a pawn and exposed the black king to attacks. Houdini had to be careful not to get into a mating net, Stockfish captured a second pawn and its queen pawns were free to march. The game was soon over.
Houdini started an attack on the black king with a knight sacrifice in game 100. Stockfish exchanged any piece it could, Houdini got the piece back and only RB vs RB remained with Houdini a pawn up on move 29. Evals were over 0.5, but after exchanging rooks the game reached an opposite color bishops ending, and the extra pawn was not enough. The engines continued to play for a hundred more moves before lowering evals enough for a draw. This was the last game of the season and the engines wanted to stay as long as they could.

Monday, April 16, 2018

Season 11, superfinal stats

Draw rate, wins



Final draw rate was 78%, very close to the draw rate of the last superfinal and slightly better than the target rate 80%.

Game termination

The three most common game termination causes were:
32% - TCEC draw rule
28% - TB position
22% - TCEC win rule


Moves per game

 

Median=62
Average=73.2

The distribution has a long right tail. There were 17 100+ move games, three of these were decisive, all black wins.
 
Time per game (hours)
  
Median= 4:12
Average=4:09

The short increment made sure the game duration would not far exceed 4 hours.

Openings

There were variable length book openings in this stage chosen by Jeroen.
The first letter of the ECO codes was distributed as follows:


If we use the opening 'family name' (using format FAMILY_NAME: VARIANT....) the top 3 are:

Sicillian - 14 times
King's Indian - 10 times
Caro-Kann - 8 times
Spanish -  8 times

This is very similar to the distribution in season 10.

Almost all of the book lines had the same ECO codes in reverse games, except for one paired game - games 11-12, one was A08 Reti: KIA, French Variation and the other was C00 French: KIA. There were 6 game pairs with different opening variants.

Reverse pairs, wins

 
There was only one biased openings with two white wins. On the other hand there was one opening where Stockfish won both games, a rare event in superfinals.


Reverse pairs, same moves  


Reverse games diverged very quickly, 36% diverged immediately after book, 68% diverged at most after two plys.


Sunday, April 15, 2018

Season 11 superfinal, games 81-90

After 90 games Stockfish leads 18-2 with 70 draws. It is now official: Stockfish is the winner of the superfinal of season 11. Houdini can not close the gap in the remaining ten games.

Stockfish had an eval close to 1 at the start of game 81. It gave a pawn and opened the king side, took over the long diagonal looking at the black king and threatened mate on the h file. Stockfish's eval was over 2 very quickly.



Houdini exchanged pieces to get its king out of immediate danger. It was still a pawn up and had passers on both the king and the queen side, while Stockfish had a bishop pair, a central passer, and the black king was still exposed. Stockfish's eval was already above 3.



After exchanging a few pawns Stockfish put a rook on the 7th rank, and together with the bishop pair they confined Houdini's king and two pieces to the back rank.



The black pawns could not be defended and Stockfish created a second passer on the king side. With the two passers advancing the game was soon over. Houdini forced the black king to move without castling in game 82. Houdini's eval was around 0.8 for a while, but it came down as the engines opened files and exchanged pieces. The game reached a RB vs RB position and ended in a tablebase draw.

The engines castled in opposite directions in game 83. Stockfish pushed a pawn on the king side, weakening the black king position. Houdini pushed pawns on the queen side, Stockfish gave a rook for a bishop to eliminate one pawn. Stockfish captured another pawn in the center, creating a queen side passer, evals started to increase. Stockfish's eval peaked around 1.5, it had 4 queen side passers in a QBN vs QRB position. Houdini's pieces were very active and could attack the white king, Houdini gave back the material and got rid of two of the passers. Evals dropped to 0, it took the engines another 30 moves to reach a tablebase draw. In game 84 the engines again castled in opposite directions and pushed pawns in a similar manner. The files remained closed on both sides of the board, and after a few exchanges the engines started to shuffle on move 27. Evals came down to 0 and 55 moves later the engines opened the queen side and the game ended in a perpetual check draw.

Evals were close to 0 throughout game 85. The engines exchanged pieces gradually until reaching a rook ending. Stockfish lost its rook to stop a black pawn queening. Then Stockfish promoted a pawn, and Houdini was forced to close the game with perpetual check. There were many early exchanges in game 86, and only QRR vs QRR remained on move 21. The engines shuffled for a while and then reduced to a drawn rook endgame.

Houdini was a pawn up in game 87 with a queen side passer. Stockfish was ahead in development and it quickly equalized. Evals were close to 0 as Stockfish concentrated forces against the weak black king side and opened it by pushing pawns. Stockfish let Houdini capture a bishop on the queen side for a dangerous attack on the black king, which was strong enough to force a perpetual check draw. In game 88 Stockfish pushed the queen side pawns forward, ignoring the lack of development on the king side and keeping the king in the center. Houdini placed its rooks on the queen side and prevented the black passer from advancing. Evals dropped to 0 after a series of exchanges, Houdini eliminated the black passer and only RN vs RN remained. The game continued until reaching a tablebase draw.

Stockfish started game 89 with an eval advantage that increased gradually and was over 1 on move 20. Houdini had a pawn advantage on the queen side and it pushed its pawns forward. After a series of exchanges Stockfish won a pawn and QRB vs QRN remained. On move 44 after 12 minutes of thought Stockfish's eval jumped to almost 3. Houdini thought for 10 minutes and its eval also jumped. Both engines did not provide a PV that showed what they planned, however in the following move the evals dropped back, very strange. The engines traded pieces and the game reached a drawn B vs N ending.  Houdini had an eval advantage in game 90, an early series of exchanges left QRB vs QRN on move 22. Stockfish was up a pawn but its king was vulnerable. After exchanging queens the evals came down, the game reached a tablebase draw.

Friday, April 13, 2018

Season 11 superfinal, games 71-80

After 80 games Stockfish leads 17-2 with 61 draws. Houdini finally got its 2nd win, this time drawing the reverse game. Stockfish still has a 15 win advantage, and with 20 more games to play it only needs a few more draws to be declared the winner of the superfinal.

Stockfish had an eval close to 1 at the start of game 71. Both engines attacked the opposite kings and after several exchanges the net outcome was that Stockfish was up a rook for a knight and pawn on move 23. Houdini's king was exposed and evals started to increase in a QRB vs QBB position. After exchanging queens Houdini had passers in the center and Stockfish had a doubled passer on the king side. Houdini managed to get a pawn to the 2nd rank protected by a bishop. It couldn't promote but Stockfish had to keep its rook guarding the file. Stockfish's evals was over 2 for a while but it soon realised Houdini was holding. After many moves Stockfish released the black pawn to promote to a queen, and the game ended in a RB vs Q draw. Another narrow escape for Houdini, seems this is the best it can do to show its strength. In game 72 Stockfish equalized quickly and Houdini's eval dropped on move 24. The engines were mostly shuffling with most pawns on the board. Houdini broke through the center, leading to many exchanges and a drawn rook ending.

Many early exchanges in game 73 left RRB vs RRB on move 32 and evals close to 0. The engines continued to exchange pawns and pieces until reaching a tablebase draw. In game 74 evals were negative for a while but Houdini was never in any danger. After exchanging queens the game reached a RB vs RB position. Both engines had advanced passers, Stockfish gave a rook to stop the black passer, Houdini forced a perpetual check draw ro avoid the queening of a white passer.

Stockfish's eval climbed above 1 at the start of game 75. The engines castled in opposite directions, there were many early exchanges and queens were off as well. Houdini didn't seem to be worse on the board, material was equal and the pawns structures were similar.



Stockfish used its bishop to block the king side and apply pressure on the black king, together with a rook on the 7th rank. The evals slowly increased, then suddenly jumped above 2.5. Houdini moved a rook on the back rank to d8 and not g8 as Stockfish recommended, Stockfish played the same response it planned and doubled rooks on the e file.



Houdini gave a rook to take out the white bishop, and the evals jumped higher very quickly. The game reached a RR vs RN position, the white pawns on the queen side started to march and Houdini saw it couldn't stop them in the long run.
In game 76 both engines castled long, evals stayed close to 0. After exchanging most pieces only RB vs RB were left on move 31 and the game ended in a tablebase draw not long afterwards.

Stockfish had an eval advantage of about 1 in game 77. The engines castled in opposite directions, both engines pushed pawns towards the opposing king and opened files. Stockfish's eval peaked close to 2, it attacked on the king side and exposed the black king. However the eval had a sudden drop, Houdini's counter on the queen side exposed the white king and was enough for a perpetual check draw.
Houdini had an eval advantage in game 78. Stockfish delayed castling its king until after queens were exchanged. On move 30 only RRB vs RRB remained with equal material. Evals hadn't moved a lot since the opening, probably a draw.



Houdini captured a pawn and the engines exchanged a pair of rooks. It was not clear how Houdini can improve its position, and the bishops of opposite color made the position look drawish. Yet the evals slowly crept up. Houdini managed to capture a second pawn and create a passer on the king side. Could this be Houdini's day?



Houdini secured its passer and then moved its rook forward where it could attack the black king and threaten the queen side pawns. Stockfish was forced to exchange rooks and then it saw a win for Houdini. Houdini took a few more moves to realise that despite the opposite color bishops ending it had a winning position.



With two passers and the king on its way to the queen side Houdini was too strong and Stockfish agreed it was a win for white. Finally an opening where Houdini wins a game and does not lose on the reverse.

Game 79 had a wild start. The king side remained undeveloped and both engines kept their kings in the center uncastled. Stockfish pushed two pawns on the queen side and one got to the 7th rank. Houdini arranged its pieces for an attack on the white king, with the pawn on a3 blocking the escape route. Stockfish's eval was over 1 but changed very little.



Both queens attacked simultaneously, and both kings looked very vulnerable. Stockfish queened its pawn and for a while played with two queens. Houdini's mate threat kept it alive, forcing white to lose one of the queens. After a series of exchanges the remaining queens were gone and the attacks were over, Stockfish had a material advantage with RRN vs RBB remaining.



The engines exchanged a pair of rooks and the evals increased slowly. It is possible that Houdini could have held, but Stockfish found a way to improve and its evals started to jump. As usual Houdini realised it was in trouble a few moves later. The black bishop pair was harassed by the white rook, allowing Stockfish to arrange its pieces and keep the black king away. 



The white pawn started to march and the game was soon over. Houdini couldn't stop it without losing material. What a game!
 In game 80 both kings remained uncastled, but both engines kept their pieces behind the pawn lines and did not attack. The engines started to shuffle with almost all pieces on the board. On move 32 the pawn lines locked across the board, with only the f pawns able to move. Nothing happened for the next 75 moves, except a few f pawn moves and a few piece exchanges. To finish the game the engines opened up the king side and exchanged all pieces but a pair of knights.

Wednesday, April 11, 2018

Season 11 superfinal, games 61-70

After 70 games Stockfish leads 15-1 with 54 draws. Stockfish added one win in white and one in black, Houdini had a narrow escape in another game. With 30 games remaining Stockfish will undoubtedly win the match, can Houdini get a win in the remaining games?

Stockfish started game 61 with an eval under 1. On the board Stockfish had a space advantage on the queen side and a bishop pair. Houdini had a queen side passer, facing all the white pieces the passer was not going anywhere.



Stockfish's eval continued to increase. Houdini gave the passer to open a file on the queen side, and tried to counter attack. Stockfish's pieces were all aimed at the black king from a distance, looking very dangerous. Houdini tried to block the center, according to the evals Stockfish was much better though it was not clear on the board.



Stockfish broke through the queen side. Houdini gave a rook for one of the bishops, and Stockfish created an advanced queen side passer. The combination of the passer with a mating threat on the king side was enough for a win.
In game 62 Houdini did not keep its bishop pair, preferring to capture an early pawn. Stockfish exchanged queens and got the pawn back, evals decreasing toward 0. The game reached a RB vs RB position on move 28. In the end only pawns remained, both engines queened one pawn but the result was still a draw.

In game 63 Stockfish had an eval advantage under 1. The queens were exchanged early, the engines continued to trade pieces until reaching a rook ending with evals at 0. The game ended in a tablebase draw. Game 64 was similar, Stockfish's eval was 0 from move 18. The engines exchanged all their pieces to get a king and pawns ending, both engines queened a pawn and another draw.

Stockfish had an eval advantage in game 65 despite keeping ints king uncastled in the center. The engines exchanged queens early, by move 32 only RBB vs RBN remained and evals were close to 0. The game ended 40 moves later in a B vs N ending.
Houdini was ahead on eval starting game 66, Stockfish quickly equalized. After a few exchanges the center opened up. Houdini had a pawn majority on the queen side, material was balanced and evals were close to 0.



Houdini seemed more dominant in the central files. and yet the evals turned negative. Possibly the white king was more vulnerable to attack with no pawns on the 2nd rank. After exchanging bishops Stockfish controlled the long a8-h1 diagonal with its queen. Houdini gave a rook to take out the central black knight and then exchanged queens to get a RB vs RR ending. Evals were still below 1.



Houdini allowed the black rooks to take over the 2nd rank, and the evals jumped over 1. After several futile checks Stockfish found a way to improve. It used its rooks to guide the white king slowly to the queen side, with the h5 pawn blocking the path to the king side.



With its king unable to help Houdini's defense soon collapsed. Stockfish brought its king forward and pushed pawns on the king side. Houdini had to move its pawns and they became targets for the black rooks. The PV at the end of the game showed how black can capture all the white pawns. 

Stockfish's eval was close to 1 in game 67, Houdini's king was uncastled in the center with almost all pieces on the board. Houdini trapped a white bishop on the king side yet the evals increased over 1. Stockfish gave two pawns and got a rook on the 7th rank, threatening the black king. It then freed its trapped bishop and got the pawns back, evals approaching 2. Houdini exchanged queens but evals continued to increase, white had passers on the king side and Houdini gave a rook for a bishop to stop them. The engines reached a RB vs BN ending and the evals stayed high but constant. Unexpectedly Houdini held the position to a draw. In game 68 Houdini mainained an eval of about 0.5 for over 40 moves. Stockfish castled its king, the engines opened the queen side and exchanged pieces. It again seemed that the black king was under threat when Houdini's rook moved to the 7th rank, though it didn't stay there for very long. In a QRB vs QRB position Houdini captured a pawn, Stockfish countered by trading down to a drawn rook ending.

Evals stayed low in game 69. The engines exchanged pieces in an open position, when all minor pieces were gone the two kings were exposed to attacks. The engines chose to draw by reducing to a tablebase draw, though there were possible perpetual check draws available. In game 70 Stockfish gave a rook to get a perpetual check draw after only 31 moves.

Monday, April 9, 2018

Season 11 superfinal, games 51-60

After 60 games Stockfish leads 13-1 with 46 draws. The last 10 games have all been draws, Stockfish may be taking a break since it is almost surely going to win the match.

Houdini was a pawn up after the start of game 51. All pieces remained on the board as the engines started to shuffle on move 19. After 35 moves Houdini gave a rook for a bishop, and the position opened up. After a series of exchanges only RR vs BNN remained, Houdini with 3 extra pawns. Evals were close to 0 despite the unusual material imbalance, a white passer cost Houdini a knight, the engines then traded pieces until reaching a tablebase draw. In game 52 the engines kept the pieces on the board for the first 35 moves. Stockfish was a pawn up with a queen side passer and eval above 1. After a few exchanges the evals dropped and the engines started to shuffle. At move 112 the engines started to open the position, after clearing the queen side the game was adjudicated.

Stockfish started game 53 with an eval advantage with all pieces on the board. The engines were close to a 3-fold repetition on move 17, instead the pawns locked the queen side and the engines exchanged a few pieces and pawns. The position opened a little and the engines exchanged most pieces, Houdini gave a knight for pawns to break the white pawn line on the king side. The game reached a bishop vs pawns ending, Houdini was 3 pawns up. The result was a draw though it took 65 shuffling moves to get there. In game 54 the engines were essentially shuffling from move 19. Evals turned negative but stayed constant. The pawns stopped moving on move 49 as the shuffling continued. On move 119 The engines started to open the position with exchanges, reaching a drawn knight ending.

Game 55 started with Stockfish a pawn up and Houdini's king in the center unable to castle. Stockfish's eval was over 1 but more or less stable. The position became static, then after a series of exchanges only QNN vs QNN were left. After exchanging a pair of knights Stockfish's queen moved forward, and Houdini managed to trap it in the corner. For a while Stockfish thought it was ahead with eval approaching 2, but it soon realized there was no way to improve. After exchanging queens the evals dropped to 0, the game continued until getting a tablebase draw. In game 56 Houdini chose a different line, not forcing the black king to move but winning a rook temporarily. Stockfish countered with an attack on the white king, forcing it to move without castling. Stockfish regained the material by move 17 and evals were close to 0. The game reached a knight ending and a draw.

Stockfish had an eval advantage close to 1 in game 57. It went a pawn up on move 16, and after a series of exchanges only BN vs BN were left on move 28. The extra pawn was not enough, the game reached a tablebase draw. In game 58 evals came down to 0 faster. Material was balanced as the engines exchanged pieces. Houdini grabbed a pawn before exchanging queens, reaching a N vs B ending on move 34. Stockfish got the pawn back and the game was adjudicated a few moves later.

There were many early exchanges in game 59, with only RRB vs RRN remaining on move 22. Stockfish's eval jumped over 1 when it took over the 7th rank, and jumped again over 1.5 after trading minor pieces. It appeared that Stockfish was ahead with an advanced queen side passer, however Houdini put its rooks on the 2nd rank and could end the game with a perpetual check. Houdini chose to play on, only delaying the draw by a few moves. In game 60 evals were low throughout the game. After move 13 there were no exchanges for 20 moves. After the position opened up the white king was exposed. Stockfish attacked and started a series of exchanges that led to a drawn B vs N ending.