[ Help | Earliest Comments | Latest Comments ]

[ List All Subjects of Discussion | Create New Subject of Discussion ]

[ List Earliest Comments Only For Pages | Games | Rated Pages | Rated Games | Subjects of Discussion ]

Comments by H. G. Muller

~~Earlier~~ ⇧Reverse Order⇩ Later⇧ Latest⇩

Piece Values[Subject Thread] [Add Response]

H. G. Muller wrote on Sat, May 3, 2008 06:59 AM UTC:

Ha, finally my registration could be processed manually, as all automatic procedures consistently failed. So this thread is now also open to me for posting. Let me start with some remarks to the ongoing discussion. * I tried Reinhards 4A vs 8N setup. In a 100-game match of 40/1' games with Joker80, the Knights are crushed by the Archbishops 80-20. So although in principle I agree with Reinhard that such extreme tests with setups that make the environment for the pieces very alien compared to normal Chess could be unreliable, I certainly would not take it for granted that his claim that 8 Knights beat 4 Archbishops is actually true. Possible reasons for the discrepancy could be: 1) Reinhard did not base his conclusion on enough games. In my experience using anything less than 100 games is equivalent to making the decision by throwing dice. It often happens that after 30 games the side that is leading by 60% will eventually lose by 45%. 2) Smirf does not handle the Archbishop well, because it is programmed to underestimate its value, and is prepared to trade it to easily for two Knights to avoid or postpone a Pawn loss, while Joker80 just gives the Pawn and saves its Archbishops until he can get 3 Knights for it. 3) The shorter time control used does restrict search depth such that this does not allow Joker80 to recognize some higher, unnatural strategy (which has no parallel in normal Chess) where all Knights can be kept defending each other multiple times, because they all have identical moves, and so judges the pieces more on their tactical merits that would be relevant for normal Chess. * The arguments Reinhard gives against more realistic 'asymmetrical platesting': | Let me point to a repeatedly written detail: if a piece will be | captured, then not only its average piece exchange value is taken | from the material balance, but also its positional influence from | the final detail evaluation. Thus it is impossible to create | 'balanced' different armies by simply manipulating their pure material | balance to become nearly equal - their positional influences probably | would not be balanced as need be. seem invalid. For one, all of us are good enough Chess players that we can recognize for ourselves in the initial setup we use for playtesting if the Archbishop or Knight or whatever piece is part of the imbalance is an exceptionally strong or poor one, or just an average one. So we don't put a white Knight on e5 defended by Pf4, while the black d- and f-pawn already passed it, and we don't put it on a1 with white pawns on b3, c2 and black pawns on b4, c3. In particular, I always test from opening positions, where non of the pieces is on a particularly good square, but they can be easily developed, as the opponent does not inderdict access to any of the good squares either. So after a few opening moves, the pieces get to places that, almost by definition, are the average where you can get them. Secondly, when setting up the position, we get the evaluation of the engine for that position telling us if the engine does consider one of the sides highly favored positionally (by taking the difference between the engine evaluation and the known material difference for the piece values we know the engine is using). Although I would trust this less than my own judgement, it can be used as additional confirmation. Like Derek says, averaging over many positions (like I always do: all my matches are played starting from 432 different CRC opening positions) will tend to have avery piece on the average in an average position. If a certain piece, like A, would always have a +200cP 'positional' contribution, (e.g. calculated as its contribution to mobility) no matter where you put it, then that contribution is not positional at all, but a hidden part of the piece value. Positional contributions should average to zero, when averaged over all plausible positions. Furthermore, in Chess positional contributions are usually small compared to material ones, if they do not have to do with King safety or advanced passers. And none of the latter play a role in the opening positions I use. * Symettrical playtesting between engines with different piece-value sets is known to be a notoriously unreliable method. Dozens of people have reported trying it, often with quite advanced algorithms to step through search space (e.g. genetic algorithms, or annealing). The result was always the same: in the end (sometimes after months of testing) they obtained piece values that, when pitted against the original hand-tuned values, would consistently lose. The reason is most likely that the method works in principle, but requires too many games in practice. Derek mentioned before, that if two engines value certain piece combinations differently, they often exchange them for each other, creating a material imbalance, which then affects their winning chances. Well, 'often' is not the same as 'always'. For very large errors, like putting AR the undervaluation of A only can lead to much more complicated bad trades, as you have to have at least two pieces for A. The probability that this occurs is far smaller, and only 10-20% of the games will see such a trade. Now the problem is that the games in which the bad trades do NOT happen will not be affected by the wrong piece value. So this subset of games will have a 50-50 outcome, pushing the outcome of the total score average towards 50%. If A vs R+N gives you 60% winning chance,(so 10% excess), if it is the only bad trade that happens (because you set A slightly under 8), and happens in only 20% of the cases, the total effect you would see (and on which you would have to conclude the A value is suboptimal) would be 52%. But the 80% of games that did not contribute to learning anything about A value, because in the end A was traded for A, will contribute to the statistical noise! To recognize a 2% excess score in stead of a 10% excess score you need a 5 times lower statistical error. But statistical errors only decrease as the SQUARE ROOT of the number of games. So to get it down a factor 5, you need 25 times as many games. You could not conclude anything before you had 2500 games! Symmetrical playtesting MIGHT work if you first discard all the games that traded A for A (to eliminate the noise they produce, and they can't say anything about the correctness of the A value), and make sure you have about 100 games left. Otherwise, the result will be garbage.

[Subject Thread] [Add Response]

H. G. Muller wrote on Sat, May 3, 2008 07:14 AM UTC:

Well, this is exactly the kind of games I played. Plus that I do not play
from a single position, but shuffle the pieces in the backrank to have 432
different initial positions. This to minimize the risk that I am putting to
much emphasis on a position that inadvertantly contained hidden tactics,
biasing the score.

If there are such positions, sometimes one side should be favored,
sometimes the other, and the effect will average out. If the posession of
one piece as opposed to another (or a set of others) would systematically
lead to more tactics in favor of that piece even from an opeing position,
I think that is a valid contribution to the piece value of such a piece.

Of course I did all games on a 10x8 board, as I wanted to have piece
values for Capablanca Chess. If I were to do it on 8x8, I would use a
setup like yours, but with Q next to K for both sides, to make the piece
mix to which it is exposed even more natural. (Of course there always is a
problem introducing A and C in 8x8 Chess that they don't fit naturally on
the board, s you have to kick out some other pieces at their expense. But
you don't have to kick out the same pieces all the time. It is perfectly
valid to sometimes give both sides an A on d1/d8, some times a Q, some
times a C, or sometimes Q+C at the expense of a Bishop. The total mix of
pieces in the game should be N AVERAGE close to what it will be in real
games, or you cannot be sure that results are meaningful.

I never went more extreme than giving one side two A and the other two C
(or similarly AA vs QQ and CC vs QQ), by substituting A->C for one side of
the Capablanca array, and C-> for the other. For the total list of
combinations I tried, see:
http://z13.invisionfree.com/Gothic_Chess_Forum/index.php?showtopic=389&st=1
(For clarity: the pieces mentioned in that list where in general the
pieces I deleted from the opening array.)

H. G. Muller wrote on Sat, May 3, 2008 08:58 AM UTC:

For completeness, I listed the combinations that are relevant for
comparison of the Q, A and C value here:

Q-BNN    (172+ 186- 75=) 48.4%
Q-BBN    (143+ 235- 54=) 39.4%
C-BNN    (130+ 231- 71=) 38.3%
C-BBN    ( 39+  86- 11=) 32.7%
A-BNN    (124+ 241- 67=) 36.5%
RR-Q     (174+ 194- 64=) 47.7%
RR-CP    (131+ 227- 74=) 38.9%
RR-AP    (166+ 199- 67=) 46.2%
RR-C     (188+ 170- 74=) 52.1%
RR-A     (197+ 162- 73=) 54.1%
QQ-CC    (131+ 55-  30=) 67.6%
QQ-AA    (117+ 60-  39=) 63.2%
QQ-CCP   (112+ 72-  32=) 59.3%
QQ-AAP   (112+ 78-  26=) 57.9%
CC-AA    (102+ 89-  25=) 53.0%
Q-CP     (164+ 191- 77=) 46.9%
Q-AP     (191+ 186- 55=) 50.6%
Q-C      (215+ 161- 56=) 56.3%
Q-A      (219+ 138- 75=) 59.4%
C-A      (187+ 182- 63=) 50.6%
A-RN     (261+ 122- 49=) 66.1%
C-RN     (273+ 101- 58=) 69.9%
A-RNP    (247+ 121- 64=) 64.6%
C-RNP    (242+ 144- 46=) 61.3%

So it is not only that C and A has been tried against each other, alone or
in pairs. They have also been tested against Q (alonme or in pairs, with or
without pawn odds for the latter), BNN, RR and RN (with or without Pawn
odds). On the average, C does only slightly better than A, on the average
2-3%, where giving Pawn odds makes a difference of ~12%. 

The A-RNP result seems a statistical fluke, as it is almost the same as
A-RN, while the extra Pawn obviously should help, and the A even does
better there than C-RNP. Note the statistical error in 432 games is 2.2%,
so that 32% of the results (so eight) should be off by more than 2.2%, and
5% (1 or 2) should be off by more than 4.5%. And A-RNP is most likely to be
that latter one.

Aberg variation of Capablanca's Chess. Different setup and castling rules. (10x8, Cells: 80) [All Comments] [Add Comment or Rating]

H. G. Muller wrote on Sat, May 3, 2008 09:15 AM UTC:

Note that a Nash equilibrium in a symmetric zero-sum game must be the globally optimum strategy. If it weren't, the player scoring negative could unilaterally change its strategy to be the same as his opponent applies, and by symmetry then raise his score to 0, showing that the earlier situation could not heave been a Nash equilibrium.

Piece Values[Subject Thread] [Add Response]

H. G. Muller wrote on Sat, May 3, 2008 10:14 AM UTC:

Sorry my original long post got lost.

As this is not a position where you can expect piece values to work, and
my computers are actually engaged in useful work, why don't YOU set it
up?

Aberg variation of Capablanca's Chess. Different setup and castling rules. (10x8, Cells: 80) [All Comments] [Add Comment or Rating]

H. G. Muller wrote on Sat, May 3, 2008 10:34 AM UTC:

As piece values are only useful as strategic guidelines for quiet positions, they cannot be sensitive to who has the move. A position where it matters who has the move is by definition nont quiet, as one ply later that characteristic will have essentially changed. So at the level of piece-value strategies, Chess is a perfectly symmetric game.

Piece Values[Subject Thread] [Add Response]

H. G. Muller wrote on Sat, May 3, 2008 10:36 AM UTC:

It seems to me that that is bad strategy. If you fail you should keep
trying until you succeed. Only when you succeed you can stop trying...

Aberg variation of Capablanca's Chess. Different setup and castling rules. (10x8, Cells: 80) [All Comments] [Add Comment or Rating]

H. G. Muller wrote on Sat, May 3, 2008 03:22 PM UTC:

Sure, this is what people do and have done for ages. It is well known that the advantage of having the move is worth 1/6 of a Pawn, (corresponding in normal Chess to a white score of 53-54%) and that, by inference, wasting a full move is equivalent to 1/3 of a Pawn.

But the point is that this does not alter the piece values. It just adds to them, like every positional advantage adds to them. In my test the advantage of having the lead move is neutralized by playing every position both with white to move and black to move.

Piece Values[Subject Thread] [Add Response]

H. G. Muller wrote on Sat, May 3, 2008 04:15 PM UTC:

To summarize the state of affairs, we now seem to have sets of piece
values for Capablanca Chess by:

Hans Aberg (1)
Larry Kaufman (1)
Reinhard Scharnagl (2)
H.G. Muller (3)
Derek Nalls (4)

1) Educated guessing based on known 8x8 piece values and assumptions on
synergy values of compound pieces
2) Based on board-averaged piece mobilities
3) Obtained as best-fit of computer-computer games with material
imbalance
4) Based on mobilities and more complex arguments, fitted to experimental
results ('playtesting')

I think we can safely dismiss method (1) as unreliable, as the (clearly
stated) assumptions on which they are based were never tested in any way,
and appear to be invalid.
Method (3) and (4) now are basically in agreement. 
Method (2) produces substantially different results for the Archbishop.

One problem I see with method (2) is that plain averaging over the board
does not seem to be the relevant thing to do, and even inconsitent at
places: suppose we apply it to a piece that has no moves when standing in
a corner, the corner squares would suppress the mobility. If otoh, the
same piece would not be allowed to move into the corner at all, the
average would be taken over the part of the board that it could access
(like for the Bishop), and would be higher than for the piece that could
go there, but not leave it (if there weren't too many moves to step into
the corner). While the latter is clearly upward compatible, and thus must
be worth more.

The moral lesson is that a piece that has very low mobility on certain
squares, does not lose as much value because of that as the averaging
suggest, as in practice you will avoid putting the piece there. The SMIRF
theory doe not take that into account at all.

Focussing on mobility only also makes you overlook disastrous handicaps a
certain combination of moves can have. A piece that has two forward
diagonal moves and one forward orthogonal (fFfW in Betza notation) has
exactly the same mobility as that with forward diagonal and backward
orthogonal moves (fFbW). But the former is restricted to a small (and ever
smaller) part of the board, while the latter can reach every point from
every other point. My guess is that the latter piece would be worth much
more than the former, although in general forward moves are worth more
than backward moves. (So fWbF should be worth less than fFbW.) But I have
not tested any of this yet.

I am not sure how much of the agreement between (3) and (4) can be
ascribed to the playtesting, and how much to the theoretical arguments:
the playtesting methods and results are not extensively published and not
open to verification, and it is not clear how well the theoretical
arguments are able to PREdict piece values rather than POSTdict them. IMO
it is not possible to make an all encompasisng theory with just 4 or 6
empirical piece values as input, as any elaborate theory will have many
more than 6 adjustable parameters.

So I think it is crucial to get accurate piece values for more different
pieces. One keystone piece could be the Lion. This is can make all leaps
to targets in a 5x5 square centered on it (and is thus a compound of Ferz,
Wazir, Alfil, Dabbabah and Knight). This piece seems to be 1.25 Pawn
stronger than a Queen (1075 on my scale). This reveals a very interesting
approximate law for piece values of short-range leapers with N moves:

value = (30+5/8*N)*N

For N=8 this would produce 280, and indeed the pieces I tested fall in the
range 265 (Commoner) to 300 (Knight), with FA (Modern Elephant), WD (Modern
Dabbabah) and FD in between. For N=16 we get 640, and I found WDN
(Minister) = 625 and FAN (High Priestess) and FAWD (Sliding General) 650.
And for the Lion, with N=24, the formula predicts 1080.

My interpretation is that adding moves to a piece does not only add the
value of the move itself (as described by the second factor, N), but also
increases the value of all pre-existing moves, by allowing the piece to
better manouevre in place for aiming them at the enemy. I would therefore
expect it is mainly the captures that contribute to the second factor,
while the non-captures contribute to the first factor.

The first refinement I want to make is to disable all Lion moves one at a
time, as captures or as non-captures, to see how much that move
contributes to the total strength. The simple counting (as expressed by
the appearence of N in the formula) can then be replaced by a weighted
counting, the weights expressing the relative importance of the moves. (So
that forward captures might be given a much bigger weight than forward
non-captures, or backward captures along a similar jump.) This will
require a lot of high-precision testing, though.

H. G. Muller wrote on Sat, May 3, 2008 04:21 PM UTC:

Oh Yes, I forgot about:

[name removed] (5)

5) Based on safe checking

I am not sure that safe checking is of any relevance. Most games are not
won by checkmating the opponent King in an equal-material position, but
by
annihilating the opponent's forces. So mainly by threatening Pawns and
other Pieces, not Kings. A problem is that safe checking seems to predict
zero value for pieces like Ferz, Wazir and Commoner, while the latter is
not that much weaker than the Knight. (And, averaged over all game
stages,
might even be stronger than a Knight.) This directly seems to falsify the
method.

[The above has been edited to remove a name and/or site reference. It is
the policy of cv.org to avoid mention of that particular name and site to
remove any threat of lawsuits. Sorry to have to do that, but we must
protect ourselves. -D. Howe]

H. G. Muller wrote on Sat, May 3, 2008 04:46 PM UTC:

Reinhard, why do you attach such importance to the 4A-9N position. I think
that example is totally meaningless. If it would prove anything, it is
that you cannot get the value of 9 Knights by taking 9 times the Knight
value. It will prove _nothing_ about the Archbishop value. Chancellor and
Queen will encounter exactly the same problems facing an army of 9
Knights.

The problem is that there is a positional bonus for identical pieces
defending each other. This is well known (e.g. connected Rooks). Problem
is that such pair interactions grow as the square of the number of pieces,
and thus start to dominate the total evaluation if the number of identical
pieces gets extremely high (as it never will in real games).

Pieces like A, C and Q (or in particular the highest-valued pieces on the
board) will not get such bonuses, as the bonus is asociated with the
safety of mutually defending each other, and tactical security in case the
piece is traded, because the recapture then replaces it by an identical
one, preserving all defensive moves it had. In absence of equal or higher
pieces, defending pieces is a useless exercise, as recapture will not
offer compensation. If you are attacked, you will have to withdraw. So the
mutual-defence bonus is also dependent on the piece makeup of the opponent,
and is zero for Archbishops when the opponent only has Knights, and very
high for Knights when the opponent has only Archbishops.

If you want to playtest material imbalances, the positional value of the
position has to be as equal as possible. The 4A-9N position violates that
requirement to an extreme extent. It thus cannot tell us anything about
piece values. Just like deleting the white Queen and all 8 black Pawns
cannot tell us anything about the value of Q vs P.

H. G. Muller wrote on Sat, May 3, 2008 05:18 PM UTC:

Well, Reinhard, there could be many explanations for the 'surprising'
strength of an all-Knight army, and we could speculate forever on it. But
it would only mean anything if we could actually find ways to test it. I
think the mutual defence is a real effect, and I expect an army of all
different 8-target leapers to do significantly worse than an army of all
Knights, even though all 8-target leapers are almost equally strong. But
it would have to be tested.

Defending each other for Archbishops is useless (in the absence of opponet
Q, C or A), as defending Archbishop in the face of Knight attacks is of
zero use. So the factthey can do it is not worth anything.

Nevertheless, the Archbishops do not do so bad as you want to make us
believe, and I think they still would have a fighting chance against 9
Knights. So perhaps I will run this tests (on the Battle-of-the-Goths
port, so that everyone can watch) if I have nothing better to do. But
currently I have more important and urgent things to do on my Chess PC. I
have a great idea for a search enhancement in Joker, and would like to
implement and test it before ICT8.

The Pizza Kings. An experimental army for Chess with Different Armies, with lots of calories.[All Comments] [Add Comment or Rating]

H. G. Muller wrote on Sat, May 3, 2008 05:24 PM UTC:

I thought this piece (W+D+A+F+N) was called a Lion, but it seems I was misinformed. I playtested this piece in a Capablanca Chess environment, and it is not that excessively strong. It is about 1.25 pawn stronger than a Queen, 1075 on my scale (on 10x8 board).

Piece Values[Subject Thread] [Add Response]

H. G. Muller wrote on Sat, May 3, 2008 06:32 PM UTC:

Well, I got that from the beginning. But the problem is not that the A
cannot be defended. It is strong and mobile enough to care for itself. The
problem is that the Knights cannot be threatened (by A), because they all
defend each other, and can do so multiple times. So you can build a
cluster of Knights that is totally unassailable. That would be much more
difficult for a collection of all different pieces. This will be likely to
have always some weak spots, which the extremely agile Archbishops then
seek out and attack that point with deadly precision.

But I don't see this as a fundamental problem of pitting different armies
against each other. After an unequal trade, andy Chess game becomes a game
between different armies. But to define piece values that can be helpful
to win games, it is only important to test positions that could occur in
chames, or at least are not fundamentally different in character from what
you might encounter in games. and the 4A-9N position definitely does not
qualify as such.

I think this is valid critisism against what Derek has done (testing
super-pieces only against each other, without any lighter pieces being
present), but has no bearing on what I have done. I never went further
than playing each side with two copies of the same super-piece, by
replacing another super-piece (which was then absent in that army). This
is slightly unnatural, but I don't expect it to lead to qualitatively
different games, as the super-pieces are similar in value and mobility.
And unlike super-pieces share already some moves, so like and unlike
super-pieces can cooperate in very similar ways (e.g. forming batteries).
It did not essentially change the distribution of piece values, as all
lower pieces were present in normal copy numbers.

I understand that Derek likes to magnify the effect by playing several
copies of the piece under test, but perhaps using 8 or 9 is overdoing it.
To test a difference in piece value as large as 200cP, 3 copies should be
more than enough: This can still be done in a reasonably realistic mix of
pieces, e.g. replacing Q and C on one side by A, and on the other side by
Q and A by C, so that you play 3C vs 3A, and then give additional Knight
odds to the Chancellors. This would predict about +3 for the Chancellors
with the SMIRF piece values, and -2.25 according to my values. Both
imbalances are large enough to cause 80-90% win percentages, so that just
a few games should make it obvious which value is very wrong.

H. G. Muller wrote on Sat, May 3, 2008 06:42 PM UTC:

Derek Nalls:
| Given enough years (working with only one server), this quantity of 
| well-played games may eventually become adequate.

I never found any effect of the time control on the scores I measure for
some material imbalance. Within statistical error, the combinations I
tries produced the same score at 40/15', 40/20', 40/30', 40/40',
40/1', 40/2', 40/5'. Going to even longer TC is very expensive, and I
did not consider it worth doing just to prve that it was a waste of
time...

The way I see it, piece-values are a quantitative measure for the amount
of control that a piece contributes to steering the game tree in the
direction of the desired evaluation. He who has more control, can
systematically force the PV in the direction of better and better
evaluation (for him). This is a strictly local property of the tree. The
only advantage of deeper searches is that you average out this control
(which highly fluctuates on a ply-by play basis) over more ply. But in
playing the game, you average over all plies anyway.

H. G. Muller wrote on Sat, May 3, 2008 08:18 PM UTC:

| And by that this would create just the problem I have tried to 
| demonstrate. The three Chancellors could impossibly be covered, 
| thus disabling their potential to risk their own existence by 
| entering squares already influenced by the opponent's side.

You make it sound like it is a disadvantage to have a stronger piece,
because it cannot go on squares attacked by the weaker piece. To a certain
extent this is true, if the difference in capabilities is not very large.
Then you might be better off ignoring the difference in some cases, as
respecting the difference would actually deteriorate the value of the
stronger piece to the point where it was weaker than the weak piece. (For
this reason I set the B and N value in my 1980 Chess program Usurpator to
exactly the same value.) But if the difference between the pieces is
large, then the fact that the stronger one can be interdicted by the
weaker one is simply an integral part of its piece value.

And IMO this is not the reason the 4A-9N example is so biased. The problem
there is that the pieces of one side are all worth more than TWICE that of
the other. Rooks against Knights would not have the same problem, as they
could still engage in R vs 2N trades, capturing a singly defended Knight,
in a normal exchange on a single square. But 3 vs 1 trades are almost
impossible to enforce, and require very special tactics.

It is easy enough to verify by playtesting that playing CCC vs AAA (as
substitutes for the normal super-pieces) will simply produce 3 times the
score excess of playing a normal setup with on one side a C deleted, and
at the other an A. The A side will still have only a single A to harrass
every C. Most squares on enemy territory will be covered by R, B, N or P
anyway, in addition to A, so the C could not go there anyway. And it is
not true that anything defended by A would be immune to capture by C, as
A+anything > C (and even 2A+anything > 2C. So defending by A will not
exempt the opponent from defending as many times as there is attack, by
using A as defenders. And if there was one other piece amongst the
defenders, the C had no chance anyway. 

The effect you point out does not nearly occur as easily as you think.
And, as you can see, only 5 of my different armies did have duplicated
superpieces. All the other armies where just what you would get if you
traded the mentioned pieces, thus detecting if such a trade would enhance
or deteriorate your winning chances or not.

H. G. Muller wrote on Sat, May 3, 2008 09:31 PM UTC:

Reinhard, if I understand you correct, what you basically want to introduce
in the evaluation is terms of the type w_ij*N_i*N_j, where N_i is the
number of pieces of type i of one side, and N_j is the number of pieces of
type j of the opponent, and w_ij is an tunable weight.

So that, if type i = A and type j = N, a negative w_ij would describe a
reduction of the value of each Archbishop by the presence of the enemy
Knights, through the interdiction effect. Such a term would for instance
provide an incentive to trade A in a QA vs ABNN for the QA side, as his A
is suppressed in value by the presence of the enemy N (and B), while the
opponent's A would not be similarly suppressed by our Q. On the contrary,
our Q value would be suppressed by the the opponent's A as well, so
trading A also benefits him there.

I guess it should be easy enough to measure if terms of this form have
significant values, by playing Q-BNN imbalances in the presence of 0, 1
and 2 Archbishops, and deducing from the score whose Archbishops are worth
more (i.e. add more winning probability). And similarly for 0, 1, 2
Chancellors each, or extra Queens. And then the same thing with a Q-RR
imbalance, to measure the effect of Rooks on the value of A, C or Q.

In fact, every second-order term can be measured this way. Not only for
cross products between own and enemy pieces, but also cooperative effects
between own pieces of equal or different type. With 7 piece types for each
side (14 in total) there would be 14*13/2 = 91 terms of this type possible.

H. G. Muller wrote on Sun, May 4, 2008 08:57 AM UTC:

Derek Nalls: | The additional time I normally give to playtesting games to improve | the move quality is partially wasted because I can only control the | time per move instead of the number of plies completed using most | chess variant programs. Well, on Fairy-Max you won't have that problem, as it always finishes an iteration once it decides to start it. But although Fairy-Max might be stronger than most other variant-playing AIs you use, it is not stronger than SMIRF, so using it for 10x8 CVs would still be a waste of time. Joker80 tries to minimize the time wastage you point out by attempting only to start iterations when it has time to finish them. It cannot always accurately guess the required time, though, so unlike Fairy-Max it has built in some emergency breaks. If they are triggered, you would have an incomplete iteration. Basically, the mechanism works by stopping to search new moves in the root if there already is a move with a similar score as on the previous iteration, once it gets in 'overtime'. In practice, these unexpectedly long iterations mainly occur when the previously best move runs into trouble that so far was just beyond the horizon. As the tree for that move will then look completely different from before, it takes a long time to search (no useful information in the hash), and the score will have a huge drop. It then continues searching new moves even in overtime in a desparate attempt to find one that avoids the disaster. Usually this is time well spent: even if there is no guarantee it finds the best move of the new iteration, if it aborts it early, it at least has found a move that was significantly better than that found in the previous iteration. Of course both Joker80 and Fairy-Max support the WinBoard 'sd' command, allowing you to limit the depth to a certain number of plies, although I never use that. I don't like to fix the ply depth, as it makes the engine play like an idiot in the end-game. | Can you explain to me in a way I can understand how and why | you are able to successfully obtain valuable results using this | method? Well, to start with, Joker80 at 1 sec per move still reaches a depth of 8-9 ply in the middle-game, and would probably still beat most Humans at that level. My experience is that, if I immediately see an obvious error, it is usually because the engine makes a strategic mistake, not a tactical one. And such strategic mistakes are awefully persistent, as they are a result of faulty evaluation, not search. If it makes them at 8 ply, it is very likely to make that same error at 20 ply. As even 20 ply is usually not enough to get the resolution of the strategical feature within the horizon. That being said, I really think that an important reason I can afford fast games is a statistical one: by playing so many games I can be reasonably sure that I get a representative number of gross errors in my sample, and they more or less cancel each other out on the average. Suppose at a certain level of play 2% of the games contains a gross error that turns a totally won position into a loss. If I play 10 games, there is a 20% error that one game contains such an error (affecting my result by 10%), and only ~2% probability on two such errors (that then in half the cases would cancel, but in other cases would put the result off by 20%). If, OTOH, I would play 1000 faster games, with an increased 'blunder rate' of 5% because of the lower quality, I would expect 50 blunders. But the probability that they were all made by the same side would be negligible. In most cases the imbalace would be around sqrt(50) ~ 7. That would impact the 1000-game result by only 0.7%. So virtually all results would be off, but only by about 0.7%, so I don't care too much. Another way of visualizing this would be to imagine the game state-space as a2-dimensional plane, with two evaluation terms determining the x- and y-coordinate. Suppose these terms can both run from -5 to +5 (so the state space is a square), and the game is won if we end in the unit circle (x^2 + y^2 < 1), but that we don't know that. Now suppose we want to know how large the probability of winning is if we start within the square with corners (0,0) and (1,1) (say this is the possible range of the evaluation terms when we posses a certain combination of pieces). This should be the area of a quarter circle, PI/4, divided by the area of the square (1), so PI/4 = 79%. We try to determine this empirically by randomly picking points in the square (by setting up the piece combination in some shuffled configuration), and let the engines play the game. The engines know that getting closer or farther away of (0,0) is associated with changing the game result, and are programmed to maximize or minimize this distance to the origin. If they both play perfectly, they should by definition succeed in doing this. They don't care about the 'polar angle' of the game state, so the point representing the game state will make a random walk on a circle around the origin. When the game ends, it will still be in the same region (inside or outside the unit circle), and games starting in the won region will all be won. Now with imperfect play, the engines will not conserve the distance to the origing, but their tug of war will sometimes change it in favor of one or the other (i.e. towards the origin, or away from it). If the engines are still equally strong, by definition on the average this distance will not change. But its probability distribution will now spread out over a ring with finite width during the game. This might lead to won positions close to the boundary (the unit circle) now ending up outside it, in the lost region. But if the ring of final game states is narrow (width << 1), there will be a comparable number of initial game states that diffuse from within the unit circle to the outside, as in the other direction. In other words, the game score as a function of the initial evaluation terms is no longer an absolute all or nothing, but the circle is radially smeared out a little, making a smooth transition from 100% to 0% in a narrow band centered on the original circle. This will hardly affect the averaging, and in particular, making the ring wider by decreasing playing accuracy will initially hardly have any effect. Only when play gets so wildly inaccurate that the final positions (where win/loss is determined) diverge so far from the initial point that it could cross the entire circle, you will start to see effects on the score. In the extreme case wher the radial diffusion is so fast that you could end up anywhere in the 10x10 square when the game finishes, the result score will only be PI/100 = 3%. So it all depends on how much the imperfections in the play spread out the initial positions in the game-state space. If this is only small compared to the measures of the won and lost areas, the result will be almost independent of it.

Simplified Chess. Missing description (8x7, Cells: 56) [All Comments] [Add Comment or Rating]

H. G. Muller wrote on Thu, May 8, 2008 02:44 PM UTC:

I don't think that 'promoting to a captured piece only' is a simplification of the rules. 'Always promote to Queen' would be a simplification. This just adds a complex rule.

H. G. Muller wrote on Thu, May 8, 2008 04:15 PM UTC:

Well, I do not consider the stalemate rule essential to Chess, and there are many variants where stalemate = loss.

You won't get rid of many draws, though, when you abolish it. To get rid of draws entirely, you could add some kind of a tie-break to the game, like penalty shootouts in soccer: In a position where FIDE rules would declare draw (50-moves, 3-fold-rep, insuff. material) you could trigger this tie-break. From the moment on it is triggered, the opponent can do two moves in a row, then you can do three moves, then he 4, etc. This would even work for King vs King, as in the end there will be no place you can hide without his King being able to capture you.

H. G. Muller wrote on Fri, May 9, 2008 10:24 AM UTC:

Rich Hutnik:
| Do you want a flipped rook to become a 'Jester' piece that can 
| represent any other piece on the board?  Guess what the rook does 
| now.  It is that.  
This has never been a problem when I was playing OTB games. In most variants the choice of promotion piece is a rather academic one anyway, as in practice almost always the strongest piece is chosen. After the promotion, if the piece is then not captured, the game is over in 5 or 6 moves...

Even in Capablanca Chess, where there are 3 nearly equivalent pieces available (Q, C and A), it took me months before I discvered that 'underpromotion' to C or A was not properly implemented in my engine Joker80. Although it was considering other promotions than Q in its search, the MakeMove routine at game level always promoted to Q, overruling the choice. This never changed the game result, and I only discovered it when Joker80 announced mate-in-1 on a promotion move, and then the game continued a few more moves before it actually was checkmate.

People that want to play variants should have a wider choice of piece equipment anyway. An inverted Rook is a warning sign that whatever it is, it is not a Rook. But nothing is more annoying to a Chess player than having a Knight on the board that doesn't move like a Knight, but as a Camel or Zebra. The solution to that is easy enough:

http://home.hccnet.nl/h.g.muller/ultima.html

Piece Values[Subject Thread] [Add Response]

H. G. Muller wrote on Mon, May 12, 2008 05:57 AM UTC:

To Derek:

I am aware that the empirical Rook value I get is suspiciously low. OTOH,
it is an OPENING value, and Rooks get their value in the game only late.
Furthermore, this only is the BASE VALUE of the Rook; most pieces have a
value that depends on the position on the board where it actually is, or
where you can quickly get it (in an opening situation, where the opponent
is not yet able to interdict your moves, because his pieces are in
inactive places as well). But Rooks only increase their value on open
files, and initially no open files are to be seen. In a practical game, by
the time you get to trade a Rook for 2 Queens, there usually are open
files. So by that time, the value of the Q vs 2R trade will have gone up
by two times the open-file bonus. You hardly have the possibility of
trading it before there are open files. So it stands to reason that you
might as well use the higher value during the entire game.

In 8x8 Chess, the Larry Kaufman piece values include the rule that a Rook
should be devaluated by 1/8 Pawn for each Pawn on the board there is over
five. In the case of 8 Pawns that is a really large penalty of 37.5cP for
having no open files. If I add that to my opening value, the late
middle-game / end-game value of the Rook gets to 512, which sounds a lot
more reasonable.

There are two different issues here:
1) The winning chances of a Q vs 2R material imbalance game
2) How to interpret that result as a piece value

All I say above has no bearing on (1): if we both play a Q-2R match from
the opening, it is a serious problem if we don't get the same result. But
you have played only 2 games. Statistically, 2 games mean NOTHING. I don't
even look at results before I have at least 100 games, because before they
are about as likely to be the reverse from what they will eventually be,
as not. The standard deviation of the result of a single Gothic Chess game
is ~0.45 (it would be 0.5 point if there were no draws possible, and in
Gothic Chess the draw percentge is low). This error goes down as the
square root of the number of games. In the case of 2 games this is
45%/sqrt(2) = 32%. The Pawn-odds advantage is only 12%. So this standard
error corresponds to 2.66 Pawns. That is 1.33 Pawns per Rook. So with this
test you could not possibly see if my value is off by 25, 50 or 75. If you
find a discrepancy, it is enormously more likely that the result of your
2-game match is off from to true win probability.

Play 100 games, and the error in the observed score is reasonable certain
(68% of the cases) to be below 4.5% ~1/3 Pawn, so 16 cP per Rook. Only thn
you can see with reasonable confidence if your observations differ from
mine.

[Subject Thread] [Add Response]

H. G. Muller wrote on Mon, May 12, 2008 06:06 AM UTC:

Note that you can also use WinBoard as a FEN editor. There are commands
(with shortcut keys) to copy FENs from and to the clipboard. And there is
an edit-position mode that allows you to conveniently drag and drop pieces
over the board, and add new ones from a popup menu when right-clicking a
square.

http://home.hccnet.nl/h.g.muller/winboardF.html

Knightmate. Win by mating the knight. (8x8, Cells: 64) [All Comments] [Add Comment or Rating]

H. G. Muller wrote on Mon, May 12, 2008 09:31 PM UTC:

Note that there has just been released a WinBoard compatible version of the variant-capable engine Dabbaba of Jens Baek Nielsen. One of the games it knows is Knightmate. You can currently watch it play a Knightmate match live against my own engine Fairy-Max, on my Chess-Live! webserver

http://80.100.28.169/gothic/knightmate.html

for the next one or two days.

If anyone knows any other WinBoard engines that can play Knightmate, let me know; then I could hold a tournament.

Piece Values[Subject Thread] [Add Response]

H. G. Muller wrote on Mon, May 12, 2008 10:12 PM UTC:

Drek Nalls:
| They definitely mean something ... although exactly how much is not 
| easily known or quantified (measured) mathematically.
Of course that is easily quantified. The entire mathematical field of
statistics is designed to precisely quantify such things, through
confidence levels and uncertainty intervals. The only thing you proved
with reasonable confidence (say 95%) is that two Rooks are not 1.66 Pawn
weaker than a Queen. So if Q=950, then R > 392. Well, no one claimed
anything different. What we want to see is if Q-RR scores 50% (R=475) or
62% (R=525). That difference just can't be seen with two games. Play 100.
There is no shortcut. Even perfect play doesn't help. We do have perfect
play for all 6-men positions. Can you derive piece values from that, even
end-game piece values???

| Statistically, when dealing with speed chess games populated 
| exclusively with virtually random moves ... YES, I can understand and 
| agree with you requiring a minimum of 100 games.  However, what you 
| are doing is at the opposite extreme from what I am doing via my 
| playtesting method.
Where do you get this nonsense? This is approximately master-level play.
Fact is that results from playing opening-type positions (with 35 pieces
or more) are stochastic quantity at any level of play we are likely to see
the next few million years. And even if they weren't, so that you could
answer the question 'who wins' through a 35-men tablebase, you would
still have to make some average over all positions (weighted by relevance)
with a certain material composition to extract piece values. And if you
would do that by sampling, the resukt would again be a sochastic quantity.
And if you would do it by exhaustive enumeration, you would have no idea
which weights to use.
And if you are sampling a stochastic quantity, the error will be AT LEAST
as large as the statistical error. Errors from other sources could add to
that. But if you have two games, you will have at least 32% error in the
result percentage. Doesnt matter if you play at an hour per move, a week
per move, a year per move, 100 year per move. The error will remain >=
32%. So if you want to play 100 yesr per move, fine. But you will still
need 100 games.

| Nonetheless, games played at 100 minutes per move (for example) have 
| a much greater probability of correctly determining which player has 
| a definite, significant advantage than games played at 10 seconds per 
| move (for example).
Why do I get the suspicion that you are just making up this nonsense? Can
you show me even one example where you have shown that a certain material
advantage would be more than 3-sigma different for games at 100 min / move
than for games at 1 sec/move? Show us the games, then. Be aware that this
would require at least 100 games at aech time control. That seems to make
it a safe guess that you did not do that for 100 min/move.
 On the other hand, in stead of just making things up, I have actually
done such tests, not with 100 games per TC, but with 432, and for the
faster even with 1728 games per TC. And there was no difference beyond the
expected and unavoidable statistical fluctuations corresponding to those
numbers of games, between playing 15 sec or 5 minutes. 
The advantage that a player has in terms of winning probability is the
same at any TC I ever tried, and can thus equally reliably be determined
with games of any duration. (Provided ou have the same number of games).
If you think it would be different for extremely long TC, show us
statistically sound proof.

I might comment on the rest of your long posting later, but have to go
now...

25 comments displayed

~~Earlier~~ ⇧Reverse Order⇩ Later⇧ Latest⇩

Permalink to the exact comments currently displayed.