Ratings & Comments

Yes, I saw the Copilot symbol from Edge in your image and realized you were using Edge. So, I tried some other things. First, I changed my ad blocker in Chrome from AdBlock Plus to uBlock Origin, and it made no difference. Note that I had turned each one off to see the ads, but I wanted to see if either was blocking some ads regardless. Then I installed NoScript, and that did make a difference. Even with everything trusted, NoScript is still stopping the bottom screen ad from showing up, but it is not stopping a Google ad from showing up in the sidebar. So, it's probably that my other browsers, which I use more often, already had NoScript installed, and Chrome didn't.

I logged into Adsense and found out that Google was conducting an experiment in Anchor ads, and I stopped this experiment. I then removed NoScript from Chrome and loaded the logs page, and I saw a Google sidebar ad but didn't see any bottom screen ad. So, I think I've gotten them turned off.

The main problem here is that I don't think anyone among the editorship understands Russian (I can speak German and Spanish, I presume H.G. speaks Dutch, and Jean‐Louis, while not presently an editor, would be able to help with French, but as far as I'm aware that's about it), so it would be difficult to be confident in the quality of such a translation (though I suppose there are other Russian speakers on this forum, who might be better placed to help in this regard?).

That said, in principle I'm all for having more Russian‐language (or any other non‐English) pages, so if we can find a way to make this work by all means :‌)

We can rely on browser translations to make sure the translation is on target. To do this, we should add a LANG attribute to some major element, such as HTML or BODY. As a test, I added LANG="nl" to some Dutch pages and was able to get an English translation. While the translations we get may be imperfect, we can at least use them to make sure that the page is explaining the rules of the game. Actually, as a further test, I tried to get a translation of a Dutch page without this, and it still worked. As one more test, I was able to get an English translation of our Russian Chess Rules page. So, even if none of us speak Russian, we may be able to check the translations.

@BnEm yes I could help with French, also with Spanish and Italian or Esperanto, but I have realized that it is not easy to do such a thing with chess variants. The problem of names of the piece, which is already an issue in English, may become a problem with other languages.

Just to illustrate an Alfil is a Bishop in Spanish, a Cavalier a Knight in French. The Queen in Russian is a Boat and the Bishop is an Elephant. The Bishop is a Fool in French. Of course, things are getting even weirder with other names of characters and animals. If I had to translate Bigorra with its more than 30 different pieces, I may come to some difficulties.

Just to illustrate an Alfil is a Bishop in Spanish, a Cavalier a Knight in French. The Queen in Russian is a Boat and the Bishop is an Elephant. The Bishop is a Fool in French.

Incorrect, Queen is Ferz while Rook is a Boat. Хрен редьки не слаще, to be said)

The problem of names of the piece, which is already an issue in English, may become a problem with other languages. <…> Of course, things are getting even weirder with other names of characters and animals. If I had to translate Bigorra with its more than 30 different pieces, I may come to some difficulties.

I’m creative, so I’ll find the right way)

Well, being creative is not the main requirement, and might actually backfire. There surely must already exist publications in these other languages about chess variants, which name many of the elementary pieces. It would be important to conform to these existing naming schemes, rather than create entirely new ones.

@Lev: yes, sorry I wrote too fast. Yes Queen is Ferz and Rook is Boat (ladya). I knew that of course as you imagine. It changes nothing on what I was trying to say. On the contrary.

I don't get the point with "Хрен редьки не слаще" even with a translator. I hope this is not disrespectful on me.

I don't get the point with "Хрен редьки не слаще" even with a translator.

Russian proverb: one trouble is not better than another.

Russian proverb: one trouble is not better than another.

Reflected in a contemporary American idiom: It's always something. :)

The problem of names of the piece, which is already an issue in English, may become a problem with other languages

Yeah piece nomenclature would definitely be the hardest part of this; as H.G. notes, there'll be some precedent in whatever exesting literature on CVs there is in a given target language, but that will almost certainly be limited in scope for most languages compared to what we have here (even, say, Die Schwalbe's relatively extensive glossary has some, from a variantist point of view, arguably major omissions). And as you say there's a certain amount of conflicting usage between languages that makes things less than straightforward.

Of course, that cuts both ways; would‐be translators have an opportunity (if they do their research appropriately) to avoid making quite as much of a mess of naming as we have in English :‌) Even if we don't go as far as attempting the likely‐futile task of trying to replicate the likes of Man and Beast in, say, French.

And depending on the pages Lev is interested in translating it may not be much of an issue at all; plenty of games on these pages use only the Orthodox sextet

If I had to translate Bigorra with its more than 30 different pieces, I may come to some difficulties

Might be an interesting exercise in itself, to see how feasible such a task would be. And whilst i don't know the established French names (assuming there are any) for Cardinal/Marshall/Amazon, most of the remaining pieces (with the exception of the Direwolf and maybe the Soldier) ought to be easily translated word‐for‐word. For Russian we might have to pay more attention to the Elephant and Ship (we could always take precedent from English and go with ‘Филь’ for the former at least), and Italian/Spanish/German might want something more distinct from ‘dame’ than ‘duchess’, but these are exceptions really.

And whilst i don't know the established French names (assuming there are any) for Cardinal/Marshall/Amazon

Some English speakers would call the first two Archbishop and Chancellor or Princess and Empress. If he's translating the rules of a game, it would be most appropriate to use the translation of whatever the piece is called in that game, as opposed to how it is more widely known. It would also be appropriate for him to put the original English name in parentheses. And if he knows a common Russian name for a piece, he might add a note about what it is commonly known in Russian as.

I would be interested to know if there is any substantial Russian literature on chess variants or fairy chess. It's possible that this kind of individuality was frowned upon in the Soviet Union and didn't get the same attention it got in the west.

Fr: Cardinal/ Maréchal/Amazone; Loup Garrou/Soldat, yes those are easy. The difficulties in Fr would be that "knight" is translated by "chevalier" in a general context and by "cavalier" (meaning a horse rider) in the context of chess. So a CV having Knight, Cavalier and Chevalier, will be difficult to translate in French. Or a CV having a Tower and a Rook, both being "Tour". A CV having a Lady, would be translated as "Demoiselle", as "Dame" is already the word for Queen. Etc.

Litterature for CV exists in Fr with the pioneer works of Boyer in the 50s (which inspired Parton), then the rich book of Giffard and Bienabé (Guide des échecs), that I recommend even if you are not at ease with Fr. Bienabé wrote a very large section about fairy chess with plenty of information on fairy pieces, from the perspective of the 20th problemists (many were French). And, of course, my own French books on the subject, I should mention.

In Spanish, Dutch, German, Italian I have some literature too were I could find the adopted names for the most used ones.

The difficulties with translating to French are mainly due to French being one of the main sources for Modern English. Because of this, French and English share many words even if we pronounce them differently or give them slightly different meanings. Russian, though, has never been as much of an influence on English, and it uses a different alphabet. So, it's probably not going to have as many clashes over piece names as French has.

The difficulties in Fr would be that "knight" is translated by "chevalier" in a general context and by "cavalier" (meaning a horse rider) in the context of chess. So a CV having Knight, Cavalier and Chevalier, will be difficult to translate in French. Or a CV having a Tower and a Rook, both being "Tour". A CV having a Lady, would be translated as "Demoiselle", as "Dame" is already the word for Queen. Etc.

In general, this is true, yes; my previous comment was referring specifically to Bigorra (and by extension the rest of the games in its family) which doesn't have such conflicts.

The difficulties with translating to French are mainly due to French being one of the main sources for Modern English

I'd say the difficulty is a little subtler: English, due to both its Lingua Franca status and its extensive acquisition of loanwords, simply has a lot of words in certain semantic domains that mean either the same or very similar things. Which is obvious when, as with French, there are actual clashes, but even in Russian I'd be a little surprised (perhaps @Lev can enlighten us?) if it had three different words for Knight/Cavalier/Chevalier.

For comparison, German might get away with that triplet using both ‘Knecht’ and ‘Ritter’, (cognate to ‘Knight’ and ‘Rider’ respectively, and with (I think) slightly different connotations), but even then only because the Chess Knight is unrelatedly named ‘Springer’ — it can thus even spare a word for ‘Horse’ (‘Pferd’ — or even ‘Ross’ if necessary, though that'd be a bit like naming two pieces ‘Horse’ and ‘Steed’ in English). It would have just as much trouble as French with ‘Rook’/‘Tower’ (both ‘Turm’), though.

At some point, creative license would no doubt become necessary.

Some English speakers would call [Cardinal and Marshall] Archbishop and Chancellor or Princess and Empress

And some would call the Amazon Ace or Terror. Yes, English CV nomenclature is a mess.

One might argue that's an accident of history: several people independently reinventing the same pieces under different names before any one convention got established. There's no reason a priori to replicate that in translation (this being the ‘opportunity’ I referred to).

Here is a free online open source fairy chess server designed to play several CVs:

https://www.pychess.org/about

I have been running the open-source FICS code on my server for a while, mainly to organize on-line tournaments for computer programs. I did add a lot of variants on it too, such as Capablanca 10x8 variants, Spartan Chess. No one would ever play those, of course, but there were hardly any human visitors of that server anyway, and I never organized computer tournaments for those on that server.

I don't know if that old source still compiles on CentOS; otherwise it would be possible to run it on the CVP server as well. But it would have to be changed a lot for allowing a wider variety of variants, and in particular easier implementation of new ones. Furthermore, the only client capable to run the variants is WinBoard/XBoard, while nowadays people lose interest for anything you cannot play directly in your browser.

Very few (or zero) server visitors being interested in Spartan Chess I think I can understand, but I am a bit surprised that the final form of 10x8 Capablanca Chess (the variant made by the famous Cuban World Chess Champion, i.e. not all the other similar 10x8 CVs) would attract very little interest too. Of course, if few people visited that server anyway...

We have had a Chess variants server since 2001. It's called Game Courier, and it supports more Chess variants than all the others combined.

I don't know if Game Courier can ever handle very fast-moving games played of CVs

If you stay on the same game instead of checking your logs, Game Courier will check whether your opponent has moved every 7.5 seconds and update the page with an audible beep when he has.

We have had a Chess variants server since 2001. It's called Game Courier, and it supports more Chess variants than all the others combined.

This is what people call a 'turn-based server'. Which is fine for correspondence chess, but incomparable to a real-time server like FICS or LiChess. People play bullet games there, meaning 2 min per game per person, and sub-second recording of thinking time. To play such a game you need to have deposited your move about 1 sec after your opponent did. (And that should then be mostly their own thinking, not the roundtrip delay of the network...)

I have made Game Courier's delay in checking for your opponent's move more variable, depending upon the time controls. I have replaced 7500 with a variable called delay, and I have set it with this PHP code. Note that the time controls are in seconds, and the delay is in microseconds.

$totaltime = $gracetime + $opptime;
if ($totaltime > 24*3600)
    echo "var delay = 7500;\n";
elseif ($totaltime > 3600)
    echo "var delay = 4000;\n";
elseif ($totaltime > 1800)
    echo "var delay = 2000;\n";
elseif ($totaltime > 900)
    echo "var delay = 1000;\n";
elseif ($totaltime > 0)
    echo "var delay = 500;\n";
else
    echo "var delay = 7500;\n";

https://playstrategy.org/variant/noCastling

PlayStrategy supports it as a separate variant.

Thanks Lev. Here is what I just posted in regard to my governance friend's second question:

"Regarding your second question, even if a player is on the edge of being mated in some random position where he's down to almost nothing but an unmoved king and rook, if castling were legal then so would be moving the king sideways one step, I'd think."

Well, providing one understands what is written there. It is so complex, I read this part tens of times and I still don't catch nothing at all. Not sure it will be good for advertising...

Maybe a [new??] suggested GC blitz time control of 5 minutes per player might be offered, if possible? Game Courier already allows for something like Increments, so Fischer or Bronstein Delay blitz time controls might be options, too (I don't know if chess servers offer such these days - I only ever used a chess server once, briefly, decades ago, after a friend gifted me a chess server's gift card he won as a [door?] prize at a chess event, that he didn't really want, and my internet was dialup then, and I constantly lost my connection...).

It means that it normally checks whether the opponent has moved every 7.5 seconds, but when tighter time controls are set, it will check at smaller intervals, ranging from every 4 seconds to every half second, depending upon how quickly the game is supposed to go.

Yes, I meant for the whole game (standard meaning of blitz time control in chess). For Fischer increments a 2 second bonus can be added after a player makes a move in a blitz game, for example. Bronstein Delay I'm less familiar with, but it can be Googled I guess.

Yes, having blitz games (the usual choice of chess server players, though they can use longer time controls on at least some servers, as an option, say for a tournament) might be a serious problem if there is no way to ensure both players are online and on Game Courier when their game begins.

edit: A Google blurb:

Bronstein Timing is similar to Fischer Timing. Players have some time at the beginning of game and then after each move they receive a bonus of a few seconds (let's assume 3 seconds for now). The difference is that if you use less then 3 seconds (the bonus), you receive back only the time you used for your move.

edit2: I think 'receive back etc.' is a poor phrase for it. Instead it should just say you receive no bonus on that turn whatsoever, if you use less than 3 seconds before making your move.

I've edited my post I'm replying to, in case anyone missed it.

It is not a variant. It is just orthodox chess played from a different starting position, a position that already is in the game tree of orthodox chess. Tournaments of such games are known as thematic tournaments, where you are obliged to play a certain opening line.

We are also not going to make pages for a 'variant' that only differs from FIDE by the Pawns in the King  file starting on e4 and e5.

It also makes a lot of difference when you have to continue a game after 1. e4 c5 or after 1. d4 d5. That doesn't make the Queen's Gambit and the Sicilian different chess variants. It also makes a lot of difference whether you get 90 minutes or 2 minutes or 30 days on your clock. Not every difference, no matter how large, creates a new chess variant.

Throne Chess has different rules from orthodox Chess. In fact so different that at no time you can enter the game tree of one from a position in that of the other. The more the game progresses, the more this difference will be felt. (Like in the more popular King of the Hill, based on a similar idea.) This in contrast to, for instance, Seirawan Chess. Where after trading Hawk and Elephant you just end up in an  orthodox game. Even from Chess with Different Armies you can eventually convert to an orthodox Pawn ending.

So you should not be fooled by the term 'modest'. That term means something very specific here, and in any case not that the change in rules cannot have huge impact. If I am not mistaking, even Ultima and Arimaa count as a modest variants, because they can be played with an orthodox Chess set on an 8x8 board. (Of course I take the position here that they are not even Chess variants...)

The setup board for Impossibly Complicated Chess. ;)

OK, you are right about the promotion, although the effect of that would (especially in Seirawan Chess) be very minimal.

You cannot get to a position with castling rights in No-Castling Chess, but you can get to a position as would be reached after castling. In orthodox Chess castling rights are not preserved forever, and usually expire when most material is still on the board. All positions that follow would also be reachable from the No-Castling initial position. But the point was really the other way around: that the No-Castling initial position could be reached from the FIDE setup. Not in the way you mention, because you could not get the Pawns back on 2nd rank, but by Nf3-Rg1-Rh1-Ng1 (repeated for the other three Rooks).

That some moves in this opening line are sub-optimal is not really relevant. In fact all opening lines but one must have sub-optimal moves in them, in Chess. The whole idea of thematic tournaments is that you want people to play from positions they would not reach voluntarily. In computer matches one often intentionally uses opening lines that are very poor for one of the players, bringing them on the edge of losing, before the engines are allowed to play the moves of their choice. And then play a pair of games with reversed colors to make it fair. This to eliminate the draw margin, which by the standards of modern engine play is so wide that virtually all games starting from an equal position end as draws. By starting on the edge the outcome can go either way (draw or loss), with about 50% chance, and a reversed pair of games has only 50% probability of ending in a draw, between equal players.

So no, poor quality of an opening line does not make a game where that opening line was played a chess variant.

Well, what do orthodox Chess players know about variants?

Some practical considerations are that an explanation of the rules is redundant; any Chess player already knows you can lose castling rights, and very likely has experienced this himself more than once in a game. Special computer programs for it do not have to be created, because all existing engines for orthodox Chess that are in common use already can play it. Special servers for it are not needed, because it can already be played on all servers for orthodox Chess. (The server would not be fully rule-checking in that case, but who cares? We have presets here that don't check any rules at all, and it never discouraged people from playing these variants. Many servers do allow you to play from setup positions anyway, and there you could even set up a position without castling rights.)

The only thing that is new is that you cannot find books with opening theory on it. But the fact that a player has not memorized an opening line does not make something a variant.

We have presets here that don't check any rules at all, and it never discouraged people from playing these variants.

That depends upon how you quantify people. It never discouraged everyone from playing these variants, but it has discouraged some people on an individual basis.

I wonder if Kramnik has realized his 'mistake' by now, or even cares? In any case if someone here wants to make a Rules Page for No Castling Chess, they can add mention of your 'clear refutation' of it being a true variant H.G. (i.e. that it's a poor opening variation that can arise from FIDE Chess) - that would be one more anecdote for an Introduction or Notes section on such a Rules Page. :) edit2: I'm not sure that we should reject this 'variant' - would we be seen as snobs?

edit: one note about your refutation H.G. - those rook and knight moves by both sides from the FIDE start position would eat into the 50-move rule count, but that's almost guaranteed to be restarted after someone should decide to move a pawn, if it were still FIDE Chess being played. :) Same goes for avoiding/allowing 3-fold repetition early on. :) However, note that it would no longer be possible to make an early agreed draw in the FIDE version, say at move one. :)

I've edited my post I'm replying to, in case anyone missed it.

Bruh can you just add No-Castling as a version of Chess on its existing page?

I'm not sure that we should reject this 'variant' - would we be seen as snobs?

Yeah, let us behave like ignorants, otherwise the ignorant masses notice we stick out...

Well, that is not my philosophy of life.

Bruh can you just add No-Castling as a version of Chess on its existing page?

That is what we usually do for small rule changes that do not need any additional explanation.

Hmmm.

Speaking of snobbery, reminds me of the era when the US extended a 'No Fly' zone over Iraq. The NY Times thought that was incorrect (Pidgin English) and used the term 'No Flight' zone in their newspaper. No one else followed their example.

Hi Lev

There are two existing Rules Pages for Chess that I am aware of. One is older, the one by Hans Bodlaender, while Fergus has made a newer Rules Page. So, not being the author of either, I cannot follow your suggestion regarding including a summary of the rules for No Castling Chess on a Rules Page for Chess myself. Hans no longer visits the site as far as I know, but maybe Fergus might think about your idea...

if this works maybe Fergus or another editor can tell me if It's allowed in a 'Pieces' section of a Rules Page

Why wouldn't it be allowed?

if they can read my comment here from the original state I had it in before submitting it

Earlier versions of comments are not saved as revisions, but we can look at the source code of your comment if that is what you mean. The main change I would recommend is to use relative URLs.

I copied and pasted from a settings file - would save time doing it that way again.

Note that the PTA also has a button for creating a HTML verbal piece description that you only have to copy-paste into the Pieces section, in order to save time when creating entirely new variants. For regular pieces this should work quite well.

Happy Leap Day!

Should be special for chessvariants dot com. :)

We can all jump with joy. ;)

Perhaps I need to clear cache?

Yes.

@Fergus you fixed it. Thank you.

I am in general against spreading misinformation; there already is so much of that on the internet, no one is longing for us to add more to it. The problem with posted piece values is that they are very often not rooted in reality, but spring purely from the imagination of the author. I don't see what value that would add to an article. Every reader should be able to make unfounded guesses without any help. And it is especially bad if people post values that contradict a large body of evidence.

If I were to edit the article on orthodox Chess, claiming that the piece values are P=1, minor=2, R=4 and Q=8, based on the theory that piece values are inversely proportional to the number of those pieces you have in a game... Should it be allowed to stand? Should it go accompanied by an explanation of how these were calculated? Or by a disclaimer like "virtually everyone in the world agrees that these values are very much off, but I present these here anyway for no other reason that I could calculate them through a method that did not require any experimental evidence"?

My brother always says:  "if you don't have anything to say, then don't do that here!". Information should flow from where there is knowledge to where there is none. It is only useful to publish information that is better / more reliable than what the reader already has. Sadly, for piece values that will usually not be the case.

The main thrust of that gives no value to human intuition when there is not yet consensus or quite conclusive enough evidence (in the eyes of the beholder). Indeed, I personally think you sometimes rely on intuition when it comes to your methods for establishing or estimating piece values - that could arguably lead to misinformation unknowingly, despite your best intentions to be thorough.

I think it is easier to determine relative values than to determine precise, absolute values. Piece values are only a guide for evaluating positions, and they do not themselves determine who wins or loses. So, if you stick to relative values, you should be good, but when you attempt to give precise, numeric values to pieces, that will be far more speculative and prone to error.

It's easier, yes. A problem can surface if ever in a game you have the choice of making 2 for 1 or 3 for 1... trades. Then, for example, in chess it would not help you that you knew that P<N<=B<R<Q, if you want to know with some degree of confidence (or at least an intuitive feeling) that N+P is normally close to worth a R, or whether it's N+2P that is is normally much closer to worth a R - that is with all other affected features of the position at hand being in some kind of balance after making such a trade.

In fact it's usually N+2P, maybe during any phase of a game (if that is also to be taken into account). A (more advanced/different?) tip I've read is that (if I recall right) B+2P are usually worth R, and N+2P are a shade less than a R. Already H.G. might argue that computer studies (not just his) put single N = single B in (8x8) chess - however he might say that things are different for such (3 for 1) trades, because more units are involved, so no loss of face for anyone necessarily in such a case, for those who intrepidly try to assign (or offer to fine-tune) fairly precise piece values. In my case, for chess variants rules pages I've made, I add the caveat that my suggested values are tentative, hopefully wakening any adult who still has a child-like faith in the written word (I'd personally make an exception for a given version of bible, but perhaps even then mistranslations might have happened in some cases).

I add the caveat that my suggested values are tentative

That should be fine. I usually ignore estimates of piece values anyway, because I don't expect them to be gospel truth, and when playing a game, I rely more on my own ability to understand and compare different pieces. But piece values could be of more interest to someone who is trying to get a program to play a game better. As someone who is programming engines to play Chess variants well, it makes sense that HG would have a keen interest in this. I recall when Steve Evans and I were working on a better Shogi ZRF together, and one thing we did, which I think was more his idea than mine, was to add code to adjust the values Zillions-of-Games assigned to different pieces.

Agreed! But I am all for publishing piece values obtained through applying your experimental method. That and other tactical or strategic tips the author has found. For example I have observed that it is wrong to move a joker, in all my apothecary games, if there are pawns still ahead because they can move forward attacking the joker while the poor sucker cannot run, as it imitates a pawn. Conversely if there are no pawns ahead moving the joker to the center can be very fruitful as it can imitate anything making it temporarily the most powerful piece on the board.

The problem with intuition is that it is notoriously unreliable. Humans suffer from an effect called 'observational bias', because one tends to remember the exceptional better than the common. This is probably the reason that GMs/world champions have grossly overestimated the tactical value of a King (as ~4 Pawns): in games where the King plays an important role it can indeed be very strong, but there are plenty of cases where a King is of no use at all (because it cannot catch up with a passed Pawn). These tend to be dismissed, as "the King played no role here, so we could not see how stong it really is".  While in fact you could see how weak it was by its lack of ability to play a role. In practice two non-royal Kings are conclusively defeated by the Bishop pair, (in combination with balanced other material, and in particular sufficiently many Pawns). In games between computer programs that most humans could not beat at all. Of course none of these GMs ever played such a game even once.

Other forms of intuition often result from application of simplistic logic, rather than observation. It is 'intuitively obvious' that a BN is worth several Pawns less than RN, as B is worth several Pawns less than R, and it is their only difference. Alas, it is not true. They are almost equivalent. It ignores the effect that some moves can cooperate better than others, and in games BN + Pawn would score convincingly better than RN (and on average even beat Q).

We should also keep in mind that piece values are just an approximation. It is not a law of nature that the strength of an army can be obtained by adding a value of individual pieces, and that the win probability can be calculated from the difference between the thus obtained army strength. And indeed, closer study shows that it is not true at all. The win probability depends on how well pieces in the army cooperate, and complement each other, and how effective they are against what the opponent has.

For example, A=BN and C=RN are more effective against a Queen than against a combination of lighter material (say R+N+2P) that in itself would perfectly balance a Queen. Because all squares attacked by the latter, even though very similar to the number of squares attacked by a single Q, are no-go areas for a C or A, even when they are protected, while they would not have to shy away from a Q attack in similar situations. This causes Q+C+A < R+B+C+A, in Capablanca Chess, even though Q > R+B as usual. The extra C and A on the Q side are effectively weaker pieces than their counterparts on the R+B side, so much that it reverses the advantage. An extreme manifestation of this effect is that 7 Knights easily beat 3 Queens on an 8x8 board. Something that cannot be explained by any value for N/Q that would make sense in a context with more mixed FIDE material.

Your claim that B+2P ~ R and N+2P < R, which I don't doubt, cannot be used to conclude that B > N because of these subtleties. Piece values are not defined as how well the pieces do against a Rook, but by how well they do against a mix of opponent pieces such as these typically occur in end-games. And I have no doubt that the average performance of the Bishop suffers from the fact that there are many cases where B+2P ~ B+P, while N+2P would have done much better (namely when the Bishops are on unlike shades).

Note that the claim lone B ~ N was not based on what I would call a 'computer study'. I have no doubt a computer was used in the process, but just as an aid for quickly searching a huge database of human GM games. Not by playing computers against each other. The fact that a computer was used thus in no way had any effect on the conclusion. In the Kaufman study the claim was detailed further by stating that the B-N difference correlated with the number of Pawns, and exact equality only occurred when each player had about 5 Pawns; for fewer Pawns the Bishop performed better, for more Pawns the Knight. It is also common knowledge that Knights typically perform poorer in end-games where there are Pawns on different wings than when all Pawns are close together. This is of course also something that transcends piece values, which are defined as the best estimate for the chances without knowing the location of the pieces. Piece values are not the only terms that contribute to the heuristic evaluation of individual positions.

But to come back to the main topic: I don't think it would be a good idea to dismis any form of a quality standard on published piece values because "people should know that they should not believe what they read". That is an argument that could be used for publishing any form of fake news. It is already bad enough that this is the case, and we should not make it even more true by adding to the nonsense. There can also be piece values that have a more solid basis, and I think readers should have the right to distinguish the one from the other. So as far as I am concerned people can publish anything, as long as they clearly state how they arrived at those values. Like "personal experience based on N games I played with these pieces" or "based on counting their average number of moves on an NxN board" or whatever. If there is a non-trivial calculation scheme involved, it is fine to publish that as a separate article, and then refer to that.

One problem with computer studies of chess [variants] is that there has been no peer review by many mathematicians, and grandmasters of chess might be thrown in. For the scientific method to work in a trustworthy way, at least according to the high priests of science etc., you need that.

There are things already about computer studies that give me red flags personally (although I am no scientist/math wizard). The claimed margin of error could be wrong, for one thing. The armies or initial position chosen for each side of a given study could make a hugely underestimated difference. The (2300 FIDE at best!?) engine(s) used have been relatively weak so far, as far as I know - chess endgames take 2700+ human opponent players to play optimally sometimes.

I'm not sure why I should not believe such computer studies in general should be just dismissed as a pile of rubbish, if people more knowledgeable were to insist on rigourous proof for studies being correct at this point in time, if you want to play hardball about publishing standards. Such standards are reserved for scientific journals in the real world anyway, not for hobbyists who do not have (much, if any) money or life and death issues at stake.

More specifically for myself, I already balk at the idea Amazon only =Q+N in value, even on 8x8. As a chess master with the memory of a number of chess world champions' and grandmasters' views, I do not trust that single B merely = N exactly on 8x8 on average. As for Archbishop almost = Chancellor, a bit hard to trust, but that is more alien to my intuition. They both cover 16 nearby cells in a radius of 2 cells, I give you that.

[edit: if you really want, to please/amuse you and others I could always [not just sometimes] put calculations I use for my tentative estimated piece values on Rules Pages Notes - I look at the answers I get and see if my intuition agrees (so far it has, pretty much). I have yet to do such calculations for my most recent large batch of Rules Pages. For what it's worth, sometimes I also borrow some of your rules of thumb, where I lack my own formulae.]

I've edited my previous post, for any who missed that.

Well, from what you say it appears that with 'computer study' you mean statistical data from games that computers played against each other (or themselves). As I would. But as I said, the B = N observation came from the Kaufman study, which was nothing of the sort. He just filtered positions with a B-vs-N imbalance from a huge database of human GM games, selecting those that were the imbalance was stable for some number of moves (to weed out tactics in progress), and counted the number of wins, draws and losses in which these games ended. Which apparently was a 50% score.

It doesn't sound like rocket science to me, but I suppose a complete idiot could bungle even the most simple tasks. And I have met other chess-engine programmers that have done similar things for themselves. (The Kaufman study did not publish more specific things, like how the N or B would do against Rooks, or whether the difference also correlates with the presence of other pieces than Pawns, and some programmers want to make their engines aware of that too, and put a complete table of every conceivable material compustion in their engine.) And they never told me they had proven Kaufman wrong.

The problem is that implying someone is a bungling idiot that even cannot do the simplest thing right, or a fraud who intentionally publishes falsehoods, is a pretty heavy accusation. Most people would hesitate to make such an accusation without having very solid evidence that the published results were indeed wrong. "It was not checked by anyone, so it must be wrong" is not really a valid line of reasoning.

You seem to have a wrong impression of the peer-review system. The 'peers' that are asked to referee a scientific publication will NOT redo the reported work. They only judge whether the described method according to which the results were obtained is a proper procedure. If the claims are in contradiction with earlier results the referees have a hard time. They would at the very least insist that the authors of the new manuscript give an explanation for why their method would be more reliable than what people previously did, and even then they stand a large probability of rejection if that doesn't convince the referees. In a sense everyone is a peer on the internet, and could have contested what others publish there, in particular the Kaufman results. But it didn't happen, and that means much more than when he would just had to fool one or two referees. And there isn't really any need for mathematicians, people that know how to count seem sufficient. You are aware that Larry Kaufman is a GM himself?

I don't really understand your third paragraph, but I am intrigued by the term "more knowledgeable". True knowledge should of course never be dismissed. But what knowledge are you talking about, here?

I agree the Amazon result is suspect; it was only based on a couple of hundred games where the Queen and a Knight where replaced by an Amazon, and the baseline pieces were shuffled to provide more game diversity. That is a very different story than GMs not being able to convert a B-N 'advantage' into a better result in a few thousand games. The remarkable thing about computer games is that it doesn't seem to matter much what the level of play is. Errors tend to cancel out, when both players make them. Even random movers systematically win more games when you give them stronger material. (Although quantitatively they don't make the most of that, as they too easily give the strong material away.)

Rather than describing the calculations in a large number of articles, which is likely to lead to a lot of duplication, you could make a separate article of it. That could lead to a more coherent presentation, and the other pages could then just refer to that.

By more knowledgeable (than me), I mean someone who might have better qualifications for evaluating the sorts of computer studies done both by yourself and Kaufman (his is, naturally, a different type of computer study, if I may call it that). That is, someone who is a mathematician and/or a chess grandmaster, neither of which I am. Even then, Kaufman may qualify, especially if he is the former besides being a GM - however other people with equally good qualifications may disagree. A body of such people, who are known to be interested/paid (to make the evaluations of the studies/results), really, would be needed to build a consensus. I've read online somewhere long ago that some GM tried to explain the result Kaufman got, maybe unconvincingly.

Most of my calculations for estimating piece values are quite short and simple, even if highly suspect to at least some readers. An article on my assortment of quick and dirty methods of calculation (that try to presently provide for a big range of pieces and board sizes/shapes) would not cover all the piece types that are possible, I suppose, and also I have never done a CVP article/item of that sort (perhaps you could for your computer studies method, too, if it would not be too lengthy).

I've edited my last post a bit, for any who missed it.

Re: "It was not checked by anyone, so it must be wrong" is not really a valid line of reasoning"... (H.G. wrote)

Something being not yet worthy of trust is a shade different than being said to be wrong (or proven to be).

In 'Secrets of Practical Chess', GM Dr.[of math] John Nunn wrote of little- or un-tested sequences of play (in over-the-board games of strong players) in the opening phase of a game, that are recommended by chess authors, not to trust them to be in your chess opening repertoire (especially if you must rely on just such sequence(s) to keep your repertoire from going under, I'd add). Meaning, I suppose, treat them like rubbish until proven otherwise. Or, let someone else be the Guinea Pig - probably the advice was especially meant for players well below GM level.

That was back in the 1990s, when commercially available computer engines were mostly still relatively weak, though. Nowadays maybe you can count on what you come up with at home using a chess engine as to be virtually golden.

I am not sure what you try to demonstrate with this example. Obviously something that has never been tested in any way, but just pulled out of the hat of the one who suggests it, should be considered of questionable value, and should be accompanied by a warning. As Dr. Nunn does, for untested opening lines. And as I do, for untested piece values. That is an entirely different situation than mistrusting someone who reports results of an elaborate investigation, just because he is the only one so far that has done such an investigation. It is the difference of someone being a murder suspect merely because he has no alibi, or having an eye witness that testifies under oath he saw him do it. That seems a pretty big difference. And we are talking here about publication of results that are in principle verifiable, as they were accompanied by a description of the method obtaining them, which others could repeat. That is like a murder in front of an audience, where you so far only had one of the spectators testify. I don't think that the police in that case would postpone the arrest until other witnesses were located and interviewed. But they would not arrest all the people that have no alibi.

And piece values are a lot like opening lines. It is trivial to propose them, as an educated guess, but completely non-obvious what would be the result of actually playing that opening line or using these piece values to guide your play. It is important to know if they are merely proposed as a possibilty, or whether evidence of any kind has been collected that they actually work.

Your second paragraph may be bang on, except it could be a circular argument to say a body of evidence has been found by chess studies, yet it is the methodology of those very studies that might be viewed as unproven.

Sure, methods can be wrong, and therefore have to be validated as well. This holds more for true computer studies using engines, than for selecting positions from a game database and counting those. The claim that piece A doesn't have a larger value B if good players cannot beat each other when they have A instead of B more often than not is not really a method. It is the definition of value. So counting the number of wins is by definition a good method. The only thing that might require validation is whether the person having applied this method is able to count. But there is a point where healthy skepsis becomes paranoia, and this seems far over the edge.

Extracting similar statistics from computer-generated games has much larger potential for being in error. Is the level of play good enough to produce realistic games? How sensitive are the result statistics to misconceptions that the engines might have had? It would be expected of someone publishing results from a new method to have investigated those issues. And the method applied to the orthodox pieces should of course reproduce the classical values.

For the self-play method to derive empirical piece values I have of course investigated all that before I started to trust any results. I played games with Pawn odds at many different time controls, as well as with some selected imbalances (such as BB-vs-NN) to see if the number of excess wins was the same fraction of the Pawn-odds advantage. (It was.) And whether the results for a B-N imbalance were different for using an engine that thought B>N as for using one that thought N>B. (They weren't.)

New methods don't become valid because more people apply them; if they all do the same wrong thing they will all confirm each other's faulty results. You validate them by recognizing their potential for error, and then test whether they suffer from this.

I guess I have to get into the specifics I personally still don't trust about computer studies, again.

First, Kaufman's type of study: saying that B=N based on large number of games stats (I only vaguely recall, but many of the players in his database may have been sub-grandmaster level - GMs are relative adults compared to 2300 players playing wargames in a sandbox vs. each other). If you want to establish the absolute truth of if B=N, solving chess from the setup and then doing some sort of a database wins/losses count for [near-]'perfect' play would be best, but that is impossible on earth right now (perfect play, if it does not result in a draw, would probably favour White).

Today's best chess engines might be used to generate, say, a 3000+ vs. 3000+ engine vs. engine database if enough games could be played over time to very statistically matter - that would be arguably second best, but even then there may be some element of doubt to the result being the truth that might be hard to assign exact probability to, perhaps (maybe even a professional statistician who is also a GM could throw up his hands and say, we simply cannot say). In any case, the time it takes to make such a database makes it impractical for now, yet again.

Coming to the type of study used for fairy chess piece values, I don't know how margin(s) of error for such a study can be confidently established, for one thing. Next, more seriously, on my mind is the exact setup and armies used in a given study. For Chess960, I saw somewhere long ago online that someone figured after their own type of study that certain setups are roughly equal, while others favour White more than in orthodox chess, say up to 0.4 pawns worth over Black (you might find this somewhere on the internet, to check me). Consider also that that's just for armies that are equal in strength exactly, being identical as in chess. You may give both sides equally White and Black, but the setup and armies vary per study, and I'd guess it's hard to always be exhaustively fair to every possible setup/army, given time constraints.

Finally, you wrote earlier that errors tend to cancel each other out with lower level play (say 2300+ vs. 2300+ engines, as opposed to 2700+ vs. 2700+), It would be very good to know how many games and studies (even roughly) you base that conclusion on, if you still recall. Also, does the cancellation ever significantly favour one side or the other very much with any given [sort of] study? I think the strength of the engine(s) used just might be the most underestimated/large factor causing possible undetected error with this type of study (and sub-GM play within Kaufman's database study, as I alluded to above).

Well, I looked up the exact numbers, and indeed hish threshold for including games in the Kaufman study was FM level (2300) for both players. That left him with 300,000 games out of an original 925,000. So what? Are FIDE masters in your eyes such poor players that the games they produce don't even vaguely resemble a serious chess game? And do you understand the consequences of such a claim being true? If B=N is only true for FIDE masters, and not for 2700+ super-GMs, then there is no such a thing as THE piece values; apparently they would depend on the level of play. There would not be any 'absolute truth'. So which value in that case would you think is more relevant for the readers of this website? The values that correctly predict who is closer to winning in games of players around 1900 Elo, or those for super-GMs?

Your method to cast doubt on the Kaufman study is tantamount to denying that pieces have a well-defined value in the first place. You don't seem to have much support in that area, though. Virtually all chess courses for beginners teach values that are very similar to the Kaufman values. I have never seen a book that says "Just start assuming all pieces are equally valuable for now, and when you have learned to win games that way, you will be ready to value the Queen a bit more". If players of around 1000 Elo would not be taught the 1:3:3:5:9 rule, they would probably never be able to acquire a higher rating.

The nice thing about computer studies is that you can actually test such issues. You can make the look-ahead so shallow and full with oversights that it does play at the level of a beginner, and still measure how much better or worse it does with a Knight instead of a Bishop. And how much the rating would suffer from using a certain set of erroneous piece values to guide its tactical decisions. And whether that is more or less than it would suffer when you improve the reliability of the search to make it a 1500 Elo player.

It would also be no problem at all to generate 300,000 games between 3000+ engines. It doesn't require slow time control to play at that level, as engines lose only little Elo when you make them move faster. (About 30 Elo per halving the time, so giving them 4 sec instead of an hour per game only takes some 300 points of their rating. So you can generate thousands of games per hour, and then just let the computer run for a week. This is how engines like Leela-Chess Zero train themselves. A recent Stockfish patch was accepted after 161,000 self-play games showed that it led to an improvement of 1 Elo...

And in contrast to what you believe, solving chess would not tell you anything about piece values. Solved chess positions (like we have if end-game tables if there are only a few pieces on the board) are evaluated by their distance to mate, irrespective of what material there is on the board. Piece values are a concept for estimating win probability in games of fallible players. With perfect play there is no probability, but a 100% certainty that the game-theoretical result will be reached. In perfect play there is no difference between drawn positions that are 'nearly won' or 'nearly lost'. Both are draws, and a perfect player cannot distinguish them without assuming there is some chance that the opponent will make an error. Then it becomes important if only a tiny error would bring him in a lost position, or that it needs a gross blunder or twenty small errors. And again, in perfect play there is no such thing as a small or a large error; all errors are equal, as they all cost 0.5 point, or they would not be errors at all.

So you don't seem to realize the importance of errors. The whole elo model is constructed on the assumption that players make (small) errors that weaken their position compared to the optimal move with an appreciable  probablity, and only seldomly play the very best move. So that the advantage performs a random walk along the score scale. Statistical theory teaches us that the sum total of all these 'micro-errors' during the game has a Gaussian probability distribution by the time you reach the end, and that a difference in the average error/move rate implied by the ratings of the players determines how much luck the weaker player needs to overcome the systematic drift in favor of the stronger player, and consequently how often he would still manage to draw or win. Nearly equivalent pieces can only be assigned a different value because it requires a smaller error to blunder the draw away for the side with the weaker piece than it does for the side with the stronger piece. So that when the players tend to make equally large errors on average (i.e. are equally strong), it becomes less likely for the player with the strong piece to lose than for the player with the weak piece. Without the players making any error, the game would always stay a draw, and there would be no way to determine which piece was stronger.

If you want a definition of near-perfect play, that still allows for the possibility of a (small) error or two, a very long game (that is well-played) that is a win for one side comes to mind.

You could decide to solve chess and it could be useful for determining piece values - just tag the number of moves a given 'game' of near-perfect chess takes to play until checkmate, and also keep track if B vs. N is involved. Optionally, you could have an engine assessing (albeit not perfectly accurately) who has the advantage (and how much) at every move. This of course is all not practically possible, in today's world at least.

The problem is that an N vs B imbalance is so small that you would be in the draw zone, and if there aren't sufficiently many errors, or not a sufficiently large one, there wouldn't be any checkmate. A study like Kaufman's, where you analyze statistics in games starting from the FIDE start position, is no longer possible if the level of play gets too high. All 300,000 games would be draws, and most imbalances would not occur in any of the games, because they would be winning advantages, and the perfect players would never allow them to develop. Current top engines already suffer from this problem; the developers cannot determine what is an improvement, because the weakest version is already so good that it doesn't make sufficiently many or large errors to ever lose. If you play from the start position with balanced openings. You need a special book that only plays very poor opening lines, that bring one of the players on the brink of losing. Then it becomes interesting to see which version has the better chances to hold the draw or not.

High level of play is really detrimental for this kind of study, which is all about detecting how much error you need to swing the result.

And then there is still the problem that if it would make a difference, it is the high-level play that is utterly irrelevant to the readers here. No one here is 2700+ Elo. The only thing of interest here is whether the reader would do better with a Bishop or with a Knight.

My impression was that periodically there is a leap in the strength of one engine someone is working on (e.g. AlphaZero), and then it outperforms other engines, say in 100 game matches, at least for a while, until the next such cycle begins.

edit: Chess.com is quoted by Google as saying AlphaZero lost 8 games to a version of Stockfish in a recent match, out of 1000 games, causing the loss of the match. Not many decisive results this time around, but wins are still possible at such a lofty level 'at the top' as we have right now, given enough games are played.

edit2: I haven't kept track of the progress quantum computing has been making, but that could lead to stronger engines and perhaps even open the door to solving chess.

That is a completely wrong impression. After a short period since their creation, during which all commonly used features are implemented, the bugs have been ironed out, and the evaluation parameters have been tuned, further progress requires originality, and becomes very slow, typically in very small steps of 1-5 Elo. Alpha Zero was a unique revolution, using a completele different algorithm for finding moves, which up to that point had never been used, and was actually designed for playing Go. Using neural nets for evaluation in conventional engines (NNUE) was a somewhat smaller revolution, imported from Shogi, which typically causes an 80 Elo jump in strength for all engines that started to use it.

There are currently no ideas on how you could make quantum computers play chess. Quantum computers are not generally faster computers than those we have now. They are completely different beasts, being able to do some parallellizeable tasks very fast by doing them simultaneously. Using parallelism in chess has always been very problematic. I haven't exactly monitored progress in quantum computing, but I would be surprised if they could already multiply two large numbers.

By now it should be clear that the idea of using 2700+ games is a complete bust: 

1. it measures the wrong thing. We don't want piece values for super GM's, but for use in our own games.
2. it does it in a very inefficient way, because of the high draw rate, and draws telling you nothing.

So even if you believe/would have proved piece values are independent of player strength, it would be very stupid to do the measurement at 2700+ level, taking 40 times as many games, each requiring 1000 times longer thinking than when you would have done it at the level you are aiming for. If you are smart you do exactly the opposit, measuring at the lowest level (= highest speed) you can afford without altering the results.

Oh, and to answer an earlier question I overlooked: I typically test for Elo-dependence of the results by playing some 800 games at each time control, varying the latter by a factor 10. 800 games gives a statistical error in the result of equivalent to some 10 Elo.

I'll agree using 2700+ level play for studies is impractical at this time. Earlier I almost did tell you to re-check something you posted saying my implying that 2700+ level play being different from 2300+ level play meant that I'd thought there were no absolute piece values - yet you're going right ahead yourself and saying 2700+ play is different (besides impractical to use for studies). I thought you'd just had some sort of an automatic reaction to try and say everything I write is wrong, and that you wrote inconsistently in place(s) unknowingly. I do not know if you were still trying to be fair. In any case, I get your overall drift, certainly as of this last post of yours.

One person who earlier wrote somewhere that 'the person with the hand that holds the piece' affects it's value was Betza, if you wish to argue with that, too. I personally believe the true piece values (for average case) should be absolute (however I think we might never be able to know them for sure). I still had a couple of other things about studies that I thought were suspect (margins of error, initial setup/armies chosen, as I wrote a bit earlier) that you didn't address, but I now recall we discussed those long ago here on CVP - it's just that I never was fully convinced.

yet you're going right ahead yourself and saying 2700+ play is different

No, I said that if it was different the 2700+ result would not be of interest, while if they are the same it would be stupid to measure it at 2700+ while it is orders of magnitude easier around 2000. Whether less accurate play would give different results has to be tested. Below some level the games will no longer have any reality value, e.g. you could not expect correct values from a random mover.

So what you do is investigate how results for a few test cases depend on Time Control, starting at a TC where the engine plays at the level you are aiming for, and then reducing the time (and thus the level of play) to see where that starts to matter. With Fairy-Max as engine there turned out to be no change in results until the TC dropped below 40 moves/min. Examining the games also showed the cause of that: many games that could be easily won ended in draws because it was no longer searching deep enough to see that its passers could promote. So I conducted the tests at 40 moves/2min, where the play did not appear to suffer from any unnatural behavior.

You make it sound like it is my fault that you make so many false statements that need correcting...

Betza was actually write: the hand that wields the piece can have an effect on the empirical value. This is why I preferred to do the tests with Fairy-Max, which is basically a knowledge-less engine, which would treat all pieces on an equal basis. If you would use an engine that has advanced knowledge for, say, how to best position a piece w.r.t. the Pawn chain for some pieces and not for others, it would become an unfair comparison. ANd you can definitely make a piece worth less by encouraging very bad handling. E.g. if I would give a large positional bonus for having Knights in the corners, knights would become almost useless at low search depth. It would never use them. If you tell it a Queen is worth less than a Pawn, the side that starts with a Queen instead of a Rook would lose badly, as it would quickly trade Q for P and be R vs P behind.

The point is that the detrimental behavior that is encouraged here can never be stopped by the opponent. Small misconceptions tend to cancel out. E.g. if you twould have told the engine that a Bishop pair is worth less than a pair of Knights, the player with the Knights would avoid trading the Knights for Bishops, which is not much more difficult than avoiding the reverse trades, as the values are close. So it won't affect how often the imbalance will be traded away, and while it lasts, the Bishops will do more damage than the Knights, because the Bishop pair in truth is stronger. But there is no way you can prevent the opponent sacrificing his Queen for a Pawn, even if you have the misconception that the Pawn was worth more.

Note that large search depth tends to correct strategic misconceptions, because it brings the tactical consequences of strategic mistakes within the horizon. Wrecking your Pawn structure will eventually lead to forced loss of a Pawn, so the engine would avoid a wrecked Pawn structure even if it has no clue how to evaluate Pawn structures. Just because it doesn't want to lose a Pawn.

Statistical margins of error is high-school stuff. For N independent games the typical deviation of the result from the true probability will be square-root of N times the typical deviation of a single game result from the average. (Which is about 0.5, because not all games end in a 0 or 1 score.) So the typical deviation of the score percentage in a test of N games is 40%/sqrt(N). Having to calculate a square root isn't really advanced mathematics.

Well, 2700+ play could be of interest one day (that is, may matter, in spite of being dismissed as if that level of play should be treated in that type of [different, i.e. dismissive] way no matter what), if someone were to imply they know the actual difference, if any, between a B and a N on average for 8x8 [chess]. Computer study results, perhaps at times implying they have established the truth of piece values, are already posted all over the internet, not just here on CVP, where readers/players are all presently presumed to be sub-2700. Even thus, readers of CVP might care out of curiosity alone to know the ultimate truth, however established, even if they do not benefit by it very often, if ever, in their own games, unless they become [near-]2700+ themselves one day. A lot of people finding it interesting to know that 8x8 checkers has been weakly solved in modern times is somewhat comparable, perhaps.

That explanation of margins of error doesn't mention a thing or two that might go wrong if any assumptions are made at any step, such as assuming the presumed materially weaker side will never win the most games in a study [especially if only a few hundred games] by a possible fluke, even if unlikely.

Then, there is my own hypothesis that the larger in value/(more powerful) a piece is, the greater a certain margin of error might need to be within a study.

Perhaps unrelated(? - cannot recall if we discussed ever), 4 [uncompensated]tempi (worth 1/3rd of a pawn each in an open position), or 4/3rds of a pawn might be a normal minimum decisive edge, at least that's in line with an old-school rule of thumb I saw in an old book 'Point Count Chess', where a pawn is 3 points and 4 points ahead is supposed decisive (again, 1 pawn = 3 tempi in an open position is an old rule from even longer ago).

Systematic errors can never be estimated. There is no limit to how inaccurate a method of measurement can be. The only recourse is to be sure you design the method as good as you can. But what you mention is a statistical error, not a systematic one. Of course the weaker side can win, in any match with a finite number of games, by a fluke. Conventional statistics tells you how large the probability for that is. The probability to be off by 2 standard deviations or more (in either direction) is about 5%. To be off by 3 about 0.27%. It quickly tails off, but to make the standard deviation twice smaller you need 4 times as many games.

So it depends on how much weaker the weak side is. To demonstrate with only a one-in-a-million probablity for a fluke that a Queen is stronger than a Pawn wouldn't require very many games. The 20-0 result that you would almost certainly get would only have a one-in-a-million probability when the Queen was not better, but equal. OTOH, to show that a certain material imbalance provides a 1% better result with 95% 'confidence' (i.e. only 5% chance it is a fluke), you will need 6400 games (40%/sqrt(6400) = 40%/80 = 0.5%, so a 51% outcome is two standard deviations away from equality).

My aim is usually to determine piece values with a standard deviation of about 0.1 Pawn. Since Pawn odds typically causes a 65-70% result, 0.1 Pawn would result in 1.5-2% excess score, and 400-700 games would achieve that (40%/sqrt(400) = 2%). I consider it questionable whether it makes sense to strive for more accurate values, because piece values in themselves are already averages over various material combinations, and the actual material that is present might affect them by more than 0.1 Pawn.

I am not sure what you want to say in your first paragraph. You still argue like there would be an 'absolute truth' in piece values. But there isn't. The only thing that is absolute is the distance to checkmate. Piece values are only a heuristic used by fallible players who cannot calculate far enough ahead to see the checkmate. (Together with other heuristics for judging positional aspects.) If the checkmate is beyond your horizon you go for the material you think is strongest (i.e. gives the best prospects for winning), and hope for the best. If material gain is beyond the horizon you go for the position with the current material that you consider best. Above a certain level of play piece values become meaningless, and positions will be judged by other criteria than what material is present. And below that level they cannot be 'absolute truth', because it is not the ultimate level.

I never claimed that statistics of computer-generated games provide uncontestable proof of piece values. But they provide evidence. If a program that human players rated around 2000 Elo have difficulty beating in orthodox Chess hardly does better with a Chancellor as Queen replacement than as with an Archbishop (say 54%), it seems very unlikely that the Archbishop would be two Pawns less valuable. As that same engine would have very little trouble to convert other uncontested 2-Pawn advantages (such as R vs N, or 2N+P vs R) to a 90% score. It would require pretty strong evidence to the contrary to dismiss that as irrelevant, plus an explanation for why the program systematically blundered that advantage away. But there doesn't seem to be any such evidence at all. That a high-rated player thinks it is different is not evidence, especially if the rating is only based on games where neither A nor C participate. That the average number of moves on an empty board of A is smaller than that of C is not evidence, as it was never proven that piece values only depend on average mobility. (And counter examples on which everyone would agree can easily be given.) That A is a compound of pieces that are known to be weaker than the pieces C is a compound of is no evidence, as it was never proven that the value of a piece is equal to the sum of its compounds. (The Queen is an accepted counter-example.)

As to the draw margin: I usually took that as 1.5 Pawn, but that is very close to 4/3, and my only reason to pick it was that it is somewhere between 1 and 2 Pawns. And advantage of 1 Pawn is often not enough, 2 usually is. But 'decisive' is a relative notion. At lower levels games with a two-Pawn advantage can still be lost. GMs would probably not stand much chance against Stockfish if they were allowed to start with a two-Pawn advantage. At high levels a Pawn advantage was reported by Kaufmann to be equivalent to a 200-Elo rating advantage.