Derek Nalls wrote on Fri, May 2, 2008 03:37 PM UTC:
Yes, your test example yields a result totally inconsistent with
everyone's models for CRC piece values. [I did not run any playtest
games of it since I trust you completely.] Yes, your test example could
cause someone who placed too much trust in it to draw the wrong conclusion
about the material values of knights vs. archbishops. The reason your test
example is unreliable (and we both agree it must be) is due to its 2:1 ratio of knights to archbishops. The game is a victory for the knights player simply because he/she can overrun the archbishops player and force materially-disadvantageous exchanges despite the fact that 4 archbishops indisputably have a material value significantly greater than 8 knights.
In all three of my test examples from my previous post, the ratios of
archbishops to chancellors and archbishops to queens were only 9:8. Note
the sharp contrast. Although I agree that a 1:1 ratio is the ideal goal, it was impossible to achieve for the purposes of the tests. I do not believe a slight disparity (1 piece) in the total number of test pieces per player is enough to make the test results highly unreliable. [Yes, feel free to invalidate my test example with 18 archbishops vs. 16 chancellors and 18 archbishops vs. 16 queens since a 2 piece advantage existed.] Although surely imperfect and slightly unreliable, I think the test results achieved thru 'asymmetrical playtesting' or 'games with different armies' can be instructive as long as the test conditions are not pushed to the extreme. Your test example was extreme. Two out of three of my test examples were not extreme.