Comments/Ratings for a Single Item

Christine,

We have two ratings systems on this site. One is so hidden that you didn't
even see it. In the example game you gave, Battle of the Six Armies, the
game has not even been made available for ratings yet. And for games that
have been made available for ratings, such as European Chess, you still
have to go to another page to see the ratings. This ratings system strikes
me as useless and should be replaced with something better.

The ratings system you are actually referring to is the page ratings
system. This is available for all pages, not just games. Since this also
uses a five-point scale, one option would be to migrate the data from this
to what I am proposing. Another option is to leave it as a separate page
rating system. What I am proposing is specifically a game rating system
that would appear only in the headers of pages describing the rules of
games.

Asking how often a game has been played provides a measure of popularity,
and it gives a measure of reliability to the rating given to a game. Also,
there remain other ways to play games besides Game Courier. There is
Zillions of Games, ChessV, Fairy Max, over-the-board play, play by postal
mail, play through other websites, and play through dedicated programs.

ah ok, i do now remember seeing something like 'make this game available
for rating'. I agree, this is probably useless, and hidden, mainly because
people who do rate, rate the 'game page' or 'article page'. This still
does give an indication about what people think, especially if a game does
get a few good ratings, then people who disagree tend to come out and rate
it.

Now, you say 'one option would be to migrate the data from this
to what I am proposing. Another option is to leave it as a separate page
rating system.'

I personally would like it to be seperate, see what others think. I can't
see a problem keeping it seperate. Especially because, i'm still having
major issues with this 'how many times have you played this game'
question. Yes, as you say, game courier is not the only way to play, there
is zillions and other engines and what have you, but, the fact remains,
even counting all these, many, many games are still not available counting
all these.

I understand how this question can tell how popular a game can be, but as
far as indicating how a great a game is, that is another question. I still
see it as being a bit unfair to many games.

Christine Bagley-Jones wrote:

I understand how this question can tell how popular a game can be, but as far as indicating how a great a game is, that is another question. I still see it as being a bit unfair to many games.

No rating system will automatically reveal how great a game is. But having a rating system gives the truly great games an opportunity to prove themselves, and that's how we can come to recognize some of them. Popularity is one measure along which great games can prove themselves.

It's in the nature of any rating system that some things being rated will have an advantage over others besides mere quality. No rating system is going to magically distinguish games on nothing but quality. Even if I dropped the question on popularity and just asked about quality, it would not work better. For one thing, some people rate games without playing them, and having not played a game makes someone less qualified to judge its quality. The main purpose behind asking how much someone has played a game is to better gauge how qualified he is to rate the game and to make it easier to separate ratings based on experience from ratings based on speculation. Another purpose behind it is to distinguish playtested games from untested games. Also, popular games, whether or not they are the best, usually have something going for them. A game's popularity may reflect its age, its presence in tournaments, or active promotion from its creator, but if it's an inherently flawed game, it probably still won't rise so high in popularity. So popularity is a sign of quality, making it something to look for when looking for games of high quality.

Well, last thing i will say about all this is, whichever way you throw it,
i see games that are not on zillions and game courier or other programs at
a disadvantage with the 'how many times have you played' question.

And finally, i just hope if a new rating system goes ahead, i really hope
it is seperate, throwing out, what, a decade of ratings, or putting them in
with the new system would be a little disappointing. Surely it can be
seperate. You say the 'hidden' rating system right now is useless, why don't you replace that with your idea, and keep the 'page rating's' as is. Why does this have to get involved?

Yes, some games will be at a disadvantage, and I'm fine with that. The point of a rating system is not to recognize every game of quality. That is going to be difficult however you go about it, because some games will just be overlooked. The purpose behind a rating system is to give games the opportunity to prove themselves and to recognize those that do.

What you're wanting to do is minimize false negatives, quality games that unfairly go unrecognized. There is no way to minimize false negatives short of thorough diligence in rating games. What this rating system will be designed to do is minimize false positives, low quality games that receive undeserved recognition. Measuring popularity is a good way to do this, because popularity is not easily won, and when it is won, it is a sign of quality. Sure, some high quality games will be overlooked. But even without a rating system, they would still be overlooked.

So, a rating system is not going to be perfect, but it will provide a needed service, which is to help visitors find games that are worth playing. For anyone looking to find the best games of all, I don't think any rating system will help, because it will still take the diligence and effort to dig through the games we have. But for those mainly looking to avoid bad games and find some good ones, a rating system will be helpful.

While trying to figure out how to make a new ratings system work, I have
been looking into the database to see what is there. One of the tables is
called Ratings, and it is for the rating system I have described as hidden
and useless. Only three games have been rated. These are an example game
David Howe made to test the ratings system, European Chess, and Shatranji.
Given the scant use of this ratings system, I think it would be alright to
discard it. What I want to replace it with will be more prominent and
easier to use, making it more likely that people will use it. What I'm
trying to decide on right now is how to represent the data in the database.
It could go in the Item table, the Person table, or both. Putting it in the
Item table makes it easier to calculate the average ratings for a game,
while putting it in the Person table makes it easier to list someone's
favorites. I'll probably go with putting it in the Item table. Since
userids are uniformly lowercase and all the keys in the Item table are
mixed case, it should be safe to add userids as keys in the Item table. The
userid would be a key to a two-character string representing how someone
has answered the two questions. To calculate an average rating, I could
then scan an Item row for lowercase keys, add up the values and divide.

Fergus, you said 'What you're wanting to do is minimize false negatives,
quality games that unfairly go unrecognized.' Sorry, this is not true. The
only thing i am trying to do is keep 10 years of members ratings. Every
game has a game page, every member can rate these pages. Sounds fair to me,
i dont see how that is unfair to any game. I have faith in the members
rating games, even non-members, for that matter.
But i see you are bent on changing things. Can i ask, what does it mean to put ratings in the 'item table'? Is this a page like this ..
http://www.chessvariants.com/index/mainquery.php?type=Any&startswithletter=Na&orderby=LinkText&displayauthor=1&displayinventor=1

Christine Bagley-Jones wrote:

But i see you are bent on changing things, without bothering also to talk to the other members apparently, haha.

Talking to the other members about what I'm planning is exactly what I'm doing here. But so far, you're the only one who has had anything to say about it. I assume from their silence that other members are okay with what I'm planning.

The only thing i am trying to do is keep 10 years of members ratings.

I plan to leave the comments sections and page ratings alone. What I intend to replace is the game ratings system that has been used for only three games the whole time it has been here.

Every game has a game page, every member can rate these pages. Sounds fair to me, i dont see how that is unfair to any game. I have faith in the members rating games, even non-members, for that matter.

It's fair in that it is equally useless for all games. While it allows people to rate and review games, it does not make use of this information in any way. What I have planned is a dedicated game rating system that can report back various types of information, such as the popularity or average rating for a game, an individual member's favorite games, or whether the inventor has played the game and how he rates it himself.

Can i ask, what does it mean to put ratings in the 'item table'?

It is MySQL jargon, and only David Howe needs to understand precisely what I mean. Anyhow, I'm now thinking it would be best to create a new table in the database, because it would be shorter than the Item or Person tables and so faster to access.

Is this a page like this .. http://www.chessvariants.com/index/mainquery.php?type=Any&startswithletter=Na&orderby=LinkText&displayauthor=1&displayinventor=1

No, it's nothing you would see on the website.

cool, well done, hehe, good work, just you know, being protective and
silly, great work fergus, your a legend.
sorry if i cause trouble, hehe, dont mean to.

you know, another idea, besides what you doing, if you are worried about
people coming to this site and not finding quality games, or finding it
hard to find them, making a link on main page to 'recommended games', or
something along those lines (some recommended top quality games) etc etc. 

You could start by just picking some obviously top games. Later you could
change it to link to the top rated games that have been rated by your
system.

We have that already. It is called Recognized Variants. But it takes effort
to add to it, and it is not as responsive to user preferences.

For the technically inclined, here is the table I'm thinking of adding:

create table Likes (
    UniqID blob,
    ItemID blob,
    UserID blob,
    Freq tinyint,
    Rating tinyint,
    primary key(UniqID),
    key (ItemID, UserID)
);

The UniqID will be a concatenation of ItemID with UserID, and its purpose
is to uniquely identify a row for quick reading and writing. The other two
keys, ItemID and UserID, may have the same value for multiple rows and are
included for finding all rows with the same ItemID or the same UserID. The
remaining two values include integers representing answers to the two
questions.

Just a few thoughts about the Likes table:

• It might be useful to include an ID for the item's creator. I know this can be found by joining through the ItemID but keeping it in this table as well would enable faster queries concerning games inventors. It would also make it easier to stop creators rating their own games (UserID != CreatorID)
• Will a user be able to revise their ratings? If so the table will need a timestamp field and possibly either a binary current/superseded field or a status field. Or previous ratings might have their own table - LikesHistory

Graeme Neatham wrote:

It might be useful to include an ID for the item's creator. I know this can be found by joining through the ItemID but keeping it in this table as well would enable faster queries concerning games inventors. It would also make it easier to stop creators rating their own games (UserID != CreatorID)

Actually, I want to encourage inventors to rate their own games. A game's inventor is usually the most experienced player of the game. As an inventor myself, I'm not expecting inventors to give top ratings to all of their games. I would give top ratings to some of my games and not to others. I don't think there is any need to add inventor ids to the table, because these can be retrieved easily enough from the Item table.

Will a user be able to revise their ratings? If so the table will need a timestamp field and possibly either a binary current/superseded field or a status field. Or previous ratings might have their own table

Yes, users will be able to revise their ratings. I don't see the need for a timestamp or other field. Old rating values will simply be replaced with new rating values.

Having read some more about MySQL, I think the table might look like this:

create table Likes (
    ItemID blob,
    UserID blob,
    Freq tinyint,
    Rating tinyint,
    primary key (ItemID, UserID),
    key (ItemID),
    key (UserID)
);

I removed UniqID, because I learned that a key can be made from multiple
fields.

Relational Database terminology:

• Candidate Key - a column or set of columns providing data that is unique for each row. Where a set of columns is involved the term Composite Candidate Key is used.
• Primary Key - the Candidate Key that is the main index for a table and which will be used as a Foreign Key in other tables. For performance purposes a Primary Key will usually not be Composite.
• Foreign Key - the Primary Key of another table.
• Key - when used without qualification will normally be understood to be shorthand for Primary Key

Self-rating - personally I would be wary of such a practice, not because I think inventors are always going to give maximum marks to their own games, but because they are too close to their own games to provide a disinterested assessment.

Overwriting of data due to revision of values leading to loss of historical data strikes me as being what would be called 'bad practice'. But I guess I'm guilty of trying to apply corporate standards in a non-corporate situation.

Graeme Neatham wrote:

Primary Key - the Candidate Key that is the main index for a table and which will be used as a Foreign Key in other tables. For performance purposes a Primary Key will usually not be Composite.

What kind of performance issues are there? With a composite primary key, will it search individual keys until it finds a match instead of directly knowing where to go?

Self-rating - personally I would be wary of such a practice, not because I think inventors are always going to give maximum marks to their own games, but because they are too close to their own games to provide a disinterested assessment.

I'm not worried about that. I think that game inventors, especially the more productive ones, distinguish between games they consider experiments and games they consider masterpieces. And especially when it comes to inventors of several games, I want to know which ones they like best. Regardless of how objective inventors can be in evaluating their own games, I want to know how they evaluate their own games.

Overwriting of data due to revision of values leading to loss of historical data strikes me as being what would be called 'bad practice'. But I guess I'm guilty of trying to apply corporate standards in a non-corporate situation.

I'm not worried about overwriting old data, because it will have no use anyway. Also, if I included old data, I would either have to expand the table dynamically or use a more complicated primary key. The latter approach could be a real problem, because the php script would not know what primary key to use to access the ratings.

Fergus, if you need any help with the SQL end of things, please let me know.

There are two existing rating and commenting systems. I will refer to the older (and heavily used) system as the rating and commenting system. The newer system, which is not used to any significant extent, I will call the newer rating system.

I agree that the newer rating system is not used (much to my chagrin, given the work I put into it) and should be wiped from the site. The older rating and commenting system seems to work acceptably, although it has its issues and limitations. I agree that we shouldn't toss years of valuable comments/ratings.

So my feeling is that we should have a single, consistent commenting and rating system. Having two parallel systems is confusing and inelegant. That said, I am at a loss for how to proceed. Somehow, we should keep what we have, but also add desired functionality, but as a single, comprehensive system.

I suppose we could export the existing comment and ratings into the proposed new system, and have a unified and elegant commenting and rating system. That will take more effort, but may be worth it.

Just my 2 cents. Please let me know if I can be of any help on the PHP or SQL end of things. I really hope we can achieve a more effective and elegant system than the existing one.

David Howe wrote:

So my feeling is that we should have a single, consistent commenting and rating system. Having two parallel systems is confusing and inelegant. That said, I am at a loss for how to proceed. Somehow, we should keep what we have, but also add desired functionality, but as a single, comprehensive system.

I suppose we could export the existing comment and ratings into the proposed new system, and have a unified and elegant commenting and rating system. That will take more effort, but may be worth it.

That could be tricky. For one thing, comment ratings are associated with individual comments instead of with individual users. If I post two comments to a page with different ratings, I now have multiple ratings for the same game. If we exported the comment ratings, we would have to identify the latest ratings made by each user. Also, what I have in mind is specifically for games, not for pages. The rating and comment system in place is available for all pages, and even on game pages, people sometimes use it to rate the page instead of the game itself. Although I have previously proposed using a five-point rating scale, I have more recently been thinking of going with Like, Dislike, and Favorite. There is also the matter of measuring popularity, which the rating and comment system does not handle at all.

Assuming I used a five-point scale, the ratings from the rating and comment system might be incorporated into the Rating field of the Likes table. When someone posts a comment, it could show his current rating for the game, and if he changed it with his comment, it would update the Rating value in the Likes table. A comment could also report the player's experience with the game at the time of the comment. So the comment table would keep track of historical data, and the Likes table would keep track of current data. The ratings in the comments could be replaced by integers, and a separate table could be used to translate them into words. Besides entering ratings with a comment, I would provide a rating form in the header, which can be used without entering a comment, and it would just update the Likes table.

Those are my thoughts on how they could be merged together. I'm still not sure it would be the best way to go. We may want to keep them separate, using one for page ratings and one for game ratings.

I'll let you know if I need help with SQL.

I generally like the idea.  I think the inventor shouldn't rate the game;
if they want to mention what they think of the game, it should go into the
page somewhere (in the introduction would be good).  I'm a little worried
about rating games when they are themed or experimental: for instance, my
only game is 4D, so probably not a good candidate for frequent play, but
(IMO of course) it is a very good setup as far as 4D games go.  Should it
get rated in the former sense or the latter?  Maybe the frequency of play
question will serve this purpose too, but maybe there should be a third
rating?  (So 'good game', 'interesting', and 'often played'?)

Here are three different options:

1) Keep the rating and comment system separate from the dedicated game
rating system I have in mind.

2) Integrate them.

3) Turn the rating and comment system into just a comment system. Keep all
the old ratings in the comment system but don't allow new ratings. Export
the ratings of game pages into the dedicated game rating system.

For the rating scale, here are some options I have in mind:

Dislike, Like, Favorite

Poor, Inferior, Good, Superior, Exceptional

Dislike, Interesting, Like, Favorite

I had a script count the number of times each rating has been used in the
comments. Here are the results:

Array
(
    [Good] => 2052
    [None] => 21618
    [Excellent] => 3167
    [Poor] => 511
    [] => 90
    [Average] => 156
    [BelowAverage] => 76
)

I then thought to do the same thing on only Game pages. Here are the
results of that:

Array
(
    [Good] => 1507
    [None] => 9939
    [Excellent] => 2068
    [Poor] => 342
    [BelowAverage] => 43
    [Average] => 69
    [] => 19
)

Whichever results I use, the same pattern emerges. Discounting None and
null, Excellent is used most often, followed by Good and Poor. BelowAverage
and Average are used much less than these three. This supports the idea
that most raters are really employing a three-point scale rather than a
five-point scale. This lends support to using the 'Dislike, Like,
Favorite' scale over the five-point scale currently in use with the
comments.

Judging from this data, I also think that Excellent has been overused. More
games have been rated Excellent than the sum of all the other ratings. A
rating of Excellent should be reserved for the very best games, and if it
were, it would be a less common rating. The way I want to use Favorite may
curb this tendency. One thing I want to do with favorites is list each
user's favorites on his information page. When selecting a game as
favorite, the user would understand that he is also adding it to a viewable
list of favorites, and he may choose his rating with a view to this.

Fergus Duniho writes:

What kind of performance issues are there? With a composite primary key, will it search individual keys until it finds a match instead of directly knowing where to go?

It is quicker to traverse a numeric key than an alpha or alpha-numeric. But it is not just a question of machine performance - development time including code production (manual and automated) and testing can be less efficient with composite primary keys. There is also the question of maintenance and the persistence of natural composite keys when compared to surrogate primary keys.

But this isn't the place to conduct a seminar in database design and I have already declared myself guilty of misapplying enterprise practice - it isn't easy, sometimes, to put aside work practices gleaned over 3 decades.

Ben Reiniger wrote:

I think the inventor shouldn't rate the game; if they want to mention what they think of the game, it should go into the page somewhere (in the introduction would be good).

Judging by the data I collected on the frequency of ratings in the comment system, there is a lot of rating inflation going on, and it can't all be due to inventors rating their own games more favorably than more objective critics. In fact, I can screen for that. Here are the results of non-anonymous ratings by people other than the inventor:

Array
(
    [Excellent] => 782
    [Good] => 698
    [Poor] => 137
    [Average] => 27
    [BelowAverage] => 18
)

Excellent is still the most common rating, followed by Good and Poor. We can also look specifically at how inventors rate their own games. Here are the results of non-anonymous raters rating their own games:

Array
(
    [Excellent] => 18
    [Good] => 13
    [Poor] => 3
    [Average] => 2
    [BelowAverage] => 1
)

This is a smaller sample size, but the pattern is not significantly different. Pride might motivate inventors to rate their own games higher, but that doesn't account for others also giving their games an Excellent rating more often than other ratings. Perhaps the other raters feel they would be hurting the inventor's feelings if they rated a game anything less than Excellent. Perhaps the word 'excellent' has lost its comparative connotations. We may use the word too liberally, even for things that aren't really that much better than other things.

One of my main concerns in designing a new rating system is to avoid the rating inflation common in the current rating and comment system. One thing I think will help is to replace qualitative rating terms (Poor, BelowAverage, Average, Good, Excellent) with personal preference terms (Dislike, Like, Favorite). I think Favorite works better than Excellent, because it is more clearly a comparative term than Excellent is. Judging from the data I just presented, I don't think that banning inventors from rating their own games would do anything to reduce rating inflation.

Furthermore, given that an inventor is usually the most experienced player of a Chess variant, or at least the one person who knows the game better than anyone else does, I think the opinion an inventor has of his own games is valuable information, and it should be included in the system.