https://web.archive.org

Comment Rating Changes || kuro5hin.org

The Wayback Machine - https://web.archive.org/web/20091126073814/http://www.kuro5hin.org:80/story/2003/10/6/172738/261

[[P]](https://web.archive.org/web/20091126073814/http://www.kuro5hin.org/print/2003/10/6/172738/261)**[Comment Rating Changes](https://web.archive.org/web/20091126073814/http://www.kuro5hin.org/story/2003/10/6/172738/261)**

By rusty in Site News

Mon Oct 06, 2003 at 05:27:38 PM EST

Tags: Kuro5hin.org ( all tags)Kuro5hin.org

It's been pretty clear for a while that the comment ratings system isn't entirely working the way we'd intended it to. I know some of you have the impression that I'm the type to say "It's perfect! There is nothing wrong! You are a scurrilous troll for suggesting there might be!" but that's actually not true. I just don't usually change things unless I think I've got something better to change it to, and for a long time, I didn't have much in the way of good ideas for what to do.

I think I've finally come up with some changes that might do more good than harm, though. They're at least worth a try. If you're reading this, then the new system should be currently running, so read on for a description of what's different and why.


Mojo, Jojo.

Mojo, I still maintain, was a great idea. If this site could be reasonably sure that one person only had one account, it would work smashingly. The problem is, we have no reasonable expectation that someone is necessarily only going to inhabit one account, and the main benefit of mojo evaporates as soon as that's the case.

The idea behind the whole thing was to make crapflooding no fun by allowing users to collectively make garbage comments disappear. A comment rated sufficiently low would be hidden from all but trusted users, even from the user who posted it. To counterbalance this effect, and make sure it was only used in cases where a large number of people agreed that the comment should go, the rating scale was balanced so that it took five '0' ratings to counteract a single '5' rating.

What actually happened, of course, is that crapflooding was done with a throwaway account, while the crapflooder maintained one or more accounts that behaved and were trusted, so they could watch their handiwork and make sure it stayed above the hidden threshold. So the main social pressure against that kind of thing was pretty much useless.

On the other hand, there was still the carrot of "Trusted User" status out there to be gotten, and kept or lost as the ratings churned. So it became obvious that an easy way to annoy someone who cared about whether they were trusted or not was to downrate a passel of their recent comments, and try to make them lose their trusted status. In fact, this became a popular way to let off steam with someone in general, and I ended up implementing a ratings-wipe feature and getting emails to help@k5 with a depressing frequency pointing out that someone had done it to someone else, yet again. This gets old pretty fast. Obviously, if nothing else showed it, the fact that I had to wade in and manually erase stuff showed that this wasn't working.

So what could be different?

...I wondered to myself. Well, for one thing, it seemed pretty clear that the main accomplishment of trusted user status was that a majority of well-meaning but quiet users were not permitted to help hide the crap, while the battle was being waged among a small number of very prolific posters, with wildly differing views of what an appropriate comment looked like. If trustedness isn't doing its job, I thought, why keep it? So the first thing I thought should change was that everyone should be able to decide whether a comment ought to be hidden.

This decision has the necessary side-effect, though, that you also have to give everyone the option of whether to see hidden comments or not. If everyone is trusted, then the only people that hidden comments are actually hidden from are anonymous visitors. I figured that with the right tools, this was an acceptable trade off, considering that the people the hidden comments were supposed to be hidden from could mostly see them anyway. But to offset the apparent pointlessness of hiding comments from nobody, I thought it would be good to add an option for how people want to handle hidden comments. So you can choose to either see them all the time (in effect, have it behave like trusted status used to), hide them all the time (behave like a regular user used to, except that you can also hide comments), or only show them until you've rated, and then hide them if the score warrants.

The system was always designed to work best with more input. A comment's score will converge as more people rate it, due to the averaging of individual ratings. After a few ratings, the comment's score starts to converge pretty fast, but the system is weak at the outset, and the first rating has a very disproportionate effect unless it is followed quickly by others. Many comments only got one or two ratings, so a whole lot of the ratings were pretty unreliable.

I came up with two approaches to this problem. The first is to simplify the rating scale to encourage more people to rate. People tended to use 1 and 5 way more than 2 through 4, so I've collapsed the scale down to 1-3. Also, to attempt to make it more intuitive what the numbers mean, I've replaced the numbers with labels. So the scale now goes "Discourage," "Neutral," and "Encourage," with the hide rating being "Hide." Those correspond numerically to 1-3 (and -1, see below about the Hide rating) and the comment's displayed score will fall between -1 and 3. Anything below 1 is still considered hidden. The idea is that it's pretty easy to decide whether you want to encourage a comment, discourage it, or you don't feel either way. I'm hoping that with fewer choices, more people will take the half-second to click the little form. Hopefully the text labels will also help simplify the rating choice, without unduly trying to force a value scale on them. You could be encouraging more people to post similar comments, you could be encouraging the comment's author, or you could be encouraging people to read the comment. You can interpret the label a lot of different ways, and they're all right.

The other approach to get more accurate ratings is to not consider a comment's score meaningful until it has a minimum number of ratings contributing to it. A score composed of a single rating doesn't mean much. But a score composed of six ratings will likely be within a half point either way of that comment's eventual final score. So a comment's score won't be "official" until it has at least six ratings contributing to it. In the meantime, the ratings will be kept and tracked, and may be viewed, but they won't be compiled into a score or affect the comment's placement or visibility in any way.

Another longstanding problem was the disparity in power between 5 and 0. In the old days, I weighted it far on the side of not hiding things, because I was afraid it would become a cheap tool for squelching those you disagree with, instead of a cleanup mechanism, like it was intended. Well, my lack of faith in humanity was mostly unwarranted, and most people have reserved their zeros for the real bottom of the barrel stuff. So I wanted to bring the ends of the scale closer to parity, so that we don't need thirty "Hide" ratings to counteract a couple of sock puppet fives. So instead of 0, the "Hide" rating will now be numerically equivalent to -1. This is also adjustable, which it hasn't been or I'd have changed it long ago. So if future adjustments seem necessary, that can easily be done.

There have been some people who have wanted to scrap the whole thing and ignore ratings altogether, even going so far as to use rewriting proxies to accomplish this. Since most of the bondage and discipline elements of enforced participation are now irrelevant, I didn't see any reason not to allow people to just ignore ratings completely. So in your "Rate?" options, you'll now see a third option -- "Hide". This will turn off your rating boxes, and will also completely hide the ratings. If you sort with "Ignore Ratings" and set your hidden comment prefs to "Show," you will have an unmoderated K5.

And finally, I added a little thing in there that makes it so that multiple ratings on one comment from a single IP can be ignored. That is, if we get three ratings from the same IP, only the most recent will count. I know that restricting by IP is very sub-optimal, as there are many situations where different people will have the same apparent IP. But on the flip side, it would make it just that little bit harder for most people to rate a comment several times with different accounts, if they also had to change IPs every time. Not impossible, but more of a pain. That's the intention here. But if it turns out to be more of an impediment to ordinary use than it is an impediment to abuse, it will be shut off. So mark this last one "experimental" for now. Also, it is off for the moment until I figure out how it will deal with old ratings that have no IP associated. But it may be updated and enabled anytime, so I figured I'd get the description up now.

Can you summarize for me here, Mr. Proust?

Ok, to sum up, that one wafting fragrance of Madeleine inspired me to make changes as follows:

  • Rating scale now goes 1-3, with text labels Discourage, Neutral, and Encourage

  • Hide rating is equal to -1

  • All users have ability to Hide-rate

  • All users may choose, in your comment preferences, whether to always hide hidden comments, always show them, or only show them until you've rated them

  • A comment's score doesn't count until there are at least six ratings contributing to it

  • Only one rating will count from one IP (experimental)

  • Users may choose to turn off ratings display altogether, and when combined with "Always show hidden comments" and "Ignore ratings" (in sort) will effectively have an unmoderated K5. You may also choose to hide hidden comments and sort on ratings without actually showing them. It's up to you.

There will almost certainly be bugs, both from these changes and from the raft of little bugfixes that were just applied, so if you see some, don't be shy about posting below. Considering we're overriding an existing rating system that worked somewhat differently, there will also probably be some general confusion in the system for a little while. It'll clear itself up as new stuff comes in. And as always, any of this is liable to change if it doesn't work, but let it work for a while and see what happens before you decide what you think.

Although any automated system will eventually be broken by those who desperately want to abuse it, but you can generally make it not worth their time. Let's hope this does just that.

Now, to address the problem of heinous trolling...

Not perfect, not quite.

Sounds cool. But why do the diary and story sidebars on the front page display UID instead of account name now?

"Be instead like Gamera -- mighty, a friend to children, and always, always screaming." - eSolutions

Ugh... that silly IP address restriction.... ( 2.42 / 54) ( #3)

by banstyle on Mon Oct 06, 2003 at 05:36:13 PM EST

Again, having that 1 rating per IP is going to wack everyone behind a proxy. That's a lot of people.

Just a thought.

__

"Everything done in weakness fails. Moral: do nothing." -Nietzsche

Instead of just "Encourage", I would like to be able to distinguish between items that are funny, insightful, informative or just plain underrated. kthx.

Play 囲碁

the sections are now displaying userid numbers instead of names.

At least for me ;-)

  • Who's Rusty? by Cro Magnon, 10/06/2003 05:41:06 PM EST ( 2.25 / 24)

    • He is modest by 0x29a, 10/06/2003 05:44:12 PM EST ( 1.50 / 14)
  • Yup by rusty, 10/06/2003 05:44:23 PM EST ( 1.90 / 21)

  • hopefully, by Run4YourLives, 10/06/2003 05:44:29 PM EST ( 1.95 / 23)

I'm glad that we collaboratively decided on this new change to our media site!

Go us, it's our birthday, etc.

\\\__________________________

The weak are killed and eaten...

  • actually, by Run4YourLives, 10/06/2003 05:43:20 PM EST ( 2.03 / 32)

    • Fair enough by A Proud American, 10/06/2003 05:44:17 PM EST ( 2.08 / 23)
  • You don't understand. by RobotSlave, 10/06/2003 07:49:44 PM EST ( 1.65 / 20)

    • Sir by A Proud American, 10/06/2003 08:20:53 PM EST ( 1.14 / 14)

Wouldn't it be nicer to sort the rating choices the other way, so that "Encourage" is on top and "Hide" on bottom? Hiding comments should be a little bit discouraged.

You remind me why I still, deep in my bitter crusty broken heart, love K5. —rusty

the edit quene stories are now highlighed pink.

I for one consider this a huge change for my self imposed importance and am really disapointed that my opinions we're considered. I mean, PINK? Out of all the other colour in the web safe pallete, why pink?

You truly are a dictator Rusty. Damn you to hell!

It's slightly Japanese, but without all of that fanatical devotion to the workplace. - CheeseburgerBrown

No more numbers...I need numbers to validate my existance! We must rebel: we can still rate comments by responding to comments with titles "X (nt)" where X is a number 0-5.

We'll see how this works out rusty-meister. Is this the chosen forum where we should bitch about it if it doesn't work, or should we default to the diary section?

My name is LilDebbie and I have a garden.

- hugin -

well done, mr. foster.

The changes sound very positive. Although I notice that I still can't rate or vote on diaries or site news.

---------

lysergically yours

... I'm going to hide all ratings and rating boxes, while keeping no comments hidden. As much as I appreciate the positive feedback people have given me with their ratings, I'm a little tired of all the controversy that surrounds ratings and therefore I'm turning them off. It's been abused too much at this site to take seriously.

On the Internet, anyone can accuse you of being a dog.

-1 too many site updates. ( 1.58 / 24) ( #26)

by terpy on Mon Oct 06, 2003 at 05:48:53 PM EST

Thanks, these look like worthwile updates.

Though, might I ask what the motivation for applying these changes on a Monday was?

----

"I'm not sure I'm down with having Hitler on my coffee mug, but whatever, it was free." -MohammedNiyalSayeed

  • Monday by rusty, 10/06/2003 06:02:08 PM EST ( 1.80 / 15)

    • Ah. by terpy, 10/06/2003 06:24:31 PM EST ( 1.80 / 15)

  • Hmmm by A Proud American, 10/06/2003 06:23:34 PM EST ( 1.84 / 19)

  • I never could get the hang of Mondays by styrotech, 10/07/2003 08:32:35 PM EST ( none / 5)

    • Thursdays. by terpy, 10/07/2003 09:00:35 PM EST ( none / 4)

It definitely takes me more time to stop to differentiate and read the labels than it did for the numbers. When I click on the drop down, I can't just pick out a digit. It might just be the newness, but it might also be that words take more effort to understand than numerals.

Also, I don't understand the difference between 'none' and 'neutral'. If I don't rate the comment, isn't that being neutral? Actually selecting something in order to express what is basically a non-opinion is weird.

I've been trying to sort by score lately on stories, but I would like to have diaries sorted differently as they have fewer posts (what I really want is the sorting mechanism to behave like the display mechanism, after so many comments, I would like to change behavior). As long as the rating changes help the sorting, it is all good.

Also, can you please fix these things next time?

  • When writing a story, the format dropdown box always defaults to whatever you have it set to on comments. Most of the time I submit stories in HTML format, but I write comments in auto-format. Often, I go to edit a story and the format box changes back to auto-format, so when I make my changes, it screws up the formatting. Is there any way to save the format option on a submission so that doesn't happen?

  • When a story is voted out, can you change the page to say that th story was voted out and give a link to the comments page instead of those who know having to hack the URL themselves and new people who don't know the trick not knowing?

  • The user info page that shows the number of comments, diaries, and stories doesn't work when you click on the name in some instances (I think it has to do with the URL and not liking that op=info version, but the .../rusty/info works fine).

  • There were other things for the story submissions, but I can't remember them right now.

\\\_____

Smile =)

Given the culinary lineage of its former colonial masters, America's "theft" of other nation's cuisines is considered by mo

Default comment rating ( 1.90 / 30) ( #31)

by R343L on Mon Oct 06, 2003 at 06:09:04 PM EST

Why not make the default in the combo box "Neutral"? Then all a person has to do is hit Rate All and they have at least voiced their (non-)opinion. Otherwise to rate a whole page (where most of the comments you don't care about and would just rate a old-3/Neutral), you have to pull down a lot of comboboxes....

Rachael

"Like cheese spread over too much cantelope, the people I spoke with liked their shoes." Ctrl-Alt-Del

interestingly enough, ( 1.86 / 30) ( #32)

by pb on Mon Oct 06, 2003 at 06:09:32 PM EST

I was thinking about how to go about making a trusted ratings site, where the identity of the user is confirmed upon registration, solely to prevent (or control) duplicate accounts. The penalties against duplicate accounts would be legal ones, which already govern fraud and identity theft in the real world.

First, obviously the person's identity would have to be established in some pre-existing and legally binding way, and if this isn't foolproof, then there should at least be recourse if deception is uncovered later. You could tie this to id numbers or driver's licenses, but I think the simplest solution is just to charge $1 (for overhead) by credit card, check, or what have you, and keep track of the names and addresses of the account holders.

This doesn't solve the problem of having multiple John Q. Smiths in the same zip-code; for that, you'd want legally binding terms in the registration agreement stating that you are who you say you are, and that this is the only account you have on this site. Anyhow, the main point is not to tie registration to an e-mail address or an IP address.

Then, during the account process, public and private keys are created for the user, and used for authentication. The server would issue you a challenge, and you'd have to have your private key (and, of course, know your password) to log in. Your private key is like your debit card, except more secure--you need it and your password to verify your identity.

I think this would virtually solve (or regulate) the duplicate account "problem", and also make the authentication system significantly more secure than is standard today. It would also not require all communications to be encrypted, but naturally the users could encrypt things as needed. And it would add in real penalties for fraud (due to the use of a check or credit card, it could be considered mail or wire fraud, or otherwise breach of contract).

---

" See what the drooling, ravening, flesh-eating hordes^W^W^W^WKuro5hin.org readers have to say."

-- pwhysall

  • Good, but... by codemonkey_uk, 10/06/2003 06:15:10 PM EST ( 2.07 / 14)

    • yeah, by pb, 10/06/2003 07:00:49 PM EST ( 1.54 / 11)

I hope it works.

And to think, I was beinging to doubt in you. :)

Thanks, Rusty.

--- Thad " The most savage controversies are those about matters as to which there is no good evidence either way." - Bertrand Russell

on the other site hulver just invented the '4' rating

  • Rating numbers by ZorbaTHut, 10/06/2003 07:02:42 PM EST ( 2.42 / 28)

The people who complained about crapflooders all deserted you last month for HuSi. These changes will only alienate your most loyal users, the trolls a crapflooders. Please reconsider your changes, and perhaps even implement some to make K5 more crapflooder friendly. Thank you.

---

Stick, thine posts bring light to mine eyes, tingles to my loins. Yea, each moment I sit, my monitor before me, waiting, yearning, needing your prose to make the moment complete. - Joh3n

Six ratings before anything shows up? That seems a bit harsh, since usually when I'm reading most of the comments only have one or two ratings. I predict the threshold will have to come down some to be useful.

And for the hidden comments, how's this for a compromise: have an option to show 20% (or however many) of the "hidden" comments, you don't have to look at most of them, but everyone gets a chance to inspect a sample of hidden comments to make sure nothing good is getting thrown out. Of course with a six-rating minimum I wouldn't expect much malicious hiding to take place, but it might become more of an issue if you start making scores take effect after fewer ratings.

o corroded one!

rather than hack the rating system which just fiddles with some of the parameters, but doesn't solve the problem. why not give us a killfile? these retards are not going to go away. then when someone declares alleigance to clan retard, they can be plonked and the rest can carry on minus mc doofus.

what prevents a killfile? technical issues, or social ones?

one interesting thing about this, though. the rating system is/was a little game for k5'ers who were so inclined to play amongst themselves. pointless waste of time, but satisfyingly destructive. now you're taking away their favourite k5 toy. what are they going to do now? what's left to break? are you hoping they get bored and go away?

"you want enlightenment? stare into the sun."

ok, I am not angry with you any more, Mr. Foster ( 1.39 / 33) ( #44)

by mami on Mon Oct 06, 2003 at 06:19:46 PM EST

first time I understand what you are saying, I am pleased. :-)

What happens if I rate someone who got a 5 under the old system? Since "encourage" == 3, wouldn't that lower his score?

Information wants to be beer.

And I thought HuSi was dumb with the 6. Sheesh.

--It is impolite to tell a man who is carrying you on his shoulders that his head smells.

Is this a bug or is it just me? Many comments are showing things like "None / 4" as the rating. Shouldn't there be a number there?

--

jimmysquid.com - I take pictures.

  • Never mind by CaptainSuperBoy, 10/06/2003 06:40:58 PM EST ( 1.38 / 13)

"discourage" unhides comments ( 1.85 / 27) ( #54)

by jjayson on Mon Oct 06, 2003 at 06:32:31 PM EST

So under the old system there was a way to rate a comment to say that it sucked balls, but that you had no opinion on if it should be hidden or not, 1. Rating a comment with a 1 made no difference in if the comment was hidden. Nowever, however, anything besides a rating of "hide" is a vote to show the comment. There is no rating to express a neutrality of hiding the comment and by rating.

It doesn't make sense that rating comment with "discourage" should unhide the comment.

It used to break down roughly like this:

  1. - (hide) comment sucks hard and hide it

  2. - comment sucks hard, but I don't care if it is hiden

  3. - (discourage) comment suck, but I don't want it hidden

  4. - (neutral)

  5. - this rating was mostly useless

  6. - (encourage)

We have lost our 1 and 4 ratings, not our 2 and 4 rating.

\\\_____

Smile =)

Given the culinary lineage of its former colonial masters, America's "theft" of other nation's cuisines is considered by mo

I like these changes, mainly because it gets rid of the one blemish that's always annoyed me: Trusted Users.

TU Status was an attempt to draw a straight line through a statistical norm, and then add incentive for being on a certain side. Ultimately doomed when dealing with humans.

This new system puts the power where it belongs, in the full distribution of the users.

Rusty, two questions:

1. If you had a device that could detect specific physical computers (rather than IPs), thus eliminating multiple-account modbombing, would that restore your development of the mojo algorithm? (Such device mods do exist -- in Perl, no less.)

2. Can you elucidate the specific comment-sorting options that most fully take advantage of the new rating system? And in what way do these options differ from the default options?

Change is a good thing, generally.

\\\_____________________________________

Burgeoning technologies require outlaw zones... deliberately unsupervised playgrounds for technology itself. -- William Gibson

oh fantastic. ( 1.02 / 48) ( #60)

by rmg on Mon Oct 06, 2003 at 06:59:06 PM EST

i suppose you'll be anonymizing drduck now?

doesn't anybody else realize what awesome power he will possess now?

WON'T YOU PLEASE THINK ABOUT THE CHILDREN ???

\\\___ intellectual tiddlywinks

Great. ( 1.81 / 22) ( #67)

by i on Mon Oct 06, 2003 at 07:04:16 PM EST

I've shut down my proxy.

Alas, you seem to have display preferences broken. I can't set my font face and size to whatever values I choose. It keeps resetting each time. Pls fx k thx.

and we have a contradicton according to our assumptions and the factor theorem

I'm slightly curious - why three ratings, instead of four (if you exclude "Hide", since it isn't really a "normal" rating) ?

With a four-level rating system, you've got to indicate some level of positiveness / negativeness, or "encouragement"/"discouragement".

With a three-level rating system, the middle level (as the new text label suggests) means "I am neutral about this comment. So neutral, in fact, that I felt it worth clicking a couple of buttons to say just how neutral". So why bother at all ? The main occasion I can think of for using the "so neutral it hurts" rating (a "3" in the 1 - 5 rating system) is when I feel a comment has been unfairly rated so far, and I want to indicate that I don't think it's quite as good or as bad as its current rating suggests (and resisting the temptation to give it a score which it doesn't deserve, just to swing the balance the other way a little). Which also brings me to the second point.

I think six ratings before a rating is displayed is too much. In fact, I think it might be counter-productive, and reduce the amount of rating carried out. Surely many people aren't going to bother if they feel that no-one will ever see that rating, since the comment in question is unlikely ever to get the six ratings it needs to be displayed. Which can end up in a vicious circle of fewer ratings....

how will you react to someone who rates like this under the new scheme?  will you remove said persons rating abilities?

As someone who just had a story in the queue crapflooded and probably removed from autopost consideration because of it, I've got to say, thanks, but, the fact that K5 has attracted the trolls is probably a guarantee that no matter what system you come up with, there's always going to be those who'll get a kick out of trying to break it.

I've had a story with an 89 go down, another with a 55 go down, and while I can understand how the latter might not make it, the fact that another story of mine with a 42 went through just makes the whole submission process too chaotic for me to want to put anymore effort into a story for this website. If my story isn't going to go through because I put it in on a Friday instead of a Monday, or isn't going to go through because partykidd and circletimessquare are manifesting some vendetta against each other in the comments section, or isn't going to go through for any number of reasons that have nothing to do with the inherent quality of my story, well, it doesn't seem like it's worth to put in the several hours of effort with all the writing, rewriting, hyperlinking and revisioning in the diaries section.

And this isn't sour grapes for K5 rejecting a story. I've had K5 accept stories, one that went straight to 95 and another that fit the autopost criterion. But neither of those stories are the ones I'm proud of, and it's hard to be motivated to write for a site when it's very possible you won't be proud of the stuff that the site accepts.

For what it's worth, I'm not angry about it, either at K5 or at you in particular. I've just got other places I can write for. Maybe the problem at the heart of it is trying to come up with some algorithm to counteract human behaviour as it pertains to K5's format. I don't think any of us are sophisticated enough to dream up the appropriate AI for that.

Oh, and one more thing. ( 2.00 / 22) ( #79)

by i on Mon Oct 06, 2003 at 07:15:05 PM EST

Ratings still show up in searches. It looks like shutting down my proxy was a bit premature.

and we have a contradicton according to our assumptions and the factor theorem

I don't like rating labels. I don't like them in principle, but more importantly, I don't like this particular choice of labels, as I think it's somewhat misleading, given the practical effects of each rating option. Though I can sort of see how one might rationalize these particular choices for this particular system, they seem to me to be rather poor, given the numerical values behind them, and a "hide threshold" set at 1.

More specifically:

  • -1, "Hide" This is the least misleading label; it accurately describes the practical effect of the rating, ie: rater thinks comment should be hidden; rating influences outcome in that direction.

  • 1, "Discourage" "Discourage" just doesn't describe the effect of this rating at all. Since the "Hide" threshold is set at 1, the proper pragmatic interpretation for this rating is rater doesn't care one way or another whether comment it hidden, rating merely moves outcome closer to decision, in cases of comments with few (at present <6) ratings.

  • 2, "Neutral" This rating is hardly neutral, from a practical standpoint. Its effective meaning is rather rater thinks comment should remain visible; rating influences outcome to that end.

  • 3, "Encourage" Though "encourage" sounds nice, it doesn't quite describe the practical effect, to wit: precisely the same as "Neutral," except rater's action has greater influence on outcome.

Given these discrepancies beteween label and effect, I have a prediction. Now, since psychology famously trumps game theory in all sorts of situations, I may well be wrong about this, but I'm predicting users will largely ignore the "Discourage" and "Neutral" options, leaving us with a system in which pretty much everyone uses two ratings, with effective values of +2/-2.

The old system, by contrast, was much more nuanced from a game-theoretical perspective, given the interplay between TU status for high ratings and comment hiding for low.

One might also argue that ratings have an effect outside comment hiding, as there are various options to order comment display according to score, but given scoop's lack of pagination, and the near universal preference for some sort of threaded comment display, those ordering options don't really have any practical effect.

In a finicky Tufte-like aside, I'd also point out the fact that labels will be somewhat confusing to new users, as the link between comment score and rating option is no longer intuitive, but that doesn't bother me so much-- ordinary people manage to get along with non-intuitive interfaces every day, after all.

Trusted user was given out mainly to people who hung around on the site all the time. That was okay, but for the fact that a lot of people who hung around on this site all the time are trolls. A lot of well meaning people who just occasionally came by the site would be put off to find that the trolls have more moderation power than they did.

Using labels instead of numbers is better, not only because it's more intuitive, but because it reinforces the social implications of rating a comment. So if you "encourage" someone to post a stupid, contentless troll or "discourage" someone from making an intelligent post you realise what you are doing is wrong. It's harder than just giving someone a meaningless number.

The IP comment rating idea is good too. Although It's not hard to find anonymous proxies, it will make it just that much harder.

BTW, I'm relatively new to this site (although I have been lurking for quite a while). One thing that doesn't make much sense to me is the emphasis on comments rather than diaries or stories. You can rate comments but not diaries. Often I read a diary and want to "encourage" the author to write more, yet posting a "keep up the good work" comment is kind of redundant. Also, comment ratings are checked for duplicate IPs but not story votes. Personally I would think that story voting is more important to protect from cheating.

  • Exactly by leviramsey, 10/07/2003 04:16:42 AM EST ( 1.54 / 11)

I'm still a TU. what perks do I have now that the plebes can give the almighty "hide"?

Bug report... ( 2.11 / 26) ( #90)

by ana on Mon Oct 06, 2003 at 07:48:15 PM EST

As the token over-45 person here, I'd like to report that the "minimum font size" thing no longer works for me. I set it up to 3 or 4 and it's there for the one click (submit changes). Next click, it's back to 2 again. I'm using 2 different versions of Mozilla on Linux and Solaris.

I have trouble reading the itty bitty default font.

Years go by; will I still be waiting

for somebody else to understand?

--Tori Amos

You forgot something ( 1.78 / 19) ( #92)

by vayg on Mon Oct 06, 2003 at 07:59:26 PM EST

What about comments made before the change? Now I can at maximum vote them THREE! Having the TWO vastly different systems like that seems just silly, because I don't like duality.

An intellectual powerhouse

I have a problem with No. 2 ;-) ( 2.22 / 35) ( #95)

by Pluto on Mon Oct 06, 2003 at 08:06:47 PM EST

So, as a lab volunteer (K5 Member) I set out to rate some of the comments herein.

Here's my dilemma:

If I feel NEUTRAL about a comment (or dispassionate; or indifferent) I don't waste my time rating it.

If I take time to rate a comment, it means I FEEL something. Thus, a 2 rating with a NEUTRAL rating will not work.

Could we please rename the 2 rating?

How about one of the following labels: Interesting

Notable

Worthwhile

Contributory

Helpful

&c

\\\_____________________________________

Burgeoning technologies require outlaw zones... deliberately unsupervised playgrounds for technology itself. -- William Gibson

Why I have a problem with this. ( 2.28 / 39) ( #100)

by mcc on Mon Oct 06, 2003 at 08:14:07 PM EST

I'm not very happy with this becuase I never really paid any attention to the whole hidden/not hidden thing. What I did pay attention to, and what I always kind of considered to be the point of the rating system, was the "Sort by rating" feature. These changes seem to me to somewhat decrease the usefulness of "Sort by rating".

The first and lesser reason for this is the death of the 2 and 4 ratings. Lacking 2 and 4 makes steer voting much, much harder, or at least less meaningful. The thing is, though, I can't remember the last time I voted 2 or 4. So I guess I'll drop this complaint.

Most importantly though: the new ratings are, well, confusing. At the least, the Hide/Encourage/Discourage/whatever needs to go, or at least be replaced with an option to view the rating options as as -1/1/2/3 instead of those words. The first reason for this is that the words make steer voting just about impossible to think through. (If you multiply 2.34 by 5, subtract "discourage", and multiply by 6, what do you get?) The second is that they make it kind of hard to keep track in your mind of what the scores are. 2.14 is.. what? Generally discouraged? Generally encouraged? Generally neutral, what? The fact you vote with words but see scores as numbers makes there seem to be no connection between the two in the reader's mind. Maybe this is something you wanted to happen, but..

This is made much worse by the whole "ratings don't count unless there are six of them" thing. Whaaat? First off, I can't speak for anyone else, but I tend a lot of the time to not rate unless there is an existing rating and I disagree with it. Maybe this is something you want to change, and you want people to vote on everything, but I don't think "you can vote on this unrated comment, but unless 5 other people do also your rating will have no effect" is going to have all that much effect. Also, out of my last sixty comments, only seven have six or more ratings on them. The mode number of ratings seems to be two. Hmm. You might as well just ban comment rating voting from the diaries altogether, and ban them from any comment that isn't top-level in its thread, since it's relatively rare any of those will get the required six votes. I suspect the main consequence of this is that you will see a big spike in pageviews, since people will be constantly looking at the "ratings" page for every single one of their comments curious how people voted, since the ratings are not by default displayed on the normal pages.

Lastly, though this is not as big a deal, the fact we have a wider range and >=1 is "hide" makes it a bit harder to tell where hide is. 1 is a kind of an arbitrary line. Either move the hide threshold to 0, or somehow "mark" hidden posts for those of us who have them visible-- like, a little red line, a "(H)" badge, something, so we can see at a glance if it's hidden without having to do math. (Yes, it isn't much math to look at a post and see is this rating >1?, but thinking this out for every one of 127 comments in a story is too much effort...)

Anyway, if your point was to turn the ratings system from personal "ratings" into numbers tacked next to every post that are meaningless except as a hiding mechanism, well, i guess that makes sense as a goal and you've succeeded, but their use as a community feedback mechanism seems pretty heavily lessened. I think i've been kind of babbling, so to summarize, my requests are these:

  1. Give us the option to see "Hide/Discourage/etc" as "-1/1/2/3"
  2. Give us the option to see the rating, even if there haven't been many ratings
  3. (Less important) Give those who have hidden comments visible the option to have a small "hidden" badge on hidden comments.

---

Aside from that, the absurd meta-wankery of k5er-quoting sigs probably takes the cake. Especially when the quote itself is about k5. -- tsubame

  • Steer voting by Khalad, 10/07/2003 12:26:02 AM EST ( 2.23 / 17)

    • steer voting by crayz, 10/07/2003 04:33:29 AM EST ( 1.28 / 7)
  • A suggestion by flo, 10/07/2003 12:48:57 AM EST ( 1.54 / 11)

Not a single comment in this story managed to get above the 3.5 TU threshold. Lord that's some harsh rating, it's like DrDuck cloned himself and went wild.

On the serious side, does TU exist anymore and does it mean anything? Also, how do these changed effect the autoposting of stories?

I don't understand spending all that money for a fancy shot ... when pregnancy ain't nothing that a good coathanger or a pair of steel toed boots can't fix<

http://www.kuro5hin.org/story/2003/8/21/221644/233

i saw the future!

so follow me, everyone... you too rusty, as pied piper circletimessquare takes all you kids into the great troll cave where your parents will never see you again ;-P

i mean c'mon, turmeric's 3rd-to-last comment here before disappearing was to express pleasure with my guide (huh!?)... so read it fool! it must be special, for turmeric complimenting me is like me and drduck going to the prom together

in short: ratings are meaningless... ignore, them don't use them... only comment... it's a social site, so be social

at the very best, ratings to me are "metacomments", indicating a protoemotional state: 5=smiley, 1=frowny... me personally i mostly just gave people high 5s when i wanted to express pleasure with a comment they made but was too lazy to write "good post! (nt)"

to me, voting 1 on comments creates an unnecessary asocial vibe. so: why not have approval voting only?

in said hypothetical universe, instead of seeing comments with things like "4.55/27", or "2.67/4", you would see things like: "5" or "21"... meaning 5 people expressed approval for your comment, or 21 people expressed approval for your comment... there is no negative modding

you can still sort on quality comments, and the "bad" comments simply become the ignored ones... trolls are best ignored, no? so let's reflect that in the mod system

approval-only modding also removes ideology from the equation... a controversial, thought-provoking, contentious comment (the best of kuro5hin, in other words) will now rise to the top of the moderation, instead of getting something stupid like a score of "3.00/45"

there should still be -1 or 0 moderation though... nominate/ pick/ otherwise promote through some nefarious scheme to supermoderator status a few ironclad super-trustworthy users (definitely not me) to do with 0/ -1 moderating what it is REALLY intended for: racist/ sexist/ homophobic comments, spammy adverts, and crapflooding... that's really all you need to moderate negatively at all, and it's pretty clear cut when those 3 rules are broken

every other sort of 0/ -1, or even 1/ 2 moderation is emotionally superfluous, suspiciously ideological, and most definitely far from universally useful for all users according to his/ her opinions... so who needs them at all?

go all the way rusty! make kuro5hin what it is when it is at it's best: a totally social site, with no more asocial 1 modding and the bad vibes that goes with them (nobody loves you drduck)

switch to an approval-only moderating system

in my opinion rusty made a good first awkward baby step towards what i feel is inevitable and a vast improvement, so thanks! ;-)

I'm making a Low Budget HDV Filipino Horror Movie in NYC

  • xactly ! by fhotg, 10/07/2003 04:01:29 AM EST ( 1.70 / 10)

I think the words are misleading. There is one word that I really enjoy - kudize - that I would feel would work great on kuro5hin!

The words I would recommend are :

marginalize (hide)

discept

abstain

kudize

I really like the word kudize and feel it would go well with the name of the site. Discept is interesting because it means you disagree in an open talkative way. Astain means it all.

Anybody else think these are good? :-)

How are we supposed to get comments that have fallen below the Hide threshold aboce it, now? I can't rate hidden comments any more, there's no ratings box (as if I didn't have TU, but I still do).

--

"My life was more improved by a single use of [ecstasy] than someone's life is made worse by becoming a heroin addict." -- aphrael

I've got ratings hidden, but in threaded view, the parent post ratings are hidden, but all of the threaded comments have their ratings shown. That kind of makes my stoic refusal to look at comment ratings impossible - and if I were to increase the nested view over 200 (it's at >100 now), I'm afraid my browser would start doing funky things. It did before.

On the Internet, anyone can accuse you of being a dog.

  • Sorry by rusty, 10/07/2003 05:53:23 AM EST ( 1.50 / 8)

that you didn't manage to fix the sig bug.

I'd rate a lot more if there were radio buttons rather than a list. A list requires 2 mouse-clicks while radio buttons only require one.

The neutral button should be the default selection, and there wouldn't need to be a "none". This would make rating easy easy easy, and consequently many more people would do it. Which would be good.

Oh, and my font preferences in Display preferences are being ignored.

--

We brought the disasters. The alcohol. We committed the murders. - Paul Keating

I don't like the 1 vote per IP ( 2.07 / 27) ( #123)

by curien on Mon Oct 06, 2003 at 10:14:31 PM EST

No sir, not at all. And even with the rDNS lookup, how likely is it that "eagle1.langley.af.mil" will be on the whitelist?

I'd be perfectly happy with a blacklist of proxies, though.

I've said it before, and I'll say it again: the ratings system needs to weight each rating according to the rater's mojo. Also, have the validity of the ratings (as judged by similarity to other ratings on the same comments) affect the rater's mojo.

--

Screw teh tiger woods! I am teh Lunix Tarballs!

Is this intentional? All diaries older than about a year are inaccessible. And drduck of all people is listed as having written 16 diaries... when did this happen?

--

jimmysquid.com - I take pictures.

  • Me too by QuickFox, 10/07/2003 04:10:36 AM EST ( 1.12 / 8)

  • Bug by rusty, 10/07/2003 05:55:59 AM EST ( 1.27 / 11)

Interesting, very interesting. ( 1.52 / 17) ( #126)

by tiamat on Mon Oct 06, 2003 at 11:54:38 PM EST

I imagine a lot of people will have a problem with this because it is new, but I think the logic behind the idea is pretty solid.

One suggestion that I might make. People should always be able to "see" hidden comments if they follow a link directly to that comment. I know it's very annoying when people say "look at this bad rating over here: [link]", but I think it would be even more annoying if I had to keep switching settings to see the comments that people were linking to.

Mind you, I have no idea if that is at all possible.

P.S.

It took about 6 minutes to post this comment.... I hope that's not a systemic problem. . .

Fewer conversation-style threads. More Fark-like, less Usenet-like.

Top-level comments get rated more often than those buried deep in a discussion, so those who are chasing ratings will post responses to the article, rather than follow-up comments.

(Looks like the db is messed up, too, as someone else mentioned. My user page claims sixteen diaries when I've only written two.)

My user info page still tells me that I'm a trusted user in a way that makes it seem like I should care. . . .

Also, this ( http://www.kuro5hin.org/special/trusted) page should probably be updated. Overhauled, rather. It still talks about the old ratings system.

Two bugs with hidden comments ( 2.09 / 21) ( #139)

by Khalad on Tue Oct 07, 2003 at 12:52:14 AM EST

  1. I have "Show Hidden Comments" set to "No." When I try to review hidden comments and I click on a comment, it doesn't show up. I guess that setting should be temporarily overridden when I'm reviewing hidden comments.

  2. I rated a comment "Hide" and it hit the six-votes threshold. Since the comment was hidden, the comment failed to refresh. The "Loading" line just stayed there without indicating that the comment actually had been loaded and determined to be hidden.

You remind me why I still, deep in my bitter crusty broken heart, love K5. —rusty

I just experienced rating a comment with "Encourage", only to cause it to be hidden (thanks to it having 6 ratings). That's a little screwy, would you agree?

Also, I have to say that the new Hide is way too powerful. It used to take 4 "Hides" to negate one max rating. Now it takes one. You've made the hide rating 4 times as powerful as it used to be, it's just ridiculous. Your assertion that "most people have reserved their zeros for the real bottom of the barrel stuff" is plainly ridiculous, as you can see simply be viewing ratings attached to this story. Is a comment is not very good (but NOT crapflooding, etc.) and it's mainly getting 2s and 1s, the Hide rating becomes stupidly powerful, such that you could easily hide any "not great" comment with just one or two of them. Then the issue of "Hidden Comments" groupthink comes into the system. Most hidden comments in the old system that made it to that page got zeroed to hell--it's MUCH harder for a comment to get out of that page than it is to get in. Now, you've made it trivially easy for anyone to push a comment there and get it zeroed. It's seriously stupid. You need to consider lowering the hide threshold to 0.5 or something, or K5 is going to be hiding shit like crazy under this new system. You can see the change starting already in View Hidden Comments--I fully expect hidden comments in increase by at least five to ten times unless you fix the gross overpowering of the Hide rating.

--

"My life was more improved by a single use of [ecstasy] than someone's life is made worse by becoming a heroin addict." -- aphrael

  • Cry me a river by ucblockhead, 10/07/2003 01:13:09 AM EST ( 1.80 / 31)

  • yeah by reklaw, 10/07/2003 05:36:07 AM EST ( 1.50 / 8)

I would really like to see a way to moderate every comment in a story down using just one drop-down list and a button, instead of having to go through seventy-odd comments by hand. As  mark twain once said: fuck that

Also, have you considered the idea of personal, unique killfiles, that each user could use to block out offensive comments and diaries?

Lastly, are you going to take any measures to lure the members of HuSi back?

---

Thought of the week: There is no thought this week.

---

I haven't been able to rate comments for ages. I have no idea why. In light of that, why should I care?

Even though I consider myself a fair and balanced K5er, I'm only keeping my clicker finger off the Hide rating out of a thin sense of obligation and moral righteousness.  I agree that it's for the best that everyone can Hide and Review Hidden, but there's also no longer any reason for a rater not to dish out as many -1's as their withered ego warrants.

I'd like to suggest a limit to the number of -1's that can be applied by any one person.  Maybe per day, maybe per story - the number can be proportional to the full amount of comments that K5 is receiving.  It would follow that a user should have full reign over their own diary, though I can't see any reason why the limit should apply to diary comments anyway.

I just really think there needs to be some distinguishing factor between Hide and Discourage besides 'I don't like it' and 'I don't like it more'.

neutral - what does it mean? ( 2.08 / 35) ( #165)

by mami on Tue Oct 07, 2003 at 03:03:24 AM EST

If someone has a rating of 2.1/15 does that mean fifteen people feel neutral about the comment? Why would they rate in the first place?

That's like saying, I don't care for Arnold, so I vote neutral. But I am quite certain, the person, who doesn't care for Arnold, either doesn't vote at all out of protest or he votes for someone else than Arnold, to get Arnold into the hiding status.

Something doesn't make sense here.

I could say that a 2.1/15 rating means that that five people think the comment should be discouraged, five people think they have no view points on the comment and five people think it's an encouraging comment? Why would I be interested in knowing that?

If someone rates discourage, why wouldn't he rate -1, hide? Is discourage meant to be to say, hey buddy, if you continue talking nonsense, I will rate you -1? And encourage, should that mean, hey girl, go on, you made my day, gimme me more of those nice comments?

May be it would make sense, if we would know how many people voted discourage, how many voted neutral and how many voted encourage. And then, what does discourage really mean?

I don't want to discourage anybody! If you think something is funny, what I don't consider funny, should I discourage you from expressing what you consider funny, just because I have another sense of humor than you do? I don't think so. If you say something that I think is outrageously inappropriate, you get a -1 anyhow.

There is nothing encouraging in having a rating of 2.5/20 for example. There is something encouraging, if you had a 4.5/20 rating though. There is nothing discouraging in having a 1.8/20, it just shows that people don't know what to think about the comment.

You see, even if you don't care for the rating, it's kind of nice to get a five, at least you know there is at least ONE person out there who understands you, sigh, sigh ... Imagine you get five fives. That's a reason to get a beer and celebrate. Now everything is lukewarm.

I miss my fives. And I miss getting angry about a 1 rating, I didn't deserve.

This rating sceme it most probably going to be very boring in the end. It smells like politically correct or apolitical, or may be just irrelevant or may be just very uninteresting.

What this means in practical terms ( 1.27 / 18) ( #176)

by grouse on Tue Oct 07, 2003 at 04:01:11 AM EST

drduck need not ever post to K5, but he can still hide my post! You sad bastard!

"Grouse please don't take this the wrong way... To be quite frank, you are throwing my inner Chi out of its harmonious balance with nature." -- Tex Bigballs

Show the numbers with the labels.  It will also help old-timers get used to "3" being a good score

none

-----

encourage (3)

neutral (2)

discourage (1)

-----

hide (-1)

Can you still show the numbers? ( 1.89 / 19) ( #187)

by NMSpaz on Tue Oct 07, 2003 at 04:39:20 AM EST

I find it easier to quickly scan a number than read a word. Could you change the labels from "Hide", "Discourage", etc... to "-1, Hide", "1, Discourage" or something? It would make it that much easier to quickly rate comments.

Trusted User status made ratings political. Previously, I found it hard to 5 somebody no matter how good their comment if I knew they would use TU to 5 comments that should have stayed hidden. This meant that it was hard to rate a comment on content alone. I'm glad that aspect has been removed, and ratings will now have more meaning.

--

"But here's the thing: if people hand me ammunition, what kind of misanthrope would I be if I didn't use it?" - Sarah-Katherine

Since above 1 == shown, below 1 == hidden, then a rating of 1 should be "neutral", 2 should be "encourage", and 3 should be "strongly encourage". Or something like that.

eudas was found dead in his parents' basement early this morning. Cause of death appeared to be multiple self-inflicted gunshot wounds, confirmation pending a full autopsy report. Detectives at the scene also report that the victim had the numeral "2" carved into his chest.

With the vote-on-a-story menu, the user drags up to vote the story up. With the rate-a-comment menu, the user drags down to vote the comment up. Since the rate-a-comment menu is all new, why not make it work in the same direction as the vote-on-a-story menu? It might reduce the rate of mistakes.

The truth may be out there, but lies are inside your head.--Terry Pratchett

Here's a possibly better solution than waiting until a post has six ratings.

Just start each post off with a single vote at the average score -- now 2. If it receives two votes of 3, it will move up to 2.66 -- it will still be below a comment with three 3's (2.75).

The effect is that all ratings count, but the early ratings are "balanced" by the initial vote. (Statisticians can think about Bayesian methods at this point).

Only one rating will count from one IP (experimental) :(

I'm behind a corporate NAT which means that it appears that there is only one IP for the whole building (and there is more than one k5 user).

A comment's score doesn't count until there are at least six ratings contributing to it :(

Make it three, g'wan.

Rating scale now goes 1-3, with text labels Discourage, Neutral, and Encourage :(

Sometimes you want to do more than encourage people, you want to show them that you are Strongly encouraging them.

The rest is fine...

"Let us kill the English, their concept of individual rights might undermine the power of our beloved tyrants!!" - Lisa Simpson [ -1.50 / -7.74]

More freedom for all: good.

Although I personally am sad that I can't artificially inflate my ego with Trusted User Status anymore. I might have to start doing useful things for society. How could you? HOW COULD YOU?

"Blessed are the cracked, for they let in the light." - Spike Milligan

Can't we just "ignore UID"?

If I could just ignore the "Living Incarnation of Pure Evil", then I'd be happy.

--

"Jumpin Jesus H. Christ riding a segway with a little fruity 1 pint bucket of Ben and Jerry's rainbow fairy-berry crunch in his hand." --

  • no kidding by Wah, 10/07/2003 07:25:02 PM EST ( 1.42 / 7)

labels like "discourage", "encourage" might make newbies think the point of rating is to quash or reward comments based on the views they express (as opposed to the quality of the expression and its appropriateness to the discussion). I really like most of these changes, though I worry about how my logging on from multiple computers with multiple IP's might alter my rating of others.

Btw, the only real complaint I have is, now Osama Bin Fabulous can Hide all my comments instead of just 1'ing them all.

Thus, a suggestion for a system to prevent no-comment no-story crapraters from hiding comments:

A new user account has zero "points". Every time one of your posts hits 2+/6 (that is, it gets its sixth rating and its total is at least 2.00), you gain 1 point. Every time a story you submit posts, you gain 5 points. Comments in Diaries, and "posting" of Diaries, will not give points. You can only have up to 25 points maximum, any beyond that are lost. It costs 1 point to "hide" rate (all other ratings are free to apply.

Opinions?

-Kasreyn

"Extenuating circumstance to be mentioned on Judgement Day:

We never asked to be born in the first place."

R.I.P. Kurt. You will be missed.

I don't understand why the button label is "Rate All" when you're only rating a single comment. When I first started reading Kuro5hin, I figured that meant you were rating that entire thread of comments, as opposed to the single comment.

I understand how it works, now, but the label is horribly unintuitive, if you think about it.

There was a comment on #scoop a few days ago that you were going to add a line showing the hash of a poster's IP addy. Any chance of seeing this code anytime soon?

By reading this message you've unwittingly exposed yourself to my powerful, moth-like pheremones.

Just a few ideas in no particular order:

  1. As others have said, hiding the result until after six ratings is a bit harsh. The main problem I can think of with this is that top level comments are rated way more frequently than lower level comments. This can be explained by assuming that people just stop reading the threads very quickly once it becomes clear they aren't very interesting to them.

You might want to implement a variable treshold, say 6 ratings for top level comments, 3 ratings for second level comments and 1 for third level etc.

  1. One thing I've always thought would be more useful than the (score/num_votes) display is a confidence interval. The score is a sample average, and basic statistics gives a simple formula for the sample variance. You could have a scoring display of [lower_limit, upper_limit], with 95% confidence, or 80% confidence to make it more exciting.

The advantage of such a display is that usually, the interval would simply shrink over time (which is very intuitive). I expect it would also be satisfying for the voter to see his vote having a small but perceptible effect. This is important because no-one votes if they don't see their vote having an effect.

I'm just slightly worried that an interval display is less obvious to someone who doesn't know what he's looking at. But then, the (score/num_votes) isn't very obvious either to a n00b.

Having an interval display doesn't stop you from using the sample average (ie score) to do the actual comment ranking. It's just a more informative display, as it indicates how confident the statistics is about the upper/lower bounds.

Frankly, this one dropdown menu to select the rating seems a little klunky to me. Anyone else familiar with the rating system on Netflix? There is a set of 5 stars. Just click on one of them and the item is rated. The page isn't submitted conventionally so there's no waiting for it to POST. Instead of actually submitting the current page it tries a few different JavaScript methods in the background to submit the rating. Of course this would cause problems for people who have JS turned off, it shouldn't be too difficult to detect and revert back to the current behavior.

I know I would rate a lot more of the comments if it only took one click. Radio buttons would be better as well, but I like the above idea the best.

I'm Pleasantly Surprised ( 1.09 / 11) ( #239)

by Ruidh on Tue Oct 07, 2003 at 10:37:08 PM EST

The last time we had a discussion of rating, I advocated a very similar system as I felt that it was too easy for hidden comments to be unhidden.

I've been thinking about what system to use for an upcoming project and Scoop has rejoined consideration with these changes. It'll be interesting to see whether the new system improves the content of the site.

"Laissez-faire is a French term commonly interpreted by Conservatives to mean 'lazy fairy,' which is the belief that if governments are lazy enough, the Good Fairy will come down from heaven and do all their work for them."

Basically, right now all scores are strictly good for is hiding a comment, or not hiding a comment.  Beyond that they're not good for much.

I don't have much time to read K5, I would like to be able to use scores on comments to do a quick filtering.  I'm aware of the sort by rating option, but that loses all the structure of the body of comments.  What I'd really like would be if the rating system were integrated with the threaded/dynamic-threaded systems.  It'd be super nice if I could do something like hidden below 1, collapsed 1-2, expanded 2-3.  That would help a lot in quickly seperating the wheat from the chaff when I do get a chance to read the comments on a story.

Quaternary, cool! ( 1.07 / 13) ( #242)

by Fen on Tue Oct 07, 2003 at 11:10:25 PM EST

Power of 2. Radix of DNA. Of the form a_1=2, a_n=2^a_(n-1) (like hexadecimal). Never liked the 1-5 thing.

--Self.

This means that you will now be storing an IP address along with a user ID in the Scoop database. I dunno about you, but that raises a privacy concern for me. I'm not a yankee, but with your messed up DMCA, copyright and homeland security laws, you don't really want to be keeping that kind of info. Contrast this with slashdot.

I might as well use my full name and place my home address and telephone nymber into my .sig if you go ahead with this. Perhaps you could time-limit the IP storage, and store IPs for a week or two at most. I doubt many stories get significant comment moderation after a few weeks, and you can still have Scoop track clusters of moderations - you could easily track modstorms and ban users if you see five mods on a month-old comment within the space of a few hours originating from the same IP.

BTW, most k5 stories have sucked for about a year now - when are you going to fix story moderation - give science and tech stories a lower threshold, and make it damn near impossible to post the usual political / flamebait wanking which has occupied the front page (if I see another story about Michael Moore I'm gonna puke).

  • That by djotto, 10/07/2003 11:53:54 PM EST ( 1.80 / 10)

    • So? by bigbird, 10/08/2003 12:16:06 AM EST ( 1.77 / 9)
  • I agree by FlipFlop, 10/08/2003 12:54:22 AM EST ( 1.80 / 10)

been around with the new comments for about an hour or so. reading some of the comments below (and rating to do my little part), i realized i hadn't given any 2's yet.

it's going to take a little getting used to, and i really should re-read the main news (i'm a skimmer sometimes, sorry ;), but so far so good, imho...

kpaul media

I have "Show hidden comments?" set to "No".

I go to "Review Hidden Comments", click on a comment, and see... nothing.

Either

a) I've specifically asked for that comment, and should be shown it regardless of preferences

or

b) I should not be able to see the "Review Hidden Comments" link, because I can't do anything with it.

(a) is a better design choice, but (b) would be easier to code.

  • This one seems simple by supahmowza, 10/08/2003 12:53:44 PM EST ( 1.14 / 7)

    • Re by djotto, 10/08/2003 02:00:36 PM EST ( 1.16 / 6)

    • Lame. by aphrael, 10/08/2003 05:26:32 PM EST ( 1.87 / 8)

      • Better yet... by RobotSlave, 10/08/2003 07:20:57 PM EST ( none / 4)
  • Uh no... by Eater, 10/08/2003 05:01:20 PM EST ( 1.50 / 6)

it looks like neurological synapsis ( 1.05 / 17) ( #263)

by chanio on Wed Oct 08, 2003 at 02:29:49 AM EST

Isn't it looking like the way that the brain cells work?

I read it somewhere.

\\\______________

Farenheit Binman:

This worlds culture is throwing away-burning thousands of useful concepts because they don't fit in their commercial frame.

My chance of becoming intelligent!

dynamic mode ratings: feature request ( 2.12 / 16) ( #279)

by glor on Wed Oct 08, 2003 at 01:38:41 PM EST

Rating a comment in dynamic mode scrolls the comment up, then scrolls it back down with the new rating. This takes a long time and moves the text on the rest of the page. Can there be an option to "rate&toggle", since upon rating you're probably done reading the comment anyway?

--

Disclaimer: I am not the most intelligent kuron.

Why don't you put an MD5 or SHA sum of the IP instead of the actual IP into the database... that way there are no privacy concerns, and you can still only get one post per IP

  • Don't Bother by endeavor, 10/08/2003 06:26:38 PM EST ( 1.37 / 8)

    • One word. by i, 10/09/2003 02:26:01 AM EST ( 2.10 / 10)

      • 'n Peppa by endeavor, 10/09/2003 11:05:20 PM EST ( none / 4)

        • Huh? by i, 10/10/2003 05:51:27 PM EST ( none / 5)

          • Well... by endeavor, 10/11/2003 03:14:05 PM EST ( none / 2)

            • Well. by i, 10/11/2003 05:21:09 PM EST ( none / 2)
  • That would be stupid by Funk Soul Hacker, 10/10/2003 02:31:25 AM EST ( 1.12 / 8)

    • Here. by i, 10/10/2003 06:09:44 PM EST ( none / 5)

I just read the first comment attached to this story, rated 2.88 - I wondered what was wrong with it until I realized 3 is the top of the heap now, not 5. That's going to take some getting used to.

-- 散弾銃でおうがいして ください

I'm gonna have a hard time considering a 2.00-rated comment as "average" instead of "crap". As far as what Dusty wrote, i liked the changes.

If you disagree post, don't moderate.

  • Oh, btw, by Vesperto, 10/08/2003 06:51:56 PM EST ( 1.16 / 6)

    • btw part 2 by Vesperto, 10/08/2003 06:58:57 PM EST ( 1.87 / 8)

Actually this is kind of a good point. I think that if a comment has a weighted score low enough that it cannot possibly be displayed when the 6 comment count is reached, it should be hidden pre-emptively. That way Mr. Moore will not have to wait so long to have his comments pushed out of the public view.

----

The ultimate plays for Madden 2003 and Madden 2004<

Rating numbers are an implementation detail, and you should banish them altogether. In lieu of the traditional "1.23 / 4", use a little icon to indicate the rating level of the thread:

  • a "+", green light, happy face, etc. for encouraged comments (say, 2.25 or higher);

  • a "o", grey light, blank face, etc. for neutral comments (1.75 to 2.25);

  • a "-", red light, sad face, etc. for discouraged comments (1 to 1.75);

  • and a "--", brown turd, crying face or whatever for hidden comments (-1 to 1).

I suppose you can keep the rating counts for fun, but the decimal ratings make the site feel more like a spreadsheet than a discussion. In the same vein, remove the little parenthetical numerals from the "Encourage" "Discourage""Hide" menus. They look clumsy and detract from the simplification you're trying to accomplish.

--

How long must I travel on

to be just where you are?

I have a feature request.  I'd like to be a able to sort comments by the number of times they've been rated (the second number).  I think the most interesting comments are the ones which have recieved the most ratings, not necessarily the highest.  And yes, I am being serious.

I'm like Jesus, only better.

Democracy Now! - your daily, uncensored, corporate-free grassroots news hour

Is the point of the neutral rating only to force the "hide" rated comments to go away if one has chosen that setting? This would be like voting "0: Don't care" on an article submission; just to push it out of the "new" state in the queue.

Or is there some other reason that I missed in the write-up. Oh, and one more thing. "Executive Summaries" go at the beginning of the article. It'd be easier to read the whole thing, if I knew going in if there was anything of relevance to me up front. "Chapter Summaries", at the end, are for kids. :)

[TINK5C] | "Is K5 my kapusta intellectual teddy bear?" | "Yes"

What about all those dialup users out there? Or people posting from the one organisation that's using NAT and a smaller pool of IP addresses to conserve external IP addresses?

Bad idea. Better idea? Show the time that something was rated along with the IP address of the person who rated. People can then work out who's doing modbombing so they can then alert [email protected]... or something along those lines.

Everything else is good though. ---

I Hate Jesus: -1: Bible thumper

kpaul: YAAT. YHL. HAND. btw, YAHWEH wins ;) [mt]

And start again, that's the only way to do anything...

No work.

10+ neutral ratings to override one hide is ridiculous. The fact that 1 hide vote and 3 discourages, don't hide it, but the fifth vote, an "encourage" hides it is bad logic.

I'd like to see a new rating called unhide which would mean: ``I don't want to encourage the poster, maybe I don't like what they're saying, but I think it's improperly hidden and want to unhide it''.

As far as 'moderation score' 'unhide' would have the same effect as a neutral, and 'hide' should have the same effect as a 'discourage'.

And whether a post is hidden or not is a different decision determined by proportion of "hide votes" to "encourages and unhide votes"

It's true that if you want to hide a post, then you wish to discourage those sort of posts, but it's not true that if you want to discourage a post that you feel it merits being hidden.

It's true that if you want to encourage a post, then you don't want it hidden (that wouldn't encourage the post), but it's not true that not wanting it hidden means you want to encourage the post.

It sure was a lot simpler when we could just assign a number from 0 to 5, or whatever, based on where we felt the post should be sorted based on the thoughtfulness of the content.

Now I need to worry about whether I want to "encourage" or "discourage", the useful sorting element is gone with the 5+ ratings need, and my "encourage" votes may actually discourage (If my moderation is the one that makes the comment reach the number votes to get a bad score), vice-versa, my "discourage" vote may be the fifth vote (to a post with 4 discourages)

Bleah, it seems a lot simpler now for me to just not bother moderating and turn off sort by moderation, on the basis that it's all confounded.

-Mysidia the insane @k5

Most of the problems you mentioned were regarding user accounts and people's own petty rivalries. The ideal solution would be to simply not display the author of a post.

Completely erradicate authors entirely and use IPs to determine how many times users may vote, rate or post in one day. Nobody will know who posted what!

Beauty in chaos.

Grrr....

w O / Your majesty, your humble supplicant ||V       beseeches an amnesty for those who ||_ abused the old rating system. |  | =====       O__ =======     V/ /\_

  1.  What does it mean when a comment is rated (none/5)?

  2.  Why is it "0, Hide" instead of "-1, Hide?"

why?

kur05hin is banned...

kwizine.net is also banned...

https://www.collaborativemedia.org is not, but when I am trying to login, I cant, so I can only read k5 and not comment it...

:-(

Ratings: -3,3,6,9.

(6/1)

Should a comment's unofficial score be shown in a light gray font or something?

(_/1)

- I'm trying to be more explicative -

Anonyminity, why? And other ideas.... ( 1.66 / 3) ( #349)

by adharma on Tue Oct 28, 2003 at 07:43:16 PM EST

Firstly, articles have an author. Comments do as well. Why not ratings? Track user ratings of both articles and comments. This holds everyone personally accountable. I would go further and offer the abiltiy to let a ratings author type a few words about their decision.

About IP tracking... I would limit the IP to an arbitrary number user accounts in regards to rating a comment or article in any 36 hour period. If a user wanted the ability to forgo the IP check, they should purchase a paid membership. Without question the IP's should be strongly encrypted.

About scale...-1,0,+1 vote yes, no or abstain. This is simple and effective. Indeed, it does not give the Rater the ability to grade their opinion, but with a the implementation mentioned in the first paragraph, it offers the opportunity for someone to explain.

About filtering...Give users a watermark or threshold for display of content. For example, "Show all comments posted within the last X minutes(or hours) AND any rated above Y AND any rated below Z". I would also go so far as to let paid members set their preferences based on Topic. (Science, Politics, Diary,etc...)

Final note.... Any system used will have its pros and cons. In my opinion, the simplest systems and the most public systems ahve the best shot if being rectified.

Take care.

-I have accquired quite a taste for a well-made mistake-