OLDC Rules Changes

[Mystic]

As many of you already know, FuriousFox has fallen off the internet. As such we are going to be implementing a few changes to the OLDC rules while we're shifting to a new maintainer. Check the announcement for the full new rules text, as it contains a lot of wording changes, but the functional changes are listed below:

1. The judging period is now two weeks. We've traditionally had a lot of late voting because some people simply don't have the ability to play the instant they come out and/or are stuck away from their computer during the first weekend where the OLDC is available. This should help alleviate that problem.
2. The judging deadline is now serious. Votes made after the deadline, even if only a minute after the mark, will not count. This means that theoretically anyone can calculate the results and they'll be the same as the official results presuming their math is correct. Hopefully this will prevent the eternal wait for official word in the future should our maintainers fall off the internet.
3. With the deadline actually meaning something, the exact time was clarified, as previously the dates of United States daylight saving time have caused problems with our foreign users because they're different from the rest of the world. The time is now explicitly said to be local time in the eastern time zone of the United States, not EST, as that doesn't take DST into account. For reference for those of you not in the United States, DST is from the second Sunday in March to the first Sunday in November.
4. Single player stages now explicitly need to be on MAP01 or load to the stage when a new game is started. This should hopefully fix a lot of annoyance while judging them.
5. For votes to count, a voter now must vote on all maps in a division (except their own). While we understand this may cause voting to be more work, there have been many examples in the past of people voting for one or two maps with abnormally high or low scores and skewing all of the results. This will mean that people with a high or low streak will affect all of the maps equally, instead of just a few.

 

[MascaraSnake]

For votes to count, a voter now must vote on all maps in a division (except their own). While we understand this may cause voting to be more work, there have been many examples in the past of people voting for one or two maps with abnormally high or low scores and skewing all of the results. This will mean that people with a high or low streak will affect all of the maps equally, instead of just a few.

Why is this not mentioned in the rules topic? Also, I assume the score dropping will persist, only that the creators can only vote for all or no maps now. Also, does that mean that if you didn't play one maps of one category in a netgame, you can't vote on the other maps of that category at all? That seems kinda strange to me. At least, in that case, people should be allowed to evaluate the map in question offline.

As I said in the other topic, I still think imposing a strict deadline does more harm than good. A more logical rule would be that once anyone (staff or not) makes a tally, voting is over (of course, only after the official deadline). That way, there would be no confusion over which votes count and which don't and we would still include any votes that might be late.

 

[JEV3]

If we want a number of people to be able to tally the votes and had the deadline depend on when they make the tally, you've essentially got as many deadlines as you have people able to count the votes. The deadline has been extended a week, and we rarely get votes further than a week after the deadline so I wouldn't be too worried.

With the number of maps we've been getting in each division, there are few enough maps to play that you can probably play and evaluate them several times over apiece before voting and it'd take no longer than to go through all of the maps in some of the older contests once. If we start getting ten maps to a division, then we might have a problem.
Additionally, if you can't vote but you have comments, you can always review the level without rating them.

 

[MascaraSnake]

If we want a number of people to be able to tally the votes and had the deadline depend on when they make the tally, you've essentially got as many deadlines as you have people able to count the votes. The deadline has been extended a week, and we rarely get votes further than a week after the deadline so I wouldn't be too worried.

No, the voting would end at the regular deadline, but any vote that was published before the first person made a tally would be counted as well. The same as it is with the submitted maps: If you submit after the deadline, there's no guarantee your vote will get in, but if you post it before someone starts making the tally, he might just as well count it. You're probably right that it won't be a huge issue with two weeks, but it kinda bugged me this time. We'll see.

With the number of maps we've been getting in each division, there are few enough maps to play that you can probably play and evaluate them several times over apiece before voting and it'd take no longer than to go through all of the maps in some of the older contests once. If we start getting ten maps to a division, then we might have a problem.

The reason that I didn't vote for multiplayer this time was that I didn't catch one single netgame. I've seen it happen before that people only caught one game and weren't able to play all maps properly. What I'm pointing at is that these people should be able to evaluate such maps without having played a real netgame (as long as they don't start talking shit).

Additionally, if you can't vote but you have comments, you can always review the level without rating them.

That's of course the best option under the current configuration, but it would be kinda sad for the ratings that the person could have given otherwise.

 

[Mr. Mystery]

As I said in the other topic, I still think imposing a strict deadline does more harm than good.

Go look at the pitiful number of votes in Match and CTF last contest, much less the ones that were counted. If anything, having a serious deadline should motivate people to vote on time if they can; and thus to do that they should play more netgames with the MP pack.

It would only help, incidentally, if the OLDC room was put down for good, because seriously - ask any of us, we don't use it and nobody bothers to look in there.

What I'm pointing at is that these people should be able to evaluate such maps without having played a real netgame (as long as they don't start talking shit).

Which defeats the point of the MP pack containing maps for multiplayer.

Once concern has occured to me about rule #5 though: if someone declines to vote on one map for some reason related to the map itself (ie, couldn't complete a lap on a circuit, ect.), do their votes for the division still get excluded anyway?

 

[MascaraSnake]

Go look at the pitiful number of votes in Match and CTF last contest, much less the ones that were counted. If anything, having a serious deadline should motivate people to vote on time if they can; and thus to do that they should play more netgames with the MP pack.

Look at how many MP votes came too late and weren't counted. If it wasn't for that, we'd have a decent number of votes for all gametypes. Having a serious deadline will motivate nobody, if anything it will discourage people who are beyond the deadline to even play.

It would only help, incidentally, if the OLDC room was put down for good, because seriously - ask any of us, we don't use it and nobody bothers to look in there.

How would that help? It certainly does no harm, and I have no idea why everybody is so stupid and doesn't use it.

Which defeats the point of the MP pack containing maps for multiplayer.

How does that defeat the point? The people in question won't play the map either way, if they may vote or not.

Once concern has occured to me about rule #5 though: if someone declines to vote on one map for some reason related to the map itself (ie, couldn't complete a lap on a circuit, ect.), do their votes for the division still get excluded anyway?

That's why you try your best to finish a lap, and if you don't manage it, you vote on what you've seen. It's the fault of the mapper to make a circuit map so ridiculously hard that you can't finish a lap.

 

[Ezer'Arch]

It's the fault of the mapper to make a circuit map so ridiculously hard that you can't finish a lap.

... and still wins the contest because he was the only who submitted a map for that division.

My point: an OLDC division will have judging only if there are at least 2 valid maps submitted. If there's no enough maps for said division, the submitter must wait for next OLDC.

The time is now explicitly said to be local time in the eastern time zone of the United States, not EST, as that doesn't take DST into account.

This means who doesn't live in USA and other DST-regions in North hemisphere must now vote 1 hour earlier. The problem was minimized but potentially persists, though.

What is the possibility of using UTC (official world's time)?

 

[Mystic]

Why is this not mentioned in the rules topic? Also, I assume the score dropping will persist, only that the creators can only vote for all or no maps now. Also, does that mean that if you didn't play one maps of one category in a netgame, you can't vote on the other maps of that category at all? That seems kinda strange to me. At least, in that case, people should be allowed to evaluate the map in question offline.

It's not mentioned in the rules topic; it's mentioned in the new voting topic text because I figured it was more important to mention it there, since our OLDC entry rules topic is long enough as it is. I also rewrote the voting topic text, if you guys want, here you go:

Contest voting topic generic draft to account for the changes:

Welcome to the {date} SRB2 Official Level Design Contest voting topic! In this contest we have {#} single player, {#} match, {#} capture the flag, and {#} circuit entries.

To vote, play a few games of each stage in a division in the intended gametype for judging, and give each map in the division a vote from 0 to 10, no decimals and with 5 as average. You can not vote on your own map, but if you vote on the rest of the stages in a division the lowest score on your map for that division will be dropped. For votes to count, the voter must vote on every map in a division (excluding their own entry if they have one). Votes where the voter obviously did not play the level in its intended gametype, or where the voter did not play the multiplayer maps with other real players will be discarded. Please be as unbiased as possible for the author or the look of the map. It's all about how good the map plays, nothing else. For reference, the scale is as follows:

10 - Epic
9 - Awesome
8 - Great
7 - Good
6 - Decent
5 - Average
4 - Mediocre
3 - Bad
2 - Awful
1 - Horrendous
0 - Unplayable

Although it's not required for votes to count, it's very helpful to make comments about what you liked and didn't like about the map, so the author can improve his skill for future attempts. If your votes are dramatically different from the rest and you don't describe why, you may be asked to justify your reasoning.

Judging will end {date} at 7 PM local time in the eastern time zone of the United States. Like the entry deadline, this counts the United States's daylight savings time between March and November. Votes submitted after this time by the forum timestamp will NOT be counted in the average, so if you want to be counted, don't vote at the last minute. After the deadline, the scores will be then averaged, and the person with the highest score in each division is the winner.

{Contest map listing and links to the files}

As a final reminder, please make sure to play the levels in the intended gametype, and above all, have fun judging!

Again, the main reason for this change is because we've had a long history of people voting extremely high or extremely low on a single map and skewing the results. I really don't appreciate how some newbie with no understanding of our voting system can come in, vote a 9 or 10 on one or two maps, and then skew the results immensely because he only voted on a few of them. Any map he didn't vote on will generally lose because the generous voter wasn't voting on their map. This has been irking me for years and while I definitely agree it sucks in some situations (I immediately think of the times where we had over 20 match stages), it should help in the long run produce more fair results.

Also, as a final reminder, this is only for getting the results to count for the purposes of the final score of the contest. This is NOT saying you aren't allowed to give a score for what you did get to play to help the authors make better work next time. It only says that your score won't count in the average at the end of the contest.

As I said in the other topic, I still think imposing a strict deadline does more harm than good. A more logical rule would be that once anyone (staff or not) makes a tally, voting is over (of course, only after the official deadline). That way, there would be no confusion over which votes count and which don't and we would still include any votes that might be late.

We just had a perfect example of why there should be a strict voting deadline. If we had a strict voting deadline in the last contest it would have been unquestionable who won and we wouldn't have needed to wait a month and a half for results. Anyone could have done the math the minute the voting was over and calculated the results. If you want your vote to count, you have TWO FULL WEEKS now to get it in. I really don't see a reason why anyone would need to worry about getting their votes in late.

There's also an obvious problem with your plan. If someone votes while people are doing mathematics, does it count? This makes it unquestionable what the accurate and correct results are. This isn't to say people couldn't give reviews and feedback after the deadline, it simply wouldn't count for the purposes of determining the winner of the contest.

Look at how many MP votes came too late and weren't counted. If it wasn't for that, we'd have a decent number of votes for all gametypes. Having a serious deadline will motivate nobody, if anything it will discourage people who are beyond the deadline to even play.

This is better than having people voting 3 weeks after the deadline with the written explicit purpose of making me win the match division. That should never, ever happen, and I really don't like the idea of someone noticing a gap in the rules and exploiting it to affect the results. If they want to vote, they should do it in the voting period, not look at the math after the contest voting is over, notice the map they wanted to win didn't win, and quickly throw in their own votes to fix it.

Once concern has occured to me about rule #5 though: if someone declines to vote on one map for some reason related to the map itself (ie, couldn't complete a lap on a circuit, ect.), do their votes for the division still get excluded anyway?

They would. They will need to come up with some kind of numerical opinion on their experience. Did they enjoy it despite being unable to complete it? What is a completely miserable experience that they want to forget? Surely they have an opinion despite being unable to finish, and that opinion is perfectly valid.

My point: an OLDC division will have judging only if there are at least 2 valid maps submitted. If there's no enough maps for said division, the submitter must wait for next OLDC.

The problem is that while the OLDC is a contest, it is MORE a method of getting feedback on your creative work. I would rather an entry auto-win and get feedback now than have to wait another two months for the author to get criticism.

This means who doesn't live in USA and other DST-regions in North hemisphere must now vote 1 hour earlier. The problem was minimized but potentially persists, though.

What is the possibility of using UTC (official world's time)?

The thing is, this actually ISN'T a functional change, it's just finally worded correctly. It USED to just say EST, and that has been wrong the entire time the contest has been running. As an aid I did actually list the difference from UTC for each deadline date involved, which should help a lot.

The reason I'd like to keep DST in there is because it would confuse LESS people to have it there, since a majority of our users are based in the United States. However, I do understand the issue and wanted to make it clear for those of you not in the United States exactly what the deadline times are so there isn't any confusion in the future, as we HAVE had a user lose their entry in the past due to that hour-long gap and have to wait 2 months.

 

[MascaraSnake]

It's not mentioned in the rules topic; it's mentioned in the new voting topic text because I figured it was more important to mention it there, since our OLDC entry rules topic is long enough as it is.

Make a submission and a voting rules topic or something. Makes things easier to have all rules in a stickied topic (or two).

We just had a perfect example of why there should be a strict voting deadline. If we had a strict voting deadline in the last contest it would have been unquestionable who won and we wouldn't have needed to wait a month and a half for results. Anyone could have done the math the minute the voting was over and calculated the results. If you want your vote to count, you have TWO FULL WEEKS now to get it in. I really don't see a reason why anyone would need to worry about getting their votes in late.

In retrospect, I should not have included KOTE's votes, but otherwise, I think counting everything before my post was just the right thing. Otherwise, I would have ignored perfectly fine scores and opinion. Sure, it's probably not going to happen with two weeks time, but I'm not quite content with retroactively imposing the new rule. Absolutely nobody would have been hurt if the scores would have been included (except the people who are not winners but wouldn't have been otherwise, but that applies the other way round too).

There's also an obvious problem with your plan. If someone votes while people are doing mathematics, does it count? This makes it unquestionable what the accurate and correct results are. This isn't to say people couldn't give reviews and feedback after the deadline, it simply wouldn't count for the purposes of determining the winner of the contest.

The first person to make a tally includes all the votes he saw. If somebody posted in the meantime, tough luck. Then again, that might be prone to manipulation because anybody could do it.

This is better than having people voting 3 weeks after the deadline with the written explicit purpose of making me win the match division.

That happened? Well anyway, as I said nobody would be able to vote once somebody has posted the results.


Still, what about the "no netgame" thing? Would you be allowed to grade a map if you didn't get the chance to play it online?

 

[Mystic]

In retrospect, I should not have included KOTE's votes, but otherwise, I think counting everything before my post was just the right thing. Otherwise, I would have ignored perfectly fine scores and opinion. Sure, it's probably not going to happen with two weeks time, but I'm not quite content with retroactively imposing the new rule. Absolutely nobody would have been hurt if the scores would have been included (except the people who are not winners but wouldn't have been otherwise, but that applies the other way round too).

I retroactively applied it because technically the rules always stated it that way, it's just never been ENFORCED that way and in this case it really mattered when determining the results. The topic always said "Judging will end January 8th at midnight GMT (January 7th, 7:00 PM EST). The scores will be then averaged, and the person with the highest score is the winner." If you average it at that time, the results are what we posted. I'd much rather that than KOTE determining he liked my map better and intentionally breaking the results =P

The first person to make a tally includes all the votes he saw. If somebody posted in the meantime, tough luck. Then again, that might be prone to manipulation because anybody could do it.

The whole point of changing the rules like this is so there is never any question again. Whatever the math says is the result. I am tired of counting hanging chads.

That happened? Well anyway, as I said nobody would be able to vote once somebody has posted the results.

KOTE just did it =P

Still, what about the "no netgame" thing? Would you be allowed to grade a map if you didn't get the chance to play it online?

Absolutely not. I understand there were a lack of netgames for this contest, but part of that is because I wasn't around for the contest this time and part of it is a clear lack of interest from the rest of the community. I know this may be annoying, but SOMEONE needs to set up a game so you can play these in an actual game, because we shouldn't be voting based on looks, we should be voting on how they actually PLAY, and while it's possible to determine half of that without actually playing it in a game proper, I know full well that it's basically impossible to know exactly what the choke points and heavy traffic areas will be without actually trying it, and that makes a HUGE amount of difference in the quality of the map.

 

[Sonict]

Maybe what we do to prevent people from editing their posts after the deadline is maybe set it up so when it is made, in two weeks, users other than administrators and mods can edit their posts in the topic. I'm sure it can be done somehow...

 

[Scizor300]

I understand there were a lack of netgames for this contest, but part of that is because I wasn't around for the contest this time and part of it is a clear lack of interest from the rest of the community. I know this may be annoying, but SOMEONE needs to set up a game so you can play these in an actual game...

I could have sworn I hosted around 4 or 5 seperate times during the OLDC voting period :|

 

[MascaraSnake]

I retroactively applied it because technically the rules always stated it that way, it's just never been ENFORCED that way and in this case it really mattered when determining the results. The topic always said "Judging will end January 8th at midnight GMT (January 7th, 7:00 PM EST). The scores will be then averaged, and the person with the highest score is the winner." If you average it at that time, the results are what we posted. I'd much rather that than KOTE determining he liked my map better and intentionally breaking the results =P

You could have, however, counted the posts made before Scizor and later I posted the unofficial results. Those didn't have that problem.

The whole point of changing the rules like this is so there is never any question again. Whatever the math says is the result. I am tired of counting hanging chads.

Well, I understand that. Unless we suddenly start getting problems with that again, it's fine by me.

KOTE just did it =P

I didn't see anything in his post that suggested that he was doing it to make you win, although I wouldn't doubt that he did that.

Absolutely not. I understand there were a lack of netgames for this contest, but part of that is because I wasn't around for the contest this time and part of it is a clear lack of interest from the rest of the community. I know this may be annoying, but SOMEONE needs to set up a game so you can play these in an actual game, because we shouldn't be voting based on looks, we should be voting on how they actually PLAY, and while it's possible to determine half of that without actually playing it in a game proper, I know full well that it's basically impossible to know exactly what the choke points and heavy traffic areas will be without actually trying it, and that makes a HUGE amount of difference in the quality of the map.

There's the problem that everyone who doesn't catch your netgame (which is in the middle of the night for me, for example) will have huge problems catching a netgame. I understand that evaluating a map (at least anything that is not obvious crap) without having played it is not really possible though. What about making an official dedicated server? Is that practical? Would anybody be able to host it?

 

[Liliam]

... and still wins the contest because he was the only who submitted a map for that division.

My point: an OLDC division will have judging only if there are at least 2 valid maps submitted. If there's no enough maps for said division, the submitter must wait for next OLDC.

The problem is that while the OLDC is a contest, it is MORE a method of getting feedback on your creative work. I would rather an entry auto-win and get feedback now than have to wait another two months for the author to get criticism.

That's a shame, because the whole reason why I even made the map in the first place was because I noticed a gap in the rules and exploited it to see the results, which for some odd reason feels like something you really wouldn't like. Perhaps a better alternative would be to allow the map anyway, but make it lose by default due to lack of competition. People can still play it and give feedback, and it won't earn me an unwarranted Contest Winner title.

 

[Ricardo]

Or make it win by default and allow the mapper to chose not to have the title if it bugs them so much.