Does the WAD judging system need change?

[Torgo]

This is a bit different from the "Does the MB need more judges?" thread, although the idea came from that thread before it split off. If you feel like there needs to be some change in the system, if it be that it shouldn't exist, or have more or less restrictions, then this is the thread to express your thoughts. Before I get into the discussion, I would like to bring you into what the judging process was like when I was actively judging. If any of the judges would like to say "hey, this part is not how we do it now" feel free to do so.

Now when we get a submission, we see if the submission breaks the rules in some way, be it a copyright music pack or a copy of a level or character without the original user's permission. If it isn't, then we go on to testing the level in srb2. If the level is completable, works in the selected gametype, has the correct level headers, and has basic effort, then we let it through submissions. If it doesn't work in the selected gametype, then we give the user the required changes that he or she must make to have the level completable. If I notice something that doesn't break the game, but I think could use a change, then I make a separate section of the private message on where I would suggest that he or she can make adjustments, but he or she is not required to do so for the wad to go into releases.

In some cases, I could have been better at not letting opinions into what I thought needed to be changed for releases, but that is all in the past.

So, do you think that system needs to be changed?

 

[Ultimate]

Personally, I don't think the system needs a change, at least not regarding what passes and what doesn't.

However, I think it might be useful for certain "rejected WADs" that appear to be made by people who are completely new at WADing, or people who are very young - to receive some form of public feedback which could help them improve at WADing in the future. I know that sometimes the judges give feedback to these WADs, but often times people won't understand or believe what you say unless they hear the same thing from multiple people.

I remember during back during the old submissions, there were some maps that were just awful, but it seemed that the creator was very young. Having all their maps rejected might put them off from WADing in the future, even though they might have been able to improve (even if slowly). I'm sure if I made a map at the age of 6-8, it would be quite similar to the "enemy cluster" maps we have seen before. And in my opinion, just creating a working map at such young an age is a pretty decent feat to achieve.

 

[Torgo]

In most cases, the wads that get rejected because of map errors are submitted again with fixes and let through. The ones that don't get submitted again are usually wads that are rule breaking. The most common in that category are ones that are copies of something that is done without permission.

 

[Mystic]

A little copypasta from the previous trashed thread. Judges exist for the following three reasons that used to plague the releases system:

1. An overabundance of non-working or useless files that have no use to anyone, frequently not even the original creator. This includes both unfinished material (which should go in editing) and garbage releases that have no use (a recent example would be someone who submitted a WAD that was simply a replacement for one of the crosshair graphics). Most files rejected for quality reasons are things that the author probably took an hour or less to make.
2. To ensure that proper procedures are being used when creating topics. A huge number of submissions are replied to a judge with "upload your file as an attachment" or another similar problem and then approved after the problem with the submission has been resolved. This helps everyone long term by making sure that links don't die and categories are being used correctly to make things easier to search for.
3. To ensure a basic level of plagiarism protection. A HUGE number of submitted files are simply people taking someone else's work and reuploading it without permission, especially with old 1.X files. While I don't like that many authors aren't around to update their files, we don't need people releasing their old work without permission either.

As an example, I'll list the problems with the last ten things in rejected submissions, as well as when for reference so you can see the actual frequency things are being killed off:

1. Single graphic change for crosshair. Doesn't really work because it's intended to be a local change and the host would have to change it for everyone. (Last week)
2. Character WAD port from 1.X without permission (2 weeks ago)
3. Ridiculously overpowered SOC that breaks the game entirely (4 weeks ago)
4. Character WAD port from 1.X without permission, as well as a broken link (1/15/2011)
5. Map pack with blatant plagiarism of other people's content. (1/11/2011)
6. Map pack recolor, as hard as that may be to believe. Again, without permission from the original author. (12/25/2010)
7. Only file in this list rejected for quality reasons. This was HMS, and was rejected in favor of having a release in #srb2fun where it isn't breaking quality rules and where people actually will get the joke. I'm still not sure whether or not we should have let this through, but this is the only thing where an actual decision based on opinion was made after a discussion about it. (12/13/2010)
8. Sound rip pack with no original content. There was a discussion about this as well, and it was rejected under the logic that it was essentially an audio version of a sprite rip. (12/4/2010)
9. Blatant plagiarism of another's map without even giving credit to the original author, never mind getting permission. (11/23/2010)
10. Map port from 1.X without permission, not to mention the port itself was completely broken. (11/22/2010)

While I agree with Ultimate's point that having their awful work rejected would cause most little kids to lose their interest, that surprisingly isn't what the submissions system is actually rejecting. It's overwhelmingly plagiarized work that gets thrown out, because the kids that you're talking about aren't likely to respect the work of others. Even in the old system where we didn't have submissions, such files would have been trashed, and quite publicly called out by several people. I think that having your file rejected because you stole someone else's work and being told that privately is probably better than having half the forums call you out on stealing, then getting you publicly mocked, and then having your topic locked and your link removed, which is what used to happen.

The other main benefit is that the average user browsing releases doesn't have to go through several files to find one that even works. All of the files were at least checked to make sure they operate correctly, and also that they're uploaded as attachments so the links don't break. As someone who was routinely frustrated trying to find something to play with the old releases section, I find this to be a dramatic improvement. Releases doesn't mean that the judges found this file to be good, just that it's playable.

 

[Sergeant Brown]

Why do I smell "Spazzo Incident" in this? Whatever.

I feel like the judging system is already good and not flawed. What I'd like however is that Sound Pack WADs get accepted.

If the system already does, then ok. If not then "oh well". It's just a minor complain after all.

I mean, SRB2 music is great and all, but sometimes you wanna change tastes.

 

[Mystic]

The problem with sound packs, especially music WADs, is that they are a copyright violation almost universally. While I wouldn't have a problem with it if someone decided to create their own original music and compile it, that's not what really happens. We don't want to have to deal with the RIAA complaining about us hosting their music, hence we don't allow it. If you want to make them yourself and use them for your own personal amusement, feel free to do so and we encourage you to do so if you enjoy it. Just don't distribute copyrighted music on our network services.

 

[Ritz]

We don't want to have to deal with the RIAA complaining about us hosting their music, hence we don't allow it.

Has there ever been any precedent for this in any mod of any game ever. Seriously, they only go after major P2P distributors and nail the occasional grandmother to set an example, they don't trawl the internet for WADs made by children on an engine nearly 17 years old with the intent of stemming a cash leak. They wouldn't even know how to run the things to verify the presence of copyrighted content! Just say you can't spare the bandwidth or something sensible like that.

So, if the wad is hosted on an exterior server, it's totally fine, right? Or are you not allowed to link to it on the forum altogether?

 

[Mystic]

Oh, of course, Ritz, it's extremely unlikely, but it's still illegal and I don't particularly want to condone such actions on a forum for a game that is itself a trademark violation. It is not okay to host the content offsite and post it here, either. Break copyright law somewhere else where I don't have to pretend to care.

It's not like making music WAD files is in any way hard, either. You can even replace the tracks in music.dta directly if you'd like to use the new audio in netgames. SRB2 won't complain a bit as long as all you're doing is replacing the music.

 

[Sun]

What about wads that have music as a part of them, but aren't the main part of the wad? Most wads with custom music seem to have music generally from SEGA games and other games. However, those are technically copyrighted, though certain fansites that SEGA has even "worked" with often have that music on their server. Besides, all music usually does is make the file size large.

Also note that it's possible illegal music could possibly be "accidentally" downloaded via the auto download thing in SRB2. Said feature is kind of broken, as it often downloads, then stops just before it finishes. It also lags the server, and annoys everyone.

 

[JEV3]

If the level is completable, works in the selected gametype, has the correct level headers, and has basic effort, then we let it through submissions.

While I think the submissions system works as-is, I think that it might be helpful to our users to actually elaborate on the bolded criteria. This is likely the place that worries our community, and also the part which confuses judges. We've let some pretty nasty stuff through submissions, so what violates basic effort?

 

[PHCC]

so what violates basic effort?

A large room with random textures and 300 crawlas and 50 eggman bosses inside it?

 

[MascaraSnake]

A large room with random textures and 300 crawlas and 50 eggman bosses inside it?

Well yes, but that's only one specific case. To be honest, we can't define this objectively. If we could, there would be no point in even having judges.

 

[Ezer'Arch]

Well yes, but that's only one specific case. To be honest, we can't define this objectively. If we could, there would be no point in even having judges.

The forum rules define how the members should behave, but we still need moderators. So let's fire the mods?

 

[JEV3]

Well yes, but that's only one specific case. To be honest, we can't define this objectively.

We don't have to define it objectively, but we might want to define it and provide some examples... common mistakes. A few of these will be all it takes to get a general idea of where the line is drawn. I mean something more specific than:

4. Make sure your submission is fun! If your file is just an ugly room with 300 Crawlas, that's not particularly fun or interesting. Don't submit your first creation, and make sure to add some polish before you submit.

That wad with several examples of awful and broken maps comes to mind, but I wonder if that's too much of a hyperbole to be used in this instance. It doesn't really need to be a WAD but we could make a list of common ways in which people violate the basic effort rule, maybe make it a new sticky.

If we could, there would be no point in even having judges.

I believe the purpose of judges in the submissions system is primarily to take care of broken and incompatible stuff which is fairly objective. Subjective evaluation barely comes into play unless we're trying to get something in the Hall of Fame.
Most of those against our submissions system feel otherwise, that there is a lot of subjective evaluation that goes into merely determining what can be released at all. The more we explain about our standards for submissions, the less closed and subjective it seems and the more fair the system.

 

[MascaraSnake]

We don't have to define it objectively, but we might want to define it and provide some examples... common mistakes. A few of these will be all it takes to get a general idea of where the line is drawn. [...] It doesn't really need to be a WAD but we could make a list of common ways in which people violate the basic effort rule, maybe make it a new sticky.

I support this idea.

 

[jcokeinfinity]

4. Character WAD port from 1.X without permission, as well as a broken link (1/15/2011)

wow, I didn't know that little mixup caused such hard feelings. And These weren't even technically submitted (I'm assuming your talking about the porting mixup)

---------- Post added at 04:13 PM ---------- Previous post was at 04:10 PM ----------

1. Single graphic change for crosshair. Doesn't really work because it's intended to be a local change and the host would have to change it for everyone. (Last week)

socs also create a non-local change in gameplay, and those are accepted

 

[MascaraSnake]

wow, I didn't know that little mixup caused such hard feelings. And These weren't even technically submitted (I'm assuming your talking about the porting mixup)

It didn't cause any "hard feelings". It's just used as an example here.

socs also create a non-local change in gameplay, and those are accepted

The keyword is "gameplay" here. You add a SOC to modify the way the game is played. That concerns everyone. What crosshair you use is personal preference and it doesn't change gameplay.

 

[jcokeinfinity]

*shrug* Okay.