This is the tag feature working as it should because if you set a linedef special to tag 0 it will tag all sectors that are set to 0.. And since 0 is the default tag, you'll get every sector that doesn't have a non-default tag.
Also you can use tag value 65535 to make the linedef special effect every sector in the map regardless of tag... kinda cool if you want a global colourmap :)