It generally does not matter what sector the special linedef is tagged to, but tagging two colormap specials to one sector is not recommended, as this can cause problems.
Trust me, I have tried this plenty of times and there is no (even indirect) way around it. (Note that even invisible FOFs apply colormap to the whole sector despite their size)I'd assume you would have to use a linedef executor to change the sector's tag which your other colormap would use. It also saves you from tagging 2 sectors to 1 colormap. However, I haven't actually tried this, and if linedef executors don't work in this particular way, then I have another idea.
Try making 2 invisible FOFs, each with a different colormap. Through linedef triggers, change the floor/ceiling of each FOF quickly (and I mean quickly) so that the new colormap shows.