That circle theory, where it is one less than the number of positions, doesn't that only count when order is important? Different combinations of colours have no order because they are different colours anyway.
If you have three men and three women in a circle, however, order is important because nothing differentiates between the three women or the three men?
And I also agree with what has just been said, since you have fixed one colour, you can't fix another.
Theoretically however, can't you unfold a cube in such a way that it is a circle anyway? It may be a weird circle, but I am sure you could. Wouldn't that make it 5! anyway? That just came into my mind though, so who knows lol.