Note that International Maritime flags are usually written with Morse Code. But those maritime flags are not really Morse Code, and have no bearing to the dashes and dots of Morse Code.
In essence these flags are reflective of the Morse Code signal. Likewise the added colour reveals the ones digit position of each letter. Since the Morse Code flag for the letter A, K, and U all have a number 1 in the ones position via 01, 11, and 21, they are all red flags. Consequently the flag for number 1 is red too.
The order of colours to number is based upon the progression of the visible spectrum. Since yellow is the 3rd colour, we get a yellow coloured flag for flags in position 03, 13, and 23, which represent C, M, and W.
Also each colour has set contrast indicator to reduce confusion. See the image below to see which contrast indicator belongs to each specific background colour.
Also note Morse Code flags are always read downwards. Although some cultures read from the left and others from the right, every culture reads downwards.