Did Caanan really deserve that harsh punishment for Ham’s mistake?

It is always difficult to understand a situation completely, when the Bible gives so few details. I see this more as a prophecy of what would happen to Canaan, rather than a punishment. It would be grossly unfair to punish a son for the sin of the father, and God specifically rules that out, as you can see in the passage below, so that is why I lean towards the prophecy of how Canaan would turn out. God obviously told Noah what would happen with Canaan, because of the characteristics of his father Ham, and Noah proclaimed it more like a punishment (or a curse) than a prophecy.

2 Chronicles 25:4   However he did not execute their children, but did as it is written in the Law in the Book of Moses, where the Lord commanded, saying, “The fathers shall not be put to death for their children, nor shall the children be put to death for their fathers; but a person shall die for his own sin.”

As to whether Canaan’s descendants were black or not, the Bible does not specifically say. I have heard geneticists say that to get all the different colours we have in the human race, Adam and Eve must have been a mid-brown colour. In that way, the very black to the extreme white of the Scandinavians, is possible. So, it is entirely possible that Canaan’s descendants were black, but not because of a curse, only because of genetics. 

I hope that this makes sense to you. God bless!