Could the the extinction of the neanderthals be due to interbreeding with humans creating nonviable hybrid offspring? 
 
I would imagine during those winters in northern Europe the neanderthal males would be pretty popular due to their high tolerance to cold temperatures.