The story goes that cannonballs used to be stored aboard ship in piles, on a brassframe or tray called a 'monkey'. In very cold weather the brass would contract, spilling the cannonballs: hence very cold weather is 'cold enough to freeze the balls off a brass monkey'.