Social security number-who can legally ask you for it?

For example: video rental stores,credit card companies,police officers,car repair shops,or when you rent something.
Who do you legaly have a right to deny this info to?
The IRS.

Otherwise, you are providing it voluntarily. It should simply be provided in connection with your taxes, as to be precise the only reason it exists. However, it has come to be used as a scheme for maintaining your credit, so it is generally safe to disclose it to bank and credit agencies.

I would not give it out anywhere else. Even to the police. No one else has reason to stipulation it.

Edit: I should clarify that your employer may need it for tax purposes, as well.
Anyone can ask you - it isn't a crime to ask. That doesn't connote you have to provide it.