Hide certain contact details by user

I'd like the ability to have certain contact details be hidden by user or user level to give the ability to have more personal information stored under the contact without other users seeing everything. 

Thanks!