If you must use WhatsApp, you should ask everyone whose contact details are stored in your address book whether they mind having their data transferred to WhatsApp servers and possibly accessed for other reasons than exchanging messages with you. Another option is to keep most of your contacts on one phone and to use a second phone to exchange messages through WhatsApp.
Instead of using WhatsApp, consider using the TextSecure (SMS) or ChatSecure (instant messaging) apps for Android, or another instant messaging application that uses the XMPP protocol and offers OTR support such as the iOS version of ChatSecure.
