You have to manually remove them one by one if they are in the common contact list or alternatively, you can do a factory reset of your iPAD and reinstall all apps with your Appstore credential login and recreate your user list and then do not allow others to use your iPAD. Unfortunately, iPad does not support multi-user logins, however if it is a must then you can use a jailbreak apps known as iUsers at http://www.imore.com/tag/iusers but do understand the risk, including warranty void or bricking your iPAD, before trying it out. Only try if you really are an expert on iOS.
Video link of iUsers : http://youtu.be/Tq8sjcOdAsc
You have to manually remove them one by one if they are in the common contact list or alternatively, you can do a factory reset of your iPAD and reinstall all apps with your Appstore credential login and recreate your user list and then do not allow others to use your iPAD. Unfortunately, iPad does not support multi-user logins, however if it is a must then you can use a jailbreak apps known as iUsers at http://www.imore.com/tag/iusers but do understand the risk, including warranty void or bricking your iPAD, before trying it out. Only try if you really are an expert on iOS.