When some things are abused you throw them away. Toys and socks work that way. I am in dire need of some new socks right now. A couple of pairs have holes in the heel and many of them have holes which my long toe (my long toe is not my big toe) loves to play peek-a-boo out of. My big toenail grows radically more than my other toe nails. Apparently I don’t clip it often enough and then cuts though my socks? My toenail neglect is sock abuse.
Anyway, just because socks are abused you don’t find people running around saying that socks are inherently evil or faulty. You may throw a pair of socks away, but not socks altogether. Authority can be, and is often abused, but that is no reason to throw it out. In our society sex is equally abused, why aren’t we so passionate to ban sex? Toys are abused by children, should we ban toys?
God gives leaders to his church. All their authority is a derived authority, not intrinsic to themselves. Obey and submit to them for your own souls. Find elders who are faithful to God’s Word and then be faithful to those elders. Put yourself in a position where your relationship to your elder will be a reflection of your relationship to God.
If you have authority issues with your elder most of the time I would advise one of two actions:
Repent. If you believe your elder is faithful to the Word of God, your issues with him are issues with God.
Move. If your elder is not faithful to God’s word find one who is and for the benefit of your soul and his joy in ministry submit to him in the Lord.