If you have institutional authority as a supervisor, manager, director, or higher executive, there is probably a difficult person over whom you have some control. A difficult person can undermine your popular authority - that is, the leadership role you've earned among the people you work with through your consistent and trustworthy behavior.
A difficult person can complain about you when you're not present, compete with you for power, impede your ability to follow through on promises, and so on. If the difficult person misrepresents what you have done or how effectively you do things in general, it will become that much harder for you to build people's faith and trust in you - the very foundations of your popular authority.