The word translated “evil” in this verse means “hurtful in effect or influence; causing great harm or misery.” This verse requires us to carefully consider the results of every action or decision. If we foresee an “evil” result from an action then we must “abhor” it.
Some would think the verb “abhor” is simply a feeling of revulsion, but it also involves action. In John 12:42 it was translated “put out”. If I have knowledge of an action that will cause unjust harm then, if I have lawful jurisdiction, I should stop the action (i.e. put out). If I cannot lawfully stop an evil act then I must take action to prevent it in the future. So “abhor” means to so detest something that I must take lawful action. To simply stand around detesting an act that brings unjust harm to another is evil in itself.
Within this verse we have a wonderful description of the fundamental character of Christ Jesus. He is a man of steel and velvet, and we should seek to be the same. Velvet in that He loves without hypocrisy. His love is pure. I have a post Here on love as it is used in this verse. Flowing from this love is a steel-like resolve to take action when faced with evil. Jesus cannot just sit by and do nothing when evil seeks to destroy, harm, and cause pain unjustly. This is why the Spirit of God aggressively resists evil in this world.
The verb “cleave” means to “join closely together, glue or weld together.” We are to join ourselves to those things that are “good” (i.e. gentle, beneficial, positive, of merit). In taking action to resist evil we are also joining ourselves to something good.