By selecting the “more options” icon at the end of the policy, you will be given the option to "Cancel Tracking" or go to "Exception Handling" for the specific policy. 

Cancel Tracking – removes the policy from the dashboard and disables tracking.

Exception Handling – opens a dialogue box enabling the user to provide exceptions for policies that are not currently in compliance.