The failure or refusal by the holder of a permit to comply with any term or condition of the permit operates as an immediate termination and revocation of all rights conferred or claimed under the permit.