When a new sheriff is elected or appointed, and has qualified, the county clerk shall give the new sheriff a certificate of that fact, under the seal of office of the county clerk. Whenever thereafter the new sheriff is authorized by statute to enter upon the duties of the office, the new sheriff shall serve such certificate upon the former sheriff, from which time the powers of the former sheriff cease, except when otherwise specially provided.