Any director, appraiser, member of the advisory council, or other officer or employee of any sanitary district may be removed for or without cause at any time by the authority appointing him, except that any director appointed by the advisory council of a district shall be removed only for cause.