Should a majority of the membership of the board (not a majority of a quorum) deem such action necessary from any cause, they may withdraw from the individual member of the board the authority and power impressed by Section 45-20-71.20, and when such withdrawal is made, and until granted again, such authority shall be vested in the board, or as it otherwise may determine, at any constituted meeting as herein provided.