Where a zoning board of appeals or adjustment already exists, it may be appointed as the board of appeals. Otherwise, the board of appeals shall consist of five (5) members, each to be appointed for a term of three (3) years and to be removable for cause by the appointing authority upon written charges and after a public hearing.