The following persons are authorized to visit the penitentiary at pleasure: the governor, judges of the supreme court and the secretary of the criminal justice department [corrections department] or his duly authorized representative; and no other persons shall be permitted to go within the walls of the penitentiary where the convicts are confined except by permission of the warden.