When the punishment of the crime shall be the confinement of the criminal in the penitentiary, the expenses shall be paid out of the state treasury, on the certificate of the governor and warrant of the auditor; and in all other cases they shall be paid out of the county treasury in the county wherein the crime is alleged to have been committed.