No portion of any permit fee once paid shall be repaid to any person, provided that in a case where the department acting under the authority of law shall have refused upon original application for the registration period to register the vehicle to which the permit relates, the town clerk, upon application therefor, shall refund the said permit fee. The exception provided in this section shall not apply to a permit fee made up in whole or in part of transfer credits.