When an inmate has testified, the deputy and guard shall return the inmate to the penitentiary with all convenient speed.