Abstract: A novel approach is proposed to eliminate the random phase-shifting errors and CCD noise in four-step phase-shifting digital holography. By carrying out the phase-shifting process repeatedly, a series of holograms for each phase step are obtained. Theoretical analysis indicates that the random phase-shifting errors and CCD noise can be completely removed if we substitute the single hologram with the average of multiple holograms in the reconstruction. The feasibility of this method is demonstrated by computer simulation, and the results show that this method is useful in improving the quality of reconstruction in phase-shifting digital holography.