Abstract: A novel imaging Fourier transform spectrometer structure was proposed, which included two confocal hyperboloid mirrors as the collector and a Schwarzchild objective for collimating, combined with the photoelastic modulator based interferometer, imaging lens and HgCdTe focal plane array, then the infrared panoramic annular spectral imaging was achieved. In this paper, the working principle and imaging features of panoramic annular imaging Fourier transform spectrometer were introduced, the design of two confocal hyperboloid mirrors based annular collector and Schwarzchild objective based collimator were discussed in detail, then the design model and optimization scheme were presented. Finally, the ray tracing and optimizing were done by Zemax. Results show that, the working band of designed imaging Fourier transform spectrometer is 8 m~12 m, the effective focal length is -0.67 mm, the lateral field of view is 40~80and the F number is 0.9, the rootmeansquare (RMS) spot radius is 20.116 m less than a pixel, all field modulation transfer function (MTF) at 16 lp/mm are higher than 0.5 with good consistency, the optical path difference(OPD) is within 0.4 scope, the imaging quality of the system is fine.