Vitreoretinal surgery describes various invasive ophthalmologic procedures to treat a wide range of retinal diseases that leads to blindness. The main hypothesis behind this thesis is that optical coherence tomography OCT) can be used to effectively guide microscopic retinal surgery. Anatomical and functional OCT images can be used to guide surgeons maneuver in real-time to reduce hemorrhaging and detect blood oxygen saturation level SO2) intraoperatively. A fiber-optic spectral domain OCT SD OCT) has been constructed and evaluated in this dissertation research. Advanced signal processing algorithms for OCT have been developed to overcome problems inherently associated with the hardware and the basic signal processing methods. First, to achieve high axial resolution and accurate ranging, a new automatic spectral calibration ASC) method for SD OCT was proposed and validated. Secondly, to reduce speckle noise and simultaneously suppress sidelobes in OCTs point spread function PSF) induced by spectral ripples in the broadband source, an algorithm that combines speckle reduction and deconvolution was developed and the effectiveness of the algorithm was demonstrated. Thirdly, to reduce the amount of data required for high-resolution image reconstruction, the feasibility of using compressed sensing CS) in SD OCT was studied. In this dissertation, another important application of OCT is the detection of microvasculature. Chick chorioallantoic membrane with vessel network was used as model tissue. The location and the dimension of blood vessel was identified from OCT image based on signal intensity, as well as interframe speckle variance. Moreover, a technique known as spectroscopic OCT SOCT) was used to reveal the spectral modification to the incident light induced by blood absorption to assess blood oxygenation level. Finally, speckle variance OCT and spectroscopic OCT images were combined and overlaid to show the locations of blood vessels and spectroscopic information simultaneously.