This cross-disciplined project will bring state-of-the-art artificial intelligence (AI), computer vision and electronic system design technologies into the medical field to assist the surgeon with image guided surgery, thereby increasing the efficiency and quality of surgery. The feature of this project is the surgical auxiliary system powered by augmented reality with real-time recognition and dynamic body model. In this image guided surgery system, AI techniques will be developed for surgical image recognition and pose estimation for body/organ and surgical equipments. Adaptive human model will also be developed, which can integrate the data from medical imaging systems, such as CT and MRI. Moreover, we will also develop the associated low-power deep learning accelerator pose estimation hardware engine.