Abstract: Tensile and shear fractures are significant mechanisms for rock failure. Understanding the fractures that occur in rock can reveal rock failure mechanisms. Scanning electron microscopy (SEM) has been widely used to analyze tensile and shear fractures of rock on a mesoscopic scale. To quantify tensile and shear fractures, this study proposed an innovative method composed of SEM images and deep learning techniques to identify tensile and shear fractures in red sandstone. First, direct tensile and preset angle shear tests were performed for red sandstone to produce representative tensile and shear fracture surfaces, which were then observed by SEM. Second, these obtained SEM images were applied to develop deep learning models (AlexNet, VGG13, and SqueezeNet). Model evaluation showed that VGG13 was the best model, with a testing accuracy of 0.985. Third, the features of tensile and shear fractures of red sandstone learned by VGG13 were analyzed by the integrated gradient algorithm. VGG13 was then implemented to identify the distribution and proportion of tensile and shear fractures on the failure surfaces of rock fragments caused by uniaxial compression and Brazilian splitting tests. Results demonstrated the model feasibility and suggested that the proposed method can reveal rock failure mechanisms.