The present study shows that naturally developed fracture surfaces in rocks display the properties of self-affine fractals. Surface roughness can be quantitatively characterized by fractal dimension D and the intercept A on the log-log plot of variance: the former describes the irregularity and the later is statistically analogues to the slopes of asperities. In order to confirm the effects of these fractal parameters on the properties and mechanical behavior of rock joints, which have been observed in experiments under both normal and shear loadings, a theoretic model of rock joint is proposed on the basis of contact mechanics. The shape of asperity at contact is assumed to have a sinusoidal form in its representative scale r_c with fractal dimension D and the intercept A. The model considers different local contact mechanisms, such as elastic deformation, frictional sliding and tensile fracture of the asperity. The empirical evolution law of surface damage developed in experiment is implemented into the model to up-date geometry of asperity in loading history. The effects of surface roughness characterized by D, A and r_c on normal and shear deformation of rock joint have been elaborated.