Cavitation erosion is an especially destructive and complex phenomenon. To understand its basic mechanism, the fatigue process of materials during cavitation erosion was investigated by numerical simulation technology. The loading spectrum used was generated by a spark-discharged electrode. Initiation crack life and true stress amplitude was used to explain the cavitation failure period and damage mechanism. The computational results indicated that the components of different materials exhibited various fatigue lives under the same external conditions. When the groove depth was extended, the initiation crack life decreased rapidly, while the true stress amplitude was increased simultaneously. This gave an important explanation to the accelerating material loss rate during cavitation erosion. However, when the groove depth was fixed and the length varied, the fatigue life became complex, more fluctuant than that happened in depth. The results also indicate that the fatigue effect of cavitation plays an important role in contributing to the formation and propagation of characteristic pits.