An investigation on the plastic behavior of AZ31 magnesium alloy under ultrasonic vibration (with a frequency of 15 kHz and a maximum output of 2 kW) during the process of tension at room temperature was conducted to reveal the volume effect of the vibrated plastic deformation of AZ31. The characteristics of mechanical properties and microstructures of AZ31 under routine and vibrated tensile processes with different amplitudes were compared. It is found that ultrasonic vibration has a remarkable influence on the plastic behavior of AZ31 which can be summarized into two opposite aspects: the softening effect which reduces the flow resistance and improves the plasticity, and the hardening effect which decreases the formability. When a lower amplitude or vibration energy is applied to the tensile sample, the softening effect dominates, leading to a decrease of AZ31 deformation resistance with an increase of formability. Under the application of a high-vibrating amplitude, the hardening effect dominates, resulting in the decline of plasticity and brittle fracture of the samples.