High-steep slope stability and its optimal excavation design in Shuichang open pit iron mine were analyzed based on a large 3D physical simulation technique. An optimal excavation scheme with a relatively steeper slope angle was successfully implemented at the northwest wall between Nos. 4 and 5 exploration lines of Shuichang Iron Mine, taking into account the 3D scale effect. The physico-mechanical properties of rock materials were obtained by laboratory tests conducted on sample cores from exploration drilling directly from the iron mine. A porous rock-like composite material was formed for the model, and the mechanical parameters of the material were assessed experimentally; specifically, the effect of water on the sample was quantitatively determined. We adopted an experimental setup using stiff modular applied static loading to carry out a visual excavation of the slope at a random depth. The setup was equipped with acoustic emission (AE) sensors, and the experiments were monitored by crack optical acquirement, ground penetrating radar, and close-field photogrammetry to investigate the mechanisms of rock-mass destabilization in the high-steep slope. For the complex study area, the model results indicated a clear correlation between the model's destabilization resulting from slope excavation and the collected monitoring information. During the model simulation, the overall angle of the slope increased by 1-6 degrees in different sections. Dramatically, the modeled excavation scheme saved over 80 million tons of rock from extraction, generating enormous economic and ecological benefits.