Abstract: Utilizing coarse aggregate containing mining waste rock for backfilling addresses strength requirements while reducing binder costs and solid waste treatment expenses. However, it is prone to aggregate segregation, which can lead to uneven deformation and damage of the backfill. We used an image segmentation method that includes machine learning to process the distribution information of aggregates on the splitting surface of test blocks. Results reveal a nonlinear relationship between aggregate segregation and variations in solid concentration and cement/aggregate ratio. Solid concentrations of 81%–82% and cement/aggregate ratios of 10%–12.5% reflect surges in fluid dynamics, friction effects, and shifts in their dominance. The uniaxial compression experiment with additional strain gauges and digital image correlation technology helped us analyze the mechanical property and failure mechanism under the influence of aggregate segregation. It turned out that the uniaxial compressive strength ranging from 1.75MPa to 12.65MPa is linearly related to the solid concentration and cement/aggregate ratio, and has no significant relationship with the degree of segregation in numerical terms. However, the degree of segregation affects the development trend of elastic modulus to a certain extent, the standard deviation of aggregate area ratio less than 1.63 clearly indicates a higher elastic modulus. In the pouring direction, the top area of the test block is prone to form a macroscopic fracture surface earlier. In contrast, the compressibility of the bottom area is greater than that of the top area. The intensification of aggregate segregation widens the difference in deformation and failure characteristics between the different area. For samples with different uniformities, significant differences in local deformation ranging from 500με to 1500με were observed during the stable deformation stage. The extreme unevenness of aggregate induces rapid penetration of cracks in the sample and forms macroscopic tensile failure, resulting in premature failure of the structure.