Abstract: Utilizing coarse aggregates containing mining waste rock for backfilling addresses the strength requirements and reduces the expenses associated with binder and solid waste treatment. However, this type of material is prone to aggregate segregation, which can lead to uneven deformation and damage to the backfill. We employed an image-segmentation method that incorporated machine learning to analyze the distribution information of the aggregates on the splitting surface of the test blocks. The results revealed a nonlinear relationship between aggregate segregation and variations in solid concentration (SC) and cement/aggregate ratio (C/A). The SC of 81wt%–82wt% and C/A of 10.00wt%–12.50wt% reflect surges in fluid dynamics, friction effects, and shifts in their dominance. A uniaxial compression experiment, supplemented with additional strain gauges and digital image correlation technology, enabled us to analyze the mechanical properties and failure mechanism under the influence of aggregate segregation. It was found that the uniaxial compressive strength, ranging from 1.75 MPa to 12.65 MPa, is linearly related to both the SC and C/A, and exhibits no significant relationship with the degree of segregation in numerical terms. However, the degree of segregation affects the development trend of the elastic modulus to a certain extent, and a standard deviation of the aggregate area ratio of less than 1.63 clearly indicates a higher elastic modulus. In the pouring direction, the top area of the test block tended to form a macroscopic fracture surface earlier. By contrast, the compressibility of the bottom area was greater than that of the top area. The intensification of aggregate segregation widened the differences in the deformation and failure characteristics between the different areas. For samples with different uniformities, significant differences in local deformation ranging from 515.00 με to 1693.70 με were observed during the stable deformation stage. The extreme unevenness of the aggregate leads to rapid crack penetration in the sample, causing macroscopic tensile failure and resulting in premature structural failure.