Size drives the application
Larger aggregate suits mass concrete and thick sections where there is room between the reinforcement. Smaller aggregate is used in thinner, congested pours — slabs, columns and beams with tight bar spacing — where a large stone would bridge between bars and leave voids.
Sub-base and drainage applications are a different question again: there the priority is a stable, well-locking or free-draining material rather than one optimised for concrete strength.

