The proposed ban is ANYWHERE in Victoria. It's a very low impact form of recreational prospecting. I would say detectorists have more impact on the land with unfilled holes etc. 4WDing and trailbikers would do magnitudes more damage (I enjoy getting out in the state forests in the 4WD, but the track damage and erosion caused by other participants is disgusting). That lobby is far larger though, so the Greens don't even bother tackling them. Sluicing is much lower hanging fruit for them to ban.
Have a drive around the goldfields some time - the landscape has been irreversibly altered from 19th century mining practices, and anywhere that sluicing would be worthwhile can pretty much be guaranteed to be a modified landscape from earlier mining, and nothing that the next decent downpour won't obscure or obliterate through naturally constantly changing waterways.