I've been dinged a time or two (not for metals). At least they didn't hit you for duty as well.
Problem I've experienced arguing the point is if it's in the "too hard" basket the freight carrier can kick it back to Customs for a determination. Customs can then decide to inspect it. And they can...