The bigger coins really do show defects a lot more readily. I am collecting the US Mint America the Beautiful 5oz set, and I have a couple with pretty significant scuffing or minor edge denting... from both reputable bullion dealers and from eBay sellers. I normally stay away from higher premium coins, but I got most of them for around $100 US with silver having depressed prices this year, and it's the only set over 1oz that the US Mint has ever tried. We shall see, but while I'm fairly impressed thus far with the look of them, I certainly don't expect to make much profit on the set due to high premiums and having to buy large AirTites and storage case.
I would echo that you really can't complain about defects on BU coins, though. You always will just get the dealer referring you to the mint... and the mint saying that is how bullion coins are. The only way to avoid that is to buy in person at coin shops or shows... or online auctions where they have good pics of the actual coin you will receive. My whole goal on the ATB 5oz series is just to get good or very good looking BU coins without major defects and get them as close to spot price as possible (not proof-like, but not badly scuffed or scratched either). I mainly buy from individuals... who were probably planning to collect the set, then realized a 56 coin 5oz set costs much more money and time than they have available. Even with buying them all brand new or from sellers with pics of the coins, I have had quality problems with a decent amount of them. The only times I've expressed dissatisfaction are when an eBay seller was clearly using pics of a different coin than the one I was sent (pics had no edge dent, deep scratch, etc... but my coin I received did). That is bogus.
I continue to be amazed that so many people bother to get silver graded or pay extra for high grades when a good looking minted coin doesn't mean it will stay that way. In addition to production defects, silver is a metal that tarnishes, tones, and spots. That is just its very nature. I don't think US Mint has ever had a problem with milk spotting, but I fully expect some ugly toning or other discolor on the ATB coins I bought after awhile also (just like you see on some ASE). I've had no reservations breaking open several of the 5oz coins I've bought out of slabs since slabs take too much space and BU coins are obviously a high grade to begin with. It seems silly to grade them, and unless they get graded MS70 for this series, they don't seem to sell for any higher price than a good looking ungraded of the same design. Many of my ungraded ones look better than the graded MS68 or 69s which I bough simply because they were the cheapest of that coin at the time.