... and another perspective:
Paper Silver currently drives the physical spot price, so shenanigans in the markets can directly affect the value of your coins bars or what-have-you.
The markets are a rigged casino, and playing in them is rather like an individual krill going for a swim (you are not worth bothering with individually, but there are plenty of very large entities which will scoop you up).
However, if you buy physical when the markets are dipped with money you can afford to put aside, and be prepared to hold it indefinitely you should be good. Just like houses, you make money on the purchase, not the sale.