The bigger or longer time horizon you take the more variables you have and more "blurrier" is the end picture. Simply expecting that one thing will only rise it's utopian.
It's same as doing exercise to have 6-pack, invest into your children education, etc. You can have 6 pack but that won't guarantee you getting every woman on Bali.

Or you can invest heavily into son's education but he will leave school soon and start playing in rock band or something.
When I've purchased first silver coin in 2010 price was around 0,2 - 0,3 (I think). Multiply by 31,1 and you will get around 6 - 9 per oz. The price increased to almost 1 or more per g. Now it's around 0,52 (15 per oz). It's still better than before (around 2010) but you don't know if price will return keep falling to hell or fly toward heaven.
Around 2000 internet start-up companies started appearing with "light speed". People bought huge amount of stocks, etc. Most of the people think that internet start-ups will only grow and grow. Near 2000 balloon exploded and bang! most of the people suffered huge financial losses.
Kondratiev wave
This Russian scientist wrote about so called "waves" or "cycles". He writes that there are waves of "up" and "downs". Times of growth and time of fall. And they appear in around 50 years (average).
Current crisis is one of the biggest after 1929. But in between there were many other crisis. And more will follow.
Sure silver (and other precious) material has limited quantity. But imagine that you have company which uses silver for producing XY. If the prices increases too much company will start searching for alternatives. Otherwise its costs will go up and then prices, demand for their product will go down, etc.
And I don't know much people who are having silver or any precious metal except jewellery. If every person, at least, in "developed" World would have at least few ozs at home, or buy it regularly, then maybe things would be different today.
What I want to say is that most things if not all in our life are based on speculation. I need average half of hour to drive from my home to job. But if some crash occur or something I can drive to my job twice more. Or even 4times more.
You should all watch The Fall of Roman Empire:
The Fall of Roman Empire
Movie is very long (at least for today's standards) and it looks kinda funny (well it's 1964). Also the movie it's not the best of it's kind. But it's having important message (one of the quotes):
"A great civilization is not conquered from without until it has destroyed itself from within."
Keep that in mind. Morale crisis is far worse than financial. Look Hoolywood movies for examples. They are mainly without any fresh ideas and making only remakes and "restarts". Sure you can't say that Hollywood wasn't copying "itself" in the past but what we have today...