- Earlier generations were spiritual giants who possessed insight that no one alive today can match. Avraham Avinu effectively deduced the entire torah, both mitzvot d'oraita and mitzvot d'rabbanan. The dor Hamdibar (generation of the desert) climbed to such spiritual heights that a common person at the Red Sea had more prophetic insight than the prophet Ezekiel had in his vision of the Chariot. (The question of why this same generation was among the most faithless and ungrateful of all the Jews is generally ignored). There are other examples, but these give the flavor.
- Today we are in galut (exile). We lack the Sanhedrin, we lack the beit mikdash, we lack Kings and prophets, we lack the air of Eretz Yisrael that makes us wise. The Jewish legal system is set up such that lacking those institutions we can never have a court that is 'greater in numbers and/or [interpretations vary] wisdom' than the preceding ones. When galut is over things may be different.
- The decline of generations is quite simply the decline of knowledge. At Har Sinai the Jews had a complete system - not just the straight halachot but the meta-halachot of the how to interpret the torah (13 middot etc.), the mystical understandings through kabbalah, and vast amounts of now forgotten oral torah. Relentless persecution has resulted in the loss of that knowledge. Even where our ancestors explained their reasoning, we cannot be sure that the reasons given were the only ones, so we dare not change their rulings.
- Alternatively, the decline in the generations is not intellectual at all, but rather the loss of cultural familiarity with halacha, also expressed as a loss of yeriat shamayim (fear of heaven). As Rabbi Chaim Soleveitchik says in Rupture and Reconstruction despite our increase in textual knowledge, loss of mimetic culture (learning by watching the previous generations) caused by such events as the Shoah makes us less able to be at home in our own culture. See also Rabbi Micha Berger's essay in Mesukim Midivash.
- There is just too much too know now. Rabbi Akiva never had to learn the variant rulings of the Bavli and Yerushalmi. Rashi never had to learn the Shulchan Aruch and supercommentaries. Rabbi Joseph Caro knew nothing of the rulings of the Yeminites. Basically, it is no more possible today to hold all of Judaism in one's head than it is to be a contemporary renaissance man - to be a genius level artist and a world class scientist is not possible today - both fields have too much to master to let one mortal do both.
Despite this idea having a 2000 year pedigree, it feels like a cop out to me. At some level this is undoubtedly the sin of pride speaking. Nevertheless it seems to me that the greatest poskim of the generation have the obligation to make the rulings they believe to be correct, and not just blindly defer to those who came before.
Part of my problem is that I am not one of those poseks, and none of them agree with me. I can't think of a public admission by a posek in the form of 'X seems wrong to me, but it is clearly the decision made by those who came before, and so I follow it regardless.' I on the other hand have to say this sort of thing far too often.