Okay, maybe everyone knows this already, so forgive me if I sound stupid, but as far as I can tell nobody is talking about this, so here goes ..............
So I'm thinking why is Luke, who was the optimistic hero of 70s sci-fi ... why is he now depressed, a recluse, indifferent to a galaxy at war, and hasn't lifted a finger to help his friends and family out, who are being killed off. What if Luke has realized that there's no point? What if Luke has realized that the Force doesn't care? Lemme explain.
Look at the rise and fall of the Jedi and the Sith. They are like two sinusoidal waves that are 180 degrees out of phase. The Jedi who are good rise to power, and the Sith who are evil pull them back down. But the Sith then become too powerful, so The Force 'creates' Anakin Skywalker from nothing in order to bring balance back to the Force. But the "balance" wasn't the balance the Jedi were hoping for. Because the Jedi had grown fat and sluggish in their uncontested power, while the Sith grew in secrecy and shadow, unable to operate in the open because of the Jedi's hegemony.
So Anakin went to the Dark Side, and this actually fulfilled the prophecy, and DID succeed in bringing balance to the Force. However, the Sith became too dominant, and Palpatine and Vader killed off too many, prompting the Force to channel Luke and bring back Vader to the good side at RotJ.
After RotJ, Luke decided to set up an academy in the wake of a thrilling and dominating victory over the dark side, culminating in elimination of its core leadership (Palpatine) and star apprentice (Vader). Good has won, it seems the Force itself has finally won, and there are no more threats. However, unexpectedly, someone murders all his padawans not even 20 years later. Yet .... This isn't just any someone. It's the son of Leia Organa herself! How??? How could it be? No one would have predicted Han and Leia's child to be the first visible Sith since the fall of Vader. This has to speak to Luke. With all the Sith gone, the Force stole someone who in ALL likelihood should have been a Jedi, and a very powerful one.
Conclusion: The Force will do what it wants. It doesn't care about good or evil, about right or wrong, or about morality. Its morality is balance. The Jedi and the Sith have been mistaken, and is the source of their rising and falling cyclically and unavoidably. One becomes too powerful, the other brings it down, repeat and repeat. These extremes are tearing the Force apart, and Rey may be the key to bringing them back together, restoring balance of light and dark within herself first. The Jedi, then, as one extreme, must end as Luke foretells. But so too must the Sith. The future is balance, harmony, and coexistence. And either mutual extinction or a merger between these two orders must occur if the galaxy is to avoid destroying itself altogether for fear of misunderstanding the Force.
I think this is what Obi-Wan Kenobi understood - at least in part - when he allowed Vader to slay him in A New Hope. He knew that the Force would restore balance, regardless of his actions; and that by taking inaction, the Force would have to compensate even more on Luke's behalf. He loved Luke and wanted to put him in the best possible situation, knowing that his own time was coming to an end anyway. Think about two ends of a scale. You have an old Jedi and a sapling on the one side, and the ruler of the Galaxy and his star warrior on the other. The situation is already imbalanced. And Kenobi knows the Force is on their side. But if he defeated Vader, even supposing he could, the situation wouldn't favor Luke as much as if he let Vader strike him down. By allowing Vader to slaughter him, he upended an already unbalanced situation, and threw the whole weight of the force into Sith hands. This made them impossibly powerful. Old Kenobi knew that the Force would act to restore itself, and that his sacrifice would actually make him and Luke even more powerful.
Think about why there was no body when Vader sawed Kenobi in two. We've seen sabers slice and dice Jedi before, yet their body parts and corpses remained. Something special happened. Then think about the Force ghosts. How could Obi-Wan talk to Luke, when there was no ghost talking to Vader or Palpatine? The Force clearly was on the Jedi's side, at least then. I think Luke detected the same effort at balance when the Force told him that someone else would come along to bring balance, when Kylo Ren butchered his padawans and razed his academy to the ground. Luke didn't kill Kylo Ren even when he easily could have, because he was spellbound by this voice and its promise, despite not understanding it. And I think that's why he went to that Jedi Temple, to understand why exactly the Force would do what it's done his entire life, play both sides it seems.
So now we are back to old Luke in Episode VIII. What if Luke is depressed because the goodness in him is so disappointed in the indifference of the Force for human morality? For family values and heroism and love? What if the Force is really just a force, an impersonal and implacable pressure that cannot be resisted, and that when channeled in one way automatically creates opposition to return balance? What if the Jedi have been wrong all along in thinking that the Force would always act on their side, the side of good, when the Force really only cares about balance? What if the Force itself is not good? Wouldn't YOU be depressed? I think this is many people's [mistaken] ideas about God.
So you have poor, old Luke who is losing his sister, his brother in law, his father, and everyone he's loved. And he realizes that this Force he's dedicated his life to understanding, given up chance at having a family, and sacrificed 40 years of his life for ... this Force doesn't care. And that even if he throws his weight into the battle, and they somehow win against the Sith, this same Force he worshipped as God will create or steal someone even more powerful to rise later on and probably destroy him, the Jedi, and maybe everything he's ever cherished and loved. Maybe once and for all. Would you entertain the thought that the Force is your enemy?
Maybe there is an advanced Jedi concept, where the way to restoring order is to do nothing about the Sith. Of course, regular people would hate the Jedi for this, but the way the situation would play out is that the Jedi doing nothing would almost force the Force to bring the Sith down. While the Sith, who are driven by imbalance and impulsivity, would either self-destruct or be drawn so close toward the Jedi in order to restore the balance, that they would cease to be Sith per se.
This is just my idea. I could be very wrong. But if I were directing this movie, this is what I would have done it this way.
(Rian, if you're reading this, I'm looking for a writing job)