Passion is something we suffer either from or through—or more recently, something we’re willing to suffer for. (From Latin root, pati, to suffer, also the source of “pathology”, “pathetic”, “pathos”, etc.)
Things we suffer from include overpowering emotions. These include six of the traditional “Seven Deadly Sins”-type: anger, envy, gluttony, greed, lust, and pride—as well as what we now term addictions, such as alcoholism or gambling.
Suffering through means the pain we attribute to martyrs and masochists: e.g. “Passion of Christ”.
Suffering for has roots in Victorian novels, where love was painful, usually due to strict societal or moral restrictions or perhaps was unrequited. So passions should follow us, not the other way around.
In the late 20th century, “passionate” which described painful love, came to—incorrectly—substitute for love—or a love-like feeling.
“Serving one’s own passions is the greatest slavery.” ~Thomas Fuller, Gnomologia
“If passion drives you, let reason hold the reins”. ~Benjamin Franklin
“My fault, my failure, is not in the passions I have, but in my lack of control of them.” ~Jack Kerouac
“It is with our passions as it is with fire and water; they are good servants, but bad masters.” ~Roger L’Estrange, Aesop’s Fables