The short answer: No. Polyamory is not only about sex.
The longer answer: I have a confession.
When I first became aware of ethical non-monogamy and polyamory, the first images that popped into my head were 1960's hippie-style "free love" and 1970's swinger key parties. My 1993 high school job was Blockbuster Video and I remember getting at least one call a night asking if we had the 1969 movie Bob & Carol & Ted & Alice (we did, but it was ALWAYS rented).
Basically, I equated "polyamory" with "sex." And I was wrong.
The misconception exists because many people assume that polyamory simply involves having loads more sex. But it's far more than that. Time and effort often goes into maintaining multiple relationships. Polyamory requires strong time management skills and plenty of planning ahead. There's even a running joke that goes "polyamory is for people obsessed with Google Calendar."
I'm Poly and yes, we have a Shared Google Calendar.
As I've written elsewhere in this FAQ, polyamory is about having having different kinds of loving, consensual, intimate relationships with more than one person at a time. The literal definition is "multiple loves." For some people, sex of some sort is crucial to poly relationships. For others, it isn't.
That means that it’s quite possible for some of those poly relationships to be close and deeply loving, but without any physical intimacy.
This might not make sense to those whose can only envision love and sex going together in the same romantic relationship. But given that 15% of traditional monogamous married couples haven't had sex in at least one year (with the infidelity rate far higher than that), perhaps it makes more sense to say that we have a cultural expectation that love and sex go together in a relationship. Clearly, it’s not exactly rare that they don’t.
Everyone does polyamory differently, and that's ok.
If someone tells you they are polyamorous, it does not automatically mean they're thinking, "Now that I've told you, I'm going to seduce you/steal your man/steal your woman/invite you to some weird sex party!”
In other words: "Yes, I'm poly. That doesn't mean I'm automatically DTF."
It just means it doesn't have to end with, "Hi, I'm married/taken/spoken for. So we can only just be friends." You can pursue it further, and even have a committed, long-term relationship with or without sex.
Just like in regular monogamous relationships.