At the root of the anti-romance and anti-sex prejudice in large parts of the SFF community lies a deep discomfort with emotion. I’ve blogged about this before in 2011.Her in-depth essay provides much food for thought and I recommend you check it out!
And indeed accusations of “This isn’t SFF” often broil down to “This story contains more emotion than I’m comfortable with”, since for some reason no one ever accuses stories about characters standing around in an SFF landscape and endlessly philosophizing about something or other of being “not SFF”. But if there’s a story that deals with relationships and emotions, even if the context is clearly speculative, then suddenly that story is accused of being not speculative enough. Bonus points if the author and/or POV characters are something other than straight white cis- and heterosexual men.