The matter of labelling bothers me a lot too, and I think that’s the reason: labels are shortcuts. It’s like in our fast world we don’t have the time to consider what we are reading, if it’s something we’d like to read. We let other people tell us what it is we are going to read and if we are likely to like it. We are robbed of our right to make up our own mind.
And you are right, of course: we are going to be labelled anyway, so at least we should choose our own labels. It’s kind of sad to think that’s all the power we are left with, but that’s today’s publishing world, I suppose, we need to cope with it.