Speaking as someone who has been told (directly and indirectly) that some things they write make no sense, therefore zero coherence; while to others they seemed coherent.
The former: mainly composers or people who have studied composition. The latter: performers (sometimes esteemed by the composers themselves, thus in "potential conflict").
I can say that personally, despite not having the means to understand a piece of contemporary music, I will give a fairly complex practical example:
Some works by Gubaidulina as Frank anticipated, but also Ligeti, Penderecki, Silvestrov, Sciarrino, and others, I perceive as coherent; it seems absurd, but it is so.
Even if one attempts (without a score, but perhaps even with the latter it would have been equally complicated because it is not easy to read a score of a specific contemporary piece) to understand the elements that constitute the work, it can happen to get lost at a certain point; however, despite this, something strictly linked to everyone's perception, or almost all of ours, allows us—even if superficially—to perceive a given work as a coherent work, intuiting, but not understanding, that there is something that "works," that is consequential. In practice, there is this sensation but without awareness. Rules are useful, even those in vogue in the 1300s, although it is wrong to say they prevailed then, because since treatises have existed, these latter are considered rules; before, as all those who study or have studied know, one spoke of performance practices, compositional practices, modes, influences, and last but not least taste, which is always linked to the composer's personality, but also to the performer's skill. I feel ignorant because I do not believe it is sufficient to write how one feels if one is not aware; sure, it can be pleasant, but there is no solidity behind it, at least I experience it this way.
If one thinks of those who perform and not "only" those who compose, a comparison is possible: performance techniques have always existed and evolve, just as compositional ones evolve and shape (change) themselves; what would an performer do who does not know the basics of their instrument's technique—could they arrive at a use that goes beyond what they consider "a certain level"? Or perhaps they would find themselves performing passages that with basic technique they would approach with less difficulty, only to then go beyond? Obviously, I understand that the example intended was of a person who manages to build their own language from nothing, but I would say that is almost a utopia, because unless this person has never listened to anything of any genre, one cannot speak of genius (understood as coming out of nowhere), and there are various nuances to this term. Anyway, if someone listens to (even involuntarily) music that uses established harmonic successions, it's not that if they write pieces that reproduce them, just because they haven't studied them, they are a great [composer]; there, personal taste, sensitivity, the ability or lack thereof to make something out of basic material comes into play, and the factors are truly many. What I can say as someone who years ago rejected rules regardless (and even those I met did not help me get rid of this idea, by too much rigidity I mean, without giving me concrete examples and saying, as is often erroneously said, "because that's just how it's done," like a parrot, without knowing why), and now for some years has been aware (even if I do not use them properly and in this I fail significantly) that they have their own (very profound) reason, is that first of all you can know how to move, you decrease writing times, you are more aware and perhaps even more satisfied for having obtained a result that does not escape "control," therefore not due solely (even if it's not a small thing) to sensitivity and one's own ability or lack thereof to express what one would like in music.