These days, as I scrape through my increasingly narrower and more tightly defined game collection, I have noticed that fantasy games seem to fall into one of four categories:
1. The D&D-Like
This game either is Dungeons & Dragons or is closely rooted to it in some way (13th Age, Pathfinder, Swords & Wizardry, Fantasy Craft, OSRIC, every other edition of D&D, and most retroclones in existence).
These games are essentially D&D or offering the D&D experience as closely as the OGL allows. A key component of these games is that they are probably going to have lots of known D&D tropes in them, especially if its in the OGL, or will offer up analogs for those things and creatures which are considered protected IP (if your game has a close analog to a mind flayer, it is probably one of these, for example). If your game has classes, level advancement, attack bonuses, escalating hit points, armor class, and the notion that a level 10 dude can mop the floor with dozens of level 1 dudes then you're probably in this space. Extra points if a beholder, drow, mind flayer or other distinctly D&D monster show up.
2. The "Does D&D Better and/or Different"
This game was designed to emulate the D&D experience but with a different ruleset. Depending on your interpretation certain games may or may not fall in to this category, but some are indisputably attempting to enter D&D territory, but with a different mechanic entirely. These are games aimed at people who's nostalgia is for the idea of a dark dungeon delve, but not necessarily centered on need for mechanical emulation; These are games that pay homage to Gygax's legacy without embracing the rules. Their fans may actually revile D&D mechanically while engorging upon the ambiance of the dungeon delve.
Examples that I would lump in this category include Dungeon World, Mythras's Classic Fantasy expansion, and some games which are arguably more subtle in their differences such as Dungeon Crawl Classics, Torchlight, and FATE's Freeport Sourcebook. GURPS Dungeon Fantasy is a curious example of a tried and true system seeking emulation. Although I lump 13th Age in the D&D-like space, one could also argue it belongs here (I think it's too close to a 4E emulator, though).
3. This is Totally Not D&D but Really wants to Compete in That Space
These are games which do not seek to emulate D&D, and in fact try to provide a decent alternative set of mechanics and creative mental space in which to work. Their hallmark is doing things differently, from stuff like "our orcs are weird," to "we don't even do orcs, elves, etc." --but a key element of these system is that they totally want you to be able to do orcs in your setting if you want to.
Fantasy AGE is resting firmly in this space, as is Mythras, Runequest, Fantasy HERO, GURPS Fantasy (but not Dungeon Fantasy ironically), The Dark Eye, and many, many classics that are now gone or hard to find (Chivalry & Sorcery, Ysgarth, and countless others). Most Generic Systems that offer fantasy expansions fit into this spot (Genesys Core and Cypher System certainly do; I'd argue that the Savage Worlds Fantasy Companion is much closer to a type 2 emulator, though some of its fantasy settings are definitely in this third category).
The key point though is that these games offer up a competitively different fantasy space in which to game. Their selling point is usually that D&D DOESN'T do "this," but that you can totally game with their systems into perpetuity just like D&D.
4. This is Fantasy, And We Hate D&D
Here's where you see the interesting stuff. Stuff which is clearly inspired by Tolkien (Awaken, Symbaroum), stuff which emulates specific genres (Conan RPG) with specially designed systems, stuff which seeks to provide a form of fantasy which neither feels like nor looks even remotely like D&D or any conventional high fantasy experience. It is often easy to distinguish these systems by simply asking whether or not it is even possible to imagine creating your own homebrew setting with them, or modifying it to run your own....chances are no, these games are as unique as their settings.
Talislanta, Skyrealms of Jorune, and Tekumel are classic examples of this corner of fantasy gaming. Worlds weird and strange, magic that defies D&D style quasi-Vancian magic, even (ironically) The Dying Earth RPG more appropriately rests here. Other games like Symbaroum and Awaken might feel more familiar but as you dive in you realize that it is their inextricably entwined rules and setting that make them unique even if they feel like a corruption of the familiar.
These games aren't really competing with D&D; they're trying to ignore it.