I play any build on Legendary, except Warrior builds. Any mage, any Archery, any Assassin, any distance fight build are OK for legendary. Assassin is a close contact build, but 6x and 15x modifiers make fights easier. Warrior builds are Ok on Master but any crafting skill, they depend on, should be finish around lvl 20+. To me hitting a regular enemy with Warhammer 5+ times doesn’t make sense, and that is Legendary.
If we like to enjoy the story, role play, levels lower then Master are better. We focus less on making, fulfilling goals, avoiding certain places, fighting for survival.
Adept, always and forever.
Artificially adjusting damage to/from the player and the player alone creates logical inconsistencies, and encourages power gaming and emphasis on specific combat tactics (stealth archery, paralysis, conjuration, illusion, etc.) over just enjoying the game any way you want.
That said, if I always perked Enchanting on every character I ever played, I would probably increase the difficulty to keep things from being too easy.
I often play no-crafting builds, and enemies very sharply scale in both hit points and damage. Go into a "level 6" dungeon where it takes you 15 two-handed power attacks to kill one of the bandits, and he can kill you in a single hit at 150 hp, and then you don't feel like Adept isn't hard anymore.
Plus yeah, role-playing becomes very difficult due to level scaling, and add to that PC mods that make enemies actually intelligent, and Adept starts to feel just fine.
I play on Master mostly because I made the commitment to test builds out to see if they're viable on that difficulty. I'll push it up to Legendary if the build turns out to be too powerful. I'm proud to say that I've had to lower the difficulty only once out of 11 builds, and that was during the fight with Antioch in the Undeath mod. My god that guy is OP!
I play on Adept most of the time. The way difficulty works is that on Novice, you dea x2 damage while only taking half damage, on Apprentice you deal x1.5 and only take 0.75 in return, on Adept you deal and take x1 damage, on Expert you only deal 0.75 damage but take 1.5 in return, on Master you only deal 0.5 damage but take x2 in return, and on Legendary you only deal x0.25 but take x3 in return. Adept is basically the middle of the group, and it's probably the easiest way of working out damage taken/damage dealt.