If I am playing a caster, I usually stop at the free thalmor gear near the shrine of Talos. The one when your leaving Helgen and make a left just before the first standing stones. Then when I get to Whiterun I take a carriage to Winterhold, run up to the mages college and steal all the gold and soul gems around the college.
I like this idea for a thread. Here are a few contributions from me:
You can get a nearly complete set of Steel Plate armor straight out of Helgen, either by killing Kharjo -- which would be appalling, as he's awesome -- or by killing Salma at Ironbind Barrow (near the Nightgate Inn, west of Windhelm). I think you can actually just let other NPCs kill her for you, as there are two powerful hostile NPCs that will fight you at the end of the barrow. It's very small too, and Salma and another character will help clear it out for you, so even for a level 1 character, it's quick and easy.
You can also get Wolf Armor after completing just a few of the Companions missions (I believe it's purchasable from Eorland once you go with Farkas for your test run at Dustman's Cairn). It's lightweight but decent quality heavy armor, and it benefits from Steel Smithing, so you don't need to invest in perks to get a decent tempering bonus.
You can get a full set of Stormcloak Officer gear by siding with Ralof in Helgen, then heading to Forelhost (southeast of Riften) and killing Captain Varmir, who camps outside the ruin. As of now, the boots are bugged, classified as Heavy Armor, so you can't gain the benefits of Well Fitted, Wind Walker, etc. while wearing this set. But it's great in the early game, and I believe patch 1.9 is supposed to fix that bug with the boots, which makes this set actually very useful even in the endgame.
For stealth-based characters, obviously joining the Thieves Guild and/or Dark Brotherhood will give you easy access to some pretty good light armor, that comes with useful enchantments. Even non-stealth characters should consider these armors, as they have solid defense stats, and some of their enchantments can be useful for non-stealth characters too.
For weapons, you can get Skyforge Steel at the same time as Wolf Armor becomes available for purchase through the Companions quest line. You can also get a Blades Sword from Fultheim at the Nightgate Inn (by killing or pickpocketing him; he's easy to 1-hit in his sleep). Or a Scimitar can be found under a boat west of Dawnstar. All three of these weapons are stronger and lighter than Dwarven swords, and can be tempered with Steel Smithing.
Valdr's Lucky Dagger makes for an awesome weapon, especially once you add Dual Flurry and/or Elemental Fury. It has a 25% chance of critical strike, and when it crits, it's a guaranteed stagger. You can get it right out of Helgen. Also, the 25% chance to crit is not considered an enchantment, which means it never runs out of charges, and you don't need Arcane Blacksmit to temper it. I believe it has the same stats as a Skyforge Steel Dagger, and can likewise be better tempered with Steel Smithing (There's actually a bug that allows you to get two of them for dual wielding; look up a tutorial on YT if you're interested.)
For mages, in addition to your suggestion of those Hooded Thalmor Robes near Helgen, and getting gems from the College of Winterhold, joining the College offers other immediate benefits: Faralda will sell you a random spell (from a select pool of possibilities) for 30 coins when you try to join. You can save in front of her and reload until you get the spell you want, at a very cheap price! Also, when you join, the robes you get will have the same Magicka Regen rate as standard Novice Robes, but also a Fortify <random school of magic> enchantment, so that's a nice upgrade. These robes offer a better enchantment than the Hooded Thalmor Robes, especially when you add the Novice Hood. I think there are a few characters who wear adept-level robes at level 1, whom you might be able to kill, but I'm not 100% sure about that.
Dawnstar hidden chest
Another decent set of light armor that you can easily acquire, that often doesn't occur to me, is simple Leather. It's a significant upgrade over the Imperial Light and Stormcloak variants from Helgen, and even though it doesn't appear in random loot and merchant inventories until level 6, it can actually be obtained right in Riverwood at level 1! You can forge it yourself, and you should have a fair amount of hides for making leather from the bear in Helgen, along with that pack of wolves on the path between the Guardian Stones and Riverwood. There are also some more wolves, elk, etc. roaming the outskirts of Riverwood, and you can of course buy leather and strips from Alvor. A set of tempered leather will offer around 30-40% better AR than untempered Imperial Light, and won't need to be upgraded for a long time.
There's also some Ebony weapons in Lost Valkygg, but that can be a serious risk for level 1 characters, probably not worth trying unless you want to bump the difficulty down...
If you side with Hadvar, you can pick up Iron and Steel Ingots at Alvor's forge without stealing them. You can also get a Ebony Weapon early on by heading to Solitude and accepting the bards college quest involving Olaf One-Eye, he's guaranteed to carry one and you don't even have to kill him on your own, an essential bard-ghost will do it for you.
Teccam has already mentioned Ironbind Barrow but I'd like to just place a little more emphasis on how great that little tomb is for entry level characters...
1. The Fiery Soul Trap Enchant! Hugely cost effective way of increasing low level damage...
2. Steel Plate Armor from Salma, she disappears forever at the end of the quest anyway, may as well let her die...
3. Salma carries an enchanted weapon so take that...
4. Become Ethereal word wall...
5. 2 X Black Soul gems (guaranteed)
6. Enchanted helmet on the trap.
7. A leveled ring of destruction from Beem Ja...
8. The Reanimate Corpse spell tome.
9. Leveled 'Honed' draugr weapon (always enchanted with cold) from Gathrik the boss...
And then you exit which leads you out onto a hill with a dwarven shrine displaying....
10. 1 enchanted two handed weapon
and
11. Two chests containing expensive jewellery...
Ironbind Barrow FTW!
For my casters I usually do 2 simple glitch/exploits (unless you are opposed to them) shortly after leaving Helgen. I loot the chest in Dawnstar (there are usually novice robes of a random school) then go to Winterhold and take a plate with me and push through the wall for a couple quick chest loots full of soul gems, staves, spell tomes, and some other loot. This gives any caster I play an awesome leg up.
Both of these can be found on youtube.
Cheers.
This thread is great, this is the kind of thing I love to exploit. I've used several of the ones mentioned, but here are some of the tricks I like that no one has brought up:
1. Not a super great bonus, but somewhat useful, loot the Amulet of Talos from Roggvir after he's decapitated in Solitude. Be aware that it's a quest item though, so don't keep it unless you're cool with not being able to drop it
2. Get a scimitar from the Dainty Sload, a ship north of Solitude. To get there follow the path that runs under the arch that Solitude is built on. It's on the same tier as Elven/Skyforge Steel and only needs steel smithing.
3. Farkas drops a Skyforge Steel Greatsword at Dustman's Cairn when he transforms into a werewolf, and you are free to pick it up as he switches to an iron greatsword
4. Eorlund Gray-Mane's blacksmith chest is accessible if you jump out of the level. It's fairly complicated, so I'll just link this youtube video. You get all of his merchant gold in addition to a pile of items and smithing materials. He won't be able to buy anything but you if you pay for training can sell back to him or just slowly walk down to Adrienne Avenicci.
5. Silver Swords found in the Companions quest line do +20 to undeads, dual-wielding them for a whopping +40 damage has gotten me through many a draugr-infested ruin. Even at low levels of enchanting you can add a pretty decent amount of fire damage to make yourself the ultimate low-level undead killer
6. Every time you open the inventory of the skeleton on the Altar of Xrib, an unmarked location near Sightless Pit, a new copy of "The Doors of Oblivion" spawns, I've sold about 200 of them before, really useful if you spend a bit of time "mining" the books. Sure, it's tedious, but it gets me tons of soul gems and spell books when I trade with Farengar or the college wizards.
about the ebony,
there is a place south of Ivarstead with at the bottom a lot of skeletons and a draugr or three with a boss in the middle. there is also a dragon and a long staircase with at the top a draugr death overlord. he is not really fast so if you just take him to the edge of the staircase, FUS RO DAH him of. the fall will kill him (if he falls straight down, an ddoesn't land on the rocks) and you have a free Ebony war axe