Acronyms
CR – Challenge Rating – Used to determine what monsters/traps/other obstacles are an appropriate challenge for a party of adventurers.

What I like about 4E:

• It brings characters to a more narrow range of power.  In other words, weak characters (often made by new players) aren’t nearly as helpless, while powerful characters don’t roll over entire CR-appropriate dungeons without even trying.  You know that awful character that didn’t contribute to the party in 3.5?   Pretty much gone.  There’s still a difference between well-built and poorly-built characters, but characters that first time players build can actually do their share for the party without someone experienced basically having to build their character for them.  Those uber-awesome characters that simply broke the game are gone, too (google Pun Pun if you don’t know what I’m talking about).  Was it cool to build those characters?  Sure.  Fun for everyone else?  Probably not.
• CR system changes & experience.  If you never DM’d in 3.5  I can’t impart with words on you how utterly awful this system was.  4E made encounter creation for DMs incredibly easier, streamlined and more effective.  Instead of having some awkward system where you have to calculate the combined CR of each encounter (3.5), in 4E each monster simply has an experience point value.  Monsters are much better balanced across similar rewards as well.  In 3,5, monsters of the same CR (supposedly equivalent challenges) could have drastically different power levels.
• Monster roles.  Again, something you wouldn’t fully understand unless you DM’d both editions.  4th designed monster stat lines with a specific role in mind (such as brute or artillery).   This makes it incredibly easier to find an appropriate monster to create the encounter feel you desire, as you can simply search monsters that fit the role.
• Skill challenges.  Finally rewarding characters for outside of combat actions is great.  Were they done poorly?  Meh.  I’d rate them as mediocre.  But at least having them as a guideline is incredibly helpful to reward characters for their out of combat actions.
• Party member roles.  In 3.5, the expected party roles weren’t clearly defined, which made it difficult to determine if all of your party’s bases were covered.  In 4th, however, party members have clearly defined roles and ways to work together.  This gives new players a clear idea of their purpose in the group, and makes a lot of new classes and mechanics open up.  For example, the 3.5 Marshal goes to bed every night wishing he’d wake up as a 4E Warlord.
• Party synergy.  4E does an incredible job at making combat about great parties instead of great characters.  If you’ve never played 4E, you probably won’t appreciate how much 3.5 lacks in this area.  Party synergy is a much more powerful tool in 4th edition, and may even eclipse individual power for importance.  In 3.5  one incredibly powerful character could (and often would) carry the party through encounters, making it difficult for DMs to scale encounters to both challenge the powerful character while not wiping the floor with the weaker characters.  My last 3.5 campaign was often a good example of this.  I played a wizard, and oftentimes other characters would literally have no positive contribution or died (or both) while my character would often emerge unscathed and handle the encounter mostly (or in some cases, literally) single-handedly.  In 4E, the focus is on cooperation.  The most powerful groups are those that can key off of the actions of their teammates.  The Warden’s slowing powers means that the monsters can’t get to the Ranger, who turns around and uses World Serpent’s Grasp to knock them prone on hit.  Which gives combat advantage and bonus damage to the Barbarian thanks to Headsman’s Chop.  Forced movement and other battlefield control make coordination much more important.
• All classes are interesting.  In 3.5, classes like the Fighter or roles like ‘healbot’ got real boring, real fast.  4E did a much better job at making them fun and diverse to play.  The Fighter and the healbot all have more active roles in combat (healers can heal and attack in the same round, or heal as part of an attack) while simple classes like the Fighter can now add options (Use Cleave to hit multiple enemies, Tide of Iron to move them around, etc).
• Rules make sense.  I’ve taught literally dozens of players both 3.5 and 4E.  The time it takes to get someone up to playing speed in 4th edition is much shorter.  Just try explaining skills in 3.5 to someone who has never played an RPG before.  Or grappling.  Spell-casting & memorization. You get the point.  Technically, yes, you can just shove a Fighter character sheet to those new players, but that really sucks for players as A) some don’t want to play the big dumb fighter and B) the Fighter really only is competitive if the DM makes it so.
• Game play doesn’t fall apart at higher levels.  Sure, you may have played a high level campaign in 3.5, but you had to do a lot of band-aid fixes and turn a blind eye to other pieces in order to make that work, even if you didn’t realize you were doing them.  There are plenty of posts around the Internet that go in to these, so I’ll just leave it at that.
• Tactical combat.  For those that love tactics and maneuvering, there is an immense difference.  In 3.5, you basically ran up to a monster and hit it until one of you fell over.  Your options might as well have been Attack, Use Item, Flee (kudos to those that get the reference).  In 4E, combat feels a lot more fluid, moving between opponents, adjusting your position across the battlefield and you have a lot more variability in what you do.  Every character has two (or more) options in addition to the basics (melee attack, ranged attack, grapple, etc) that give them a variety of options instead of saying, “I attack.  Done.” every turn.
• Formatting is much cleaner.  While perhaps a minor thing, once you’ve gotten used to it, the 4E rules blocks, power descriptions and the like are laid out in a much more intuitive fashion and the important information is much easier to find quickly.  The character builder, compendium and adventure tools are superb additions if you don’t mind the premium.

What I don’t like about 4E:

• Customization is much more limited.  You could make a character that did pretty much (if not literally) whatever you wanted in 3.5.  Any crazy combination you could dream of was possible.  In 4E, you can basically put up to two classes together, and dabble in another.  You get two prestige classes on top of that.  Now, with that being said, the customization level for 4E is still immense (and much better than 3.5 if you’re just looking at core).
• Feat taxes.  These (Expertise feats, Improved Defenses, etc) are dumb.  Essentially, there are certain feats that your character is ‘required’ to take in order to keep up stat-wise.  There’s no point in allowing choices if they’re going to be inferior to just taking these math feats.
• Crazy stuff is harder to do.  The rules do a much better job of handling the things they want you to do, but as a side effect the random crazy stuff is harder to pull off (Polymorph and Shrink Item, for example).  It’s hard to put my finger on this exactly, but I think if you’ve played for any decent length of time you know exactly what I’m talking about.  While it sounds small, some of these moments were the best stories.
• Every class follows the same “framework”.  In order to bring characters all to the same playing field, they brought them all on to the same framework.  This has made things much, much better balanced, but at the expense of removing the individuality of a lot of classes.  Weird classes like the Factorum are pretty much gone and a lot of the coolest prestige classes (Chameleon) lost a lot of their flavor in translation.  The drastic difference in flavor between the Wizard and Fighter isn’t nearly as distinct.

In the end, I feel that 3.5 offers more player choices and flexibility, but 4E is, mechanically, a better game in almost every way.  For players new to the RPG genre, I would generally suggest 4E as a much better learning experience than 3.5.  I find that the biggest reason to continue playing 3.5 is for those that enjoy fine-tuning every minor detail of their characters, although for many, this is not necessary.