Browsing the Apple forums I happened upon a post with the most excellent comment from crisB regarding speeding up Xcode 4.
My Xcode 4 was PAINFULLY slow...
Basically you need to remove the workspace file from the contents of your project.
Here's a before and after shot. My project's file was 50MB's and my Xcode 4 was running EXTREMELY slooooooow; I would have the spinning beach ball for several MINUTES!!
So this Automator workflow / droplet basically deletes the "project.xcworkspace" from the file you select or drop.
PS What the heck happened to May!!!