If you can put together an app on Heroku, you can configure a web app server. I don't really see how Herkou saves much work over AWS and Linode. It might take a few minutes or a few hours, but if you know how to SSH in and run "rails server" you've got a platform for your MVP.
You can then easily evolve your app server in whatever direction you need to.
Heroku gives you a load balancer and auto-scaling. Time not spent in having to configure and maintain that piece of infrastructure is time spent in working on the MVP.
Another benefit of writing code targeting platforms like Heroku is that when your app gains traction you can easily scale since the app would've been (forced to be) written keeping horizontal scaling in mind.
What are the odds that your MVP needs load balancing and auto-scaling? It's far more likely that your app could run in a single process on a single box for a very long time.
As always, it's a matter of thinking about that up-front.
At what point is this a project that can be considered self-sustaining? (i.e. pays for its own development). Can you reach that point with a single box?
And if you can, do you have some leeway for growth after that point so you can actually develop a more scalable solution?
For some projects, that results in being OK with a single proc/single box approach. For other projects, you realize that unless you acquire ${LARGE_USER_COUNT} users quickly, you might as well give up on it - so you have to build in some scalability.
There's also expected growth rate - if for whatever reason you think your project might easily grow from single box to "needs more than one" in a short amount of time, you better plan for that.
Devil's advocate: The app doesn't get traction in the first place because it's slow and lacking features (because you spent time scaling instead of coding features).
It is not an all or nothing proposition. Building an app keeping in mind scaling doesn't mean you go for complex initial setups. All you need to do is make some simple design decisions to build the app so that it can scale to multiple web front-ends if need be. IMHO - this is usually the most common first step in scaling.
Also building your app thinking about a bit of scaling upfront gives you more time to work on your app, because you don't even have to setup and configure your web servers - getting an app running in Heroku is the most painless way of app hosting that I've seen.
You can then easily evolve your app server in whatever direction you need to.