A few of us have talked about renting some dedicated hosting. Primarily collocation hosting. It'd mean upfront purchase and ownership of server, and only a monthly cost for rack space, power, internets, bandwidth.
Personally, I'm very tired for getting pw3nd by shared hosting providers. I do understand that it can't be perfect, but you still have to have a decent uptime, and you can't lie. I'd go into detail about how 1and1 has screwed with me over the years but I'll be brief (tried to cancel webhosting but keep Exchange, but told me I had to get a different Exchange account and transfer my mail, or else it would be lost since the Exchange was 'attached' to the webhosting. Seems like a billing problem, not MY problem. Also, if I call you to tell you your server has been down for 30 minutes, and I call and you fix it while I'm on hold, don't passively treat me like I'm bothering you because everything is working)
Also, their .net app restrictions are not the best.
Okay, it wasn't too brief. But I've played out the numbers assuming I purchase the server, and it'll cost you less than good meal per month.
Update: +1 BJ +1 SP +1PS
Check out http://ntsource.com/web-hosting/colocation-web-hosting.html
Update: BJ and PeterS have brought up cloud hosting options like Amazon's EC2 and GoGrid. I am not very familiar with these, and they seem to charged based on application. But if it were possible to have a single app running windows server 2008 VM, I could see it working.