Kind of.. There are a few more things you would have to think about. The cache for example uses the filesystem as well. Since 5.4 you can actually use memcache to avoid this which would allow a nice scale up...

Putting all files in a database isn't something I recommend at all. There's quite some overhead in doing that. I'd like to have this option, especially because I build partially protected sites.

However, you can easily avoid this. If you really have huge sites you can probably afford some nice hosting where files are located on a SAN. This is what we always did when things got huge. If your wallet is a bit smaller, use NFS and it works as well.

As far as I know, I haven't seen any popular cms that puts all files into the database. We've got that feature in some systems, but only partially. You always have a few files, no matter what. And even if it's just a css files for the theme. You have to put these files on two servers as well. BLOB isn't going to work for everything.. And yes, you can manually copy these files to all servers, but if you have such huge sites you don't want to implement such an unprofessional thing.. You would end up using NFS again.

If it's just about failover, you can easily rsync the files and create a database dump to a second servers...

A whole lot of possible solutions I think (:

Remo's point is totally right, I'd like to add another point. You can always use a hoster that hosts your cms on a cloud or cluster.

You can look for "cloud hosting" too.
There are really some hosters that offer your clustered redundant and failsafe hosting plans.