spotsearch

Friday, September 28, 2007

Some Big Things from Small Companies

Google's main advantage over other companies (and Yahoo in particular) is its distributed infrastructure. By abstracting away the complexity of building scalable, fault-tolerant software using their Google FileSystem (GFS) and MapReduce programming layer, they greatly increase the productivity of their developers and reduce their time-to-market with scalable code.

The open source community has not stood still, and there are some great projects aiming to do similar things. For example, at Kosmix, where I work, we've decided to release our Kosmos File System as open source (download kfs here). Hbase for example, has had support from Powerset and Zvents. In addition, my friends Ethan and Doug at Zvents have been working on cool new things to-be-announced.

The folks at Yahoo have also done a great job of supporting open source scalable software as well, with their strong support of hadoop and Powerset has been using it on Amazon's EC2 computing service. . All of the pieces are falling together for start-ups to build their own scalable infrastructure inexpensively.

What about Google? They'd be smart if they one-upped everyone and released their scalable infrastructure to the public, perhaps as a hosted service. This would make it easy for them to become the defacto supplier of scalable computation, making start-up acquisition and integration easier for them, in addition to expanding their offerings. Then, anyone developing a viral Facebook app would find a natural home for building their app to handle the computation load.

0 Comments:

Post a Comment

<< Home