Login

Facebook's data management system is 'like Candy Land' for engineers

Facebook Nasdaq NYC Stock

How does Facebook manage the insane amount of data that one billion users pour into the service nearly every day? Wired explains how the company manages and analyzes the staggering amount of data using custom-built software solutions like Prism and Scuba. Facebook has the world's largest Hadoop cluster — a group of servers connected using Hadoop's open-source software — with more than 4,000 machines containing over 100 petabytes of data. Even more impressive, it isn't Facebook's only cluster. The problem of managing the constantly swelling system requires some of the greatest engineering and computing minds to solve, but as database administration and storage systems manager Santosh Janardhan told Wired, "if you're a technical guy, this is like Candy Land."

The Verge
X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new Verge username and password

As part of the new Verge launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to Verge going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new Verge username and password

As part of the new Verge launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to Verge going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.
Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.