Scalability -- Doing It Right

Descubierto vía http://highscalability.com/

Nice Ideas/Tips para hacer escalable un site via Nati Shalom http://natishalom.typepad.com/nati_shaloms_blog/2008/05/twitter-as-an-e.html

vamos ideas para hacer algo escalable a lo grande, grande ...

Asynchronous event-driven design: Avoid as much as possible any synchronous interaction with the data or business logic tier. Instead, use an event-driven approach and workflow

Partitioning/Shards: Design the data model to fit the partitioning model

Parallel execution: Use parallel execution to get the most out of available resources. A good place to use parallel execution is the processing of users requests. Multiple instances of each service can take the requests from the messaging system and execute them in parallel. Another good place for parallel processing is using MapReduce for performing aggregated requests on partitioned data

Replication (read-mostly): In read-mostly scenarios (LinkedIN seems to fall into this category well), database replication can help load-balance the read load by splitting the read requests among the replicated database nodes

Consistency without distributed transactions: This was one of the hot topics of the conference, which also sparked some discussion during one of the panels I participated in. An argument was made that to reach scalability you had to sacrifice consistency and handle consistency in your applications using things such as optimistic locking and asynchronous error-handling. It also assumes that you will need to handle idempotency in your code. My argument was that while this pattern addresses scalability, it creates complexity and is therefore error-prone. During another panel, Dan Pritchett argued that there are ways to avoid this level of complexity and still achieve the same goal, as I outlined in this blog post.

Move the database to the background: There was violent agreement that the database bottleneck can only be solved if database interactions happen in the background. (NOTE: I recently wrote a more detailed post explaining how you can effectively move the database to the background.