Links to Study

Chord: Building a DHT (Distributed Hash Table) in Golang

Cassandra modelling

Cassandra Data Modeling Best Practices, Part 1

Basic Time Series with Cassandra

Write heavy workload. Choose DB with LSM indexed DB

Which NoSQL DB is more advantageous for IoT data?

Art of choosing a datastore

General difference between sql and no-sql

SQL vs NoSQL: The Differences - SitePoint

dropbox system design by Naren

Imgur

Netflix stream processsing

Patterns of Streaming Applications

SHA perfomance metrics

MD5, SHA-1, SHA-256 and SHA-512 speed performance | Automation Rhapsody

Youtube scaling

Indexing / Search Engine

ngnx

Inside NGINX: Designed for Performance & Scalability

Haproxy architecure

http://www.haproxy.org/download/1.2/doc/architecture.txt

Strore session in centralied location

Scalability for Dummies - Part 1: Clones

Load Balancer

Making applications scalable with Load Balancing

HAProxy Config

Load Balancing, Affinity, Persistence, Sticky Sessions: What You Need to Know - HAProxy…

Scaling storage

Scale timeseries databases in netflix

Scaling Time Series Data Storage — Part I

Netflix timeseries db

Scaling Time Series Data Storage — Part I

Kafka Design

Kafka Consumer Assignments - Signal

Live stream architecure.

Location based search + quadtree

Improved Location Caching with Quadtrees

Generating Global Unique ID

1. clocks across different computer are not synchronous all the time
2. NTP sometime roll back the time if it detect the computer clock is diverging (due to clock skew drift issues)
3. Using get_time_of_day() usec+counter+machine_name might be globally unique but slow because get_time_of_day() is slow. Also it might not be monotonically increasing (see #2)
4. get UUID in windows / DB’s can generate system specific globlal UID
5. if you need monotonically increasing global ids (you want to sequence events), best option is to just have a single server from where everyone ask for a unique id
6. gettimeofday() gives UTC time (sec,microsecond). but gettimeofday is slower. So you can just use machine_name+counter if you want speed. you have to backup the counter state once in a while.

Flikr approach: (distributed unique primary keys)

Use mysql auto incr counter with only storing the last incremented number in DB (so that the size of the db dont grow). Use REPLACE INTO command

Ticket Servers: Distributed Unique Primary Keys on the Cheap

Generating unique IDs in a distributed environment at high scale.

Zookeeper.

1. Writes goes to the leader. OK is only sent when quorum writes happes.

2. Read happens with any server. read can be stale for a while(eventual consitent)

3 Watch can be set for any ephimeral node

4. can be used for service discovery

How Apache Zookeeper Works internally?