Find a system design coach

May 27, 2018

Introduction

I had this past Saturday system design study, I had a few times to feel so tired and sleepy. I know that I may have problems down the road. I need to find a coach. I start to search who will be my best coach.

First two hours learning 

May 28, 2018

I just could not believe that I had two hours training from my friend. Two hours learning makes big difference.

Here is my notes I took in the training.

Second time coaching

May 29, 2018

It takes time for me to get ideas how things work out. I need to work on one thing a time. That is the reason I got a coach No. 3, who has a lot of industry experience. Basically he went over grokking system design lecture notes and discussed with me. If I have any question, he gave me answers and make sure that I can understand the concept by his explanation. He could explain things so clearly and I totally quickly understand.


Content dilivery network:  CSS, spritemap

Amazon S3/ cloudfront

CDN -> Amazon /cloudfront

Shard -> horizontal/ row/ databases
range-based shard

sharding - horizontal, columns
Direct-based partition
partition key values
Index - sacrife the space to get shortest time
Explain SQL plan -> look up index needed or not
Proxy - reverse proxy
enforce/ request/ rule/ login information / map/ API - url reroute

Queue - RabbitMQ, producer/ consumer

Data/ service   logging shipping/ database server
microservice
Horizontal
encyms   Deployment/ encyms

SQL/NoSQL JASON    redis
NoSQL, structure/ unchange
Cassandra/ couchDB

HDFS or cluster FS

RAID - redundant array of independent disks

I/O in parallel - read/ write

RAID - cluster / Redundancy

I was strongly advised to look at open source tool called Swagger / OSI open specification

Go over netflix stream system design

7.  File processing workflow
8.  Data Deduplication

2. Range based partition

10 caching   Least recently userd (LRU)

RAID - partition key -> 256 buckets
Blob - save image

Elastic search - NoSQL search , document - tuning score, criterias: two words how close they are, subject

high speed cache - google CDN

8. Metadata sharding
Partition
user id    cassandra

Memcache   k-V memcache


Given an example about tag on video, threshold > 3,
caching 20%

Redis - can be applied to anywhere
Cluster -
Elastic search
Cassandra - big data handling, very good write speed
build index x ( problem to build index)


Storage - log/ by minute/ 60 seconds
search

Overall, learning is much better with a friend to go over the system design of Netflix stream, and I had chance to ask a few questions. I was not afraid to ask questions, and then the friend gave me very good experience based on his work experience. The time spent is 60 minutes.


My task is to understand those tools, and be able to freely apply them in the system design. Redis, swagger, Elastic search, Cassandra, load balancer, Amazon/ Google/ F5 load balancer. The coach shows me how easily he can come out ideas how to design step by step.