Dinner and learn: Bigtable in action (Google Cloud Next '17) | Aug 30, 2021

Aug. 30, 2021

Here is the link. 

I like to focus on one slide 

23:44 / 55:07

DBYO: don't build your own

• abstractions and integrations already exist for popular use cases:
• graph database
• time-series database
• time-series database
• geospatial querying

Integration with Cloud Dataflow

Integration with Cloud Dataproc

Server Density | Monitoring use case | David Mytton | Co-Founder & CEO

Agenda

• Old system: PHP + MongoDB
• Early 2016 - Challenges
• OpenTSDB + Bigtable selected
• The migration
• New system: OpenTSDB + Bigtable
• Lessons learned #1
• Lessons learned #2

Old system: PHP + MongoDB

27:55/ 55:07

Early 2016 - challenges

• Time sink - Spending a lot of time maintaining infrastructure
• We want to build more - Small team & feature limitations of custom code
• Unpredictable scaling - New shards, new replicas, impact of migration data around cluster
• Expensive hardware - SSDs + RAM, Optimising most common uses cases needs data in memory - very expensive 

30:26/ 55:07

Challenges 

OpenTSDB + BigTable selected

• Managed service - Zero time spent maintaining underlying infrastructure
• OpenTSDB actively developed - More features than we would have time to build ourselves, but we can contribute upstream
• Linear scalability - Predictable model for reads/writes and costs
• Specialist datastore - Specifically designed for time series, so more efficient and lower cost