Linkedin profile: Data engineer | Zach Wilson

 

Staff Data Engineer (commercial products)Staff Data Engineer (commercial products)

Airbnb · Full-timeAirbnb · Full-timeFeb 2021 - Present · 1 yr 7 mosFeb 2021 - Present · 1 yr 7 mosSan Francisco Bay AreaSan Francisco Bay Area

• • Decreased the landing time of Airbnb's unit economics dataset from 3 days to 1 1/2 days. Built a long-term roadmap to further increase the quality of this critical dataset.
    
    Improved the quality of online systems at Airbnb to be easily compilable by Spark pipelines to compute metrics offline.
    
    Lead a team of 6 engineers to deliver on the MIDAS initiative to increase the data quality of the Commercial Products org.
    
    Upleveled smart pricing at Airbnb by improving the feature engineering and latency of the data used to train the smart pricing model.Decreased the landing time of Airbnb's unit economics dataset from 3 days to 1 1/2 days. Built a long-term roadmap to further increase the quality of this critical dataset. Improved the quality of online systems at Airbnb to be easily compilable by Spark pipelines to compute metrics offline. Lead a team of 6 engineers to deliver on the MIDAS initiative to increase the data quality of the Commercial Products org. Upleveled smart pricing at Airbnb by improving the feature engineering and latency of the data used to train the smart pricing model.
• • Skills: Big Data · Scala · Apache Spark · SQL · Machine Learning · Apache Superset · Apache Airflow · Mentoring · Team Leadership · Data Visualization · Data Analysis · Java · Python · Linux · Git · Googling

NetflixNetflix

1 yr 10 mos1 yr 10 mos

• • Senior Software EngineerSenior Software Engineer

    Oct 2018 - Mar 2020 · 1 yr 6 mosOct 2018 - Mar 2020 · 1 yr 6 mosSan Francisco Bay AreaSan Francisco Bay Area

    • • Built a machine learning feedback system that allowed security engineers to label corporate user behavior as risky or not risky.
        
        Built Asset Inventory - a graph database solution that is a map of all of Netflix's cloud infrastructure.
        
        Built a machine learning feedback system that allowed security engineers to label corporate user behavior as risky or not risky. Built Asset Inventory - a graph database solution that is a map of all of Netflix's cloud infrastructure.
    • • Skills: Big Data · Scala · Apache Spark · Cybersecurity · SQL · Machine Learning · Apache Airflow · Team Leadership · Data Visualization · Data Analysis · Java · Python · JavaScript · HTML · Cascading Style Sheets (CSS) · Node.js · React.js · D3.js · Linux · Git · PostgreSQL · REST APIs · Spring Framework · GooglingSkills: Big Data · Scala · Apache Spark · Cybersecurity · SQL · Machine Learning · Apache Airflow · Team Leadership · Data Visualization · Data Analysis · Java · Python · JavaScript · HTML · Cascading Style Sheets (CSS) · Node.js · React.js · D3.js · Linux · Git · PostgreSQL · REST APIs · Spring Framework · Googling
  • Senior Data EngineerSenior Data Engineer

    Jun 2018 - Oct 2018 · 5 mosJun 2018 - Oct 2018 · 5 mosLos Gatos, CaliforniaLos Gatos, California

    • • I built a pipeline that measures the cloud infrastructure impact on AB tests, saving Netflix millions by allowing them to make smarter AB test rollout decisions. I built a pipeline that measures the cloud infrastructure impact on AB tests, saving Netflix millions by allowing them to make smarter AB test rollout decisions.
    • • Skills: Big Data · Apache Spark · Cybersecurity · SQL · Machine Learning · Apache Airflow · Data Visualization · Data Analysis · Java · Python · Linux · Git · Googling

Data EngineerData Engineer

FacebookFacebookAug 2016 - May 2018 · 1 yr 10 mosAug 2016 - May 2018 · 1 yr 10 mosMenlo Park, CaliforniaMenlo Park, California

• • - Managed a 10 PB+ Hive data warehouse
    - Consolidated and conformed company-wide growth metrics (across WhatsApp, Instagram, Messenger, and Facebook) into a single, company-wide view.
    - Optimized machine learning feature set generation pipelines (200+ TB/day) from having a 4 day latency to having a 1 day latency. While also dropping compute costs for those pipelines 4x.
    - Reduced core notification data set latencies from 36 hours to < 8 hours.
    - Migrated 50% of notifications pipelines from using Hive to use Spark, Presto, or real-time streaming.
    - Cut compute cost from notifications pipelines by 40% over the course of 9 months.- Managed a 10 PB+ Hive data warehouse - Consolidated and conformed company-wide growth metrics (across WhatsApp, Instagram, Messenger, and Facebook) into a single, company-wide view. - Optimized machine learning feature set generation pipelines (200+ TB/day) from having a 4 day latency to having a 1 day latency. While also dropping compute costs for those pipelines 4x. - Reduced core notification data set latencies from 36 hours to < 8 hours. - Migrated 50% of notifications pipelines from using Hive to use Spark, Presto, or real-time streaming. - Cut compute cost from notifications pipelines by 40% over the course of 9 months.
• • Skills: Big Data · Apache Spark · SQL · Machine Learning · Data Visualization · Data Analysis · Java · Python · JavaScript · React.js · Hadoop · MapReduce · Linux · Git · Googling

Leslie Lamport

 Leslie B. Lamport (born February 7, 1941 in Brooklyn) is an American computer scientist. Lamport is best known for his seminal work in distributed systems, and as the initial developer of the document preparation system LaTeX and the author of its first manual.^[2] Lamport was the winner of the 2013 Turing Award^[3] for imposing clear, well-defined coherence on the seemingly chaotic behavior of distributed computing systems, in which several autonomous computers communicate with each other by passing messages. He devised important algorithms and developed formal modeling and verification protocols that improve the quality of real distributed systems. These contributions have resulted in improved correctness, performance, and reliability of computer systems.^{[4][5][6][7][8]}

Early life and education[edit]

Lamport was born into a Jewish family in Brooklyn, New York, the son of Benjamin and Hannah Lamport (née Lasser).^[9] His father was an immigrant from Volkovisk in the Russian Empire (now Vawkavysk, Belarus)^[10] and his mother was an immigrant from the Austro-Hungarian Empire, now southeastern Poland.

A graduate of Bronx High School of Science, Lamport received a B.S. in mathematics from the Massachusetts Institute of Technology in 1960, followed by M.A. (1963) and Ph.D. (1972) degrees in mathematics from Brandeis University.^[11] His dissertation is about singularities in analytic partial differential equations.^[12]

Career and research[edit]

Lamport worked as a computer scientist at Massachusetts Computer Associates from 1970 to 1977, SRI International from 1977 to 1985, and Digital Equipment Corporation and Compaq from 1985 to 2001. In 2001 he joined Microsoft Research in California.^[11]

Distributed systems[edit]

Lamport's research contributions have laid the foundations of the theory of distributed systems. Among his most notable papers are

• "Time, Clocks, and the Ordering of Events in a Distributed System",^[5] which received the Principles of Distributed Computing (PODC) Influential Paper Award in 2000,^[13]
• "How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs",^[14] which defined the notion of sequential consistency,
• "The Byzantine Generals' Problem",^[15]
• "Distributed Snapshots: Determining Global States of a Distributed System"^[16] and
• "The Part-Time Parliament".^[17]

These papers relate to such concepts as logical clocks (and the happened-before relationship) and Byzantine failures. They are among the most cited papers in the field of computer science,^[18] and describe algorithms to solve many fundamental problems in distributed systems, including:

• the Paxos algorithm for consensus,
• the bakery algorithm for mutual exclusion of multiple threads in a computer system that require the same resources at the same time,
• the Chandy–Lamport algorithm for the determination of consistent global states (snapshot), and
• the Lamport signature, one of the prototypes of the digital signature.

The Man Who Revolutionized Computer Science With Math

 Here is the link.