Abhishek Shah

Building an Audience Estimation System

Sat, 08 Jun 2024 12:00:00 -0700

Cocoon

Cocoon is a User-Count Estimating Query Engine on top of ZDB-On-Flash. This system is an integral part of Ads Infra at Meta serving 40K qps under 100ms P90 latency.

How does Cocoon store its data

Columnar Data Stored in Ordered Format
UserID Format

How is the Data Stored in ZDB

Multiple UserIds per Row: Each ZDB Row Value has multiple UserIds in it. Optional Attribute(s). Efficient Fetching.
Static Ranking: Static Rank of UserIds stored for even distribution. Bi-Directional.
Compression: Compressed using Elias Fano Format. Quasi-Succinct. ~ Theoretical Best.
Sorting: ZDB Keys are sorted. Static Ranked UserIds are sorted across ZDB Keys and also, within each ZDB Value.

How is the Query Done

Round 1: Fetch 0th Bucket for each term in Query.
Round 2 (Optional): Fetch more buckets to improve estimate.

Estimation & Extrapolation

Query-Time Sampling (Not Data-Load Time Sampling)

Sampling of Data at Load Time leads to very poor estimation. Hard to determine how much to sample.
We load all the data into Store. At Query Time, we determine how much we want to read till we are confident of a good estimate.

Extrapolation

In the int64 space, we determine what value we have reached at and what the hit count is.
We then extrapolate.
Srank-based ordering removes biases based on how userIds are created.

Horizontal Scaling

The data to process at query time is still a lot.
Horizontal Scaling to the rescue! Divide the data and have parallel processing done on it using a TW Tier.

Leaf Layer

32 Leafs per tier.
Each Leaf reads only its own partition of data.
2 Leafs can never read the same data… Ever.
32 Data Partitions exist.

Aggregator Layer

Parses Query
Throttles Callers
Fans out query to Leafs
Handles Leaf Fail Over

System Architecture Diagram

+-------------------------------------------------------------+     +-------------------------+
|                  [Aggregator / Indexer Tiers]               |     |       Graph API         |
|  +------------------+ +------------------+ +--------------+  |     | +---------------------+ |
|  |                  | |                  | |              |  |====>| |Cocoon PHP Component | |
|  |   Aggregator     | |   Aggregator     | |  Aggregator  |  |     | +---------------------+ |
|  |       ||         | |       ||         | |      ||      |  |     +------------^------------+
|  |    Indexer       | |    Indexer       | |   Indexer    |  |                  |
|  +--------^---------+ +--------^---------+ +------^-------+  |     +------------v------------+
+-----------|--------------------|------------------|----------+     |    Supported Features   |
            |                    |                  |                | - Reach & Frequency     |
+-----------v--------------------v------------------v----------+     | - Reach Estimate        |
|                          ZippyDB                             |     | - Bid Suggestion        |
|    +----------------+          +----------------+            |     | - Outcome prediction    |
|    | ZDB node PRN   |<========>|  ADB node LLA  |            |     | - Pacing                |
|    +-------^--------+          +--------^-------+            |     | - Analysis              |
|            |                            |                    |     | - Audience Insights     |
|            +-----------+    +-----------+                    |     +-------------------------+
|                        |    |                                |
|                    +---v----v---+                            |
|                    |ZDB node ATN|                            |
|                    +------------+                            |
+--------------------------------------------------------------+

Cocoon Ingestion Pathways

1. Daily Delta Bulk Updates (per data source)

FB user profile
AdEnv (device, os, placement)
Friend connection (of page, group, application and event)
Location and Geo info
Interests
Instagram info

2. Real-time Updates (Dispatchers per data source)

Custom audience change dispatcher
Partner category change dispatcher
Look alike change dispatcher

Building Our Own CDN

Sat, 08 Jun 2024 12:00:00 -0700

Building Our Own CDN

Brief History

Pre-2011: Business contracts were maintained with external CDN companies.
Traffic Allocation: Guaranteed each external CDN a stable share of the streaming traffic.
Simplicity of Steering: All of the Video CDN Steering logic fit entirely within:
- 5 Java Classes
- ~60 lines of code
Scale: Handled 1/3 rd of peak internet traffic in the US.
Strategic Shift: A decision was made to build an in-house CDN.

Predictable Viewing Patterns

The foundational architecture of this custom CDN system relies on a predictable data distribution rule: a small fraction of content generates the bulk of internet traffic.

Mon, 01 Jan 0001 00:00:00 +0000

Maya Platform at Coupang

The Maya platform virtualizes compute and network for our tenants on shared physical infrastructuire. This is currently a major in-flight program with dynamic tenancy on the bare-metal path. Tenants self-serve VMs (with GPU passthrough) or bare-metal instances; instances in the same nodepool share a tenant VPC (private network) and base image. Conceptually similar to AWS VPC (on the Nitro substrate) plus AWS Managed Node Groups, but built for our internal fleet.

Mon, 01 Jan 0001 00:00:00 +0000

ABHISHEK SHAH

Sunnyvale, CA | (650) 630-9280 | shahabhishek@gmail.com | LinkedIn: /in/abhishekshah

SUMMARY

Senior engineering leader with 25+ years architecting and operating cloud compute platforms at hyperscale. Currently Senior Director at Coupang Intelligent Cloud (CIC), leading three engineering organizations — API Platform, Kubernetes Platform, and Virtualization — that together deliver Coupang’s internal GPU cloud and broader compute substrate. Lead Software Architect of CIC, founding member of the program, technical owner of CIC’s Kubernetes platform (Cortex) and the CompositeApplication CRD primitive (3 patents filed). Prior tenures at Netflix (designed and led the OpenConnect CDN control plane), Facebook/Meta (led the ad-audience platform processing a trillion updates per day), Google (built the original Kubernetes L4 SDN and DNS — code paths still running in every Kubernetes cluster today), and Roblox (next-gen pub/sub at 5M msgs/sec). Operate with influence across software, infrastructure, security, and partner-engineering organizations; partner-engineering relationships at the architecture level with NVIDIA, AWS, and Run:AI. Coupang Bar Raiser for senior and principal hiring.