Cilium Codebase Overview

Cilium is an open-source, eBPF-based networking, security, observability, and service mesh solution for Kubernetes and containerized environments. It powers cloud-native infrastructure with high-performance dataplane capabilities, replacing traditional components like kube-proxy while providing L3-L7 policy enforcement, load balancing, and deep visibility.

What is this project?

Cilium (cilium/cilium) is a CNCF-graduated project implementing a fast, scalable dataplane using eBPF bytecode loaded into the Linux kernel. The userspace agent (written in Go) compiles and manages eBPF programs for packet processing at hooks like XDP, TC, and tracepoints. It delivers flat Layer 3 networking (native or overlay), identity-aware L7 policies, distributed load balancing, multi-cluster connectivity (ClusterMesh), and observability via Hubble, all integrated with Kubernetes CRDs.

graph TD
    A[Kubernetes API] --> B[Cilium Operator<br/>pkg/operator/]
    B --> C[Cilium Agent Daemon<br/>daemon/main.go]
    C --> D[eBPF Programs<br/>bpf/]
    D --> E[Kernel Hooks<br/>XDP/TC/Socket]
    E --> F[Policy/LB/Observability]
    G[Hubble UI/Relay<br/>pkg/hubble/ ] --> F
    style D fill:#f9f,stroke:#333,stroke-width:2px

Why does it exist?

Traditional Kubernetes networking (e.g., iptables-based kube-proxy) suffers from scalability limits, high latency, and IP fragility for policies. Cilium fills this gap by leveraging eBPF for kernel-level efficiency: O(1) lookups via hash maps, socket-level redirects avoiding per-packet NAT, and identity abstraction decoupling security from volatile IPs. It scales to millions of services/pods without performance cliffs, supports advanced features like bandwidth control and DSR, and provides unified visibility absent in legacy CNI plugins.

Trade-offs: eBPF requires modern kernels (4.9+); fallback modes exist for older ones. Userspace compilation adds startup latency but enables dynamic updates without restarts.

Who uses it?

Targeted at platform engineers and SREs managing Kubernetes at scale (e.g., multi-cluster, hybrid cloud). Used by enterprises like Datadog, DigitalOcean, and banks for production workloads. Use cases:

• Replacing Flannel/Calico/ kube-proxy for east-west/north-south LB.
• Enforcing identity-based L7 policies (e.g., HTTP/gRPC/ Kafka).
• Multi-cluster federation with global service discovery.
• Observability (metrics, flows, L7 traces) via Hubble.
• Gateway API, Ingress/Egress, Service Mesh (without sidecars via eBPF).

Key Concepts

To dive into the code, master these:

1. eBPF Dataplane: Core innovation. Userspace in pkg/datapath/ loads/maps ELF bytecode from bpf/ into kernel hooks. Maps (e.g., sockmap, conntrack) enable stateful processing. See bpf/cilium_l3.c for L3 forwarding.

2. Security Identities: Pods get stable numeric IDs (not IPs) via pkg/identity/. Policies in pkg/policy/ compile to eBPF allow/deny lists. Decouples scale from IP churn.

3. Distributed LB (CiliumProxyReplacement): pkg/service/ uses Maglev hashing in eBPF maps for kube-proxy-free LB. Socket redirects in pkg/endpoint/.

4. ClusterMesh: pkg/clustermesh/ syncs identities/services across etcd KV stores for global policies/LB.

5. Hubble Observability: pkg/hubble/ relays eBPF events (flows, layers) to gRPC/ Kafka for UI/metrics.

Project Structure

The monorepo is organized around components, with Go packages in pkg/, eBPF in bpf/, and binaries in root-level mains:

cilium/
├── bpf/                 # eBPF C/bytecode sources (libbpf, maps)
├── daemon/              # Cilium agent entrypoint (main.go)
├── operator/            # Kubernetes Operator (main.go, pkg/operator/)
├── pkg/                 # Core libraries
│   ├── cilium          # CLI, API
│   ├── datapath        # eBPF loader/loader.Loader
│   ├── endpoint        # Per-pod state/policy
│   ├── identity        # Security identities
│   ├── policy          # L3-L7 enforcement
│   ├── proxy          # Envoy-based L7 proxy
│   └── service         # LB implementation
├── Documentation/       # Guides, images
└── contrib/             # Charts, Helm, manifests

Clever patterns:

• Generator-based eBPF: bpf/ uses Go templates (bpf/lib/common.h) for host<->kernel structs.
• Regeneration: make targets regenerate eBPF/Go from templates.
• BPF-to-Go maps: Auto-generated via bpftool for zero-copy sharing.

Start with daemon/main.go → pkg/datapath for agent bootstrap, then bpf/ for perf secrets. Use go mod tidy and make for builds.