Skip to content
Clusterf**ctional — Research

netzlive.cluster

netzlive_cluster is the library that lets one NETZlive application call functions in another. redispatch asks NMK for Betriebsmittel, netzkoordinator asks user-management to send an email, the UI asks redispatch for master data. None of those calls go over HTTP: the applications are BEAM nodes in one Erlang cluster, and a call is an :erpc.call/5 to whichever node currently advertises the function.

The library does four things:

  1. Forms the cluster. Each node asks the Kubernetes API which pods carry the label netzlive/pod-type=cluster-service and connects to them. → Cluster-Bildung
  2. Advertises services. A service is a module whose functions are marked with defapi. Starting it registers a description of it — name, version, functions, call defaults — in a replicated registry every node can read. → Registry
  3. Calls services. A client module declares defdelegate_api stubs. Calling one looks the service up in the local copy of the registry, picks the newest compatible version, and runs the function on that node, with timeouts and retries. → Dienste und Clients, Ein Aufruf
  4. Carries errors across the node border. Exceptions are wrapped in a ServiceError on the serving node so the caller can read them without having the exception module loaded.

Repo: EnBWAG/netzlive.cluster. Published to the private netzlive hex organisation as netzlive_cluster. Paths on these pages are prefixed with their repo — netzlive.cluster/, redispatch/, and so on.

The whole thing on one picture

Section titled “The whole thing on one picture”
sequenceDiagram
    participant K8s as Kubernetes API
    participant A as Node A (provider)
    participant B as Node B (consumer)

    loop every 10 s, on every node
        A->>K8s: GET pods?labelSelector=netzlive/pod-type=cluster-service
        B->>K8s: same
        K8s-->>B: pod list → node names
        B->>A: Node.connect
    end

    Note over A: Supervisor starts MyApp.Service.PingV3
    A->>A: Monitor process: Phoenix.Tracker.track(service struct)
    A-->>B: Tracker delta (≤ 1.5 s): service joined

    Note over B: PingClient.ping()
    B->>B: Registry.lookup(name, "~> 3.0") → newest match, its node
    B->>A: :erpc.call(A, PingV3, :ping, [], timeout)
    A->>A: defapi wrapper: run, catch, wrap errors
    A-->>B: {:ok, :pong}
    B->>B: unwrap → {:ok, :pong}

Components

Section titled “Components”

Everything below NL.Cluster.Supervisor, started by the library’s own application callback in netzlive.cluster/lib/nl/cluster.ex:13-36:

Process Module Job
NL.Cluster.V3.PubSub Phoenix.PubSub Transport for the registry’s replication, plus local service_up/service_down events
NL.Cluster.V3.Registry NL.Cluster.Registry (netzlive.cluster/lib/nl/cluster/registry.ex) A Phoenix.Tracker: CRDT of every service in the cluster
Netzlive.Cluster legacy supervisor (netzlive.cluster/lib/netzlive/cluster.ex:143-149) Second, separate registry for the deprecated API — see below
Cluster.Supervisor libcluster, with NL.Cluster.Strategy.Kubernetes Discovers and connects nodes

Per service, one more process: an NL.Cluster.Monitor (netzlive.cluster/lib/nl/cluster/monitor.ex), which is what a service module’s child_spec/1 actually starts (netzlive.cluster/lib/nl/cluster/service.ex:214-219). It holds the registration — when it dies, the service disappears from the registry.

There is no central broker. Every node holds a full copy of the registry and makes its own routing decision.

Two APIs in one package

Section titled “Two APIs in one package”

The package contains two complete implementations:

Namespace Status Registry process PubSub
NL.Cluster.* Current. Introduced in 2.5. NL.Cluster.V3.Registry NL.Cluster.V3.PubSub
Netzlive.Cluster.* @moduledoc deprecated — “will be removed in v3.0.0” Netzlive.Cluster.Registry Netzlive.Cluster.PubSub

Both run on every node that depends on the library. Their registries are separate trackers, so a service registered with one API is invisible to clients of the other. That is the reason the upgrade guide (netzlive.cluster/guides/upgrading/v2.5.md) tells providers to register each service twice — once per namespace, the NL one with a bumped major version — until every consumer has moved its clients over.

3.0 is meant to delete Netzlive.Cluster. A v3.0.0-rc.0 tag exists (2025-08-25), but it is not an ancestor of main; main is still on the 2.x line. Who decides when 3.0 ships could not be determined from the repo.

The pages in this section describe NL.Cluster. Where the legacy API behaves differently in a way that matters, it says so.

Where to go next

Section titled “Where to go next”