89 lines
3.3 KiB
Markdown
89 lines
3.3 KiB
Markdown
Provision a Proxmox server with L3 networking and EVPN for VM networks.
|
|
|
|
# Configuration
|
|
|
|
The NetBox device and interfaces should define at least the following properties:
|
|
|
|
{
|
|
"asn": { "asn": 65000 },
|
|
"cluster": "pve",
|
|
"cluster_type": "proxmox",
|
|
"name": "pve-2",
|
|
"interfaces": [
|
|
{
|
|
"comment": "Link to fabric for BGP."
|
|
"name": "lan0",
|
|
"mac_address": "AA:BB:CC:DD:FF:00",
|
|
},
|
|
{
|
|
"comment": "Link to fabric for BGP."
|
|
"name": "lan1",
|
|
"mac_address": "AA:BB:CC:DD:FF:01",
|
|
},
|
|
{
|
|
"comment": "Addresses on this interface are announced via BGP to fabric."
|
|
"name": "lo",
|
|
"type": { "value": "virtual" },
|
|
"ip_addresses": [
|
|
{
|
|
"address": "10.0.0.102/32",
|
|
"dns_name": "pve-2.example.net",
|
|
"role": { "value": "loopback" },
|
|
},
|
|
{
|
|
"address": "1000::102/128",
|
|
"dns_name": "pve-2.example.net",
|
|
"role": { "value": "loopback" },
|
|
}
|
|
],
|
|
},
|
|
{
|
|
"comment": "A separate SSH instance will run in mgmt VRF."
|
|
"name": "mgmt0",
|
|
"mac_address": "AA:BB:CC:DD:EE:00",
|
|
"vrf": { "name": "mgmt" },
|
|
"ip_addresses": [
|
|
{ "address": "10.0.1.102/24", "vrf": { "name": "mgmt" } },
|
|
{ "address": "1001::102/64", "vrf": { "name": "mgmt" } }
|
|
],
|
|
},
|
|
],
|
|
}
|
|
|
|
## EVPN
|
|
|
|
Each server needs a unique ASN. EVPN routes to individual VMs are announced to fabric via BGP over `lan*` interfaces. To allow receiving EVPN routes on the connected switch interfaces, the switch config context should define `ifaces_evpn`:
|
|
|
|
"ifaces_evpn": ["swp11", "swp12"]
|
|
|
|
## Firewall
|
|
|
|
Firewall rules are common to the whole cluster and are represented as NetBox services. Services must be attached to a specific device, so a “fake” VM is created to hold all rules for the cluster. Each service should then be added to the `services` custom field for the cluster. All traffic between cluster nodes is allowed by default.
|
|
|
|
## Certificates
|
|
|
|
The role configures ACME settings based on the `dns_name` of the loopback IP address. Certificate has to be ordered manually once after setup.
|
|
|
|
## Ceph
|
|
|
|
In the config context for device or cluster two additional options may be defined. Specifying
|
|
|
|
ceph-version: squid
|
|
|
|
will enable the no-subscription Ceph repository and install the given version.
|
|
|
|
## LDAP
|
|
|
|
To sync users and groups with LDAP, add the following to the config context:
|
|
|
|
sync-ldap: realm
|
|
|
|
This will configure a daily cron job to sync users. The cluster password file should contain `ldap_user` and `ldap_pass` keys. The servers are autodiscovered using the `domain` property from the config context.
|
|
|
|
# Cluster
|
|
|
|
After setup, join the device into an existing cluster with
|
|
|
|
ip vrf exec default pvecm add pve-1.example.net --link0 10.0.0.101
|
|
|
|
where pve-1 is an existing cluster node and `--link0` specifies the loopback address of the new device.
|