Skip to content

perf(serverdb): render supernet filters as uncorrelated subqueries - #479

Merged
kofrezo merged 4 commits into
mainfrom
dk_improve_supernet_filter
Oct 19, 2026
Merged

kofrezo merged 4 commits into
mainfrom
dk_improve_supernet_filter

Conversation

@kofrezo

@kofrezo kofrezo commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

A filter on a supernet attribute, such as route_network=regexp(...), was rendered as a correlated EXISTS over server_inet_attribute with a second sublink on the server table to match the networks by hostname. Postgres evaluated it per candidate server, probing the server's address by key and only then testing containment, so the GiST index on server_inet_attribute.value could not drive the join.

Render the condition as "server.server_id IN (SELECT ...)" instead, driven from the matching networks' prefixes, the same shape #471 gave inherited supernet paths. Apply the hostname condition directly to the "supernet" alias, which is already a server row, instead of through a sublink. Both call sites share one helper so the join graph stays in a single place.

A filter on a supernet attribute, such as route_network=regexp(...),
was rendered as a correlated EXISTS over server_inet_attribute with a
second sublink on the server table to match the networks by hostname.
Postgres evaluated it per candidate server, probing the server's
address by key and only then testing containment, so the GiST index on
server_inet_attribute.value could not drive the join.

Render the condition as "server.server_id IN (SELECT ...)" instead,
driven from the matching networks' prefixes, the same shape #471 gave
inherited supernet paths. Apply the hostname condition directly to the
"supernet" alias, which is already a server row, instead of through a
sublink. Both call sites share one helper so the join graph stays in a
single place.
@kofrezo kofrezo self-assigned this Oct 15, 2026
@kofrezo kofrezo added enhancement ai Code fully or partially AI generated. labels Oct 15, 2026
@kofrezo
kofrezo requested a review from TuxPowered42 October 15, 2026 13:23
@kofrezo
kofrezo marked this pull request as ready for review October 15, 2026 14:22
@kofrezo

kofrezo commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

To give some context: The query in doubt being slow was route_network=regexp("servers[0-9]+-officeh-int") state=not(retired).

We traced it down via PostgreSQL log to this one (our database with our servertypes):

SELECT server.server_id, server.hostname, server.intern_ip, server.servertype_id
  FROM server
  WHERE EXISTS (SELECT 1 FROM server AS supernet JOIN server_inet_attribute AS server_addr ON (server_addr.server_id = server.server_id) JOIN server_inet_attribute AS net_addr ON (net_addr.value >>= server_addr.value AND
    net_addr.attribute_id = server_addr.attribute_id AND net_addr.server_id = supernet.server_id) WHERE supernet.servertype_id = 'route_network' AND supernet.server_id IN ( SELECT server_id FROM server WHERE hostname::text ~
    'servers[0-9]+-officeh-int'))
    AND (server.servertype_id IN ('tunnel_interface', 'hdmi_transmitter', 'floating_address', 'voip_phone', 'loadbalancer', 'hardware_external', 'hypervisor', 'room_tablet', 'router', 'switch', 'vm', 'blade_enclosure', 'vpn',
      'media_device', 'vm_external', 'camera', 'project_network', 'hardware', 'hw_internalrouter', 'doorlock', 'ipmi', 'nat_gateway', 'vlan_interface', 'accesspoint', 'printer', 'natbox', 'hw_loadbalancer'))
  ORDER BY server.hostname; 

It was especially costly on the CPU side due to the many joins doing filtering.

And this is where the LLM took over we simply asked it to do a root cause analyze in the query executor and suggest a faster way which it did by reordering the joins.

Result looks sane and is being tested.

@kofrezo
kofrezo requested a review from TuxPowered42 October 19, 2026 11:51
@kofrezo
kofrezo force-pushed the dk_improve_supernet_filter branch from a4a3ac7 to efe2f87 Compare October 19, 2026 12:10
@kofrezo
kofrezo merged commit 7e67833 into main Oct 19, 2026
5 checks passed
@kofrezo
kofrezo deleted the dk_improve_supernet_filter branch October 19, 2026 12:23
kofrezo added a commit that referenced this pull request Oct 19, 2026
)

A filter on a supernet attribute, such as route_network=regexp(...),
was rendered as a correlated EXISTS over server_inet_attribute with a
second sublink on the server table to match the networks by hostname.
Postgres evaluated it per candidate server, probing the server's
address by key and only then testing containment, so the GiST index on
server_inet_attribute.value could not drive the join.

Render the condition as "server.server_id IN (SELECT ...)" instead,
driven from the matching networks' prefixes, the same shape #471 gave
inherited supernet paths. Apply the hostname condition directly to the
"supernet" alias, which is already a server row, instead of through a
sublink. Both call sites share one helper so the join graph stays in a
single place.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai Code fully or partially AI generated. enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants