Cloud-native Postgres observability: from client apps to underlying cloud resources
In the current era of cloud computing, the days of managing single-app systems with standalone databases are long gone. Instead, we find ourselves responsible for overseeing complex systems containing hundreds of services and dozens of databases. Adding to the challenge is the dynamic nature of these systems, where databases can migrate between nodes, and their storage performance can be adjusted on the fly. In this presentation, we will explore strategies for maintaining observability of Postgres databases in complex cloud environments. Topics include leveraging eBPF for generating comprehensive service maps and instrumenting Postgres calls across various client applications to specific databases. Additionally, we explore automatic discovery mechanisms for topologies within High Availability (HA) Postgres clusters, facilitated by different Kubernetes Operators. Furthermore, the talk addresses the importance of monitoring underlying cloud resources, encompassing virtual machines (VMs), storage volumes, and network components.