Vacuum do PostgreSQL – Manutenção do Catálogo Bacula

Vacuum do PostgreSQL – Manutenção do Catálogo Bacula

Executar periodicamente os comandos vacuum analyze e vacuum full no PostgreSQL são essenciais para manter o desempenho e a integridade do banco de dados. A importância de cada um desses comandos é explicada a seguir.

1. vacuum analyze

  • Atualização de estatísticas: O comando vacuum analyze vai além do vacuum básico, pois também atualiza as estatísticas do banco de dados. Essas estatísticas são vitais para o otimizador de consultas do PostgreSQL tomar decisões inteligentes sobre como executar consultas de maneira eficiente. Sem estatísticas atualizadas, o otimizador pode fazer escolhas sub-ótimas de plano de consulta.
  • Melhoria do desempenho das consultas: Ao atualizar as estatísticas, o vacuum analyze ajuda o banco de dados a tomar decisões mais informadas sobre como acessar os dados, o que, por sua vez, melhora o desempenho das consultas.

2. vacuum full

  • Remoção de registros obsoletos: o PostgreSQL utiliza um mecanismo de armazenamento chamado Multiversion Concurrency Control (MVCC) para controlar transações concorrentes. Isso significa que os registros não são removidos imediatamente quando são excluídos ou atualizados, mas marcados como obsoletos. O vacuum é responsável por remover esses registros obsoletos, recuperando espaço em disco e melhorando o desempenho.
  • Prevenção de bloat: o bloat ocorre quando as tabelas do banco de dados acumulam registros obsoletos que não foram limpos. O vacuum ajuda a prevenir o bloat, garantindo que o banco de dados não cresça desnecessariamente e que as consultas sejam executadas de maneira mais eficiente.
  • Reorganização do espaço em disco: o vacuum também pode reorganizar os dados no disco, melhorando a eficiência do acesso aos registros e reduzindo a fragmentação do disco.

Em relação ao Bacula, lentidões de consultas (por exemplo) de seleção de log do job pelo Bweb já foram solucionadas em cenários específicos de produção pela execução de um vacuum. Como o Bacula está constantemente reciclando backups e “prunando” informações do catálogo, é recomendado executar o vacuum analyze semanalmente, e um vacuum full a cada três meses.

Seguem exemplos de Job Admin do Bacula para execução automatizada.

3. Exemplo de Jobs Admin de vacuum e vacuum analyze no Bacula

O vacuum analyze deve ser executado em uma agenda semanal como no exemplo a seguir.

Job {
  Name = "admin-vacuum-weekly"
  Type = Admin
  JobDefs = BackupsToDisk
  RunBeforeJob = "vacuumdb -z -q -j 2 -d bacula"
  Schedule = admin-weekly
  ...
}

Schedule {
  Name = "admin-weekly"
  Run = Mon at 14:00
}

O vacuum full deve ser executado a cada três meses como no exemplo a seguir.

Job {
  Name = "admin-vacuum-analyze-trimestral"
  Type = Admin
  JobDefs = BackupsToDisk
  RunBeforeJob = "vacuumdb -f -q -d bacula"
  Schedule = admin-trimestral
  ...
}

Schedule {
  Name = "admin-trimestral"
  Run = Jan, Apr, Jul, Oct at 14:00
}

Conforme as Figuras 1 e 2, as configurações retromencionadas podem ser feitas graficamente pelo Bweb.

Figuras 1 e 2. Configuração dos Jobs Admin de vacuum analyze e full (Bweb).

Mesmo selecionando apenas a base “bacula” para a execução do vacuum, é possível que o comando vacuumdb imprima alertas como os que seguem.

2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shdepend" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_authid" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_database" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_db_role_setting" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_tablespace" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_auth_members" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shdescription" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_subscription" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shseclabel" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_replication_origin" --- only superuser can vacuum it

Estes alertas podem ser ignorados.

A partir da versão PostgreSQL 16, no entanto, existe um novo tipo de permissão que pode ser concedida ao usuário do bacula para mitigar esses alertas como no comando exemplificativo do psql a seguir.

grant pg_vacuum_all_tables to bacula;

Ref.: https://www.cybertec-postgresql.com/en/grant-vacuum-analyze-postgresql-16

 

Leave a Reply

PostgreSQL Vacuum – Bacula Catalog Maintenance

PostgreSQL Vacuum – Bacula Catalog Maintenance

Periodically executing the vacuum analyze and vacuum full commands in PostgreSQL is essential to maintain the performance and integrity of the database. The importance of each of these commands is explained below.

1. vacuum analyze

  • Updating Statistics: The vacuum analyze command goes beyond basic vacuum as it also updates the database’s statistics. These statistics are vital for PostgreSQL’s query optimizer to make intelligent decisions on how to execute queries efficiently. Without updated statistics, the optimizer may make suboptimal query plan choices.
  • Improving Query Performance: By updating statistics, vacuum analyze helps the database make more informed decisions on how to access data, which in turn improves query performance.

2. vacuum full

  • Removing Obsolete Records: PostgreSQL uses a storage mechanism called Multiversion Concurrency Control (MVCC) to control concurrent transactions. This means that records are not immediately removed when they are deleted or updated but are marked as obsolete. Vacuum is responsible for removing these obsolete records, reclaiming disk space, and improving performance.
  • Preventing Bloat: Bloat occurs when database tables accumulate obsolete records that have not been cleaned up. Vacuum helps prevent bloat, ensuring that the database does not unnecessarily grow and that queries are executed more efficiently.
  • Reorganizing Disk Space: Vacuum can also reorganize data on disk, improving the efficiency of access to records and reducing disk fragmentation.

Regarding Bacula, slow queries (for example, selecting job log entries in Bweb) have already been resolved in specific production scenarios by running a vacuum. Since Bacula is constantly recycling backups and pruning catalog information, it is recommended to run vacuum analyze weekly and vacuum full every three months.

Below are examples of Bacula Admin Jobs for automated execution.

3. Example of Admin Jobs for vacuum and vacuum analyze in Bacula

Vacuum analyze should be executed on a weekly schedule as shown in the following example.

Job {
  Name = "admin-vacuum-weekly"
  Type = Admin
  JobDefs = BackupsToDisk
  RunBeforeJob = "vacuumdb -z -q -j 2 -d bacula"
  Schedule = admin-weekly
  ...
}

Schedule {
  Name = "admin-weekly"
  Run = Mon at 14:00
}

Vacuum full should be executed every three months as shown in the following example.

Job {
  Name = "admin-vacuum-analyze-trimestral"
  Type = Admin
  JobDefs = BackupsToDisk
  RunBeforeJob = "vacuumdb -f -q -d bacula"
  Schedule = admin-trimestral
  ...
}

Schedule {
  Name = "admin-trimestral"
  Run = Jan, Apr, Jul, Oct at 14:00
}

As shown in Figures 1 and 2, the aforementioned configurations can be done graphically through Bweb.

Figures 1 and 2. Configuration of vacuum analyze and full Admin Jobs (Bweb).

Even when selecting only the “bacula” database for vacuum execution, the vacuumdb command may print warnings like the following.

2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shdepend" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_authid" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_database" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_db_role_setting" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_tablespace" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_auth_members" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shdescription" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_subscription" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shseclabel" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_replication_origin" --- only superuser can vacuum it

These warnings can be ignored.

Starting from PostgreSQL 16, however, there is a new type of permission that can be granted to the Bacula user to mitigate these warnings, as shown in the example psql command below.

grant pg_vacuum_all_tables to bacula;

Ref.: https://www.cybertec-postgresql.com/en/grant-vacuum-analyze-postgresql-16

Leave a Reply

Vacuum de PostgreSQL – Mantenimiento del Catálogo Bacula

Vacuum de PostgreSQL – Mantenimiento del Catálogo Bacula

Ejecutar periódicamente los comandos vacuum analyze y vacuum full en PostgreSQL es esencial para mantener el rendimiento y la integridad de la base de datos. La importancia de cada uno de estos comandos se explica a continuación.

1. vacuum analyze

  • Actualización de estadísticas: El comando vacuum analyze va más allá del vacuum básico, ya que también actualiza las estadísticas de la base de datos. Estas estadísticas son vitales para que el optimizador de consultas de PostgreSQL tome decisiones inteligentes sobre cómo ejecutar consultas de manera eficiente. Sin estadísticas actualizadas, el optimizador puede tomar decisiones de plan de consulta subóptimas.
  • Mejora del rendimiento de las consultas: Al actualizar las estadísticas, el vacuum analyze ayuda a la base de datos a tomar decisiones más informadas sobre cómo acceder a los datos, lo que a su vez mejora el rendimiento de las consultas.

2. vacuum full

  • Eliminación de registros obsoletos: PostgreSQL utiliza un mecanismo de almacenamiento llamado Multiversion Concurrency Control (MVCC) para controlar las transacciones concurrentes. Esto significa que los registros no se eliminan inmediatamente cuando se borran o actualizan, sino que se marcan como obsoletos. El vacuum se encarga de eliminar estos registros obsoletos, liberando espacio en disco y mejorando el rendimiento.
  • Prevención de la fragmentación: La fragmentación ocurre cuando las tablas de la base de datos acumulan registros obsoletos que no se han limpiado. El vacuum ayuda a prevenir la fragmentación, asegurando que la base de datos no crezca innecesariamente y que las consultas se ejecuten de manera más eficiente.
  • Reorganización del espacio en disco: El vacuum también puede reorganizar los datos en el disco, mejorando la eficiencia del acceso a los registros y reduciendo la fragmentación del disco.

En cuanto a Bacula, las demoras en las consultas (por ejemplo, la selección de registros de trabajo en Bweb) se han resuelto en escenarios de producción específicos mediante la ejecución de un vacuum. Dado que Bacula está constantemente reciclando copias de seguridad y “podando” información del catálogo, se recomienda ejecutar vacuum analyze semanalmente y vacuum full cada tres meses.

A continuación, se muestran ejemplos de trabajos administrativos de Bacula para ejecución automatizada.

3. Ejemplo de trabajos administrativos de vacuum y vacuum analyze en Bacula

El vacuum analyze debe ejecutarse en una programación semanal como se muestra en el siguiente ejemplo.

Job {
  Name = "admin-vacuum-weekly"
  Type = Admin
  JobDefs = BackupsToDisk
  RunBeforeJob = "vacuumdb -z -q -j 2 -d bacula"
  Schedule = admin-weekly
  ...
}

Schedule {
  Name = "admin-weekly"
  Run = Mon at 14:00
}

El vacuum full debe ejecutarse cada tres meses como se muestra en el siguiente ejemplo.

Job {
  Name = "admin-vacuum-analyze-trimestral"
  Type = Admin
  JobDefs = BackupsToDisk
  RunBeforeJob = "vacuumdb -f -q -d bacula"
  Schedule = admin-trimestral
  ...
}

Schedule {
  Name = "admin-trimestral"
  Run = Jan, Apr, Jul, Oct at 14:00
}

Según las Figuras 1 y 2, las configuraciones mencionadas anteriormente se pueden realizar gráficamente a través de Bweb.

Figuras 1 y 2. Configuración de trabajos administrativos de vacuum analyze y full (Bweb).

Incluso si se selecciona solo la base de datos “bacula” para la ejecución de vacuum, es posible que el comando vacuumdb imprima advertencias como las que se muestran a continuación.

2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shdepend" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_authid" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_database" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_db_role_setting" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_tablespace" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551

: BeforeJob: WARNING:  skipping "pg_auth_members" --- only superuser can vacuum it
2023-09-07 08:37:55 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shdescription" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_subscription" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_shseclabel" --- only superuser can vacuum it
2023-09-07 08:37:56 ebacula-dir JobId 40551: BeforeJob: WARNING:  skipping "pg_replication_origin" --- only superuser can vacuum it

Estas advertencias pueden ignorarse.

A partir de la versión PostgreSQL 16, sin embargo, existe un nuevo tipo de permiso que se puede otorgar al usuario de Bacula para mitigar estas advertencias, como se muestra en el comando de ejemplo de psql a continuación.

grant pg_vacuum_all_tables to bacula;

Ref.: https://www.cybertec-postgresql.com/en/grant-vacuum-analyze-postgresql-16

Leave a Reply