Logstash config and enrichment backup using script

Logstash config and enrichment backup using script

Question:

I want to take backup of the logstash config ( /etc/logstash/conf.d ) and enrichment files. I can do it manually. But do we have script for it ?

Answer:

You can use below script and modify the server details. This script will take a backup of entire logstash folder as it cover tables and configs in all the analysers as well as alerts, reports & dashboard 

#!/bin/bash

sudo mkdir -p /tmp/backup | sudo chown vunet:vunet /tmp/backup
sudo mkdir -p /tmp/backup/analyser1-backup | sudo chown vunet:vunet /tmp/backup/analyser1-backup
sudo mkdir -p /tmp/backup/analyser2-backup | sudo chown vunet:vunet /tmp/backup/analyser2-backup

echo “Copying the contents of analyser logstash folder in tmp”

ssh shipper2 “sudo chmod 755 -R /etc/logstash/ | scp -r /etc/logstash/ shipper:/tmp/backup/shipper2-backup”
ssh analyser “sudo chmod 755 -R /etc/logstash/ | scp -r /etc/logstash/ shipper:/tmp/backup/analyser1-backup”
ssh analyser2 “sudo chmod 755 -R /etc/logstash/ | scp -r /etc/logstash/ shipper:/tmp/backup/analyser2-backup”

echo “Copying contents of cairo,logstash,opt folder of shipper in tmp”

#sudo chmod 755 -R /home/vunet/workspace/cairo/ | sudo cp -r /home/vunet/workspace/cairo/ /tmp/backup/shipper-confbackup/
sudo chmod 755 -R /opt/data_config_store/ | sudo cp -r /opt/data_config_store/ /tmp/backup/shipper-confbackup/
sudo chmod 755 -R /etc/redis/ | sudo cp -r /etc/redis/ /tmp/backup/shipper-confbackup/
#sudo chmod 755 -R /opt/kibana/ | sudo cp -r /opt/kibana/ /tmp/backup/shipper-confbackup/
#sudo chmod 755 -R /var/www/html/ | sudo cp -r /var/www/html/ /tmp/backup/shipper-confbackup/
sudo chmod 755 -R /home/vunet/scripts/ | sudo cp -r /home/vunet/scripts/ /tmp/backup/shipper-confbackup/
sudo chmod 755 -R /etc/logstash/ | sudo cp -r /etc/logstash/ /tmp/backup/shipper1-backup/

echo “Taking a backup of index-patterns”

source $VUNET_VIRT_ENV/bin/activate; export PYTHONPATH=$VUNET_BASE_PATH/cairo/

python /home/vunet/workspace/cairo/tools/manage_vienna_data.py –server analyser –port 9200 get 1 1 /tmp/backup/shipper-confbackup index-pattern

python /home/vunet/workspace/cairo/tools/manage_vienna_data.py –server analyser –port 9200 get 1 1 /tmp/backup/shipper-confbackup dashboard

python /home/vunet/workspace/cairo/tools/manage_vienna_data.py –server analyser –port 9200 get 1 1 /tmp/backup/shipper-confbackup alert

python /home/vunet/workspace/cairo/tools/manage_vienna_data.py –server analyser –port 9200 get 1 1 /tmp/backup/shipper-confbackup report

sudo tar -czf /tmp/backup-“$(date +”%Y-%m-%d”)”.tar.gz /tmp/backup/
sudo chmod 755 -R /tmp/backup-“$(date +”%Y-%m-%d”)”.tar.gz
scp -r /tmp/backup-“$(date +”%Y-%m-%d”)”.tar.gz shipper2:/data/backup/

echo “Wait for 5 secs..”
sleep 5

sudo rm /tmp/backup-*

Answer2:
There is a script available for this very purpose. The script is cairo/tools/backup_wrapper.sh. This is driven by a configuration file. A sample file is available at cairo/tools/collect-config-rules.yml



    • Related Articles

    • How To Decrypt The Backup Files Created By Frontend Config

      When we configure backup from the front end option, an encrypted file is getting created in the specified location. How do we decrypt this file and use it for recovering any specifics like objects or logstash configs etc., ANS ...

    • How to decrypt the backup files created by frontend config

      When we configure backup from the front end option, an encrypted file is getting created in the specified location. How do we decrypt this file and use it for recovering any specifics like objects or logstash configs etc., Ans: ...

    • Backup Settings In Frontend

      In the latest version of 8.5r5, we have the backup option configurable in the front end. Does this backup only the dashboards and other vienna objects or it backsup the config files like logstash aswell. Also how do we recover these backup files that ...

    • Backup settings in frontend

      question:In the latest version of 8.5r5, we have the backup option configurable in the front end. Does this backup only the dashboards and other vienna objects or it backsup the config files like logstash aswell. Also how do we recover these backup ...

    • Error in adding data enrichment entry 581

      Solution Document Error in adding data enrichment entry Overview General/Customer specific General Author Niveditha  Reviewer Seema Approver Sendil Release date 05/09/2022  Product Version 8.5r9 Audience: CSG/TechWarriors/PAC/Platform/Product teams ...