We had an issue with the URL monitoring, on checking we found out that the network team started blocking HTTP traffic from their location (where our stand alone shipper is), so instead of using the stand alone shipper we used our main shipper (Cloud) to monitor them. It was working fine for 3 days, again it showed down and when the heartbeat was restarted, it started working fine….now again it is showing down and issue seen is ‘Too many open files’, ‘no such host
Solution:
On the shipper where heartbeat is running
As vunet user do
crontab -e
* * * * * /bin/ps -eaf | /bin/grep -w heartbeat | /bin/grep -v god | /bin/grep -v grep | /usr/bin/awk ‘{ print $2}’ | xargs -I {} sudo /usr/bin/prlimit –pid {} –nofile=9000:9000
This cron job increases the open file limit for heartbeat process