NFS Mount Point Disk Space Usage Breached The Threshold 357

NFS Mount Point Disk Space Usage Breached The Threshold 357




    Solution Document

          NFS Mount Point Disk Space Usage Breached The Threshold

       Overview
 

General/Customer specific

Sify TSL

Author

Rukmini

Reviewer

Seema

Approver

 Rukmini

Release date

01/07/2022

Product Version

      


           Audience: CSG/TechWarriors/PAC/Platform/Product teams


What’s the Issue? 

NFS mount point disk space usage breached the threshold

Describe the Issue in Detail

In TSL environment on all  analysers nfs mount point disk space usage breached the threshold (80%), nfs mount point is used to store archival data .


If it gets increased day by day it may hamper services like Elasticsearch, logstash.. which consumes disk space.



Solution Summary

   1) Check data rentention setting at front end is wirking properly

   2) Check and compare current index size and index size of initial setup

   3) Compare no of devices in current and initial setup


Solution Steps In Detail    

Check UI data retention  for each index is working properly


Data retention should work as per data retention settings in data store module. Active data , Inactive data and Archived data,  these details will provide number of days we want indices to be active,inactive or archived

Example:As mentioned in above image netflow data should be active for 10 days and archived for 30 days after that netflow index will be deleted automatically as per setting.

To validate data retention is working properly or not search netflow index in storage under data store module and verify number of active indices as well as  archived indices as per setting are present.




If indices are not present as per data retention settings delete unwanted indices (Note:Validate each index as per retention settings.)

If all indices are present as per data retention settings, then issue may be with data polling got increased compare to initial settings (More devices are  added in data sources which will increase number of polling documents).Compare older indices size with current indices size. And if Indices size is more then more disk space is required.

In sify TSL env index size was more compared to initial settings because of addition of more devices.

Nfs mount point storage wwas increased to accomodate pooling of more devices.

Mount point space usage should be kept less than 80%.

Use df -hT   command to find mount point details.


        More Help: 

  


       References: 




    • Related Articles

    • How do we find the disk usage for each ES Node in a cluster

      Question: Is there a command that provides the disk usage by ES for each node in a cluster? Also, how much is actual data size because we may be replicating it for High Availability? Answer: ​​Run below command from any of the analyzer where ...

    • Unable to Mount Partition

      Getting error while mounting the partition Error – Already mounted or busy Answer: check if there is any process using that device (/dev/sda1). It is often a fsck process which runs automatically on system startup. You can check it quickly e.g. by ...

    • Logbeat started reading very much old files as well when mount point is placed

      Question: In RBL, for a given case, the mount point got removed where logs are updating. Now when mount point is placed logbeat started reading very much old files as well. How can we exclude old files?  Answer: ignore_older argument is available in ...

    • Unable to delete archive index when we are facing space issue in archive partition 399

         Solution Document   Unable to delete archive index when we are facing space issue in archive partition        Overview General/Customer specific General Author Rukmini Reviewer      Nishanth Approver  Ravi  Release date 13/09/2021 Product Version ...

    • Older Indices are not getting archived/deleted #658

      Solution Document ICICI || Older Indices are not getting archived/deleted Overview General/Customer specific General Author Akash Thosare Reviewer Approver NA Release date 18/08/2022 Product Version 8.5r5 Audience: ...