How File Descriptor Leaking may kill your application?

Photo by Fikri Rasyid on Unsplash

What is a File Descriptor?

A file descriptor (FD) is a handle the operating system uses to track all the opened files or other I/O resources, such as a pipe or socket by a process. The programs use FDs to perform various operations on the file, such as reading, writing, or seeking.

In simple words, when a process opens a file, the operating system creates an entry to represent that file and stores the information about that opened file.

How to see the list of File Descriptors in your OS

In Unix-like OS, you can use the command lsof to list all the FDs.

lsof
# also you can see the FDs of a specific process
lsof -p 123

Example Program

The program below opens a file and runs an infinite loop so the process won’t get terminated.

public static void main(String[] args) throws IOException, InterruptedException {
    FileInputStream fis = new FileInputStream("file.dat");
    // fis.close(); Files isn't closed
  
    while (true) {
        Thread.sleep(100);
    }
}

Run the above code, open a terminal and execute the below command.

lsof | grep file.dat

You will see the result similar to below. This shows that the process is holding the file ‘file.dat’. This is the file descriptor.

What is File Descriptor Leaking?

Given below is an example program that’s written in Java which leads to File descriptor leaking.

public static void main(String[] args) {
  try {
      while (true) {
          BufferedReader reader = new BufferedReader(new FileReader("example.txt"));
          String line;
          while ((line = reader.readLine()) != null) {
              System.out.println(line);
          }
      }
  } catch (IOException e) {
      e.printStackTrace();
  }
}

In the above program, we read a file example.txt in an infinite loop, we created a BufferedReader to read the file. However, we haven’t closed the reader. Hence it leads to an FD leak.

Memory leaks are the most common resource leaks. If memory is not freed when it is no longer needed, a process might be unable to allocate memory for other tasks. Like memory leak, another type of resource leak is the File descriptor leak. File descriptor leaking is a situation where a program does not correctly release or close a file descriptor after it is no longer needed, causing the operating system to run out of available file descriptors which eventually leads to errors. The default maximum number of file descriptors allowed for a process depends on the OS. In general, most modern operating systems set a default limit of 1024 file descriptors per process.

Unlike memory leaks, when a program running out of memory crashes, programs that leak FDs usually do not crash. Instead, they stop performing normal tasks, which might not be immediately noticeable.

How to prevent File Descriptor Leaking?

Here are some general guidelines to help avoid file descriptor leaking in your program.

1. Always close any file or other resource stream when its usage is over. Java provides the feature try-with-resource to achieve this.
2. Avoid using unnecessary file descriptors. For example, if you only need to read from a file, don’t open it in write mode as well.
3. Don’t open more file descriptors than you need at any given time. If you’re processing a large number of files, consider processing them in smaller batches to avoid running out of available file descriptors.
4. Consider using a connection pool or other resource management framework that handles the opening and closing of file descriptors for you.
5. Monitor your program’s resource usage and ensure that it’s not consuming too many file descriptors or other resources over time.
6. Consider increasing the maximum number of file descriptors allowed for a process if your program needs to handle a large number of open files.

Conclusion

This article covered the definition, examples and prevention methods for file descriptor leaking. To avoid file descriptor leaks in programs, it’s important to properly manage and close file handles and other resources when they are no longer needed, as well as to monitor resource usage to ensure that the program is not consuming too many file descriptors over time.