I'm opening lots of files with fopen() in VC++ but after a while it fails.
Is there a limit to the number of files you can open simultaneously?
The C run-time libraries have a 512 limit for the number of files that can be open at any one time. Attempting to open more than the maximum number of file descriptors or file streams causes program failure. Use
_setmaxstdio to change this number. More information about this can be read here
Also you may have to check if your version of windows supports the upper limit you are trying to set with
_setmaxstdio. For more information on
_setmaxstdio check here
Information on the subject corresponding to VS 2015 can be found here
If you use the standard C/C++ POSIX libraries with Windows, the answer is "yes", there is a limit.
However, interestingly, the limit is imposed by the kind of C/C++ libraries that you are using.
I came across with the following JIRA thread (http://bugs.mysql.com/bug.php?id=24509) from MySQL. They were dealing with the same problem about the number of open files.
However, Paul DuBois explained that the problem could effectively be eliminated in Windows by using ...
Win32 API calls (CreateFile(), WriteFile(), and so forth) and the default maximum number of open files has been increased to 16384. The maximum can be increased further by using the --max-open-files=N option at server startup.
Naturally, you could have a theoretically large number of open files by using a technique similar to database connections-pooling, but that would have a severe effect on performance.
Indeed, opening a large number of files could be bad design. However, some situations call require it. For example, if you are building a database server that will be used by thousands of users or applications, the server will necessarily have to open a large number of files (or suffer a performance hit by using file-descriptor pooling techniques).