-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
directory_iterator_construct: Avoid provoking undefined behaviour #77
Conversation
The directory_iterator(const path& p) constructor calls detail::directory_iterator_construct passing nullptr (as 0) for the ec parameter. detail::directory_iterator_construct may call directory_iterator::increment(*ec) which dereferences the nullptr provoking undefined behaviour. On GCC, and probably many other compilers, it seems that this merely causes a null reference to be passed to directory_iterator::increment which in turn, turns it back into nullptr when calling detail::directory_iterator_increment which knows how to deal with the nullptr and is happy. Unfortunately, directory_iterator::increment(system::error_code& ec) is marked noexcept (unlike the version that takes no arguments) but detail::directory_iterator_increment throws exceptions when ec == nullptr. This results in std::terminate being called. The simplest way to fix this is for detail::directory_iterator_construct to just pass on the potentially-nullptr ec argument to detail::directory_iterator_increment rather than going via the potentially-noexcept directory_iterator::increment member function. (Discovered in the field with Boost 1.63 and a faulty disk. I managed to reproduce the problem by teaching dir_itr_increment to pretend that readdir_r_simulator returned an error when it saw a certain filename.)
I've reproduced the same problem and confirmed the fix with boost 1.66 too. Unfortunately I was unable to get the current state of Boost master to compile in order to run the test suite there. Hopefully travis/appveyor will tell me if I've broken anything. |
The AppVeyor test failure appears to be:
The remove_symlink_tests test doesn't even seem to use directory_iterator, so it looks like this failure may not be related to my patch. |
Is there anything else that I need to do? Is my explanation not clear? |
This change might be the cause of a new problem. int main()
}` this now gives rise to complains when this is ran with valgrind : operations.hpp 905 0x40409D /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths main /home/ldco/Projects/TryOuts/FileSystem0/src main.cpp 8 Uninitialised value was created by a heap allocation 0x4C2E01F /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so malloc 0x4E47DE9 /home/ldco/boost/lib/libboost_filesystem.so.1.69.0 boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator&, boost::filesystem::path const&, boost::system::error_code*) 0x405D12 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths boost::filesystem::directory_iterator::directory_iterator(boost::filesystem::path const&) /home/ldco/boost/include/boost/filesystem operations.hpp 905 0x40409D /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths main /home/ldco/Projects/TryOuts/FileSystem0/src main.cpp 8 0x3 1 UninitCondition Conditional jump or move depends on uninitialised value(s) 0x4E46531 /home/ldco/boost/lib/libboost_filesystem.so.1.69.0 boost::filesystem::detail::directory_iterator_increment(boost::filesystem::directory_iterator&, boost::system::error_code*) 0x405D70 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths boost::filesystem::directory_iterator::increment() /home/ldco/boost/include/boost/filesystem operations.hpp 941 0x407357 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths void boost::iterators::iterator_core_access::increment<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator&) /home/ldco/boost/include/boost/iterator iterator_facade.hpp 556 0x406947 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths boost::iterators::detail::iterator_facade_base<boost::filesystem::directory_iterator, boost::filesystem::directory_entry, boost::iterators::single_pass_traversal_tag, boost::filesystem::directory_entry&, long, false, false>::operator++() /home/ldco/boost/include/boost/iterator iterator_facade.hpp 666 0x4040E6 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths main /home/ldco/Projects/TryOuts/FileSystem0/src main.cpp 10 Uninitialised value was created by a heap allocation 0x4C2E01F /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so malloc 0x4E47DE9 /home/ldco/boost/lib/libboost_filesystem.so.1.69.0 boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator&, boost::filesystem::path const&, boost::system::error_code*) 0x405D12 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths boost::filesystem::directory_iterator::directory_iterator(boost::filesystem::path const&) /home/ldco/boost/include/boost/filesystem operations.hpp 905 0x40409D /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths main /home/ldco/Projects/TryOuts/FileSystem0/src main.cpp 8 ` gcc : gcc (SUSE Linux) 7.3.1 20180323 [gcc-7-branch revision 258812] ===> so conditional jump based on uninitialized values ==> that sounds like undefined behaviour ? |
Should be fixed in bbe9d17, thanks. |
That code runs fine for me under valgrind with boost 1.63 + my patch (which is what I did most of my testing on) and boost 1.66 + my patch (which is what I'm currently shipping.) I haven't upgraded to anything newer yet. I suspect that your problem is not related to this pull request. Mike. |
the problem I mentioned occurs in boost 1.69, and was still working ok in boost 1.68, the fact I ended up in this ticket is due to the release notes : Fixed undefined behavior in boost::filesystem::directory_iterator implementation. (PR#77) |
Yes, it is not related to this PR. The problem was present in code since ba4aaf3, which is the initial release of Boost.Filesystem v3. It did not reproduce on Linux before Boost 1.69 (more precisely, until 48b8d75) because a different code branch was used on Linux. The problem could reproduce on other platforms before that revision. |
I have patched my boost environment with the mentioned fix above, and can confirm that valgrind is happy again :-) |
1 similar comment
I have patched my boost environment with the mentioned fix above, and can confirm that valgrind is happy again :-) |
The directory_iterator(const path& p) constructor calls
detail::directory_iterator_construct passing nullptr (as 0) for the ec
parameter.
detail::directory_iterator_construct may call
directory_iterator::increment(*ec) which dereferences the nullptr provoking
undefined behaviour.
On GCC, and probably many other compilers, it seems that this merely causes
a null reference to be passed to directory_iterator::increment which in
turn, turns it back into nullptr when calling
detail::directory_iterator_increment which knows how to deal with the
nullptr and is happy.
Unfortunately, directory_iterator::increment(system::error_code& ec) is
marked noexcept (unlike the version that takes no arguments) but
detail::directory_iterator_increment throws exceptions when ec == nullptr.
This results in std::terminate being called.
The simplest way to fix this is for detail::directory_iterator_construct to
just pass on the potentially-nullptr ec argument to
detail::directory_iterator_increment rather than going via the
potentially-noexcept directory_iterator::increment member function.
(Discovered in the field with Boost 1.63 and a faulty disk. I managed to
reproduce the problem by teaching dir_itr_increment to pretend that
readdir_r_simulator returned an error when it saw a certain filename.)