Nothing Special   »   [go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

directory_iterator_construct: Avoid provoking undefined behaviour #77

Merged
merged 1 commit into from
Sep 6, 2018

Conversation

mikecrowe
Copy link

The directory_iterator(const path& p) constructor calls
detail::directory_iterator_construct passing nullptr (as 0) for the ec
parameter.

detail::directory_iterator_construct may call
directory_iterator::increment(*ec) which dereferences the nullptr provoking
undefined behaviour.

On GCC, and probably many other compilers, it seems that this merely causes
a null reference to be passed to directory_iterator::increment which in
turn, turns it back into nullptr when calling
detail::directory_iterator_increment which knows how to deal with the
nullptr and is happy.

Unfortunately, directory_iterator::increment(system::error_code& ec) is
marked noexcept (unlike the version that takes no arguments) but
detail::directory_iterator_increment throws exceptions when ec == nullptr.
This results in std::terminate being called.

The simplest way to fix this is for detail::directory_iterator_construct to
just pass on the potentially-nullptr ec argument to
detail::directory_iterator_increment rather than going via the
potentially-noexcept directory_iterator::increment member function.

(Discovered in the field with Boost 1.63 and a faulty disk. I managed to
reproduce the problem by teaching dir_itr_increment to pretend that
readdir_r_simulator returned an error when it saw a certain filename.)

The directory_iterator(const path& p) constructor calls
detail::directory_iterator_construct passing nullptr (as 0) for the ec
parameter.

detail::directory_iterator_construct may call
directory_iterator::increment(*ec) which dereferences the nullptr provoking
undefined behaviour.

On GCC, and probably many other compilers, it seems that this merely causes
a null reference to be passed to directory_iterator::increment which in
turn, turns it back into nullptr when calling
detail::directory_iterator_increment which knows how to deal with the
nullptr and is happy.

Unfortunately, directory_iterator::increment(system::error_code& ec) is
marked noexcept (unlike the version that takes no arguments) but
detail::directory_iterator_increment throws exceptions when ec == nullptr.
This results in std::terminate being called.

The simplest way to fix this is for detail::directory_iterator_construct to
just pass on the potentially-nullptr ec argument to
detail::directory_iterator_increment rather than going via the
potentially-noexcept directory_iterator::increment member function.

(Discovered in the field with Boost 1.63 and a faulty disk. I managed to
reproduce the problem by teaching dir_itr_increment to pretend that
readdir_r_simulator returned an error when it saw a certain filename.)
@mikecrowe
Copy link
Author

I've reproduced the same problem and confirmed the fix with boost 1.66 too. Unfortunately I was unable to get the current state of Boost master to compile in order to run the test suite there. Hopefully travis/appveyor will tell me if I've broken anything.

@mikecrowe
Copy link
Author
mikecrowe commented Jul 2, 2018

The AppVeyor test failure appears to be:

libs\filesystem\test\operations_test.cpp(1394): test 'fs::exists(link)' failed in function 'void __cdecl `anonymous-namespace'::remove_symlink_tests(void)'

The remove_symlink_tests test doesn't even seem to use directory_iterator, so it looks like this failure may not be related to my patch.

@mikecrowe
Copy link
Author

Is there anything else that I need to do? Is my explanation not clear?

@b-spencer
Copy link
b-spencer commented Sep 6, 2018

This fixes the second issue of #53, fixed by aa2a58a. But, the first issue in #53, fixed by 1303b3e, still seems outstanding. I will look at rebasing #53.

@killerbot242
Copy link
killerbot242 commented Jan 9, 2019

This change might be the cause of a new problem.
Consider the following code snippet :
`#include <boost/filesystem.hpp>
#include

int main()
{
const std::string path{"/tmp"};

boost::filesystem::directory_iterator iter(path);
while (iter != boost::filesystem::directory_iterator()) {
    ++iter;
}
return 0;

}`

this now gives rise to complains when this is ran with valgrind :
`
0x0
1
UninitCondition
Conditional jump or move depends on uninitialised value(s)


0x4E46531
/home/ldco/boost/lib/libboost_filesystem.so.1.69.0
boost::filesystem::detail::directory_iterator_increment(boost::filesystem::directory_iterator&, boost::system::error_code*)


0x4E4804A
/home/ldco/boost/lib/libboost_filesystem.so.1.69.0
boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator&, boost::filesystem::path const&, boost::system::error_code*)


0x405D12
/home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths
boost::filesystem::directory_iterator::directory_iterator(boost::filesystem::path const&)

/home/ldco/boost/include/boost/filesystem
operations.hpp
905


0x40409D
/home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths
main
/home/ldco/Projects/TryOuts/FileSystem0/src
main.cpp
8


Uninitialised value was created by a heap allocation


0x4C2E01F
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so
malloc


0x4E47DE9
/home/ldco/boost/lib/libboost_filesystem.so.1.69.0
boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator&, boost::filesystem::path const&, boost::system::error_code*)


0x405D12
/home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths
boost::filesystem::directory_iterator::directory_iterator(boost::filesystem::path const&)
/home/ldco/boost/include/boost/filesystem
operations.hpp
905


0x40409D
/home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths
main
/home/ldco/Projects/TryOuts/FileSystem0/src
main.cpp
8


0x3 1 UninitCondition Conditional jump or move depends on uninitialised value(s) 0x4E46531 /home/ldco/boost/lib/libboost_filesystem.so.1.69.0 boost::filesystem::detail::directory_iterator_increment(boost::filesystem::directory_iterator&, boost::system::error_code*) 0x405D70 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths boost::filesystem::directory_iterator::increment() /home/ldco/boost/include/boost/filesystem operations.hpp 941 0x407357 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths void boost::iterators::iterator_core_access::increment<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator&) /home/ldco/boost/include/boost/iterator iterator_facade.hpp 556 0x406947 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths boost::iterators::detail::iterator_facade_base<boost::filesystem::directory_iterator, boost::filesystem::directory_entry, boost::iterators::single_pass_traversal_tag, boost::filesystem::directory_entry&, long, false, false>::operator++() /home/ldco/boost/include/boost/iterator iterator_facade.hpp 666 0x4040E6 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths main /home/ldco/Projects/TryOuts/FileSystem0/src main.cpp 10 Uninitialised value was created by a heap allocation 0x4C2E01F /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so malloc 0x4E47DE9 /home/ldco/boost/lib/libboost_filesystem.so.1.69.0 boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator&, boost::filesystem::path const&, boost::system::error_code*) 0x405D12 /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths boost::filesystem::directory_iterator::directory_iterator(boost::filesystem::path const&) /home/ldco/boost/include/boost/filesystem operations.hpp 905 0x40409D /home/ldco/Projects/TryOuts/FileSystem0/Deliv/Debug/FileSystem0-Paths main /home/ldco/Projects/TryOuts/FileSystem0/src main.cpp 8 `

gcc : gcc (SUSE Linux) 7.3.1 20180323 [gcc-7-branch revision 258812]
valgrind : valgrind-3.13.0

===> so conditional jump based on uninitialized values ==> that sounds like undefined behaviour ?

@Lastique
Copy link
Member
Lastique commented Jan 9, 2019

Should be fixed in bbe9d17, thanks.

@mikecrowe
Copy link
Author

This change might be the cause of a new problem.
Consider the following code snippet :
`#include <boost/filesystem.hpp>
#include

int main()
{
const std::string path{"/tmp"};

boost::filesystem::directory_iterator iter(path);
while (iter != boost::filesystem::directory_iterator()) {
    ++iter;
}
return 0;

}`

That code runs fine for me under valgrind with boost 1.63 + my patch (which is what I did most of my testing on) and boost 1.66 + my patch (which is what I'm currently shipping.) I haven't upgraded to anything newer yet.

I suspect that your problem is not related to this pull request.

Mike.

@killerbot242
Copy link

the problem I mentioned occurs in boost 1.69, and was still working ok in boost 1.68, the fact I ended up in this ticket is due to the release notes : Fixed undefined behavior in boost::filesystem::directory_iterator implementation. (PR#77)
That's how I made the link, I will past my boost install with the fix mentioned above, and report back.

@Lastique
Copy link
Member
Lastique commented Jan 9, 2019

Yes, it is not related to this PR. The problem was present in code since ba4aaf3, which is the initial release of Boost.Filesystem v3. It did not reproduce on Linux before Boost 1.69 (more precisely, until 48b8d75) because a different code branch was used on Linux. The problem could reproduce on other platforms before that revision.

@killerbot242
Copy link

I have patched my boost environment with the mentioned fix above, and can confirm that valgrind is happy again :-)

1 similar comment
@killerbot242
Copy link

I have patched my boost environment with the mentioned fix above, and can confirm that valgrind is happy again :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants