Nothing Special   »   [go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving large number of files just results in hanging #456

Open
dalanmiller opened this issue Nov 21, 2021 · 2 comments
Open

Moving large number of files just results in hanging #456

dalanmiller opened this issue Nov 21, 2021 · 2 comments
Labels
enhancement New feature or request

Comments

@dalanmiller
Copy link

Firstly, dziekuje ci! This is such a helpful tool! I've been poorly trying to write a python script for a couple years to do the same thing but this is just so much better!

I noticed that after scanning many thousands of files and attempting to move a slice of them to a new directory, after selecting 'Ok', the program just hangs. Looking at the directory I can see that files are indeed moving there but there's no indication that the program is still functioning or making progress or has hit some sort of issue.

Perhaps a progress bar status window would be best shown here so that it's clear what's happening? And/or a small window to indicate whether there are any issues with files as a outcome / results log?

Just a couple ideas!

Screen Shot 2021-11-21 at 8 54 41 pm

@gabefair
Copy link
gabefair commented Oct 4, 2022

I also noticed that it gets stuck at 22,173,422 files

image

Allow me to explain why there are so many files. I had a drive that needed to be recovered. The file structure was lost and while the files were saved, the directory was not. So I have one drive full of files. I am comparing a drive with a fresh, clean, empty windows 11 install against my drive of recovered files in-order to isolate just the user files that I need saved. But after 1 hour of letting it scan, it got stuck.

@GeckoEidechse
Copy link

On Windows, running task manager while moving duplicate files I noticed a lot of disk usage and just moving 2 files with a total of 6GB took multiple minutes.

This would indicate that when moving a file, instead of simply updating the path where it is located, the old file is actually read into memory and written to a new file. Looking at the creation and last modified date of the moved file confirms that theory.

I'm not sure what the reason for this approach is, as unless one is moving files across disks/shares, simply updating the path the file is located at is faster many times over.

This gets especially bad in the case where one has a small number of large files. Let's say a movie collection where each files is a few GBs. If we say we e.g. have 100 files with 1GB each, "moving" the files by writing them from scratch (which is what czkawka currently does on Windows v5.0.2) would on a conventional harddisk with 160MB/s write speed take around 10 minutes to complete. Moving the file by just updating the path for comparison would take around 2-3 seconds. That's a 300x difference!

 

Now I'm pretty sure there is some reason that moving files is done in the way it is in czkawka so I'm curious what the reason is ^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants