Metadata/EXIF data is assigned to the wrong files when adding a large number of files
When I add a large number of files to Advanced Renamer, the EXIF/metadata reading becomes unreliable. Some files are read correctly, but a large number of files are assigned metadata that belongs to another file.
For example, when I add multiple video files, a given file/row is populated with metadata from a different file/row. As a result, renaming based on metadata becomes unreliable and can produce incorrect filenames.
This issue is critical because it can silently cause files to be renamed using the wrong metadata. It should be fixed as a priority.
I have tested several versions, including the older version 3.87, and the problem exists there as well. Recent versions still have the same issue.
For example, when I add multiple video files, a given file/row is populated with metadata from a different file/row. As a result, renaming based on metadata becomes unreliable and can produce incorrect filenames.
This issue is critical because it can silently cause files to be renamed using the wrong metadata. It should be fixed as a priority.
I have tested several versions, including the older version 3.87, and the problem exists there as well. Recent versions still have the same issue.
Reply to #1:
Welcome saleh,
Can you share a recipe to duplicate the problem -- or a small set of files that exhibit the problem?
Even a screenshot can help. (But you have to link to a site like https://postimages.org/ because this form doesn't let us post images directly.)
Also: *** the "Pair renaming" setting might be messing things up. *** try checking and unchecking it to see if it makes a difference for your files.
~~~
I created 1000 files "tinyfile 001.mp3" thru "tinyfile 999.mp3" (plus one other) -- each with several mp3 tags and a unique <title> tag.
Aren 4.27 read them flawlessly. I then randomly blanked tag info from a few files and Aren still worked as expected.
If you don't want to share files publicly, contact the support email and perhaps you can share them privately with Kim.
Regards,
Randy
Welcome saleh,
Can you share a recipe to duplicate the problem -- or a small set of files that exhibit the problem?
Even a screenshot can help. (But you have to link to a site like https://postimages.org/ because this form doesn't let us post images directly.)
Also: *** the "Pair renaming" setting might be messing things up. *** try checking and unchecking it to see if it makes a difference for your files.
~~~
I created 1000 files "tinyfile 001.mp3" thru "tinyfile 999.mp3" (plus one other) -- each with several mp3 tags and a unique <title> tag.
Aren 4.27 read them flawlessly. I then randomly blanked tag info from a few files and Aren still worked as expected.
If you don't want to share files publicly, contact the support email and perhaps you can share them privately with Kim.
Regards,
Randy
Reply to #2:
The problem occurs specifically with MTS video files. When I add them as one large batch at once, the metadata is not read reliably, and files are often assigned metadata that belongs to other files. However, when I split the same files into smaller batches, they are renamed correctly. The issue therefore seems to occur mainly when a large batch is added at once rather than in smaller groups.
I also tried unchecking Pair renaming, but unfortunately it did not resolve the problem.
I am not entirely sure whether I have described the issue accurately enough, but I would be grateful if you could look into it. If you are unable to reproduce the problem, I can try to send precise screenshots showing the issue in detail.
Thank you for your time and support.
Best regards,
The problem occurs specifically with MTS video files. When I add them as one large batch at once, the metadata is not read reliably, and files are often assigned metadata that belongs to other files. However, when I split the same files into smaller batches, they are renamed correctly. The issue therefore seems to occur mainly when a large batch is added at once rather than in smaller groups.
I also tried unchecking Pair renaming, but unfortunately it did not resolve the problem.
I am not entirely sure whether I have described the issue accurately enough, but I would be grateful if you could look into it. If you are unable to reproduce the problem, I can try to send precise screenshots showing the issue in detail.
Thank you for your time and support.
Best regards,
Reply to #3:
How large a batch? I avoid doing more than 10K, of any type of file, as speed and working memory become a factor. That's just my personal observation. For video files, maybe 2500 is a sensible limit?
Regardless, the behavior you describe would be a bug but I can't reproduce it.
Screenshots would help. Is it any particular tags? Any particular values -- Unicode especially?
At this point, you may have to contact email support and arrange with Kim to provide him with an Short, Self Contained, Correct (Compilable), Example ( https://sscce.org/ )
Regards,
Randy
How large a batch? I avoid doing more than 10K, of any type of file, as speed and working memory become a factor. That's just my personal observation. For video files, maybe 2500 is a sensible limit?
Regardless, the behavior you describe would be a bug but I can't reproduce it.
Screenshots would help. Is it any particular tags? Any particular values -- Unicode especially?
At this point, you may have to contact email support and arrange with Kim to provide him with an Short, Self Contained, Correct (Compilable), Example ( https://sscce.org/ )
Regards,
Randy
Reply to #3:
MTS files in particular has been an issue in the past. Metadata from MTS files are not natively supported by Advanced Renamer. Instead data is extracted through ExifTool. As far as I know, MTS files are usually very large files and the layout of the format may result in the program reading the whole file to extract the metadata. I think especially the duration of the stream is difficult to read without reading the whole file.
How big are the files, are they 4GB+ each? Are the files located on an external drive, like a network drive? Does it take a long time to load the files? Could one of them be corrupted causing issues down the chain of files?
MTS files in particular has been an issue in the past. Metadata from MTS files are not natively supported by Advanced Renamer. Instead data is extracted through ExifTool. As far as I know, MTS files are usually very large files and the layout of the format may result in the program reading the whole file to extract the metadata. I think especially the duration of the stream is difficult to read without reading the whole file.
How big are the files, are they 4GB+ each? Are the files located on an external drive, like a network drive? Does it take a long time to load the files? Could one of them be corrupted causing issues down the chain of files?
Reply to #5:
Hello Kim,
I believe I have discovered part of the problem.
I suspect that disk read speed is the root cause. When files are read from a network drive, the issue occurs even when the batch is not particularly large — approximately 250 files or slightly more. I tried copying the same folders to the local C: drive (SSD), and the problem did not occur there.
In summary, it appears that the speed at which the file itself is read is what triggers the issue, causing a file to take on the metadata of another row in the same list. It seems the program may set a fixed time limit for reading, and if that limit is exceeded, it moves on before the correct data has been read, which leads to the problem.
The issue is not limited to MTS files — I also tested with MP4 files and encountered the same problem.
I apologize for not describing the issue precisely enough earlier. Please let me know if you need any further details or screenshots.
With best regards,
Hello Kim,
I believe I have discovered part of the problem.
I suspect that disk read speed is the root cause. When files are read from a network drive, the issue occurs even when the batch is not particularly large — approximately 250 files or slightly more. I tried copying the same folders to the local C: drive (SSD), and the problem did not occur there.
In summary, it appears that the speed at which the file itself is read is what triggers the issue, causing a file to take on the metadata of another row in the same list. It seems the program may set a fixed time limit for reading, and if that limit is exceeded, it moves on before the correct data has been read, which leads to the problem.
The issue is not limited to MTS files — I also tested with MP4 files and encountered the same problem.
I apologize for not describing the issue precisely enough earlier. Please let me know if you need any further details or screenshots.
With best regards,
Reply to #6:
Ah, perhaps it is a classic correlating async/multi-threading data problem.
Interestingly, Aren uses 3 threads just sitting open, empty, and idle (not a problem nor rare, just an observation).
~~~~~~~~~~~~~~~~~~
I'm sure Kim will fix this but I can make no promises for him. ;)
Meanwhile, as a workaround, you can write -- or foolishly trust AI to write -- or pay me big $Bank$ to write -- a simple script that reads each file with ExifTool directly and then constructs a CSV file*** with the desired tag data.
It's not too hard to make this multi-threaded in PowerShell or Python which you will want to do for performance.
Then, in Aren you'd use the Import CSV data function and rename your files from that.
Be sure to check all four "Don't analyze" checkboxes in the "Media files" settings, or Aren will use much time reading the metadata anyway.
Regards,
Randy
~~~
*** But don't use a comma as the separator! Use a semicolon typically and make sure that whatever character you use never appears in the data.
Ah, perhaps it is a classic correlating async/multi-threading data problem.
Interestingly, Aren uses 3 threads just sitting open, empty, and idle (not a problem nor rare, just an observation).
~~~~~~~~~~~~~~~~~~
I'm sure Kim will fix this but I can make no promises for him. ;)
Meanwhile, as a workaround, you can write -- or foolishly trust AI to write -- or pay me big $Bank$ to write -- a simple script that reads each file with ExifTool directly and then constructs a CSV file*** with the desired tag data.
It's not too hard to make this multi-threaded in PowerShell or Python which you will want to do for performance.
Then, in Aren you'd use the Import CSV data function and rename your files from that.
Be sure to check all four "Don't analyze" checkboxes in the "Media files" settings, or Aren will use much time reading the metadata anyway.
Regards,
Randy
~~~
*** But don't use a comma as the separator! Use a semicolon typically and make sure that whatever character you use never appears in the data.
Reply to #6:
I am implementing some changes for version 4.28 that hopefully fixes the issue.
I am implementing some changes for version 4.28 that hopefully fixes the issue.