Algorithm to detect duplicate videos ?
Algorithm to detect duplicate videos ?
hello,
is what someone has created a program to detect the videos in duplicate. or has developed an efficient algorithm in this direction ...
ps: I already test programs in that sense. I looking for a free solution.
is what someone has created a program to detect the videos in duplicate. or has developed an efficient algorithm in this direction ...
ps: I already test programs in that sense. I looking for a free solution.
Re: Algorithm to detect duplicate videos ?
Interesting but very complex. difficult to understand. I am not a mathematician. I looking for a simpler algorithm (FreeBasic) based on the research of the same color matching point ...
it seems that the subject was not studied here, I'll see in other forums ...
but thank you for the link ...
it seems that the subject was not studied here, I'll see in other forums ...
but thank you for the link ...
Re: Algorithm to detect duplicate videos ?
Well, ok. It depends on what you want to do. What is the problem? You have 2 video streams the same resolution, frame rate etc? Are they the same length or are you looking to locate where they overlap? If the streams have the same resolution etc then pick say 10 random pixel positions eg. (163,573), (437,294)... then find the colours in each frame at those pixel locations. When you are compairing the 2 streams you only have to check 10 colours per frame instead of 10's or 100's of thousands per frame. Sure it's approximate, you can do better if you do some more math.
It is extremely unlikely you are going to find, off the bat, code that does identical to what you want for free just out there on the internet somewhere. And even if you did you would likely end up having to learn an extensive API just to use the thing.
It generally ends up far quicker and easier to write your own code, even if it means reinventing the wheel.
It is extremely unlikely you are going to find, off the bat, code that does identical to what you want for free just out there on the internet somewhere. And even if you did you would likely end up having to learn an extensive API just to use the thing.
It generally ends up far quicker and easier to write your own code, even if it means reinventing the wheel.
Re: Algorithm to detect duplicate videos ?
Just randomly sample 100+ bytes in each file and build a table of values. Then compare all of the values for matches. Also compare based on file date/time, size, and name.
IMO
-Vince
IMO
-Vince
Re: Algorithm to detect duplicate videos ?
sean_vn
in general. before creating a feature I do a search to avoid wasting time reinventing the wheel. I give an example. Recently I was looking for a way to clean html code, leaving only the text. I found a function pascal language in three minutes on the net. then the creation of the program in lazarus took a few minutes. if I created it myself the function. it would probably take a lot longer ...
vdecampo
good idea. I had thought but it is a video file that can be of different resolutions. so this is suitable ...
though. thank you for your answers.
in general. before creating a feature I do a search to avoid wasting time reinventing the wheel. I give an example. Recently I was looking for a way to clean html code, leaving only the text. I found a function pascal language in three minutes on the net. then the creation of the program in lazarus took a few minutes. if I created it myself the function. it would probably take a lot longer ...
vdecampo
good idea. I had thought but it is a video file that can be of different resolutions. so this is suitable ...
though. thank you for your answers.
Re: Algorithm to detect duplicate videos ?
one method i would be tempted to use is too simply look at 'the rate' that the data changes over time
think of it as a kind of 'beat detection' , almost every video will have a unique 'data beat' even when examined roughly
There are 101 variables which will defeat any narrow thru the keyhole type sampling * assuming you are comparing videos from different sources, then you have resolution, frame rate, display ratio, compression method etc etc
By using a high level snooper, ie standing back from the technical details you will be able to apply a 'common sense kind of algorithm' to get a common sense kind of answer which will give you a good idea of 'how similar' any two videos are to a person viewing them
think of it as a kind of 'beat detection' , almost every video will have a unique 'data beat' even when examined roughly
There are 101 variables which will defeat any narrow thru the keyhole type sampling * assuming you are comparing videos from different sources, then you have resolution, frame rate, display ratio, compression method etc etc
By using a high level snooper, ie standing back from the technical details you will be able to apply a 'common sense kind of algorithm' to get a common sense kind of answer which will give you a good idea of 'how similar' any two videos are to a person viewing them
Re: Algorithm to detect duplicate videos ?
Must add, Google Optical Flow
with this method you could pick out two similar videos even if one was in black and white, played backwards, mirrored and at a jaunty angle and quite possibly via a fish eye lens too !...our brains use multiple algorithms which are dynamically assigned priorities according to experience...we can be fooled but the task of fooling an experienced observer is non trivial
with this method you could pick out two similar videos even if one was in black and white, played backwards, mirrored and at a jaunty angle and quite possibly via a fish eye lens too !...our brains use multiple algorithms which are dynamically assigned priorities according to experience...we can be fooled but the task of fooling an experienced observer is non trivial
Re: Algorithm to detect duplicate videos ?
http://duplicatevideosearch.com/
I found and tests Duplicate Video Search, efficient, the test version was actually find many videos in double ...
but will not pay for that, I should prefer an open source solution, I continue my research ...
I found and tests Duplicate Video Search, efficient, the test version was actually find many videos in double ...
but will not pay for that, I should prefer an open source solution, I continue my research ...
Re: Algorithm to detect duplicate videos ?
Resolved. the solution is a basic ffmpeg to create thumbnails in jpg SELECTED videos in resolution of 80x60. Then I tested some duplicate image detection programs. The most effective was a free program called ImgSearch. the final isolation of double image files and video files association with visual selection and deleting double ...
DOS is still alive and effective for this kind of treatment in batch ...
Note: I have recovered many gigas occupied by duplicate files videos ...
I would have to find the time to automate all this in a program ...
DOS is still alive and effective for this kind of treatment in batch ...
Note: I have recovered many gigas occupied by duplicate files videos ...
I would have to find the time to automate all this in a program ...
Re: Algorithm to detect duplicate videos ?
You mean Windows NT's command line interface (cmd.exe), do you? Plain DOS is only of very limited usefulness nowadays; e.g. for flashing firmware, hardware diagnostics and other low level/realtime stuff.[/quote]Coolman wrote:DOS is still alive and effective for this kind of treatment in batch ...
You can also try this program instead: https://tn123.org/simimages/ (does also compare videos; not ony images as suggested by its name)Coolman wrote:I would have to find the time to automate all this in a program ...
It's not open source though, but it's freeware.
Re: Algorithm to detect duplicate videos ?
St_W wrote:You mean Windows NT's command line interface (cmd.exe), do you? Plain DOS is only of very limited usefulness nowadays; e.g. for flashing firmware, hardware diagnostics and other low level/realtime stuff.Coolman wrote:DOS is still alive and effective for this kind of treatment in batch ...
You can also try this program instead: https://tn123.org/simimages/ (does also compare videos; not ony images as suggested by its name)Coolman wrote:I would have to find the time to automate all this in a program ...
It's not open source though, but it's freeware.[/quote]
I use a lot back to launch batch script which perform many tasks such as backups, file copies displacement ... for ffmpeg (download here http://ffmpeg.zeranoe.com/builds/ , the 64 bits is faster) I created the following script:
Code: Select all
cls
echo off
set crep=%cd%
set thumb="%crep%"\_Thumbnails_\
echo.
echo Creation Thumbnails en cours. Patientez svp...
if not exist "%thumb%" mkdir "%thumb%"
for %%A IN (*.mp4) do (
if not exist "%thumb%%%A.jpg" ffmpeg -ss 00:02:00 -i "%%A" -s 80x60 -vframes 1 "%thumb%%%A.jpg" -an -loglevel quiet
)
echo.
echo.
echo Fini
pause
echo on
must be positioned at the root of the directory containing the videos, then run the script, a Thumbnails directory will be created at this level with the preservation of the image of the video to the necessary 0:02:00 position to avoid the generic and increase the detection of duplicates. Then it's simple enough to be used to isolate ImgSearch duplicate images that correspond with the name of the videos ...
ps: the script is configured to take into account that the mp4 format, if you want to add others eg avi, replace (* .mp4) by (* .mp4 * .avi) ...
there there's another script that level to move the videos whose images correspond to a different directory ...
if you want more explanation, just ask but I think you have grasped the concept ...
ImgSearch a small program of about 135 kb programmed with Delphi is surprising efficiency. pity that the author does not provide the source code if I could adapt to automate the entire process...
I tested SimilarImages but it does not take into account the videos in mp4 format, so I drop ...
Re: Algorithm to detect duplicate videos ?
You can do some research on this software:
http://couponcode0.com/duplicate-video-search.html
http://couponcode0.com/duplicate-video-search.html