Become a fan of Slashdot on Facebook

typodupeerror
• #### compare against the static baseline. (Score:2)

compare both images against the original, not each other.
count number of pixels different from the original, then calculate max and average difference between either image and the original.

decide which parameter means more to you.

go forward from there.

• #### Re: (Score:2)

adding to that, you can run the following algorithm on the diff images.
1. blur image by an arbitrary value,
2. darken the image by an arbitrary value.
3. repeat until image is all black.

count the number of repetitions. given various values for steps one and two, you can tune the algorithm to find images that have large areas of mismatch.

possibly not useful to you, but have found it good for validation testing for image manipulation software.

• #### How about audio? (Score:2, Interesting)

I would very much like to do the same with audio. I have so many duplicate tracks in my music collection in different formats and bitrates.
• #### Re: (Score:2)

If you're running a mac and have all your files in an itunes library, then Dupin [dougscripts.com] is extremely useful. It matches on name, size, length, bit rate, or all at once.

It's pretty useful, and the freeware version lets your delete from drive as well as library.

If you're on windows, I searched for years and couldn't find anything :(

• #### Look at the DCT coefficients (Score:4, Informative)

<uhmmmm&gmail,com> on Thursday July 16, 2009 @07:09PM (#28724301) Homepage

JPEG works by breaking the image into 8x8 blocks and doing a two dimensional discrete cosine transform on each of the color planes for each block. At this point, no information is lost (except possibly by some slight inaccuracies converting from RGB to YUV as is used in JPEG). The step where the artifacts are introduced is in quantizing the coefficients. High frequency coefficients are considered less important and are quantized more than low frequency coefficients. The level of quantization is raised across the board to increase the level of compression.

Now, how is this useful? The reason heavily quantizing results in higher compression is because the coefficients get smaller. In fact, many become zero, which is particularly good for compression - and the high frequency coefficients in particular tend towards zero. So partially decode the images and look at the DCT coefficients. The image with more high frequency coefficients which are zero is likely the lower quality one.

• #### Image Quality Metrics. (Score:2)

Something like $\frac{1}{N} \sum_{i=1}^{N}(x_i-y_i)^2$, where $x$ and $y$ are arrays of pixels, and \$N is the number of pixels in each array?

• #### Is there a way to find out the compression engine? (Score:2)

Does JPEG header have the compression method listed as well as compression ratio? If not, is there any way to figure out what kinda compresison engine is used base on how an image is constructed?

If so, simply do some testing against some of the most popular compression engine base on the artifact to determines what engine is used, then find out their compression ratio (perhaps a simple files size might work?). Then simply pick the images with the best quality base on engine used and ratio?
• #### variation (Score:2)

Compute the variance of the Fourier coefficients within each block and then calculate the average for each image. The better quality image should have lower variance. If a block has a lot of edges, then the higher frequency coefficients should have much higher values than the lower ones. If a block is uniform, then the lower frequency coefficients should have higher values. So if you have a good image, it will be easy to see the difference between uniform parts and edges. That is the coefficients of th
• #### It depends what you want.. (Score:2)

find dupes on the internet http://tineye.com/ [tineye.com]
find dupes on your HDD http://www.bigbangenterprises.de/en/doublekiller/ [bigbangenterprises.de]

• #### Just sort by the size (Score:2)

JPEG is pretty efficient at compressing images -- the only way they get smaller on average is by increasing the quality loss. Therefore, the larger of the two images in bytes is probably the better looking copy.

• #### Subjective... (Score:2)

Well, your problem is that image quality is subjective. Can computers make good subjective judgements? Not really.

Let's say you count the number of pixels that are different? Well, what if JPEG usually slightly alters the brightness? You could weight the difference, but what if JPEG sometimes moves an edge by a pixel?

I think if you study a bit about how JPEG works, you might find that you can computationally determine how much information that is lost; but that does not mean that your computed number in

• #### Expert's answer (Score:2, Interesting)

Exploit JPEG's weakness.

JPEG encodes pixels by using a cosine transform on 8x8 pixel blocks. The most perceptually visible artifacts (and the artifacts most suceptible to cause troble to machine vision algorithms) appear on block boundaries.

b. Use the value of the 8-pixel period response in X and Y direction as your quality metric. The higher, the worse the quality.

This is a crude 1st approximation but works.
• #### Some things aren't doable yet (Score:2)

Aside from the mathematical tests some have suggested, my gut tells me this is going to be almost impossible. There are tasks that a human can perform that just aren't doable given the present state of our software systems. The gap has as much to do with our understanding about how we perceive through our senses as it does with algorithms and calculation methodologies. We just don't know yet enough about the underlying processes to make a computer do it.

The same goes for other areas where AI is sorely la

• #### Sorting steps to find originals (Score:3, Informative)

on Friday July 17, 2009 @11:43AM (#28730659) Homepage Journal

You probably don't necessarily want to find the "best quality" image, but rather the image that was closest to the original.

I take it you're either trying to eliminate the low-quality duplicates or thumbnails from a really large collection of pr0n, or trying to write an image search engine that tries to present the "best" rendition of a particular image first.

1. As a quick first pass (after you've run through to collect all the similar images into separate groups), you'd obviously want to find the version of the image with the highest resolution. This might let you easily throw out thumbnails or scaled down versions you might come across. Of course, some dorks will upscale images and post them somewhere, so you might still want to hang on to some of them for the second stage.

2. For the second pass, you'd likely want to scan through the metadata first, especially stuff exposed by EXIF. So you'd want to give higher scores to EXIF data that makes it sound like it came directly off a digital camera or scanner, and bump down the desirability of pictures that appeared to have been edited by any sort of photo editing software.

3. Then maybe you want to look at something that would rank down watermarks or other modifications.

4. Another step would be to compare compression quality, but I think that's what most of the other posts are concentrating on. But this is a difficult step because it can be easily fooled, since idiots can re-save a low quality image with the compression quality cranked all the way up so the file size becomes high even though the actual image quality is worse than the original. You probably need to run it through one of those "photoshop detectors" that could tell you whether the image has been through smoothing or other filters in a photo editor. The originals (especially in raw format and maybe high quality JPEG) will have a certain type of CCD noise signature that your software might be able to detect. In the same vein, a poorly-compressed JPEG will have lots of JPEG quantization artifacts that your software might be able to detect as well. Otherwise, you're kinda left with zooming in on pics and eyeballing it.

5. Finally you might be left with a group of images that are exactly the same but have different file names... you probably want some way to store some of the more useful bits of descriptive text as search/tag metadata, but then choose the most consistent file naming convention or slap on your own based on your own metadata.

Hopefully this gives you a start to important parts of the process that you might have overlooked...

#### Related LinksTop of the: day, week, month.

The reason computer chips are so small is computers don't eat much.

Working...