It would be nice to see RAW files from different bodies that we could then download and evaluate. Of course the best comparisons would come from comparing the same scene shot with different bodies. There isn't much point to doing this with jpegs, but it would be ok IF the scene was the same.
Well, I'm thinking of a link to the RAW files on an FTP site or something like that. If the images are of the same subject and opened by each of our favorite converters, the comparisons would be "apples to apples".
Wouldn't still be apples and apples [not that the 'apples vs. oranges' cliché usually invoked does make any sense; what else should I compare but different things?], since even the one standard converter we would have to agree upon does apply different settings for different files from different cameras.
Before doing any experiment we should be clear about
a) what are we looking for
b) what we expect to see
c) what lesson can be learned.
In short: We need a theory to test.
__________________ Dierk Haasis
[DH² Publishing] - Writing and Imaging
The theory that I am testing is "which equipment will produce images that suit my needs". It's pretty subjective, and there will be compromises with any arrangement of hardware and software, so I'm just looking for a better solution than I have now.
The "apples" in the analogy is the scene, or subject. The different things we are comparing are the cameras.
Let's say we put a 50mm on a D3 and a 1Ds MarkIII, and on a D2Xs we would put a 35mm. We point the cameras at X subject and keep the lighting the same, then make an agreeable exposure. We can then process the resulting RAW files in Lightroom according to what we think makes each image look best to us.
At that point we are able to assess which image we like with regards to detail, apparent sharpness, latitude, etc. The same test can be done with a variety of lenses and subjects until a satisfactory conclusion is reached about which tools I choose to work with in the near future. At the moment I'm using a D2Xs and I would like to stay with the flexibility of a 35mm style SLR, that's why I'm comparing these 3 apples instead of also including medium format options.
That wouldn't test the RAW files but - and this I consider a good and valuable test I've done myself over and over through the years - how much work we need to put into a specific file.
Essentially you are then testing the RAW converter, that is, how good are its presets. If these standard settings are taken from the files - as do all camera manufacturer's converters like Capture NC or DPP - we are also testing how good the processing of the cameras are.
If you wnat to know whjat tool is the best for your [future] work, image quality is a given, not a test criterion. While some tasks for some photographers will be so specialised they need one particular image qualia [shouldn't that be singular?] with very strict limits - essentially sharpness -, for which they have to shop around, current comparable cameras deliver, well, comparable image quality. the differences between dSLRs are much smaller than the differences we once had between Ektachrome 100 and Kodakchrome 64.
The photos the cameras produce are greyscale TIFFs with a lot of technical tags to let a software know how to interpret the greyscale. Since the underlying technology of all dSLRs except Fuji's offers are the same, it's the post-processing that makes the difference. Admittedly there are some limiting factors, like the nature of the AA/IR/UV filter and the algorithms to counter its effect, or the metering system [incl. WB metering] but most of these are corrected fro in software - for in-camera JPEGs in the camera's IC, for RAW in the converter.
For practical decision: Why do I use Nikon cameras, not Canon? It had never anything to do with 'better' metering or 'better' images [in film days IQ was still bound to the system though it was the lenses more than the bodies]. It wasn't even about the 'better' system, they have always been equally well developed, what one lacked it made up with something they other couldn't offer. It was and is all about handling - how does it feel when I work with it, comfortable, utilitarian, annoying.
Back to the original question, I don't see any real use in comparing RAW files. I don't even see how it helps - as an objectifiable test - to compare RAW files sent through a converter unless we can pinpoint settings for every single one of which we can be assured they get out the best of the data.
__________________ Dierk Haasis
[DH² Publishing] - Writing and Imaging
You nailed it, Dierk. It really boils down to something along the lines of "Does this hardware get along with my way of doing things, and do I like what I see?".
Seems awfully subjective in this high tech world of measurements, noise charts, etc., doesn't it?
But it really still is. There's a lot of art in with the science still. I'm glad about that, actually.
To your original point, Curt... Your best hope is to find a reviewer who shoots the same scene with every different camera he tests. I don't know of anyone who's done that for a while and has been posting raw files, though Drew here has been posting Raw files for his tests lately. It's probably not feasible for a community such as this one to pool their camera resources and come up with a list of comparable Raw files. Outside of that, then the best thing you can do is just process every Raw file you come across and see what comes out. We have Raw Processing Challenges from time to time here....
RAW files from different bodies 
Linear Mode
