Spirit of Photography

Mentoring and Learning => Learning Photography => Topic started by: keithsnell on June 27, 2009, 09:31:29 AM

Title: Digital Photography and the Zone System
Post by: keithsnell on June 27, 2009, 09:31:29 AM

Those of you that have been following the discussion between Iliah Borg and I on DPReveiw have probably figured out by now that my brief overview on how to use the Zone System in the "Understanding and Controlling Exposure" tutorial was an oversimplification.  That shouldn't be a surprise to anyone.  To be semantically correct, I should have described this method as a "Modified Zone System."  You will see this type of "Modified Zone System" used often in digital photography.  

We've also had a discussion on DPReview about how deeply one needs to delve into a subject before we are satisfied that we have a "practical" knowledge of the subject and can go about the business of making beautiful images.  The answer is that it varies from person to person.  The practical answer is that great photography is a culmination of many different skills, and your photography is limited mostly by your "weakest link."  If you are strong on the artistic side, but limited on the technical side, then your photography can only progress so far.  There is a balance that each photographer must define for his or herself.  

In truth, the Zone System, as defined by Adams, cannot be directly applied to digital photography without some modifications.  However, his concepts of assigning exposure to Zones can be very useful in helping us to define a systematic method for controlling our exposures.  You will find two schools of thought with respect to the "best" way to implement a modified Zone System in digital photography.  

One school of thought is that we can use the concepts in the Zone System in order to make a conscious decision about how to optimize our exposure based on placing a chosen brightness level in the scene into a predetermined range as it would be presented by our "standard" work flow.  Although this technique doesn't necessarily "maximize" the image quality that can be extracted from the raw file it is, without debate, a much better method for controlling exposure than simply pointing your camera towards the scene and using matrix metering to determine the exposure.  This is the method presented in my Understanding and Controlling Exposure tutorial.

There is a second, more precise method of using the Zone System that requires much more technical knowledge of how your camera sensor and raw processor works.  This method requires that you analyze the raw data from your camera to determine the absolute limits of dynamic range available from your sensor, and understand how this data is recorded in the raw file.  You must then understand how your raw processor takes this raw data and manipulates it to present the "output" in the various "working spaces" (color space and gamma curves), and finally, you must understand how to modify the behaviour of your raw processor in order to force it to interpret how you have chosen to record the data (to maximize dynamic range) and output this data in the way that you want it presented.  There is no debate that this is a more precise method of controlling exposures that can help you extract the maximum dynamic range from your camera.

Which method you chose to use is based on how much effort you want to expend to really learn how your camera and software systems work, and how important it is for you to extract every last bit of image quality available from your sensor.  For many photographers, the "simplified" Zone System I present in the tutorial will be sufficient to significantly improve their photographic skills.  As your skills improve, or for those photographers with the technical aptitude, you might want to explore the concepts described in the second, more rigorous application of the Zone System to digital photography.

I've included links to three very good references that deal with how to use a "modified" Zone System in digital photography.  Although they present different methods for using a modified zone system, none of these methods are "wrong."  They all provide useful concepts to help the photographer progress from simply using the "default" matrix metering for a scene, to using a "simplified" zone system, and then to a more advanced understanding of how to apply the concepts in the Zone System to digital photography.  These references are presented in order of complexity, and each one will give you a better understanding of the concept.

Exposing for the Highlights, Adapting the Zone System to Digital Photography, http://hannemyr.com/photo/zonesystem.html  (http://hannemyr.com/photo/zonesystem.html) by Gisle Hannemyr

A Digital Zone System? http://super.nova.org/DPR/ZoneSystem/DigitalZoneSystem.pdf (http://super.nova.org/DPR/ZoneSystem/DigitalZoneSystem.pdf) by Chuck Gardner  Highly Recommended reading.

Zones and Digital, Two Methods of Exposing http://www.libraw.org/node/46 (http://www.libraw.org/node/46) by Iliah Borg  NOTE:  There are many other related articles on Iliah's site if you follow the links at the bottom of the article.

I invite you to read the references, and then perhaps we can have a discussion about the topic.  Often, discussing the ideas can help solidify our understanding of the concepts and how they can best be applied in our chosen implementation of a "modified" Zone System.

Keith

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on June 30, 2009, 01:12:01 PM

Bill Janes provided valuable input (via email) with respect to this topic.  I hope he doesn't mind that I am posting his response here:

I would also include EJ Martin's essay on Noise, Bit Depth, Dynamic Range:
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html
The premise that the purpose of exposing to the right is to preserve the number of tones in the highlights is incorrect. There are plenty of tones in the highlights even if you underexpose according to ETTR. The real rationale has to do with shadow tones and noise.

In particular, check out his analysis of ETTR:
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR
Adobe Camera Raw Forum Thread, Exposure to the right and tone placement. This is a thread I started on the ACR forum some time ago. Bruce Fraser and Jeff Schewe were my antagonists. I got some things wrong, but so did these "experts". Some claimed that the zone system does not apply to digital. With a camera having good noise characteristics (and perhaps a bit depth of 14), underexposure is better tolerated than overexposure to the extent that important highlight details are clipped.

http://forums.adobe.com/thread/311940

IMHO, one should expose for the brightest tone that needs to be reproduced in the scene. Correlation with the camera JPEG engine, the the raw converter and the raw file are essential.

Regards,

Bill

Title: Re: Digital Photography and the Zone System
Post by: girod on July 11, 2009, 07:49:25 PM

Thank you very much Keith.

Could you please teach us the second method of exposure (kowing the sensor and its dynamic range, etc) and how to implement it.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 12, 2009, 09:27:39 AM

Quote from: girod on July 11, 2009, 07:49:25 PM

Thank you very much Keith.

Could you please teach us the second method of exposure (kowing the sensor and its dynamic range, etc) and how to implement it.

girod

Girod,

I would be happy to explain the second, more precise method of analyzing your camera/sensor exposure characteristics so that you can implement this knowledge in a Modified Zone System.  I think it would work best to cover this in a "discussion" format.  There is enough variability in current camera models and raw processing software that trying to write a comprehensive tutorial on the subject would probably require too much "generalization," and result in less precision in addressing a particular camera model, software combination.  

So, if you are up to it, I propose that we start a discussion thread that explains the steps necessary to analyze your camera/sensor exposure characteristics.  Once we better understand the "raw" data coming from your sensor/camera, we can then determine how to optimize your exposures, given your complete camera and raw processing software system.

So to begin, what camera model and raw processing software will you be using?

We will start by recording a series of exposures of a "known" target in consistent lighting conditions in order to determine the "clipping' point or "highlight headroom" available from your camera/sensor.  For the purposes of this test, let's use a color-neutral gray card (something like the whitebal card or an equivalent) as our "target."  We will need to control as many variables as possible in order to get consistent results, and of course the most important variable is the amount of light that will be illuminating the target.  If you have a studio setup where you can control the amount of light, that would be idea; however, we can "make do" with the diffuse light from a window as long as the lighting conditions remain relatively constant.  

Place the gray card on the floor in front of the window in an area where the amount of light hitting the card is relatively constant across the length and width of the card.  Set up your camera on a tripod so that the gray card fills the frame.  Use a lens that minimizes the amount of light fall off (vignetting) on the edge of the frame, set to an aperture of about f8.  Set the camera/lens to manual focus and focus the lens on infinity.  Set your camera on manual exposure, spot meter (set for the center of the frame), base ISO, daylight white balance (to eliminate the "variable" of auto white balance in our later analysis), and of course, to record raw data.  

Take a series of exposures in 1/3 stop increments spanning the range from 0 EV to + 4.0 EV by changing the shutter speed.  (Using auto exposure bracketing if possible.  If you use auto exposure bracketing, ensure the camera is changing the shutter speed and not the aperture to arrive at the different exposure results.)  Take at least three complete series of exposures covering the range from 0 EV to +4 EV, this will help us eliminate any variability caused by inconsistent lighting.  

Once you have recorded your series of exposures, we will be ready to analyze the raw files using Rawnalyze software, which can be downloaded here:  http://www.cryptobola.com/PhotoBola/RawnalyzeDownload.shtm

Please let me know if you are familiar with Rawnalyze software.  If you are already familiar with the software, I won't need to go into as much detail about how to analyze the data to determine clipping points in the raw data.

Please let me know when you are ready to proceed with analyzing the files you recorded in order to determine the "clipping" or saturation point of the sensor.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 12, 2009, 11:05:42 AM

I am really very ecstatic about this Keith, thank you so much.

Here's what I have (all Nikon): D700, 50mm/1.4D, 105mm/2.8G VR, 14-24mm/2.8, 70-200mm/2.8 VR; SB900; CaptureNX2; digital grey card (pocket size from digitalimageflow); Benro C269 M8 tripod + B1 ballhead.

I don't have a Studio and I am 100% clueless about Rawnalyze.

For the last month, I've been using your ETTR exposure technique using 14-bit NEF; in-camera settings -  neutral picture control, +3 sharpening, sRGB and none else . When the occasion calls, I've used Guillermo's D700 UniWB. Then I postprocess in CaptureNX2 - mainly sliding the "Shadow protection" maximally to 100 (unless significant noise comes out) to bring more pixels towards the right; apply a conservative S-shaped tonal curve to my taste, sharpens with USM (40,5,2) and or high filter pass, crop as needed and save as JPEG (excellent compression). Presently, I mainly output to the web thru Nikon's mypicturetown.com. So far, I've been happier with this process and getting satisfying feedback from my web viewers - families and friends only.

Which of the above lenses you recommend I use for the initial series of exposure? Have you done this process with your D700 or D3?

I really appreciate very much your good heart, willingness and patience to teach us in a very humbling effective manner.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 12, 2009, 12:25:24 PM

Quote from: girod on July 12, 2009, 11:05:42 AM

Here's what I have (all Nikon): D700, 50mm/1.4D, 105mm/2.8G VR, 14-24mm/2.8, 70-200mm/2.8 VR; SB900; CaptureNX2; digital grey card (pocket size from digitalimageflow); Benro C269 M8 tripod + B1 ballhead.

I don't have a Studio and I am 100% clueless about Rawnalyze.

Which of the above lenses you recommend I use for the initial series of exposure? Have you done this process with your D700 or D3?

girod

Girod,

Good, let's get started.  I would recommend using the 105mm for your exposure tests, since it has more consistent exposure across the frame than the other lenses.  Lens choice isn't really critical, but it is best to start with the most consistent data possible. 

Most people don't have a studio, so the method of using diffuse lighting from a window will work just fine.  (Any window light that is not direct sunlight shining through the window will qualify as diffuse light, as long as the intensity is consistent long enough to complete one series of exposures.) 

Rawnalyze is relatively easy to use.  I'll talk you through the steps when we get to that point.  For now, just download the program onto the computer hard drive you will be using to analyze the exposures.  No installation is required, since the program will run directly from the exe file.

I have used this process to "characterize" the exposures from my D3.  Our results should be fairly consistent, so we can use the results from my D3 as a "sanity check" against what you are seeing from the D700.  As a caveat, camera meters (especially with lenses as an additional variable) can vary by as much as much as 1/3 stop.  Given these are both "high end" cameras, I would expect the results to be very close.

Once we determine the saturation in the "raw" data, we'll move on to comparing this against what you are seeing in Capture NX.

Please let me know when you have finished taking your exposure series and are ready to look at the data in Rawnalyze.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 12, 2009, 03:38:14 PM

I'm ready Keith, I hope.

As per your instruction, I took 3 series of exposures from 0 to +4EV at 1/3 stop increments (105mm; Manual exposure f8, variable shutterspeed; manual focus at infinity) using window natural lights on pocket size grey card (occupying the whole frame) at around 3 to 4PM. The 1st and 2nd series have some shadows (from my hair) in 2 and 1 frames respectively, the 3rd series is completely free of any shadow. Will this do or repeat a fully clean 3 series?

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 12, 2009, 05:23:03 PM

Girod,

The 3rd series should be fine.  If we see unexpected results, we will reference the applicable shots from the other two series to try to determine why.  I should have also had you take an exposure at +2.5 EV, so if you still have your camera set up, please take a shot or two at +2.5 EV.

I will explain in the next post how to look at the data in Rawnalyze, but need to take a quick break to get dinner on the table for two starving children.  :-)

There is a very nice downloadable manual for Rawnalyze on the (Rawnalyze) site  if you want to "read ahead."  We will be using the "histogram" view without white balance multipliers applied.  I will talk you through the steps in the next post.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 12, 2009, 06:08:23 PM

I too, just came back from a grocery errand.

It's now 7PM here in New Orleans but there's still some light thru my window (not as bright as 3 or 4 hours ago). I will take +2.5EV shots. If this is not appropriate enough for the test, I will repeat the whole series tomorrow PM when I come home from work.

As I look now on the reflected light on my floor thru the window, I think the daylight is now inadequate.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 12, 2009, 06:20:03 PM

Girod,

No problem,  we can do the +2.5 EV exposure tomorrow, and can examine the other exposures using Rawnalyze in the meantime.

To open Rawnalyze, simply click on the Rawnalyze.exe file you downloaded onto your hard drive.  You will probably get a warning that the program has an "unverified signature" but just click OK or Proceed (the program won't harm your computer).

Once you have the program open, you should see a window like this:  

(http://spiritofphotography.com/coppermine/albums/userpics/10001/Rawnalyze_main_screen.jpg)

Click the "browse for" button and navigate to the file shot with +3.0 EV and open this file in the "open" dialog.  The file should open in the "composite" view as shown in the screen shot below.  Click on the "raw clipping" box in Rawnalyze and the "composite" image being shown will display magenta in the areas where raw data is "clipping."  You should see practically the entire image turn magenta, with perhaps some light green areas remaining just around the edges.  Again, the magenta area is "clipped" or where the sensor has exceeded 100% saturation in one of the color channels.

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B3_0_EV_showing_clipping.jpg)

Now click on the "histogram" view.  This view will display the color channels of the raw data (without white balance multipliers applied).  You should see something like this:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B3_0_EV_histogram.jpg)

The yellow dotted lines on the right edge of the histogram displays indicate where "clipping" is occurring in the raw data.  You should see significant clipping in the green channel, and potentially some clipping in the blue channel (depending on the actual color temperature of the lighting in your exposure testing.)  Toggle back to the "composite" view to compare the "area" of the image that is "clipped."  Please pay particular attention to any areas in the image that show "banding" as shown in the screenshot below.  This "banding" is a particular "feature" of the D3 and D700 sensor that we will talk more about later.  (It is probably caused by the "multi-channel readout" in the D3/D700.)

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B3_0_EV_composite_showing_channel_readout.jpg)

Now toggle back to the "histogram" display and click "mapped" and select aRGB 1998 in the dropdown menu under "mapping function."  Click "Ctrl + 0 [zero]" to apply the "as shot" white balance.  This will display how much "clipping" is in the file after the data is "mapped" to the adobeRGB space with the "as shot" white balance applied.  The screen should look something like this:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B3_0_EV_histogram_mapped.jpg)

You will be able to see the substantial amount of clipping that would occur when processing this file in Capture NX.

Next, "browse for" the file shot at +2.7 EV and go through the same process.  You should see only small portions of the image "clipped" if any.  General consensus is that the green channel on the D3 and D700 "clip" or reach 100% saturation at +3.0 EV.   As I explained in my earlier "histogram revisited" post, the green channel is inherently the most sensitive in "daylight" conditions, and so this is the channel we are concerned about the most.  As a general rule, the raw data in the green channel will clip in "daylight" color temperatures well before the red or blue channels.

So, we have in essence discovered our "clipping" or saturation point for the D3/D700 sensor, which is +3.0 EV above a "midtone" exposure according to the spot meter.  You can look at the other shots you took at +2.7 EV and + 3 EV to validate that the clipping point in the green channel is in fact at +3.0 EV (or very, very close to that value).

Please let me know if this makes sense to you, and if it does, we will continue on.  If not, please ask questions and I will try to explain better.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 12, 2009, 09:48:54 PM

Done Keith and what I've seen are exactly what you have described except:
1) No blue clipping at +3EV

2) The "binding" did not show up after I toggled from "Histogram" back to "composite", the same "magenta" showed up instead

I'm all set to continue.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 12, 2009, 10:14:48 PM

Quote from: girod on July 12, 2009, 09:48:54 PM

Done Keith and what I've seen are exactly what you have described except:
1) No blue clipping at +3EV

2) The "binding" did not show up after I toggled from "Histogram" back to "composite", the same "magenta" showed up instead

I'm all set to continue.

girod

Excellent!

The blue clipping will depend on the color balance of the light source.  I suspect my test shots were taken under "cooler" light than yours; however this won't change our conclusions significantly, so we can go on with our analysis.

The "banding" is the effect shown in this screen shot, which is caused by some of the channels in the "multi-channel" readout being "clipped" while adjacent channels are just below the clipping threshold. 

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B3_0_EV_composite_showing_channel_readout.jpg)

The view you are seeing in Rawnalyze is a "zoomed in" view.  Please try scrolling around the +3.0 EV image to see if you can find this type of banding at the edges of the image.  If you don't see any banding in the +3.0 EV image, please scroll around the +2.7 EV image in the "composite" view with "raw clipping" selected and see if you find any banding in that image.  You should be able to find evidence of this banding in one of your +3.0 EV or +2.7 exposures if you scroll around a bit.

It is getting late here, so I will break for now, and we can take a look at these exposures in Capture NX tomorrow to see how the "raw" data is handled by NX.  This will help us understand the "saturation" point in an image mapped to a gamma 2.2 space (adobeRGB) and how much "headroom" we have left to recover "blown" highlights in the image.

I'll check back with you again tomorrow and we can continue our discussion.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 12, 2009, 10:51:40 PM

Thanks Keith. Goodnight and till tomorrow.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 13, 2009, 04:37:22 PM

Hi Girod,

Did you get a chance to look at the +3.0 EV image again in Rawnalyze?  If so, did you scroll through the image and see any banding?  If not, you can "trust me" that the banding exists, and we can continue on with our discussion. 

I wanted to point out the banding, because it is one of the reasons for a "practical" limit of +3.0 EV on the D3/D700.  You will hear many opinions on the internet about the "headroom" available in a D3/D700 file.  The truth of the matter is that this "headroom" is practically limited at +3.0 EV, and will impact the way we expose our images if we want to optimize the image capture.  If you overexpose an area in your image by +3.0EV, and view the image at 100%, a discerning viewer will be able to see the banding in the image.  Highlights are not accurately or cleanly recoverable beyond this point. 

Most raw processing software has the ability to interpolate image data to recover highlights if only one color channel is blown; however, since the "saturation point" is slightly different for the readout channels of the D3/D700, this causes the interpolated image data to show banding as well. 

The whole point of this part of our testing was to help us understand the "upper limit" of the raw image data.  My testing indicates this "upper limit" is at +3.0 EV, and that I should control my exposures accordingly.

Am I making sense so far?

Keith

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 13, 2009, 05:19:21 PM

Girod,

Now that we've looked at the raw data in Rawnalyze, I'd also like you to look at the +3.0 EV, +2.7 EV and +2.5 EV images in Capture NX2.  You should see that all three images show varying degrees of "clipping."

It is important to understand that when most raw processing software "renders" or maps raw image data to a gamma 2.2 space such as sRGB or adobeRGB, they are set to render +2.5 EV at or near saturation in that space.  (My testing indicates that +2.5 EV is rendered at about 252/255, or 99% saturation for the D3 in CaptureNX2 (in sRGB space).  The "lost highlights" indication in CaptureNX2 indicates "clipping" at 254/255)  So what happens to the data that is between +2.5 EV and +3.0 EV?  That data is in what should be considered "headroom."  You will hear other definitions of "headroom" used on the internet.  The engineering definition of "headroom," and the one that makes the most sense in the context of "highlight recovery," is "the amount by which the signal capabilities of a system exceed a designated level, known as Permitted Maximum Level (PML). Headroom can be thought of as a safety zone allowing transient peaks to exceed the PML without exceeding the signal capabilities of a system."

It might help to consider "headroom" the same as "margin" between the point the manufacturer has defined to render as 100% saturation (just over +2.5 EV) and the point of actual physical saturation of the sensor (+3.0 EV).)  You can recover the image data captured in this "headroom" by using the "highlight protection" slider in NX.  The "highlight protection" function works by "remapping" all of the tonal values above the mid-tone so that more of the raw data can be "mapped" within the 128 - 255 tonal range available in a gamma 2.2 space.  This is often referred to as "highlight compression," since the difference between tonal values in the highlights is "compressed" due to the remapping.  This can be counter productive if your goal is to render distinct detail within a specific range in the highlights (for example the detail in white lace or a white wedding dress, feather detail on a white bird, etc.).  By "compressing" the highlights, you have in effect lowered the local contrast in the highlights and made this detail less distinct.  Do you see why "highlight compression" is not always desirable?

Another method of recovering the highlights is by setting negative exposure compensation with the exposure slider.  This in effect remaps all of the raw data to "lower" respective tonal values in the rendered image, making the entire image appear darker (but displaying the highlights that were initially above 254 in the rendered image).  You can use negative exposure compensation and then compress the shadows to bring them back into the image by using the "shadow protection" slider in NX2.  Of course this has the same negative effect of reducing the local contrast in the areas between the black point and mid-tone, which also results in a "flatter" looking image.  Just like with shooting transparency film, you should sometimes make a conscious decision to "let shadows fall where they may."  It often provides a more pleasing image if we let shadows remain dark (with portraits being an exception).

The point of this discussion was to help you understand the difference between "clipping" in the raw data, and "clipping" in the rendered image, and to understand some of the methods for optimizing the rendered image.  

I'll come back to this point many times, but it is worth stating here that the reason for using a modified Zone System to control our exposures is so that we can make a conscious decision of where we want the tonal values in our image to be displayed, and then optimize our exposure and image rendering (development) accordingly.

Am I making sense still?

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 13, 2009, 07:23:44 PM

Wow, this is truly a very liberating education for me Keith. You have the rare gift to teach, I really appreciate it very much.

So many things make sense to me now like, I see this "banding" so often when I try to recover highlights with the "Highlights Protection" slider; same thing with shadow recovery (though I did not know of combining it with the negative exposure slider) as you have described.

I came home late, the light was not available anymore to shoot for the +2.5EV. I'll do this hopefully tomorrow.

Meantime, can we please continue.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 13, 2009, 07:56:23 PM

Girod,

I'm glad it is making sense.  When you shoot the +2.5 exposures tomorrow, please also shoot one other series of images.  This time please shoot a series of images of a white towel.  Use the same setup as before, and take images of the white towel with 0 EV, and then from +1.7 EV to + 3.0 EV in 1/3 stop increments.  This test will allow us to see what "detail" we can render from images shot at these various exposures.  The purpose is to help us nail down where "Zone VII" is on the exposure scale.  Zone VII is defined as "the highest zone that will still retain good details" (such as textured snow, finely patterned white material, etc.)  When using a modified Zone System to base our exposures on in the field, we need to understand how to optimize our exposures to "retain good detail" in the important highlight areas of the image.

By the way, one of the "shortfalls" of a simple ETTR technique using the histogram, is that photographers are often tempted to "protect" highlights in Zones IX and X.  This would be an incorrect exposure according to the Zone System.  Zone IX is defined as a bright white area with no detail or significant texture, and Zone X is defined as a bright light source or "paper base white, with no image data recorded."   If we wanted to optimize the overall quality of our image (not induce noise in the shadows) we would most likely make a conscious decision to let Zones IX and X be "clipped" in the rendered image.  Keeping in mind of course, the "banding" issue at +3.0 EV.  As long as we are not overexposing large portions of the image, the banding shouldn't be noticeable.

Unfortunately I need to take a break to help get children to bed, but would like to continue this discussion later.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 13, 2009, 09:05:47 PM

I am assuming that when I take shots of a white towel, it should be focused optimally. I'll take those shots tomorrow.

Warmest goodnight to the kids. I have a 4 and a 2 year old girls, they are my magical inspiration to really get to know my D700 optimally for every scenario.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 13, 2009, 10:04:47 PM

Quote from: girod on July 13, 2009, 09:05:47 PM

I am assuming that when I take shots of a white towel, it should be focused optimally. I'll take those shots tomorrow.

girod

Yes, good point.  We want to be able to judge the "detail" in the shot, so yes, you should ensure the image is focused optimally.  

One thing I forgot to mention in my initial instructions here:  "Set your camera on manual exposure, spot meter (set for the center of the frame)..." was that you should use the "eyepiece shutter" to eliminate stray light coming in through the viewfinder that may affect your metering.  In the range of exposures you were working with, using the eyepiece shutter could make about 1/3 stop difference in exposure.  (So your +3.0 EV exposures might have actually been closer to +2.7 EV if you weren't using the eyepiece shutter and took the shot without your eye to the viewfinder.)  If, however, you had your eye up to the viewfinder when taking the exposures, then you were also blocking the stray light, and the exposures should be accurate.  

You might also want to use mirror lockup and a cable release for the "white towel" shots.   Given the range of shutter speeds you are likely to encounter, mirror lockup should help prevent camera shake.

While you are taking your "white towel" shots, we can also "skip ahead" to the next test sequence and take the "black point" shots as well.  The shadow range is a bit more forgiving, so we don't need to expose at 1/3 stop increments.  1/2 stop increments would probably be sufficient.  The purpose of this test is to help us understand what exposure will give us Zone III, where we typically place the "darkest area with visible detail," and how much we can "recover" shadows without objectionable noise or posterization in the image.  You should start this sequence (exposure shots of the white towel) at -2.0 EV and continue in 1/2 stop increments all the way down to -5.0 EV.  

Once we understand how a "nominal" exposure will place the tones in our rendered image, and how much "margin for recovery" we have at each end, we can start to make informed decisions about how to optimize our exposures in the field to best capture the dynamic range in the scene.  Understanding "nominal" placement of dark tones and how much we can recover in the shadows if needed is part of this "informed" decision making process.

I should add a couple of caveats:
1)  You will read all sorts of claims on the internet about "shadow recovery" capabilities of various cameras and software.  The bottom line is that the level of noise and posterization that one is willing to accept in an image is a personal choice, and often dependent upon the particular scene.  Each of us needs to perform some level of testing on our cameras/sensors to determine "acceptable" limits for our intended use of the image.  (More noise is acceptable if the image will be displayed less than full size on the web, etc.)  These "acceptable limits" will also change based on ISO, with progressively less "latitude" available at higher ISOs.

2)  The "white towel" test to determine the level of recoverable shadow detail will give us only a rough approximation (and an optimistic one) of what level of shadow recovery is possible.  The most demanding test will be recovering "skin tones" from the shadows.  You will find that your limits of "acceptable" recovery will be influenced by how well the color balance is retained and recovered in the lower reaches of the shadows.  Some cameras are better than others in this regard.  The D3/D700 is not the best in this regard (D2X probably is) but not the worst either.  As a follow-on to your "white towel" tests, you might want to refine your "acceptable" levels of shadow recovery by performing a subset of this test using skin tones...

We're not to far from our goal now.  Just a little more discussion about "mid-tones," tone curves, and "real world" dynamic range.

Keith

Title: Zone VII "White Towel" Test
Post by: keithsnell on July 14, 2009, 04:59:42 PM

Girod,

For the benefit of other website visitors that haven't had the opportunity to perform these tests themselves, I thought I would post the results of my "white towel" exposure testing.  Please let me know if your results agree with these.

Again, the purpose of this phase of my testing was to determine where "Zone VII" is on the exposure scale.  Zone VII is defined as "the highest zone that will still retain good details" (such as textured snow, finely patterned white material, etc.)  When using a modified Zone System to base our exposures on in the field, we need to understand how to optimize our exposures to "retain good detail" in the important highlight areas of the image.

We used a "white towel" as our target because it has plenty of fine detail and texture that will allow us to judge when we have successfully exposed the image at the "highest zone to retain good detail."  Since we had already determined by examining the raw data that  the saturation point for the D3/D700 sensor is +3.0 EV from the "mid-tone" meter reading , I used that as the "upper end" of the test range for this series of images.  

I set up the camera on a tripod, focused on the white towel in even diffuse lighting on the floor below the tripod, set an aperture of f8, closed the eyepiece shutter, spot metered and set the exposure at +3.0 EV (I checked the meter by "defocusing" the image to ensure it wasn't effected by uneven lighting on the towel), and took the exposure using "mirror up" and a cable release.  I repeated this process in -1/3 stop increments down to +1.7 EV above mid-tone.

I then opened each image into Capture NX and used the "highlight protection" slider to pull the histogram back down into the image in an attempt to recover the highlights.  I used the "show lost highlights" view to judge how much I needed to move the "highlight protection" slider.  I set the highlight protection slide at the point where the "show lost highlights" view no longer displayed any clipping.  I then examined each image for the level of detail present.

Here is the +3.0 EV image, with the highlight protection slider set at 80/100.  Both the histogram and "show lost highlights" view imply that all the highlights have been recovered; however, examination of the image at 100% view tells a different story:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/white_towel_%2B3.jpg)
+3.0 EV, "highlight protection" applied

(http://spiritofphotography.com/coppermine/albums/userpics/10001/white_towel_%2B3_zoomed.jpg)
+3.0 EV, "highlight protection" applied, zoomed to 100%  (cropped)

Here is the +2.7 EV image, again with the highlight protection slider set so that the histogram and "show lost highlights" view imply that all the highlights have been recovered.  I applied a white balance adjustment to this image, so the highlight protection slider had to be set to 85/100 to remove all the "clipping."  Here's the "detail" available in this image:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/white_towel_%2B2_7_zoomed.jpg)
+2.7 EV, "highlight protection" applied, zoomed to 100%  (cropped)

Here is the +2.3 EV image.  Same process, highlight protection slider set at 31/100:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/white_towel_%2B2_3_zoomed.jpg)
+2.3 EV, "highlight protection" applied, zoomed to 100%  (cropped)

Here is the +2.0 EV image.  Same process, highlight protection slider set at 6/100:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/white_towel_%2B2_0_zoomed.jpg)
+2.0 EV, "highlight protection" applied, zoomed to 100%  (cropped)

Adobe Camera Raw has traditionally done a better job of recovering highlights than CaptureNX, so out of curiosity I opened the +2.3 EV file in Adobe Camera Raw 5.3, set the exposure compensation to -1.0, and performed a click balance white balance adjustment.  Again the histogram and "clipped highlights" display indicated that all highlights had been "recovered."  Here's the detail in the file:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/white_towel_%2B2_3_zoomed_ACR_5_3_whitebal_and_-1_0expcomp.jpg)
+2.3 EV, processed in ACR 5.3, -1.0 EV on exposure slider, zoomed to 100%  (cropped)

The ACR conversion does appear to recover highlights better than the NX2 conversion of the +2.3 EV file, providing a smoother tonal transition in the areas that are recovered; however, this rendition falls way short of the detail contained in the +2.0 EV image from NX2.  I tried converting the +2.7 EV file in ACR 5.3, and the rendition was unacceptable, showing the same non-recoverable artifacts as the NX2 renditions.  It appears that ACR 5.3 will provide 1/3 stop additional "margin" in recovering highlights, although at the cost of significant image detail.  

Someone that has been following this discussion might now be asking "if my sensor doesn't saturate or "clip" until +3.0 EV, why in the heck do I need to expose highlights at +2.0 EV in order to retain detail in the highlights?"  The answer is that those "details" span a 1.3 stop range in the image (in pretty much a "bell curve" type of distribution for the image of the towel).  Looking at the +2.0 EV image of the towel again, it certainly doesn't appear to our eye that this is a 1.3 stop range of brightness, but it is.  This is consistent with just about any "real world" scene where you want to "retain detail" in the highlights.  

Lets look at the raw data histogram for the EV +2.3 image (without whitebalance multipliers applied) in Rawnalyze:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B2_0_EV_Rawnalyze.jpg)
White towel spot metered at +2.3 EV, raw data histogram from Rawnalyze

You can see from this screen shot of the raw data histograms that even when metering to expose the image at 0.7 EV below the saturation point of the sensor, there is still a substantial percentage of the green and blue channels that are "clipped" in the raw data.  (The "saturation point" of the sensor is annotated by the dotted yellow line on the right side of the histogram.)  Lowering the exposure to +2.0 EV moves this "clipped" data into the range that can be accurately recorded by the sensor.

Therefore, if my goal is to place "important" highlights into Zone VII, "the highest zone that will still retain good details," then I will expose these highlights at +2.0 EV using the camera's spot meter.  This is consistent with the Modified Zone System that I have been using for three generations of digital cameras, including Nikon, Canon and Kodak digital SLRs.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 15, 2009, 07:25:09 PM

I am very sorry Keith that I am unable to do my assignments. Yesterday I came home very late from work and very tired that I just dropped dead on the bed. And today, I just came home with the daylight gone.

For the benefit of other viewers, please continue the momentum. I will catch up.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 15, 2009, 08:32:22 PM

Girod,

No problem.  I understand what it is like to be busy (and tired).  You can read along with my posts, and ask questions if anything I say doesn't quite make sense.  I do find that most people learn much better by doing than by reading, so you might want to do the tests for yourself when you get a chance.

This gives me a good opportunity to say that we will be taking my 3-year old son and 1-year old daughter tent camping for the next two days, so I won't be posting on Thursday or Friday.  I'll pick up where we left off once we return from our camping trip.  (Hopefully I will get a chance to photograph a bit on the trip too.)

Thank you again for asking your initial question and giving me the incentive to get some of this information posted on the site.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 20, 2009, 04:26:06 PM

Hello Keith,

Finally I was able to do the shots following all your instructions (spot meter, tripod, eyepiece shutter closed, mirror-up and cable release) and here are my results or at least, my interpretations:

1) Grey card, +2.5EV: the data are very near the right wall of the histogram; no clipping at all.

2) White point white towel test:
    a. at 0EV: the data are centered at the midtone (Zone V)
    b. +2EV: near the right wall, no clipping
    c. +2.3EV: clipping starts (only Green channel, No WB correction)
    d. +3.7EV: Clipped but 100% recovered with highlight slider (52/100) without banding at 100% 
                      viewing.
    e. On Rawnalyze: without WB correction, the Green channel is clipped at +2.3; with WB correction,
                                aRGB (1998) - NO clipping of all RGB channels upto +3.7EV

2) Black point white towel and skin tests: at (-)5EV, the data are in the left wall of the histogram, photo
    is black or dark with no details visible; after applying shadow recovery slider (100/100), the data and
    photo moved to Zone III to IV, some and very acceptable noise at 100% viewing.
 

What do you think Keith?

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 20, 2009, 05:02:30 PM

Quote from: girod on July 20, 2009, 04:26:06 PM

1) Grey card, +2.5EV: the data are very near the right wall of the histogram; no clipping at all.

This is what I would expect.

Quote from: girod on July 20, 2009, 04:26:06 PM

2) White point white towel test:
    a. at 0EV: the data are centered at the midtone (Zone V)
    b. +2EV: near the right wall, no clipping
    c. +2.3EV: clipping starts (only Green channel, No WB correction)
    d. +3.7EV: Clipped but 100% recovered with highlight slider (52/100) without banding at 100%  
                      viewing.

Did you mean +3.7 EV or +2.7 EV?  (It seems odd that you skipped from +2.3 EV to +3.7 EV.)

If you meant +2.7 EV, then I think the difference in "recoverable" highlights can be attributed to the blue channel.  I noticed after taking my shots that the window light I shot under was very cool.  It was a "blue sky day" here in Colorado, and since we are at higher altitude (7,000ft) the color temperatures are even cooler.  With that cool of light (equivalent to cool "shade," the blue component of the raw data (without white balance multipliers applied) was actually slightly greater than the green channel.  This caused BOTH the green and blue channels to clip.  Highlight recovery can interpolate data when only one channel is clipped, but when two channels are clipped it doesn't have enough information to successfully interpolate the missing data.  In "direct sunlight" or cloudy conditions, the light would not be as cool, and therefore the clipping would be primarily in the green channel (and therefore more "recoverable" with interpolation).  My results would be the "conservative" results.

I would be very interested in comparing the "detail" in your +2.7 EV shot against the "detail" in the +2 EV shot.  Remember that our goal was to "retain detail in the highlights" by placing them in Zone VII.

Given your results, I will try to find time to reshoot my sequence in "warmer" light (by shooting in full sunlight or on a cloudy day, instead of the cooler "shade" lighting I shot the initial sequence in).  I'd like to compare the detail available from interpolation (i.e., with only one clipped channel) against the detail available from "unclipped" data.

Quote from: girod on July 20, 2009, 04:26:06 PM

    e. On Rawnalyze: without WB correction, the Green channel is clipped at +2.3; with WB correction,
                                aRGB (1998) - NO clipping of all RGB channels upto +3.7EV

Did you really mean +3.7 EV?  That seems odd.

It also seems odd that Rawnalyze was showing clipping at +2.3 EV without WB correction, but no clipping with WB applied.  Note that the "clipping" indications are different if WB is applied.  You won't see the yellow dotted line, but instead will see a "spike" on the right side of the histogram.  The "height" of this spike indicates the amount of clipping.

Quote from: girod on July 20, 2009, 04:26:06 PM

2) Black point white towel and skin tests: at (-)5EV, the data are in the left wall of the histogram, photo
    is black or dark with no details visible; after applying shadow recovery slider (100/100), the data and
    photo moved to Zone III to IV, some and very acceptable noise at 100% viewing.

This is fairly close to what I observed.  My level of "acceptable" noise was at -4.7EV, whereas the noise at -5.0EV became too noticeable for me.  I did however, shoot a few other test scenes with a range of tones from white in sun to black in shadow, and found that I could apply "shadow protection" to recover the "black in shadow" (which was -5.0 EV from midtone) without any objectionable noise.  The bottom line is that in a "real world scene" with a black textured object in shadow I can successfully recover the shadows (If I choose to do so) without introducing objectionable noise in the shadows.  Successful shadow recovery is dependent on accurately placing Zone VII in the image.  If I underexpose too much (in an attempt to "protect" highlights in Zones IX and X) then I won't be able to recover shadows without introducing noise.  This reinforces the need to make a conscious decision of where to place the tones, and meter accordingly, during the initial image capture.

Make sense?

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 20, 2009, 10:12:02 PM

Thanks Keith.

The recoverable highlights was from +2.3 to +3.7. The light was a direct sunlight (~4PM, New Orleans) coming thru my window. You're right the clipping was solely in the Green channel.

There was more detail in +2EV compared to the +2.7. But at 100% viewing, the fibers of the white towel are still very detailed at +2.7

I was looking for the yellow dotted line in Rawnalyze even with WB applied. You're right, with the WB applied the "spike" showed up in the Green channel starting at +2.7

I will reshoot my entire White point white towel test series. Meanwhile, can we please move on.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 21, 2009, 10:03:14 AM

Girod,

A couple of thoughts:

1)  There is a bit of inconsistency between our results.  One reason is because we were testing under light with different color temperature (shade vs direct sunlight).  However, this doesn't explain all of the differences we are seeing.  I believe there might also be a bit of inconsistency both due to our interpretation of the results and perhaps a misunderstanding of how to "apply" white balance in Rawnalyze.

Here is a screen shot of the histograms displayed in Rawnalyze for the +2.0 EV image with white balance applied:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B2_0_EV_Rawnalyze_WB_applied.jpg)
"white towel" exposure test at +2.0 EV, histogram in Rawnalyze with "selection" WB applied

Using the "composite" view, I made a large selection (right click and drag) from the center of the image, then clicked Ctrl + W to apply a white balance correction to the image using the "selection" to calculate a neutral white balance rendition.  The "hash marks" on the horizontal axis of the Rawnalyze histogram display indicate 1/3 stop increments.  With white balance applied the red and green channels are essentially just below clipping and the blue channel is clipping slightly.  

Here is a screen shot of the Rawnalyze histogram of the +2.3 EV image with white balance applied:

(http://spiritofphotography.com/coppermine/albums/userpics/10001/%2B2_3_EV_Rawnalyze_WB_applied.jpg)
"white towel" exposure test at +2.3 EV, histogram in Rawnalyze with "selection" WB applied

Note that at +2.3 EV, with the "proper" white balance applied, all three color channels indicate clipping.  In my case (because I was shooting in "cool" light) I was unable to recover the detail in the towel because both the green and blue channels were clipping in the raw data.

I hope you don't think I'm harping on this too much.  I really want to make sure we are on the same page here because the most "unforgiving" aspect of digital photography is when data is "clipped," i.e., the sensor is saturated.  If the data is clipped, the software can "interpolate" (guess at) the data to try to reconstruct the missing detail, but this interpolation is a poor approximation of the "real" data.  If it isn't important to display this portion of the image with all of the detail that is available in the original scene, then that is OK, but it doesn't belong in Zone VII if it isn't important.  

It is widely accepted by followers of the Zone System that "perceivable detail" belongs in Zones III through VII.  For a high-contrast scene (using a digital camera), you would expose to place important highlight detail in Zone VII and then adjust mid-tone and shadow placement as needed during post-processing.  We need to understand how to expose correctly to make that happen on a consistent basis.  Mid-tone and shadow placement are less important during the initial image capture because we have more latitude in adjusting placement of those tones in post-processing.

2)  I think the results of our tests under different color temperatures of light reinforce that our success at interpolating data from clipped highlights will vary depending on the "warmth" or "coolness" of the light source.  With "direct sunlight," as long as only one channel is clipped, the software can interpolate the data in the other two color channels to "reconstruct" a portion of the missing data in the clipped channel.  The success of this interpolation will depend on how consistent the data was in "real life" in the clipped channel.  With cooler light, we are in danger of clipping both the green and blue channels at the same time.  The challenge is to know when we are truly clipping the blue channel, and not be fooled by the white balance multiplier being applied to the data during the in-camera or CaptureNX2 processing.  

A general rule of thumb for the D3 and D700 is that in "shade" under a blue sky, the blue channel will clip at about the same time as the green channel.  The blue channel coefficient (for the D3/D700) for "shade" is 1.0859, so if you see blue channel clipping when you have "shade" set for white balance (or when you are using auto WB and shooting in shade under a blue sky) then you should expect that the "raw" blue channel data is truly clipping.

There are differing opinions with respect to how "hot" we can expose the highlights and still obtain usable results.  It is true that in some situations, you can expose (Zone VII) highlights at +2.3 EV, or even +2.7 EV, and still obtain usable results by using "highlight protection" or other methods to recover the highlights.  My goal is to be able to consistently obtain optimum exposures in challenging lighting conditions.  I know that I have a lot of latitude in the shadows with the D3 (and D700) and so adopting a "conservative" method of exposing the highlights has served me very well.  Exposing highlights with visible detail at +2.0 EV will almost always enable me to produce a viable image from the recorded data.  Exposing any hotter than this is "treading on thin ice."

I would very much like to continue this discussion, but I need to take a break now to finish getting ready to teach a workshop in Crested Butte, Colorado.  Please check back with me on Tuesday of next week and we can continue our discussion.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 21, 2009, 03:26:40 PM

Thank you very much for your patience Keith.

I've just redone the "White point white towel test" (New Orleans, 3PM direct sunlight on the floor thru my window) and here are my interpretations:

In CNX2:
1) 0EV - data centered at Midtone (Zone V) and no clipping upto +3.0
2) +3.3 - clipping starts (Green channel)
3) +4.0 - Highlights clipped (still Green) fully recovered with 74/100 highlights slider, no banding that I could discern at 100% viewing

In Rawnalyze:
1) +2.0 - No WB: R&B at middle of left of histogram, Green at midtone to middle of Left of histogram; With WB: R&B at Zone V, G at Zone V to middle of right of histogram.

2) +2.3 - tiny spike of Green clipped both with and without WB.

3) +2.7 - Green clipped without WB; R&G clipped with WB applied.

I think the above is consistent with your tests on your D3 - that for D3/D700, +2.0 EV (??stretchable maybe upto 2.3 - 2.7 with UniWB) is the safe Zone VII.

If you think that the above findings on my D700 have inconsistencies relative to your expectations, I could send to you the NEF files.

Do you think we can now go forward to "Tone curves", maximally capturing the real-world dynamic range, correct and maximal use of the CNX2 and others that you deemed important in this method of exposure.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 21, 2009, 04:03:36 PM

Girod,

Thank you very much for your patience, and for sticking with me in this discussion.  I would like to continue the discussion, unfortunately I am up against deadlines for preparing for our upcoming workshop in Crested Butte.  I will have to defer any further discussion until we return from the workshop next Tuesday.

If you have time to do some experimentation on your own, you might want to place a gray card in a "real world" high-contrast scene (one that contains highlights where you would want to retain detail, with bright areas in sun and dark areas in shadow), and use spot meter to determine the "range" of the scene by metering on the highlights, mid-tones and shadows.  You will want to note the exposure values of the different areas within the scene so that you can refer to them later.  Next, use two different methods to expose the scene.  First, meter on the mid-tones with spot meter (meter on both the gray card, and other tonal areas within the scene that you judge to be "mid-tone) and take the exposure.  Try an additional "mid-tone" exposure with -0.5 EV exposure compensation.  Next, meter on Zone VII and take the exposure (at +2.0 EV, then if you wish, at a higher EV).  Examine where the tonal values fall for the highlight, mid-tone and shadow areas when the images are rendered in Capture NX2.  

After you have completed your experiments with a "high-contrast" scene, repeat the same experiments with a "low contrast" scene, where all the tonal values fall within Zones III through VIII.  

There are two components to this exercise.  1)  Observing how different metering techniques affect the results  2)  Observing the "non-linear" placement of tones in the rendered images using the various tone curves within Capture NX2.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 29, 2009, 02:39:14 PM

Welcome home Keith.

I'm in the process of implementing (or experimenting) the 2 methods of exposure (Zone VII exposure and the centered-exposure) that you have so patiently illustrated, for both the high contrast and low contrast real world scenarios. I will try the -0.5EV in centered-exposure and the +2.3, +2.7, +3.0 EVs in Zone VII exposure. I will also try these using the UniWB (for D700) + custom linear tonal curve.

Meanwhile, I have some questions:
1) "Examine where the tonal values fall for the highlight, mid-tone and shadow areas when the images are rendered in Capture NX2."  - by looking at the CNX2 histogram? or looking for the actual number of tonal values? where? in Rawnalyze? how?

2) "Observing the "non-linear" placement of tones in the rendered images using the various tone curves within Capture NX2." - the CNX2 picture control (Neutral, Standard, Vivid etc)?

3) "..modify the behaviour of your raw processor in order to force it to interpret how you have chosen to record the data (to maximize dynamic range) and output the data in the way that you want it presented." - how can I do this in CNX2?

4) Could you please explain more on midtones and tone curves?

5) How do you create a custom linear curve for the D700. I did it this way: In CNX2, opened "Picture Control Utility", selected "Neutral" then "Use Custom Curve" (this, I've read in dpreview forum, apparently defaults to a linear "flat" curve); exported then loaded to my D700. Is this right?

Thanks,
girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 29, 2009, 03:52:32 PM

Hi Girod,

In answer to your questions:
1)  The easiest way is to use the "watch point" function in NX2 to see where the tonal values fall within the RGB scale.  

2)  First, 0 EV exposure does not place the resulting rendered value at the midpoint, but above the midpoint.  Typically a -0.5 EV exposure compensation is required to place a tone at mid-tone in the rendered image using a linear tone curve in NX.  If you are using a non-linear tone curve (i.e., one of the supplied tone curves with NX), then even more negative exposure compensation will be required to place a tone at mid-tone in the rendered image.  Second, what tone curve you select will determine how much the other tonal values are shifted in relation to mid-tone.  Using a "standard" or "vivid" tone curve will shift the shadow and highlight values in relation to mid-tone, applying an increasingly aggressive "S" curve to the image.  

3)  If you record a mid-tone at 0 EV and capture NX places that mid-tone value higher than mid-tone in the rendered image, then you will need a curves adjustment or use the midpoint slider (under quick fix, levels and curves, or LCH editor to place the mid-tone in its "proper" position.  (If indeed, that is where you want the midtone to be.)  This is most noticeable when skin tones in portraits look too "hot."  On the other hand, if you expose Zone VII highlights at +2 EV, you are in effect shifting the mid-tones lower on the scale.  This actually places them about where they should be (around 118 in RGB), in effect compensating for the "hidden" exposure compensation NX applies to the midtones in a normally exposed image.

4)  "Mid-tone" is generally considered to be 118 on the RGB scale.  Therefore you would expect that if you used "highlight protection" NX would only shift the highlights and leave the midtones and shadows alone, and if you use "shadow protection" NX would only shift the shadows and leave the midtones and highlights alone.  This doesn't seem to be the case.  What appears to be happening when you use "shadow protection" is that NX keeps the white point set, but compresses both the shadows and midtones, and even some of the highlights up toward the white point of the image.  In other words, the "rendering" of mid-tone in NX is variable, depending on what tone curve you select and how you apply shadow and highlight protection or adjust the black and white points of the image.

5)  You created your custom linear curve correctly.

Title: Re: Digital Photography and the Zone System
Post by: girod on July 29, 2009, 07:47:28 PM

Each step in this lesson is progressively exhilarating for me. Thanks a lot Keith.

I opened a NEF file (with in-D700 custom linear tonal curve) in CNX2, retained the linear curve in CNX2; applied shadow and highlight recovery optimally, applied a conservative S-shaped tone curve and then slid the midtone lever very slightly to the left of the histogram upto a value around 118 - I am very pleased with the result. Is this right, keeping the midtone value at around 118 is most important?

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 30, 2009, 07:43:33 AM

Quote from: girod on July 29, 2009, 07:47:28 PM

I opened a NEF file (with in-D700 custom linear tonal curve) in CNX2, retained the linear curve in CNX2; applied shadow and highlight recovery optimally, applied a conservative S-shaped tone curve and then slid the midtone lever very slightly to the left of the histogram upto a value around 118 - I am very pleased with the result. Is this right, keeping the midtone value at around 118 is most important?

girod

Hi Girod,

Keeping the mid-tone value at around 118 is important if those tonal values are an important part of the image.  If you were using one of the tone curves that shipped with your D3 and were not aware that NX2 shifts the mid-tones to a brighter position on these tone curves by default, then you might incorrectly assume that your images are "over exposed."  As I mentioned, this is especially evident in portraits, where the skin tones might look too "hot" or over exposed.  It is important to recognize that these too-bright mid-tones are mostly due to the "tonal shift" and not because of over exposure.  That knowledge, along with the knowledge of how the default tone curves (aggressive S curves) and shadow and highlight protection functions are affecting your image will enable you to optimize the rendering of your captured image.  

(As an aside, the optional D2X tone curves do not apply this tonal shift, which is why many portrait shooters prefer the D2X tone curves over the ones that shipped with the D3.  Nikon recommends setting the "brightness" to -1 when using the D2X tone curves in order to match the "out of camera" rendering from the D2X.)  

Notice that I said the perceived over-exposure is "mostly" due to the tonal shift.  NX2 (not the camera) also applies what some people have called a "hidden" exposure compensation of about +0.5 EV to the image.  Therefore, using a linear tone curve corrects for the "tonal shift" but the mid-tone will still be rendered slightly brighter than you "exposed" it.  If your goal is to correctly render the mid-tone at 118 in RGB, you will need adjust the midpoint in post processing, or expose with -0.5 exposure compensation.  (The -0.5 EV exposure compensation correctly places the mid-tone, but still implements the "tonal shift" of mid-tones in relation to other tones in the image.)  In general, I find that the -0.5 EV exposure compensation often results in the most "pleasing" rendition of the image.

In the "old days" of digital processing, the built-in tone curves in Nikon cameras were much "flatter" (like the linear tone curve) and much more "accurate," without the tonal shifts, brightness adjustments and aggressive S curves that you are seeing with the "default" curves in the D3 and D700.  The reality is that these "flat" tone curves didn't look pleasing right out of the camera, which made post-processing necessary for almost every image in order to make it look pleasing to the eye.  Nikon has incorporated years of experience into the "new" tone curves, automatically applying many of the "corrections" that used to be required in order to produce a pleasing image from a raw file.  I used to have to "brighten the mid-tone" on many of my images to make them look appealing.  Now this is applied automatically by what some call "hidden exposure compensation" and the tonal shifts in the "in camera" tone curves.  The end result is that the majority of images do in fact look better "out of camera" than they would have in the "old days."  However, if you truly want to be in control and "optimize" the output, then you need to understand what is going on "behind the scenes."  

Note that much of this discussion is predicated on the belief that "proper" placement of mid-tones is 118 in a gamma 2.2 RGB space (sRGB and AdobeRGB).  This is based on Ansel Adams' placement of "mid-tone" at 18% gray, which equates to a luminosity of 50 in Lab space, which equates to 118 in RGB gamma 2.2 space.  I'm not 100% convinced that "mid-tone" should be 118 in a gamma 2.2 RGB space.  Notice that the "mid-point" in your curves adjustment in your raw processing software is 128, not 118.  This is true for NX2, Photoshop, and every other image processor that I am aware of.  I believe that if Adams were alive today, he would be an avid digital photographer, and would choose to define 128 as "mid-tone" in a gamma 2.2 RGB space.  Defining "mid-tone" as 118 complicates matters unnecessarily, and causes us to perform mental gymnastics that are not necessary if we accept 128 (just +0.5 EV brighter on the scale) as the "mid-tone," as Nikon seems to have done in their latest generation of digital cameras.  For now, I will defer to the popular opinion that "mid-tone" is 118, but recognize that many image processors treat 128 as the mid-tone when applying curves adjustments, etc.

The "most important" thing we have talked about in this discussion is "proper" exposure of the initial image capture in order to optimize the quality of the raw data.  That said, you also need to understand the default "behavior" of your raw processor, and how to adjust this "behavior" to optimize the image rendering to match the way you intended to render the image.

I hope this helps.  I know there are many "variables" to consider during the image capture and post-processing; however, understanding the behavior of our cameras and software will help us optimize our image capture and rendering.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on July 31, 2009, 04:42:04 AM

Hello Keith,

This whole lesson on knowing the principle behind proper exposure is a revelation for me. As a hobbyist beginner, it lifted a tremendous barrier in my understanding and practice of photography.

All I have to do now is shoot as much as I can so I will know my D700 (and lenses) better.

You are a Great Teacher Keith, thank you very much.

girod

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on July 31, 2009, 07:40:45 AM

Hi Girod,

Thank you for sticking with me throughout the discussion.  The discussion was helpful for me as well, since it helped to reinforce my understanding of how to optimize the exposure and post-processing for images shot with the D3.

Happy shooting,

Keith

Title: Re: Digital Photography and the Zone System
Post by: Shane on February 23, 2012, 05:07:31 PM

I realize this is an old topic but I think my response/question is pertinent to the original post. If not feel free to initiate a new posting.

I have been trying to get organized with my different DSLR bodies with respect to exposure "calibration" and highlight clipping. I have been using UniWB and Linear Tone Curve since 2007 and developed a "gut" feeling for exposure settings and clipping based on experience and the histogram. However, this worked well for the initial few years when I only maintained two different bodies but now with several bodies and different wavelength conversions it was time to get organized and perform some "calibration". This is when I dug this very useful post up from my notes and decided to try a structured approach to the problem.

First some background. I am working with a D700, D300S, D200, D200UVIR and D200IR 830nm. I currently run linear tone curve and UniWB on all of these. Some of the UniWB files I have created from scratch and (due to laziness) some have been downloaded. Typically I use Rawnalyze, Histogrammar and exiftool to help with evaluating RAW data. Being newer (and less familiar) I decided to start with the D700 and D300S. This where the first "issue" arose.

Test was performed outside on a "medium" bright but solid overcast day. I used a Sekonic L-558 to meter the Kodak grey card before during and after the test to verify that lighting was not changing.

Based on evaluating exposure and clipping using Rawnalyze, with the D300S it appears that clipping starts to occur at +3.7EV over the metered grey card. I also ran the D700 and got a similar result of around 3.7EV. However, I need to rerun the D700 as the light started to change slightly during the test.

Based on Rawnalyze, the 0EV exposure is around 3.7EV below the G clip point which is also consistent with my final result where clipping actually occurred at just above +3.7EV .

This was in contrast to the +2.7/3.0EV reported clipping point in your D700 test. Although I will rerun the D700 when lighting is more consistent I don’t expect to see much of a change. Any comments would be welcome.

Shown are D300S at 0EV, +2.7EV and +3.7EV. However, preliminary tests for the D700 seem to be quite similar.

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on February 23, 2012, 08:17:00 PM

Hi Shane,

How were you metering?  Spot meter with the camera's meter?  Spot in the center of the frame?  Was the camera analog exposure scale showing 0 for the 0 EV shot?  You verified no exposure compensation was set?

It's always tough to determine why there might be differences when I can't observe your test procedures.  It doesn't seem like your results are consistent with what others have reported (and what I have observed), but at this point I'm not sure why. 

Keith

Title: Re: Digital Photography and the Zone System
Post by: Shane on February 24, 2012, 09:50:25 AM

Hi Keith

ISO 200, f/8, Spot metered (center spot), zero on the analog meter for the 0EV shot (same meter reading from the Sekonic), Exp Comp. zero, 70mm lens, slightly defocused.

The only difference I can see with regards to your test is that I am using a UWB but this shouldn't result in that kind of difference*. You might notice that even our exposure times are quite similar and by coincidence I realised that our focal length was identical.

* Unless perhaps the lighting environment is extremely blue or red rich (afterthought - I am thinking of perceived exposure not analog metering). You can see from my Rawnalyze that this is not the case.

The 0EV shot as shown in Rawnalyze is very important. As you can see it sits about 3.7EV below the saturation point. It would be helpful if you could show your 0EV shot for the D3, based on your results it should be 2.7EV below saturation.  I also noticed that you specified to use a Daylight WB and to avoid Auto WB however your initial Rawnalyze images show that an Auto WB was used, later images indicate Manual WB, was this an oversight?.

I found your overall approach to "calibration" to the Zone system quite interesting and will be continuing with that but would like to resolve this issue first..

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on February 24, 2012, 02:46:51 PM

Hi Shane,

I'll see if I can dig up the 0 EV shot.  I'm not sure if I still have that series on the computer, but I'll look.

With respect to the white balance setting, it definitely has an effect if you are using an external raw editor (for the white towel shots, etc); however, since Rawnalyze enables us to look at the files without WB multipliers applied, I don't think it's much of an issue when examining the files in Rawnalyze.

I'll see if I can dig up my old files from this test.

Keith

Title: Re: Digital Photography and the Zone System
Post by: girod on February 24, 2012, 05:41:50 PM

Hello Keith and Shane,

Iliah Borg have just published their new program to analyze raw file and meter calibration: rawdigger.com or here: http://www.luminous-landscape.com/forum/index.php?topic=62739.0

With my D700 and D7000 + RawTherapee (v4.0.7.1, Windows 7, 64bit) raw developer, I too have been using +3.7EV for Zone VII; AWB and I don't rely anymore on the in-camera Histogram.

Title: Re: Digital Photography and the Zone System
Post by: Shane on February 24, 2012, 08:04:25 PM

Girod
thanks for responding. I did notice the release of the new rawdigger but haven't downloaded it yet. I prefer to wait a little usually until the bugs have been sorted. How do you find it compared to Rawnalyze?

Title: Re: Digital Photography and the Zone System
Post by: girod on February 24, 2012, 11:59:37 PM

Shane,

I have not tried rawdigger yet.

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on February 25, 2012, 01:02:54 PM

Thanks for the heads up on rawdigger.  I'll have to give it a try.

The objective is to get consistent results in a way you are comfortable with, so if +3.7EV for Zone VII works for you then go for it.  Personally I find that +3.7 EV results in too many blown highlights in "real world" shooting, but you should use whatever method gives you consistent results that meet your requirements.

Shane, I looked for the old D3 files on two computers and didn't find them.  I'll try to find time to dig out my old laptop and look for them there.

Title: Re: Digital Photography and the Zone System
Post by: keithsnell on February 25, 2012, 01:13:47 PM

Hi Jaime and Shane,

It's worth noting that in the link that Jaime (Girod) provided, Bill Janes concludes "So one can spot meter the area in the scene where one wants to retain highlight detail and increase exposure by 3 stops, placing the metered area at clipping."  This is in reference to the D3, and is consistent with my own testing (and Iliah's).  I'm positive that Iliah's initial testing of the D3 pegged the raw clipping point (for the green channel) at +3 EV.

Still, if +3.7 EV works for you, then that's what matters.

Keith

Title: Re: Digital Photography and the Zone System
Post by: Shane on February 26, 2012, 09:36:19 AM

Hi Keith

I haven't got to the point in this test sequence of determining how far I can push the exposure and still retain detail in VII, so in that respect I don't know if +3.7EV will be the final result. At this stage I am trying to determine why the test indicates the RAW clip point to be +1EV higher than your D3 RAW clip point.

Thanks for looking for the old file but perhaps if you have time, it would be quicker to just reshoot a Grey card at 0EV and check in Rawnalyze to see where the G channel falls.