Author Topic: Digital Photography and the Zone System  (Read 21494 times)

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Digital Photography and the Zone System
« on: June 27, 2009, 09:31:29 AM »
Those of you that have been following the discussion between Iliah Borg and I on DPReveiw have probably figured out by now that my brief overview on how to use the Zone System in the "Understanding and Controlling Exposure" tutorial was an oversimplification.  That shouldn't be a surprise to anyone.  To be semantically correct, I should have described this method as a "Modified Zone System."  You will see this type of "Modified Zone System" used often in digital photography.  

We've also had a discussion on DPReview about how deeply one needs to delve into a subject before we are satisfied that we have a "practical" knowledge of the subject and can go about the business of making beautiful images.  The answer is that it varies from person to person.  The practical answer is that great photography is a culmination of many different skills, and your photography is limited mostly by your "weakest link."  If you are strong on the artistic side, but limited on the technical side, then your photography can only progress so far.  There is a balance that each photographer must define for his or herself.  

In truth, the Zone System, as defined by Adams, cannot be directly applied to digital photography without some modifications.  However, his concepts of assigning exposure to Zones can be very useful in helping us to define a systematic method for controlling our exposures.  You will find two schools of thought with respect to the "best" way to implement a modified Zone System in digital photography.  

One school of thought is that we can use the concepts in the Zone System in order to make a conscious decision about how to optimize our exposure based on placing a chosen brightness level in the scene into a predetermined range as it would be presented by our "standard" work flow.  Although this technique doesn't necessarily "maximize" the image quality that can be extracted from the raw file it is, without debate, a much better method for controlling exposure than simply pointing your camera towards the scene and using matrix metering to determine the exposure.  This is the method presented in my Understanding and Controlling Exposure tutorial.

There is a second, more precise method of using the Zone System that requires much more technical knowledge of how your camera sensor and raw processor works.  This method requires that you analyze the raw data from your camera to determine the absolute limits of dynamic range available from your sensor, and understand how this data is recorded in the raw file.  You must then understand how your raw processor takes this raw data and manipulates it to present the "output" in the various "working spaces" (color space and gamma curves), and finally, you must understand how to modify the behaviour of your raw processor in order to force it to interpret how you have chosen to record the data (to maximize dynamic range) and output this data in the way that you want it presented.  There is no debate that this is a more precise method of controlling exposures that can help you extract the maximum dynamic range from your camera.

Which method you chose to use is based on how much effort you want to expend to really learn how your camera and software systems work, and how important it is for you to extract every last bit of image quality available from your sensor.  For many photographers, the "simplified" Zone System I present in the tutorial will be sufficient to significantly improve their photographic skills.  As your skills improve, or for those photographers with the technical aptitude, you might want to explore the concepts described in the second, more rigorous application of the Zone System to digital photography.

I've included links to three very good references that deal with how to use a "modified" Zone System in digital photography.  Although they present different methods for using a modified zone system, none of these methods are "wrong."  They all provide useful concepts to help the photographer progress from simply using the "default" matrix metering for a scene, to using a "simplified" zone system, and then to a more advanced understanding of how to apply the concepts in the Zone System to digital photography.  These references are presented in order of complexity, and each one will give you a better understanding of the concept.

Exposing for the Highlights, Adapting the Zone System to Digital Photography, http://hannemyr.com/photo/zonesystem.html by Gisle Hannemyr

A Digital Zone System? http://super.nova.org/DPR/ZoneSystem/DigitalZoneSystem.pdf by Chuck Gardner  Highly Recommended reading.

Zones and Digital, Two Methods of Exposing http://www.libraw.org/node/46 by Iliah Borg  NOTE:  There are many other related articles on Iliah's site if you follow the links at the bottom of the article.

I invite you to read the references, and then perhaps we can have a discussion about the topic.  Often, discussing the ideas can help solidify our understanding of the concepts and how they can best be applied in our chosen implementation of a "modified" Zone System.

Keith
« Last Edit: July 12, 2009, 09:33:39 AM by keithsnell »

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #1 on: June 30, 2009, 01:12:01 PM »
Bill Janes provided valuable input (via email) with respect to this topic.  I hope he doesn't mind that I am posting his response here:

I would also include EJ Martin's essay on Noise, Bit Depth, Dynamic Range:
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html
The premise that the purpose of exposing to the right is to preserve the number of tones in the highlights is incorrect. There are plenty of tones in the highlights even if you underexpose according to ETTR. The real rationale has to do with shadow tones and noise.

In particular, check out his analysis of ETTR:
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR
Adobe Camera Raw Forum Thread, Exposure to the right and tone placement. This is a thread I started on the ACR forum some time ago. Bruce Fraser and Jeff Schewe were my antagonists. I got some things wrong, but so did these "experts". Some claimed that the zone system does not apply to digital. With a camera having good noise characteristics (and perhaps a bit depth of 14), underexposure is better tolerated than overexposure to the extent that important highlight details are clipped.

http://forums.adobe.com/thread/311940

IMHO, one should expose for the brightest tone that needs to be reproduced in the scene. Correlation with the camera JPEG engine, the the raw converter and the raw file are essential.

Regards,

Bill

girod

  • Full Member
  • ***
  • Posts: 116
Re: Digital Photography and the Zone System
« Reply #2 on: July 11, 2009, 07:49:25 PM »
Thank you very much Keith.

Could you please teach us the second method of exposure (kowing the sensor and its dynamic range, etc) and how to implement it.

girod

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #3 on: July 12, 2009, 09:27:39 AM »
Thank you very much Keith.

Could you please teach us the second method of exposure (kowing the sensor and its dynamic range, etc) and how to implement it.

girod

Girod,

I would be happy to explain the second, more precise method of analyzing your camera/sensor exposure characteristics so that you can implement this knowledge in a Modified Zone System.  I think it would work best to cover this in a "discussion" format.  There is enough variability in current camera models and raw processing software that trying to write a comprehensive tutorial on the subject would probably require too much "generalization," and result in less precision in addressing a particular camera model, software combination.  

So, if you are up to it, I propose that we start a discussion thread that explains the steps necessary to analyze your camera/sensor exposure characteristics.  Once we better understand the "raw" data coming from your sensor/camera, we can then determine how to optimize your exposures, given your complete camera and raw processing software system.

So to begin, what camera model and raw processing software will you be using?

We will start by recording a series of exposures of a "known" target in consistent lighting conditions in order to determine the "clipping' point or "highlight headroom" available from your camera/sensor.  For the purposes of this test, let's use a color-neutral gray card (something like the whitebal card or an equivalent) as our "target."  We will need to control as many variables as possible in order to get consistent results, and of course the most important variable is the amount of light that will be illuminating the target.  If you have a studio setup where you can control the amount of light, that would be idea; however, we can "make do" with the diffuse light from a window as long as the lighting conditions remain relatively constant.  

Place the gray card on the floor in front of the window in an area where the amount of light hitting the card is relatively constant across the length and width of the card.  Set up your camera on a tripod so that the gray card fills the frame.  Use a lens that minimizes the amount of light fall off (vignetting) on the edge of the frame, set to an aperture of about f8.  Set the camera/lens to manual focus and focus the lens on infinity.  Set your camera on manual exposure, spot meter (set for the center of the frame), base ISO, daylight white balance (to eliminate the "variable" of auto white balance in our later analysis), and of course, to record raw data.  

Take a series of exposures in 1/3 stop increments spanning the range from 0 EV to + 4.0 EV by changing the shutter speed.  (Using auto exposure bracketing if possible.  If you use auto exposure bracketing, ensure the camera is changing the shutter speed and not the aperture to arrive at the different exposure results.)  Take at least three complete series of exposures covering the range from 0 EV to +4 EV, this will help us eliminate any variability caused by inconsistent lighting.  

Once you have recorded your series of exposures, we will be ready to analyze the raw files using Rawnalyze software, which can be downloaded here:  http://www.cryptobola.com/PhotoBola/RawnalyzeDownload.shtm

Please let me know if you are familiar with Rawnalyze software.  If you are already familiar with the software, I won't need to go into as much detail about how to analyze the data to determine clipping points in the raw data.

Please let me know when you are ready to proceed with analyzing the files you recorded in order to determine the "clipping" or saturation point of the sensor.

Keith
« Last Edit: July 12, 2009, 09:31:38 AM by keithsnell »

girod

  • Full Member
  • ***
  • Posts: 116
Re: Digital Photography and the Zone System
« Reply #4 on: July 12, 2009, 11:05:42 AM »
I am really very ecstatic about this Keith, thank you so much.

Here's what I have (all Nikon): D700, 50mm/1.4D, 105mm/2.8G VR, 14-24mm/2.8, 70-200mm/2.8 VR; SB900; CaptureNX2; digital grey card (pocket size from digitalimageflow); Benro C269 M8 tripod + B1 ballhead.

I don't have a Studio and I am 100% clueless about Rawnalyze.

For the last month, I've been using your ETTR exposure technique using 14-bit NEF; in-camera settings -  neutral picture control, +3 sharpening, sRGB and none else . When the occasion calls, I've used Guillermo's D700 UniWB. Then I postprocess in CaptureNX2 - mainly sliding the "Shadow protection" maximally to 100 (unless significant noise comes out) to bring more pixels towards the right; apply a conservative S-shaped tonal curve to my taste, sharpens with USM (40,5,2) and or high filter pass, crop as needed and save as JPEG (excellent compression). Presently, I mainly output to the web thru Nikon's mypicturetown.com. So far, I've been happier with this process and getting satisfying feedback from my web viewers - families and friends only.

Which of the above lenses you recommend I use for the initial series of exposure? Have you done this process with your D700 or D3?

I really appreciate very much your good heart, willingness and patience to teach us in a very humbling effective manner.

girod

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #5 on: July 12, 2009, 12:25:24 PM »
Here's what I have (all Nikon): D700, 50mm/1.4D, 105mm/2.8G VR, 14-24mm/2.8, 70-200mm/2.8 VR; SB900; CaptureNX2; digital grey card (pocket size from digitalimageflow); Benro C269 M8 tripod + B1 ballhead.

I don't have a Studio and I am 100% clueless about Rawnalyze.

Which of the above lenses you recommend I use for the initial series of exposure? Have you done this process with your D700 or D3?

girod

Girod,

Good, let's get started.  I would recommend using the 105mm for your exposure tests, since it has more consistent exposure across the frame than the other lenses.  Lens choice isn't really critical, but it is best to start with the most consistent data possible. 

Most people don't have a studio, so the method of using diffuse lighting from a window will work just fine.  (Any window light that is not direct sunlight shining through the window will qualify as diffuse light, as long as the intensity is consistent long enough to complete one series of exposures.) 

Rawnalyze is relatively easy to use.  I'll talk you through the steps when we get to that point.  For now, just download the program onto the computer hard drive you will be using to analyze the exposures.  No installation is required, since the program will run directly from the exe file.

I have used this process to "characterize" the exposures from my D3.  Our results should be fairly consistent, so we can use the results from my D3 as a "sanity check" against what you are seeing from the D700.  As a caveat, camera meters (especially with lenses as an additional variable) can vary by as much as much as 1/3 stop.  Given these are both "high end" cameras, I would expect the results to be very close.

Once we determine the saturation in the "raw" data, we'll move on to comparing this against what you are seeing in Capture NX.

Please let me know when you have finished taking your exposure series and are ready to look at the data in Rawnalyze.

Keith


girod

  • Full Member
  • ***
  • Posts: 116
Re: Digital Photography and the Zone System
« Reply #6 on: July 12, 2009, 03:38:14 PM »
I'm ready Keith, I hope.

As per your instruction, I took 3 series of exposures from 0 to +4EV at 1/3 stop increments (105mm; Manual exposure f8, variable shutterspeed; manual focus at infinity) using window natural lights on pocket size grey card (occupying the whole frame) at around 3 to 4PM. The 1st and 2nd series have some shadows (from my hair) in 2 and 1 frames respectively, the 3rd series is completely free of any shadow. Will this do or repeat a fully clean 3 series?

girod

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #7 on: July 12, 2009, 05:23:03 PM »
Girod,

The 3rd series should be fine.  If we see unexpected results, we will reference the applicable shots from the other two series to try to determine why.  I should have also had you take an exposure at +2.5 EV, so if you still have your camera set up, please take a shot or two at +2.5 EV.

I will explain in the next post how to look at the data in Rawnalyze, but need to take a quick break to get dinner on the table for two starving children.  :-)

There is a very nice downloadable manual for Rawnalyze on the (Rawnalyze) site  if you want to "read ahead."  We will be using the "histogram" view without white balance multipliers applied.  I will talk you through the steps in the next post.

Keith
« Last Edit: July 19, 2009, 07:32:40 AM by keithsnell »

girod

  • Full Member
  • ***
  • Posts: 116
Re: Digital Photography and the Zone System
« Reply #8 on: July 12, 2009, 06:08:23 PM »
I too, just came back from a grocery errand.

It's now 7PM here in New Orleans but there's still some light thru my window (not as bright as 3 or 4 hours ago). I will take +2.5EV shots. If this is not appropriate enough for the test, I will repeat the whole series tomorrow PM when I come home from work.

As I look now on the reflected light on my floor thru the window, I think the daylight is now inadequate.

girod

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #9 on: July 12, 2009, 06:20:03 PM »
Girod,

No problem,  we can do the +2.5 EV exposure tomorrow, and can examine the other exposures using Rawnalyze in the meantime.

To open Rawnalyze, simply click on the Rawnalyze.exe file you downloaded onto your hard drive.  You will probably get a warning that the program has an "unverified signature" but just click OK or Proceed (the program won't harm your computer).

Once you have the program open, you should see a window like this:  



Click the "browse for" button and navigate to the file shot with +3.0 EV and open this file in the "open" dialog.  The file should open in the "composite" view as shown in the screen shot below.  Click on the "raw clipping" box in Rawnalyze and the "composite" image being shown will display magenta in the areas where raw data is "clipping."  You should see practically the entire image turn magenta, with perhaps some light green areas remaining just around the edges.  Again, the magenta area is "clipped" or where the sensor has exceeded 100% saturation in one of the color channels.



Now click on the "histogram" view.  This view will display the color channels of the raw data (without white balance multipliers applied).  You should see something like this:



The yellow dotted lines on the right edge of the histogram displays indicate where "clipping" is occurring in the raw data.  You should see significant clipping in the green channel, and potentially some clipping in the blue channel (depending on the actual color temperature of the lighting in your exposure testing.)  Toggle back to the "composite" view to compare the "area" of the image that is "clipped."  Please pay particular attention to any areas in the image that show "banding" as shown in the screenshot below.  This "banding" is a particular "feature" of the D3 and D700 sensor that we will talk more about later.  (It is probably caused by the "multi-channel readout" in the D3/D700.)



Now toggle back to the "histogram" display and click "mapped" and select aRGB 1998 in the dropdown menu under "mapping function."  Click "Ctrl + 0 [zero]" to apply the "as shot" white balance.  This will display how much "clipping" is in the file after the data is "mapped" to the adobeRGB space with the "as shot" white balance applied.  The screen should look something like this:



You will be able to see the substantial amount of clipping that would occur when processing this file in Capture NX.

Next, "browse for" the file shot at +2.7 EV and go through the same process.  You should see only small portions of the image "clipped" if any.  General consensus is that the green channel on the D3 and D700 "clip" or reach 100% saturation at +3.0 EV.   As I explained in my earlier "histogram revisited" post, the green channel is inherently the most sensitive in "daylight" conditions, and so this is the channel we are concerned about the most.  As a general rule, the raw data in the green channel will clip in "daylight" color temperatures well before the red or blue channels.

So, we have in essence discovered our "clipping" or saturation point for the D3/D700 sensor, which is +3.0 EV above a "midtone" exposure according to the spot meter.  You can look at the other shots you took at +2.7 EV and + 3 EV to validate that the clipping point in the green channel is in fact at +3.0 EV (or very, very close to that value).

Please let me know if this makes sense to you, and if it does, we will continue on.  If not, please ask questions and I will try to explain better.

Keith
« Last Edit: July 12, 2009, 07:10:44 PM by keithsnell »

girod

  • Full Member
  • ***
  • Posts: 116
Re: Digital Photography and the Zone System
« Reply #10 on: July 12, 2009, 09:48:54 PM »
Done Keith and what I've seen are exactly what you have described except:
1) No blue clipping at +3EV

2) The "binding" did not show up after I toggled from "Histogram" back to "composite", the same "magenta" showed up instead

I'm all set to continue.

girod

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #11 on: July 12, 2009, 10:14:48 PM »
Done Keith and what I've seen are exactly what you have described except:
1) No blue clipping at +3EV

2) The "binding" did not show up after I toggled from "Histogram" back to "composite", the same "magenta" showed up instead

I'm all set to continue.

girod

Excellent!

The blue clipping will depend on the color balance of the light source.  I suspect my test shots were taken under "cooler" light than yours; however this won't change our conclusions significantly, so we can go on with our analysis.

The "banding" is the effect shown in this screen shot, which is caused by some of the channels in the "multi-channel" readout being "clipped" while adjacent channels are just below the clipping threshold. 



The view you are seeing in Rawnalyze is a "zoomed in" view.  Please try scrolling around the +3.0 EV image to see if you can find this type of banding at the edges of the image.  If you don't see any banding in the +3.0 EV image, please scroll around the +2.7 EV image in the "composite" view with "raw clipping" selected and see if you find any banding in that image.  You should be able to find evidence of this banding in one of your +3.0 EV or +2.7 exposures if you scroll around a bit.

It is getting late here, so I will break for now, and we can take a look at these exposures in Capture NX tomorrow to see how the "raw" data is handled by NX.  This will help us understand the "saturation" point in an image mapped to a gamma 2.2 space (adobeRGB) and how much "headroom" we have left to recover "blown" highlights in the image.

I'll check back with you again tomorrow and we can continue our discussion.

Keith

girod

  • Full Member
  • ***
  • Posts: 116
Re: Digital Photography and the Zone System
« Reply #12 on: July 12, 2009, 10:51:40 PM »
Thanks Keith. Goodnight and till tomorrow.

girod

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #13 on: July 13, 2009, 04:37:22 PM »
Hi Girod,

Did you get a chance to look at the +3.0 EV image again in Rawnalyze?  If so, did you scroll through the image and see any banding?  If not, you can "trust me" that the banding exists, and we can continue on with our discussion. 

I wanted to point out the banding, because it is one of the reasons for a "practical" limit of +3.0 EV on the D3/D700.  You will hear many opinions on the internet about the "headroom" available in a D3/D700 file.  The truth of the matter is that this "headroom" is practically limited at +3.0 EV, and will impact the way we expose our images if we want to optimize the image capture.  If you overexpose an area in your image by +3.0EV, and view the image at 100%, a discerning viewer will be able to see the banding in the image.  Highlights are not accurately or cleanly recoverable beyond this point. 

Most raw processing software has the ability to interpolate image data to recover highlights if only one color channel is blown; however, since the "saturation point" is slightly different for the readout channels of the D3/D700, this causes the interpolated image data to show banding as well. 

The whole point of this part of our testing was to help us understand the "upper limit" of the raw image data.  My testing indicates this "upper limit" is at +3.0 EV, and that I should control my exposures accordingly.

Am I making sense so far?

Keith

keithsnell

  • Administrator
  • Hero Member
  • *****
  • Posts: 1407
Re: Digital Photography and the Zone System
« Reply #14 on: July 13, 2009, 05:19:21 PM »
Girod,

Now that we've looked at the raw data in Rawnalyze, I'd also like you to look at the +3.0 EV, +2.7 EV and +2.5 EV images in Capture NX2.  You should see that all three images show varying degrees of "clipping."

It is important to understand that when most raw processing software "renders" or maps raw image data to a gamma 2.2 space such as sRGB or adobeRGB, they are set to render +2.5 EV at or near saturation in that space.  (My testing indicates that +2.5 EV is rendered at about 252/255, or 99% saturation for the D3 in CaptureNX2 (in sRGB space).  The "lost highlights" indication in CaptureNX2 indicates "clipping" at 254/255)  So what happens to the data that is between +2.5 EV and +3.0 EV?  That data is in what should be considered "headroom."  You will hear other definitions of "headroom" used on the internet.  The engineering definition of "headroom," and the one that makes the most sense in the context of "highlight recovery," is "the amount by which the signal capabilities of a system exceed a designated level, known as Permitted Maximum Level (PML). Headroom can be thought of as a safety zone allowing transient peaks to exceed the PML without exceeding the signal capabilities of a system."

It might help to consider "headroom" the same as "margin" between the point the manufacturer has defined to render as 100% saturation (just over +2.5 EV) and the point of actual physical saturation of the sensor (+3.0 EV).)  You can recover the image data captured in this "headroom" by using the "highlight protection" slider in NX.  The "highlight protection" function works by "remapping" all of the tonal values above the mid-tone so that more of the raw data can be "mapped" within the 128 - 255 tonal range available in a gamma 2.2 space.  This is often referred to as "highlight compression," since the difference between tonal values in the highlights is "compressed" due to the remapping.  This can be counter productive if your goal is to render distinct detail within a specific range in the highlights (for example the detail in white lace or a white wedding dress, feather detail on a white bird, etc.).  By "compressing" the highlights, you have in effect lowered the local contrast in the highlights and made this detail less distinct.  Do you see why "highlight compression" is not always desirable?

Another method of recovering the highlights is by setting negative exposure compensation with the exposure slider.  This in effect remaps all of the raw data to "lower" respective tonal values in the rendered image, making the entire image appear darker (but displaying the highlights that were initially above 254 in the rendered image).  You can use negative exposure compensation and then compress the shadows to bring them back into the image by using the "shadow protection" slider in NX2.  Of course this has the same negative effect of reducing the local contrast in the areas between the black point and mid-tone, which also results in a "flatter" looking image.  Just like with shooting transparency film, you should sometimes make a conscious decision to "let shadows fall where they may."  It often provides a more pleasing image if we let shadows remain dark (with portraits being an exception).

The point of this discussion was to help you understand the difference between "clipping" in the raw data, and "clipping" in the rendered image, and to understand some of the methods for optimizing the rendered image.  

I'll come back to this point many times, but it is worth stating here that the reason for using a modified Zone System to control our exposures is so that we can make a conscious decision of where we want the tonal values in our image to be displayed, and then optimize our exposure and image rendering (development) accordingly.

Am I making sense still?

Keith
« Last Edit: July 19, 2009, 09:07:57 AM by keithsnell »