Nikon D800 processing images

[memoriesbyadams]

Hi all, I run with mostly D800 kit and its highest resolution. RAW onto CF and backup jpg onto SD cards. I struggle at times with the camera needing to take a timeout to process the big files, usually get about 10 - 15 shots close together before it goes into denial. Means I have to carry to cameras especially with say a large wedding procession as by the time the bride comes along the cameras getting close to break time. Anyone else experience this and how do you handle it if so? p.s. all my cards a Sandisk Ultra Pro's.









Every day opens a new adventure - www.memoriesbyadams.com

[ameerat42]

Hi Mems...
Isn't it a bit of a peeve that that camera has a 4 fps continuous shooting mode? - Possibly
uncalled for, considering its maximum image size.

It sounds to me like the cards are up to it, so I'd say that at 36 MPx shots, saved at
highest quality, you're just running in to the normal limits of the camera.

I do not know, but if you shot at slightly lower resolution, say 24 MPx for some situations,
would the resulting processing speed pick up?

Welcome to AP, BTW.

[ricktas]

Once you are using the fastest cards, then you just have to deal with it. Your camera has an amount of RAM and once that is full, it can only offload the images to the memory card as fast as it can. Reaching the point where the camera starts to slow down is nothing new, and its not just a D800 issue, but happens with every camera.

I would suggest not using the 'spray and pray' method of taking photos and rely more on considered shooting. ie Take the photos you need rather than 20 photos hoping to get a good one out of them.

And every decent wedding photographer uses 2 camera bodies (at least).

[arthurking83]

....

I would suggest not using the 'spray and pray' method of taking photos and rely more on considered shooting. .....

best advice.

second best advice could be to try a fast Lexar CF card of some type. 1000x at a minimum.

note: I have a personal gripe against Sandisk cards as they always seem to let me down .. and usually at the most inopportune times!

According to Rob Galbraith tho, Lexars fast CF cards seem to work better inside a D800 series, then Sandisk cards do.


I get about 15 shots in a burst on my D800E and I'm only using a fast-ish(ie. not the fastest!) 800x Lexar CF card.
(note that I don't use jpg mode at all, only raw mode).

Recovery time isn't too bad. My camera doesn't lock up .. as such .. but does need about 10sec or so to recover to a point where you could shoot another 2 sec burst at 4fps.

Keep an eye on the 'r' number when you half press the shutter. Camera with an empty buffer will show an r12 value both through the viewfinder and also on the top LCD when you half press.
This value is the buffer's remaining image allowance.
So at the start, it will show you what it thinks the buffer can hold in a burst. If you shoot at the max 4fps mode, this number will reduce in real time to show you how many shots you have left before it slows right down to about 1fps or so.
So about half way through to the approx 15-17 max continuous shots in one long burst, the R value will drop, rise again by one integer again for a moment, keep dropping and then rise again. That increase in number(eg. r06 -> r07) just means that the buffer cleared an image and there's room for one more shot.

So after about 10sec or so, you should have about an r08 or so allowance(as per above).

there could also be the possibility that your Sandisk card is fake.
or that's it's less compatible with the D800, than say a Lexar seems to be.

You could try a faster card, and make sure that by faster it's not just the read speeds that are fast.
Reads speeds on cards ratings are generally not that useful, it's the write speed that more important for most uses.

It looks as tho the D800 series seems to top out at about a 70-ish Mb/s write speed, so if you get a card capable of being written too at say 80-100Mb/s, you've probably maxed out what little performance that the D800 can muster.
Any more speed above that point will only be wasted, unless you have very little time when transferring to a computer too.

But .. like Rick said, learn to be more selective about WHEN you use the spray and pray technique. It should be remembered that the D800 was never designed to be a speed camera. That's why Nikon had the D4 camera model! ;)

[memoriesbyadams]

haha Spray and Pray, nice slur on my photography style there, not sure you actually know me at all then really. As I mentioned in my post, specific instances where you get say a long bridal party of 10 of more girls coming in, close together, you shoot them land and port, close and wide and the grooms first look and crowd reaction and then have buffer space for the bride all on a camera that can only take ten or so shots close together. other have been more helpful in suggesting lowering the resolution rather than commenting on something they assume re my style. Cheers. and P.s. very few DECENT wedding photographers carry two bodies at the same time, having a spare handy yes but not slung, thats so old school hence why im trying to avoid it.



Once you are using the fastest cards, then you just have to deal with it. Your camera has an amount of RAM and once that is full, it can only offload the images to the memory card as fast as it can. Reaching the point where the camera starts to slow down is nothing new, and its not just a D800 issue, but happens with every camera.

I would suggest not using the 'spray and pray' method of taking photos and rely more on considered shooting. ie Take the photos you need rather than 20 photos hoping to get a good one out of them.

And every decent wedding photographer uses 2 camera bodies (at least).


HAHAHA,

- - - Updated - - -

Awesome idea, why didnt I think of that. :) cheers


Hi Mems...
Isn't it a bit of a peeve that that camera has a 4 fps continuous shooting mode? - Possibly
uncalled for, considering its maximum image size.

It sounds to me like the cards are up to it, so I'd say that at 36 MPx shots, saved at
highest quality, you're just running in to the normal limits of the camera.

I do not know, but if you shot at slightly lower resolution, say 24 MPx for some situations,
would the resulting processing speed pick up?

Welcome to AP, BTW.

- - - Updated - - -

Keep an eye on the 'r' number when you half press the shutter. Camera with an empty buffer will show an r12 value both through the viewfinder and also on the top LCD when you half press.

Sounds advice there and you know what you can do with your "spray and Pray" comment.



best advice.

second best advice could be to try a fast Lexar CF card of some type. 1000x at a minimum.

note: I have a personal gripe against Sandisk cards as they always seem to let me down .. and usually at the most inopportune times!

According to Rob Galbraith tho, Lexars fast CF cards seem to work better inside a D800 series, then Sandisk cards do.


I get about 15 shots in a burst on my D800E and I'm only using a fast-ish(ie. not the fastest!) 800x Lexar CF card.
(note that I don't use jpg mode at all, only raw mode).

Recovery time isn't too bad. My camera doesn't lock up .. as such .. but does need about 10sec or so to recover to a point where you could shoot another 2 sec burst at 4fps.

Keep an eye on the 'r' number when you half press the shutter. Camera with an empty buffer will show an r12 value both through the viewfinder and also on the top LCD when you half press.
This value is the buffer's remaining image allowance.
So at the start, it will show you what it thinks the buffer can hold in a burst. If you shoot at the max 4fps mode, this number will reduce in real time to show you how many shots you have left before it slows right down to about 1fps or so.
So about half way through to the approx 15-17 max continuous shots in one long burst, the R value will drop, rise again by one integer again for a moment, keep dropping and then rise again. That increase in number(eg. r06 -> r07) just means that the buffer cleared an image and there's room for one more shot.

So after about 10sec or so, you should have about an r08 or so allowance(as per above).

there could also be the possibility that your Sandisk card is fake.
or that's it's less compatible with the D800, than say a Lexar seems to be.

You could try a faster card, and make sure that by faster it's not just the read speeds that are fast.
Reads speeds on cards ratings are generally not that useful, it's the write speed that more important for most uses.

It looks as tho the D800 series seems to top out at about a 70-ish Mb/s write speed, so if you get a card capable of being written too at say 80-100Mb/s, you've probably maxed out what little performance that the D800 can muster.
Any more speed above that point will only be wasted, unless you have very little time when transferring to a computer too.

But .. like Rick said, learn to be more selective about WHEN you use the spray and pray technique. It should be remembered that the D800 was never designed to be a speed camera. That's why Nikon had the D4 camera model! ;)

[ricktas]

Hi,


haha Spray and Pray, nice slur on my photography style there, not sure you actually know me at all then really.

Umm, why would I know you at all? You had one post on the site, and I replied based on the information you provided in your post. I have never heard of you, do you assume I should know who you are, are you someone important? So it was not a slur on your photography style, but rather a reply based on the information you gave us in your post, nothing more.

[memoriesbyadams]

Sure it was, it implied I just take lots of photos and hope that one turns out. Poor form really. Then again that is why most wedding photographers are hated by their clients, pushy, rude and no people skills.


Hi,



Umm, why would I know you at all? You had one post on the site, and I replied based on the information you provided in your post. I have never heard of you, do you assume I should know who you are, are you someone important? So it was not a slur on your photography 'sty'e, but rather a reply based on the information you gave us in your post, nothing more.

- - - Updated - - -

Oh and that wasnt a slur at all. :P


Sure it was, it implied I just take lots of photos and hope that one turns out. Poor form really. Then again that is why most wedding photographers are hated by their clients, pushy, rude and no people skills.

[I @ M]

After a careful read of your requirements for the style of photography you are doing the solution is pretty clear to me.

If I were in the same boat I would stop using yesterday's technology ( D800 ) and move to the current era and buy one or two D5s to ensure a massive buffer for when you are faced with multiple shots in quick succession.

Armed with one body whilst your assistant carries the second ( with alternate FOV lens ) should ensure success under the most stressful situations. :nod:

[Lance B]

You haven't advised what your settings are for your RAW files. Are you shooting full 14 bit uncompressed or 14 bit lossless compressed or what? Makes a huge difference to file saving speed.

[Warbler]

"Spray and Pray" is a bit of a pejorative term. I'm not surprised the OP took it that way. I did wonder a bit about the question though. Buffer is not something that most of us don't know about. Not trying to stir things up. Just sayin.....

[swifty]

Lance brought up a good point. I wasn't sure of the buffer depth on the D800 series but found this table with a quick search:
https://photographylife.com/nikon-d810-buffer-size
From your description it looks like you might be using uncompressed RAW in either 12 or 14 bit.
Going to lossless compressed nets you a few extra frames that might be the difference between lockup and a critical shot. I don't see a down side to shooting in lossless compressed.
Since you're using a D800 series setup, perhaps consider the D810 as a new main body and your D800 becomes your backup.
It looks like Nikon doubled the buffer on the D810.

[arthurking83]

Yeah D800 and D810 have different buffers, and almost certainly different write speeds to actual camera(which would mean a different chip used for that job).


....
Going to lossless compressed nets you a few extra frames that might be the difference between lockup and a critical shot. I don't see a down side to shooting in lossless compressed.
....

I reckon there's about 1/3ev in the highlights if you push process hard enough.
if the photographer was more careful about protecting highlights, and lost exposure in the shadows .. I think the comment that there'd be no downside to shooting lossless/compressed raw would be accurate.
Being a wedding photographer tho, the implication would be bride .. white dress, possible blown highlights at some critical moment.
There is the possibility that lossless compressed could lose something in those whites.

[swifty]

Oops. Apologies. I was under the impression Nikon lossless compression was really lossless but Arthur has pointed out otherwise.

[arthurking83]

Note that there is a summary(below) if you can't be bothered to read the why's and whatnots below:


Oops. Apologies. I was under the impression Nikon lossless compression was really lossless but Arthur has pointed out otherwise.

I'll double check again if I can one day.
I may have confused the last time I checked for any of this with some testing I did with the D300 tho!
(like I said I'd have to re-double check to be sure).
I do remember Mongo had a Q about compression a few years back, and I checked some stuff for him. That was definitely D800, but I can't actually recall if I checked for anomalies in exposure limitations too for that testing.
Massive file size differences tho.
I do remember that you could get up to a 50% file size saving(80-ish Meg to 40-ish Meg) .. which should allow for faster write times of each individual file to the card .. ie. seemingly larger buffer by way of comparison.


ps. while I was writing this reply, I went back to eyeball some of the images I worked on when checking stuff out for Mongo(BTW, that was about mid 2015)
One of the scenes I shot, was a backlit curtain with very small amount of blown highlights.
I can definitely see just a slightly better recovery of the highlight area on the 'no compression' image vs the 'lossless compression' image. All images 14bit too, never tried 12bit.
There is a humanly noticeable difference in the ability to recover highlight detail from lossy compressed images too compared to none or lossy compressed images.

ps. recovery attempt was a measly Nikon ViewNX2 centric -2Ev compensation too. Not some uber great, perpetuatingly costly software to work with.

AND, in saying all that, I'd even be tempted to work with the lossy compression version of file size saving. The loss of any highlight data is probably going to be insignificant, if it's a problem at all, or even if the operator has the ability to reproduce exact same exposure levels on every shot anyhow.
So the actual exposure made in any given situation will probably make more of a difference to any potential loss of fidelity, rather than the compression level used in camera.


SUMMARY:
you have two possibilities:
1. you need to shoot faster and more, for longer, and you're pretty good with exposure and not anally retentive about the last nth degree of exposure latitude.
SHOOT: compressed, in either mode, lossy or not.
File sizes by way of a vague approximation: lossy ~ 35Mb, lossless ~40Mb, no compression ~78Mb.

2. you can work around some of the speed limitations of the shooting device, but you just can't allow yourself to give up on even an nth root of an nth degree of quality in your images .. don't use compression.

[Steve Axford]

A curious answer, Arthur. All the opinion I have read on the web, except yours, would indicate that lossless compression is just that - lossless. I remember writing compression programs many years back and it is very easy to tell the difference between lossless and lossy. It would surprise me if a manufacturer like Nikon could lie about something like that.
That leaves a question. Why would Nikon include an uncompressed format if it was exactly the same as the compressed. The web answer seems to be that the compression takes a lot of processing. Hence if you want speed, use uncompressed. This is opposite to what you suggest. Perhaps someone with a D800 could test it with a fast card? Since memoriesbyadam has several and he is interested in the answer, perhaps he could try it?

[swifty]

Having done a quick google, it appears that the general consensus is indeed lossless means it is completely reversible but a few have also suggested it means 'visually lossless'.
Whereas there are some demonstrations to show subtle differences between 12 and 14bit I haven't found one for uncompressed vs lossless compression.
It's definitely quicker to compress compared to the writing to card speed of the difference in file size hence lossless compression is the way to go to extend your buffer.
In other words, the bottleneck is the write to card from buffer step so even the extra process and time to compress the file helps clear the buffer quicker by significantly reducing the file size.
It has also been suggested that in post processing, it is faster to render using uncompressed vs lossless compression because there isn't the need to uncompress the file before reading it.

[Steve Axford]

Which leaves the question. Why have lossless compressed and uncompressed? In fact why have 3 choices when lossless compression should clearly be the best? This makes no sense to me.

[Lance B]

I shoot 14 bit lossless compressed. I used to shoot uncompressed but having done careful checks, I can find no difference between 14 bit lossless compressed and 14 bit uncompressed.

[swifty]

If the googled info is correct,
Uncompressed for faster post processing as data is as is. No need to 'unpack' the files so to speak when reading the files.
Lossless compressed to reduce file size so they can be written to card faster from the buffer and to save more files per card whilst maintaining max quality.
Lossy compression to further reduce file size. I guess lossy compression exists for the same reason low quality jpegs exists.

[arthurking83]

All I did was to shoot an image as per a normal (0 Ev) exposure according to the cameras meter.

On in 14 bit uncompressed, and the other 14 bit lossless compressed(also compressed too just to compare)
The images are generally well exposed, other than one small area where they have completely blown out. images are of a backlit curtain with a small area of direct window.

To attempt to regain some of that lost detail, in ViewNX2 I used -2Ev compensation.

I can see slightly better recovered detail in the uncompressed images than I do in the lossless compressed images.

I did have 2x samples of each image too .. I actually have more I think(3 or 4??)

ps. just because I'm seeing some slight and very subtle differences, in no way means that Nikon's not truthful about their lossless compression not being lossless!
All I see is some very slightly better recovery in the uncompressed version of those images in the highlights.

@ Steve: I'll bet my last cent that 99.9999% of those tests and reviews you've seen re Nikon hardware have all used any other software other than Nikon's own to review their images.
Software can account for a large amount of difference when viewing and processing images.
While it's old hat and old school, and very limited in it's ability, I think VNX2 is fine for basic edits on raw images. It will only allow up to +- 2Ev compensation(where many other's seem to be unlimited nowadays!)

Therefore I use + or - 2Ev compensation as 'my guide'.

Back when I captured those images, I didn't even think to study the effects of extreme processing(2 Ev is extreme for me) and all I concentrated on was file sizes.

Finally: the difference I see between the images is barely noticeable too!
The only way you can see it is by comparing two images directly, and I couldn't even estimate the differences in terms of Ev .. well under 1/3Ev.
But the histograms show slightly better recovery in the blown highlights(less over exposure) on the uncompressed image than it does in the two compressed images.

In terms of speed(again using VNX2) I see zero time difference in the way it opens, edits or saves the files.

[Steve Axford]

I had a look at what I could find on compression algorithms and it seems likely that all compression potentially changes the image slightly at the extremes. I imagine that a totally black (or white) field with only noise, would ideally be read as a totally black field plus mathematical noise. Then, when decompressed, the totally black field would have noise added, which would be mathematically correct, but in fact different. I understand that Fourier transforms and inverse transform are used (I dropped out of maths at about that point) and that could imply a level of uncertainty. The choice of where to put the line which says two things are the same is open to some discussion. (It's 1.99999999999, lets call it 2)

[swifty]

I think we may never know since Nikon doesn't document this aspect.
Not doubting your results, AK but are the two pictures identical to begin with?
The reason I ask is that it is actually generally very difficult to shoot two identical images. Even such things as aperture not stopping down to exactly your specified aperture affects things or small fluctuations in light, especially natural light.
But the fact you had a few sets and your results can be repeatable with each set would lend credibility.
Either way, I think the take home might be the differences, if present are too small to worry about and at least satisfy a representation of 'visually lossless'.
As to post processing speed, if anyone is interested you would probably need to run a batch process of 100 or something in larger quantities before possibly seeing differences. Which again, even if there are differences it would be too small to care about for most of us.

[arthurking83]

.... Which again, even if there are differences it would be too small to care about for most of us.

This is the key point.

The images aren't 100% identical, but close at 99.999%.
Even tho the exposures were identical, the actual images were slightly different to begin with, and the weird part is that the uncompressed version was actually slightly brighter(more blown out) than the lossless image.
Now when I say slightly brighter, they appear to be identical(to my eye), but the histogram shows slightly brighter tonal numbers overall, then the lossless version.

The reason I made the initial comment to begin with is that even tho the uncompressed image is the brighter of the two(with blown highlights), in the -2Ev recovery it was the image that (only just) displayed slightly better highlight detail :confused013
Common sense would dictate that the darker of the two comparative images would recover better.

Problem is when converted to jpg, the differences aren't there.. it's only in the raw file, viewed as a raw file in VNX2.
(ie. VNX2 can be set to view either the (faster to load) jpg preview file, or the (slower to load) actual raw file data .. which it has to decode to view(obviously).
ps. I haven't tried tiff file conversion to see if the very slight differences are still obvious.

But, like I said earlier too .. you can't really see any differences when casually flicking between the two different images, it's only the subtle change in histogram that diverts your eye, and then you go 'looking harder'.
Then I hover the mouse over an area that is not blown out in the uncompressed image, and the RGB values are listed for those pixel areas ... say: 248, 244, 250, just as a random example for the uncompressed image. And the lossless image at those same pixel coordinates read RGB values of 255, 255, 255 by way of comparison.
If I use the H key to display blown highlights, there are less on the uncompressed images than there are on the lossless images.

As per your comment above .. I think it applies for almost all of us in the real world .. that is a variation of (say) 1/12 Ev in an image isn't going to make or break the image.


In a few days, I should have the time to check the issue out with a more controlled test. (It'll give me something to do if I can't find anything else).

[Steve Axford]

Arthur, you seem to assume that VMX2 is the best software to use, yet you also assume that almost no reviewer would use it. If almost nobody uses it, why base test results on it.

[arthurking83]

1/. Arthur, you seem to assume that VMX2 is the best software to use,

2/. yet you also assume that almost no reviewer would use it.

3/. If almost nobody uses it, why base test results on it.

1/. never said that. But why use any software? why would you prefer to use what you do? Am I not allowed my own preference.
(btw: there are technical reasons why I use it too, but unrelated to the above issue)

2/. I've never read any review based on any Nikon camera, where the reviewer has mentioned the use of VNX2.
Many folks out there in Nikon land despise VNX2 for it's limited ability(agreed, it is limited in it's tools), and instability(never seen that in many years of using it)
My assumption is based on the above reasons.

3/. Why not? I guess it begs the question, why use any software to base tests of any raw conversion? Is one software particularly greater than any other? if so, is that not a 'preferential' based choice anyhow?(ie. refer back to 1/.)

[Steve Axford]

I agree. You can use any software you like for your processing. But, for a test like this it would make sense to use software that people actually use. You claim to find a difference in the results, yet nobody else does. That alone should start the alarm bells ringing. Perhaps you have discovered a major fault in the Nikon lossless compression. One that nobody else has found and one that would create quite a stir as it implies that Nikon have been less than honest in their claims. But perhaps you have made a mistake. That option would be worth checking before you suggest that there is a real difference between lossless and uncompressed. I know that there is a difference between Sony compression and uncompressed. But they don't claim to use lossless compression. The difference is quite small, but enough to cause some major criticism and force Sony to include uncompressed. If DPreview were to get a sniff that Nikon lossless compression wasn't really lossless, they would be crowing so loudly that you would be deafened.