Question on Post Production Organisation

[superbee15]

This may seem an unusual question, but I have just this week invested in Adobe Products (Lightroom Photoshop with Bridge)

I currently have an Album organised on PC which I am happy with and this is just for our family use. I use PC Screen Saver to show photos while PC is idle. The album is all jpeg shots from previous point and shoot and for the last 4 years a Canon DSLR. All shots from the Canon are in jpeg with just a few recent months that I still have some RAW files for as well. I tend to use RAW only in recent time and then output as a jpeg from PP.

Since having the Canon I have done some post processing on some shots in Canon DPP. If just a jpeg file I may change in DPP save results and this is the file for display (i.e. overwrite the original file. When I started using RAW files in last 12 months I would change in DPP save as jpeg and then delete the RAW (except for a few days in more recent times where I still for some reason not deleted the RAW). I did this so to improve space on hard drives.

Now I have Lightroom which does not change the original shot (not sure what PS and Bridge do to original) it's made me question the structure I use for organisation of shots. I still want a jpeg file for display (RAW file does not display to my knowledge) but I am thinking I maybe should consider keeping the RAW file. However over time this will significantly increase the size of the album. If I keep the RAW files should they be in same folder or maybe in a different directory and just output the resulting jpeg to the normal album structure which means the file will appear in the screen saver displays. Or is keeping the RAW file just a waste.

I cannot do some post production on a jpeg in lightroom without saving a jpeg as a different file name for it to display correctly in screen saver. So therefore I would need to delete original?

What are others doing that may be of help?
When using RAW do you still keep when finished work on it?
Or am I over thinking this?

[ameerat42]

Last Q 1st: Yes, you are, BUT what's w:eek:rse!!! There is no reason AT ALL that can justify deleting a raw file.
Stop that first!

(May sound "blunt", but gets the "point" over.)

NEXT: Buy yourself a couple of 1TB external hard drives. Use these to store what you think you HAVEN'T got space for now.
Now why a "couple" of drives of 1TB? So that you don't have all your files in one basket-drive.

I am talking about the Western Digital Passport for physical size only, but the equivalent TOSHIBA CANVIO is just as good
and will cost you $79 at Officeworks. (The WD is $105, from memory.) Add do not get the version with "added software"
that costs more, because it is useless garbage you can get free elsewhere if you even think you need it.)

Now as to workflow, here's what I recommend. (I was going to say "suggest", but I now won't.)
1. Develop your raw file and generate various size jpegs (etc) for other uses. As you have said, a different filename for each version.
2. SAVE your original raw file somewhere for any future use.
3. That's it.

Enjoy your external HDDs. (These, BTW, are USB3, so if you have those ports on your computer you can enjoy the 10X speed increase.)
:D

[superbee15]

Thanks for comment ameerat. blunt and got the point lol. I had really decided to do that pending comments here. Also had decided to get a bigger drive to cover music, so now that's 2 reasons to act.

I also remember getting cranky at SWMBO about 20 year ago for cleaning out the photo stack and getting rid of all the negatives - it seems I really did the same!:o
Maybe I'll just store RAW and output files together - at least I will know where to find them.

"Now as to workflow (http://www.ausphotography.net.au/forum/showlibrary.php?title=Indexes:Workflow_and_DAM_Software), here's what I recommend. (I was going to say "suggest", but I now won't.)" - please do, I can always use another point - I also do not know what the term "Workflow" actually covers.:)

[Warbler]

If you're deleting RAW files because of space worries, I'm betting you don't back up either. You'll be sorry if that's the case.

[arthurking83]

If you're deleting RAW files because of space worries, I'm betting you don't back up either. You'll be sorry if that's the case.

Yep!
Many folks on here have had HDD failures, and without that backup all images collected would have been lost forever!
You don't kill yerself for years to collect a Terabyte or two of images just to have them all lost due to a silly piece of hardware.
Drives are cheap, get one spare for the PC and an external one too .. and have two copies of your images at all times.

(workflow simply means: from the point of you starting to do something to the point of when you are finished doing something to reach an endpoint)

As for workflow:
As Am indicated, it's probably best not to delete the raw files.
Keep them and be ruthless in culling images you just don't need to keep(to save space). And the good images that you do want to keep .. on two different HDDs.

I do the same as you do for displaying jpgs as screen savers, although I don't do screen savers, I use them for desktop backgrounds only.

So my 'workflow' would be something like this:
If I wasn't working yesterday, I'd be out shooting some landscapes in a small town called Maldon(In Victoria).
Camera's card is placed into the card reader, and I open Nikon's Transfer to get images off card, onto a HDD.
Raw files all get stored in specific folders on a dedicated HDD(or three) where the folders are called something relevant, I use the Transfer software to do this.
eg. (I should have been out yesterday 21-05-16.. so) the folder I place those images in will be called 20160521_Maldon. Date first, other vital info next for making it easy to navigate the million and one folders on the HDD).

Note too: if you habitually change the camera you shoot with, it's probably best to also use a dedicated camera directory structure too. I do and also use a dedicated year structure too.
So the above images that should have been shot in Maldon on the 21st May 2016, would have been stored on a dedicated HDD with a folder structure that looks like this.
(letter of drive):\Photos\2016\D800E\20160521_Maldon\
All raw files go directly into that folder and I also make a jpg subfolder to store any saved images. I prefer to keep jpgs separate from raw files.
Reason: it's easy to manage the cleaning up process at the end of every year.
I only ever keep raw files and delete all irrelevant jpgs and or tiffs.
Problem is that I have some images that at tiff of jpg only(that is scanned, or saved from another source or whatever) I don't want to delete those jpgs and tiffs .. just the ones that are copies of the raw files I've archived ... the jpg subfolder makes it easy to find and delete .. even if I were to do it manually, but there are many good programs that help to do this kind of housecleaning.

Once all the processes have been made to the raw file, I save it as a jpg ... save it into the jpg folder and then from there I can upload it to the net(eg. to here on AP) AND I also transfer another copy from that jpg folder into a dedicated Pictures folder for any of my PC screen images.
I don't delete the saved jpgs in the jpgs folder(in the original folder from the shoot) until the end of the year.
Reason is(and I did used to do that) I usually go back to that series of image at a later date. It's good to go back to it with a fresh set of eyes, or lets say with a new processing technique in mind(or new updated software of some type).

[superbee15]

If you're deleting RAW files because of space worries, I'm betting you don't back up either. You'll be sorry if that's the case.

No backup I have always done in fact I have 2 backups.

- - - Updated - - -

Thanks arthur for detailed look at your process as I was keen to understand some different approaches.
I started going back over my album about 3 weeks ago and renaming all directories with date (like your convention) and name. My collection is probably nowhere near the size of your collections so the time to do this is no all that long.

I also plan to go back over some shots and maybe treat lightly with Adobe software.

I will still use (at this point) Canon to do transfer from camera and put in in normal default which is a date directory for the 1st cut and then move to my structure for more detailed process. I just need to decide on RAW file storage process.

I would also assume the you would also import into Lightroom (or program like it) those shots that you want to work on as against all shots?

Also does Photoshop and Bridge change the original or act the same way as Lightroom?

[arthurking83]

I don't use Lr. I have a copy but haven't installed it again(and it's v4 anyhow, so it's old-ish).
But I never understood this strange concept of 'importing' images as a step in the workflow :confused013

I can't think of any other program that does this(other than Lr) where you can't simply navigate to the folder where you transferred the images too using a folder exploring tree structure somewhere within the program.

Another thing to look into developing, is a system of tagging that makes sense to you.
I'm sure DPP would have something like that too.

Nikon's software has a system where you simply press a keyboard key(or combo of keys) and it tags the image under view with that relevant rating.

There does appear to be a 'standardised' system in place for ratings tags using a value of 1-5 for a star rating and a value of 0-9 for colour tag ratings.
I tag my raw images in Nikon's software and it seems to translate well into other raw file viewing/editing software too. Some slight differences but tagged all the same! :th3:

Why tag your images? .. if you're unsure if you think you should keep the image or not, then tags make it easy to sort the wheat from the chaff .. at a later date!

eg. say you tagged an image with a 5 star rating(where 5 star is very high image rating) and you tagged other images with 2 or 3, or no star ratings. Almost all software allows you to filter the images you can view using those ratings tags.
So at the end of the year, it's time to clean up your images folder to 'save space' and seriously archive the images you definitely want to keep!
This is basically what I do at the start of every new year.
I'll go back through the years collection of images sort the view using a filter of displaying only the lowest quality images.
That is, I don't want to see the images I want to keep, I want to view only the images I don't want to keep!
Press Shift-A on the keyboard to select all images and send them to the recycle bin, or delete them all in one go.

After all those files deleted, what remains are the images you do want to keep ... because they weren't included in the filtered set that was displayed.

While the ratings tool is a seemingly innocuous step in the workflow, I reckon it's probably one of the most important tools in the long run .. unless you simply don't ever need to sort through images at all.

But!! definitely keep and filter/sort your raw images. Tag them in any way you can think of at the time, whether that means descriptive tags(ie. bird, portrait, son, Sydney Harbour Bridge, etc) or by rating or whatever, which will become valuable in later years.

[ameerat42]

...

I can't think of any other program that does this(other than Lr) where you can't simply navigate to the folder where you transferred the images too using a folder exploring tree structure somewhere within the program.

Another thing to look into developing, is a system of tagging that makes sense to you...
I'm sure DPP would have something like that too...

Now that you mention "other programs", I remember that I had Canon's ZoomBrowser years ago. I found it confusing, and
I could not see my images in Win Explorer because they had been "imported" somewhere. Soon ditched it and used then
ACDSee, but that got old and now I use the free and very adept FastStone Viewer. Of course, you can simply use Win Explorer
to transfer images from camera/card to computer. It's just that the ACDSee and FStone progs let you see them a lot more easily
and also info about them. You can decide which to cull even at "import" time.

Both these progs will "read" almost any form of raw file (ie, the embedded jpeg therein) and display an image for you. They also do
basic editing (which at a pinch I have used for jpegs only, though claims abound as to their ability to "process raw files":confused013).
They also do tagging, etc, but I don't use that feature.

Here's the FastStone site, and as a caution, DON'T use other than the Viewer program, as they are trials only.
http://www.faststone.org/

Oh, and it will not display hidden files/folders, so you have to un-hide them first.

[J.davis]

I use LR and my flow is, import to LR from SD card reader, go through the pics before actually importing and turf the dodgy ones, import the pics, once imported, go to a 1:1 preview and throw away the duds. Now I can edit my pics. Once edited, I put them in their folders and make Jpeg copies.

[Mark L]

No backup I have always done in fact I have 2 backups.

So you can now delete all photos not taken this year from the computer you work on to free up space. You don't need three copies.:)
DPP is pretty useless with JPEG, use PS. DPP will save the changes you make to a RAW file but only when you revisit via DPP. PS or LR will not notice the changes you make in DPP if you open the PPed RAW file there. You have to save the DPP changes as TIFF or JPEG for other programmes to recognise the changes.


So I get everything off the camera onto computer using Canon's ZoomBrowser (AM just doesn't now what it's useful for:p).) and it has a rating system AK mentioned.
Then delete heavily. :(
Then copy the RAW files I intend to work on into another Folder (mine's called 1s for PP). Then close ZoomBrowser, not much use know.
I know have a folder of RAW files to work on.

[Dug]

My Lightroom RAW files go in a main folder called "originals". This contains subfolders that I import my raw files to based on subject matter.
This is my way of narrowing down where to find things latter rather than use the other Lightroom filter and search methods.
I also like it this way because I can make sense of things also outside the Lightroom environment.

I believe it is best to have a main folder (even if it contains many subfolders) so that it is easier if you want to migrate your Lightroom library to another computer later.

Reasons to keep your RAW files
1/ It is the versions of the image that contains the most information for editing and the most adjustment latitude.
2/ You will always improve in your editing and you may want to do a complete rework of favorite images at a later date.
3/ Newer versions of RAW support come out for your editor with new camera releases, is means you may be able to get more from older RAW files than you could a few years ago when you first processed them.
4/ You might want to do a for print version of an image later, that was originally exported for jpeg web content. ie the new file version can be a .tiff this time, be non compressed, be AbobeRGB colour space instead of sRGB for the web.

Be critical of what RAW files to keep and you will save a lot of space, also be sure that you are picking "delete from drive" and not "remove" when you cull images.

I have a folder called "Jpegs for forum export" that I export images for that purpose to. This way I know they will all be of a curtain size and colour space and not mixed with exported images for other purposes.

Your RAW file folder is going to grow and grow in time, bigger megapixel cameras don't help with this either so it will be a good idea to have it on a drive with space to spare for it.

Just come up with something that makes sense to you.
It is a good thing you are thinking about this at an early stage.

[Mark L]

Be critical of what RAW files to keep and you will save a lot of space, ...
Just thought I'd quote this as I really like it.
It is worth waiting a day or two to reconsider you first thoughts and then delete anyway.:)

[arthurking83]

..... Of course, you can simply use Win Explorer
to transfer images from camera/card to computer. It's just that the ACDSee and FStone progs let you see them a lot more easily
and also info about them. .....

Kind'a!
not 100% accurate tho Am.
What you need to do is install the camera maker's raw codec.
M$ also has a camera codec pack, but it installs a host of other raw codecs that are both useless and bloating .. camera makers codec software is usually better(at least in terms of Nikon raw file types)

What happens if you do this:
Once the codec is installed on a Win machine, it then allows you many more options to view almost all camera metadata, and usually allows editing of such metadata in various ways.
Part of my 'processing' of images(raw images of course) is that I use M$s Photo Gallery to sort through a myriad of images for various metadata info simply to edit it, and add it where it hasn't been added.
Note that you don't need Photo Gallery to do this, you can also add/edit most metadata directly through Windows Explorer too(if you're a masochistic type!).
If you want to view specific metadata in Explorer whilst viewing the image in the preview pane too, simply add the relevant columns to the Explorer pane.

I have a thread on that somewhere .. I may edit this reply with a link if I can find that thread.


..... DPP will save the changes you make to a RAW file but only when you revisit via DPP. PS or LR will not notice the changes you make in DPP if you open the PPed RAW file there. You have to save the DPP changes as TIFF or JPEG for other programmes to recognise the changes.
.....

Sorry Mark, but as with Am .. not 100% accurate.
What Canon's software does with an edited raw file is to update the embedded jpg file(as well as add/edit any metadata with the relevant software).
Basically the same as Nikon's old software did/does.
Most programs that use an embedded jpg preview file will see the changes made to the raw file in DPP. (eg. as per above Windows Explorer, FSViewer and so forth)

Where Adobe fails in the way it's software operates is that it doesn't allow any option to use that embedded preview file(as the other programs do).
It's fair enough to want a(eg. Adobe) program to render the image in a specific manner(eg. Adobe again!), but the problem is that it's not Adobe's image .. it's YOUR image.
You really should be allowed any and every option with that image.
A simple free program like FSViewer allows that .. why not a program that you've paid for(and still probably paying for!).

I haven't reinstalled LR basically for this reason amongst a couple of other reasons.

Anyhow .. something that you may or may not notice if you use Lr.
When you originally open Lr and then only import raw files that have already been transferred to the storage device .. rather than use LR to transfer the images to the computer! it initially generates the thumbnails in a grid for you where you can select/deselect them for importing.
This is how I used to do it.
When I used Lr(or tried too) .. I'd transfer them all with Nikon's software which renamed them and added some metadata during that transfer.
I'd then sort them with Nikon's very basic software and maybe tweak WB or whatever and add ratings or whatever ...
Long story short.
On the obviously edited images(eg. WB) where lets say a very blue incorrectly whitebalanced raw file was corrected with an appropriate WB setting on that grid of thumbnails the thumbnail itself was rendered with that from the embedded thumbnail of the Nikon software edited version .. not the true raw thumbnail!

You have to have a basic understanding of what a raw file actually contains and to keep it quick, it basically contains about 4 embedded jpg files.
Some you just can't get too, but there are preview thumbnails as well as the full res preview file.
What LR appears to do is on that initial thumbnail page it's rendering the embedded thumbnail image of the raw file.
Once you import the image tho, it then does it's own thing! From that point onwards Lr then takes control of the image it will render on screen.

What's the point of this info? Knowing that Adobe software doesn't directly edit the raw file data, and it uses a system of external files that pertain to the raw file .. it's important to note that the thumbnail that it initially rendered in the grid is still there.

So in my workflow example earlier, where I edited the WB setting of a raw file in Nikon's software .. FSViewer, VNX2, Windows Explorer and many other software will show that file as I edited it in Nikon's software. Open Lr tho and it now renders that same raw file in it's own way.

So the part of your quote that I underlined is the other way around:
If you use LR to edit your images, they only display that way if they are viewed only using Adobe software.
If you edit the raw file in DPP .. many other software can see that edited rendering. Not all, as many work the same way as Adobe software does.

[ameerat42]

AK. What's not accurate, the info, or viewing the pics and metadata?
Note that there's a full stop after the Win Explorer statement and that sentence is only about
transferring files (which might be visible as details or icons but not thumbnails).

Faststone must have some codecs already, as it can at least display the embedded jpegs without
having to install more codecs. FastStone can display EXIF and histogram from the Σ raw files just
as installed.

I wonder if Superbee is confused with all the replies by now:D

[arthurking83]

AK. What's not accurate, the info, or viewing the pics and metadata?
....

Yep!

Install the codec from the manufacturer and all is revealed.
Thumbnails, metadata, viewing .... basically everything!
In Windows Explorer.
So when you connect the camera's card, the thumbnails on the card will be of the (raw)images .. not just the annoying icons of software linked to that file type.

eg. I have a couple of .X3F files and they're associated with a program which displays a flower as it's icon.

Note that once the relevant codec is installed, then you can edit the metadata in the raw file too.



.... I wonder if Superbee is confused with all the replies by now:D

If they aren't then I'd be worried(if I were them) .. otherwise they're probably as mad as we are :p

Here's the link to my cataloging thread I mentioned earlier (http://www.ausphotography.net.au/forum/showthread.php?141274-Tagging-cataloging-searching-hints-and-tips&highlight=metadata).

Once the codec is installed, if you right click a raw file go to properties and then details in the properties box, you hover your mouse over any of the values detail and if it is then bordered with a dark frame, it's editable. If there is no dark border available it's not editable.

With relevant metadata inserted into the raw file, you can then do a search for that metadata directly in Explorer and it will come up as a hit in the search window.(same with Cortana .. but who uses Cortana! ;))

[ameerat42]

OK, so it's for File/Win Explorer. Ta.

[Mark L]

DPP will save the changes you make to a RAW file but only when you revisit via DPP. PS or LR will not notice the changes you make in DPP if you open the PPed RAW file there.



Sorry Mark, but as with Am .. not 100% accurate.

mmm, think that my underlined is fairly accurate and address what superbee ask in the very first line of the OP,

...., but I have just this week invested in Adobe Products (Lightroom Photoshop with Bridge)

[superbee15]

I wonder if Superbee is confused with all the replies by now:D

Ahh yes :confused013lol

It will take a few reads and a little more research but basically I have got a lot from it all. And I was heading in a somewhat correct path.

I guess the one thing not covered was the question on PS and if it changes the original file or has the same process a Lr where the original is not changes?

[ameerat42]

So, are raw files "changed" with "processing"?

No. Not if you mean "Is all the original info still there?"

The "changes" you make when processing a raw file are kept in a "special storage area"
of the raw file. Sometimes this is called a "sidecar file".

When I process my Σ raw files, the resulting "stored changes" include the original data,
and also the latest change made.

AFAIK, the actual mechanics change with "brand" of raw file format, like Canon, Nikon...
But also AFAIK, there are only a couple of ways that is done.

You were talking about LR, so here's a link about it and raw files (2nd paragraph):
http://improvephotography.com/12010/important-lightroom-setting/

[arthurking83]

mmm, think that my underlined is fairly accurate and address what superbee ask in the very first line of the OP,

OK, apologies for 'confusing' the matter in any way.
But what will happen if you:

1/. initially open the raw files in DPP.
If you make any edits here, and WB is a great way to see this as it's usually an obvious edit .. DPP will write the alteration to the raw file.
When it does this it also updates the embedded preview files(of which there are either 3 or 4).

if you continue using DPP then all edits are written to the raw file(as most of us know).

2/. But, if you now import those files including the edited files using Lr(that I know of, but I think Bridge will do the same thing here) ...
On the initial import page where all the thumbnails of the raw files are displayed, Adobe(or more accurately, ACR) will see those edited thumbnails.
When you eventually open each of those files to edit them, ACR will then change the thumbnail for each of those files that have been opened.
You don't have to make any edits, it just writes the XML file for each of those opened files and then uses the rendering based on it's own XML details.

It's kind'a confusing as you will see one version of an image on the thumbnail grid, then suddenly you see a different looking image of that same file, even tho you haven't done anything to it(other than to simply open it in any ACR program).

I think the basic workings are, that if there is no xml file for any of the raw files, it just uses an embedded preview file.
Once the raw file is opened(edited in ACR or not), it creates the xml file and once that's done ACR will use that to render the image and or thumbnail.

Note that Lr is basically ACR with many extensive tricks.

I think I only ever used LR to import files(from card to PC) once, without using the manufacturers software. More so to see how it worked(ie. test). Didn't like it.


But one thing that Adobe doesn't do is to change the original file. It uses a process where it writes the changes to a small external file(.xml) extension, and reads the changes in the file made from that.
Most software use this system too now(which I actually don't like).

[Dug]

This video explaining setting up Lightroom for the first time and what lightroom is doing and why, has come up on the B&H site in the last few days.
https://www.youtube.com/watch?v=9ozz4EJMYec

Very well explained and essential viewing all the way through for anyone starting out with Lightroom and also demystify a few things for those also who have used it for a while.

Probably the biggest short coming of Lightroom is that at the beginning it is not intuitive how it works, what is happening and for what reason compared to how other programs do things.
By the end of the video it becomes clear to the beginner that there is method to it all and it starts to make sense as a way to handle having thousands of images.