We know the E-M5II will have sensor shift function to create high resolution images. A similar multishot technique could be used to reduce noise as a new Olympus patent explaines. Two (or more) images shot at different exposures can be used to create one single image with very low noise. In that case there is no sensor shift in action but just two same images shot at different exposures just like you do when shooting HDR.
And in a second patent Olympus also explains how to get rid of blurred images. That yes could be used in conjunction with the sensor shift mechanism as this is likely to create some blurred images when objects are moving.
50 Responses
Could they really patent image stacking technique? There must be something else to it. The technique itself is very old and has been used for decades by astronomers.
I’ve been tinkering with it for noise free landscape shots myself. And I think some Sony cameras have this implemented as well. So, what is it really that they patented?
Correct. Image stacking is public domain. Frankly, I think that deblurring may be public domain, too. I long ago wrote how you could do that, and I did not patent that. It came out of the work we were doing on QuickCam. Moreover, as I’ve long said, if you have an accelerometer in the camera that’s accurate and has small enough intervals and you know the data that corresponds to the time the shot was made, you can calculate out the blur made during the image. I originally proposed this as an alternative to IS. Why? Because it doesn’t require a mechanical component and can be done more cheaply, as long as you have a good enough accelerometer and a fast enough CPU.
If I had a nickel for every idea I didn’t patent, I’d at least be able to buy a mocha.
that will be Jpeg only, right? it would be nice if we could store that shake information to NEF file, and let the x86 CPUs do the calculations in post, but currently we have issues with even getting accurate color from our NEFs 😉
Like in a smartphone 🙂
Google has implemented that (along with some other stacking techniques) in their camera app: http://googleresearch.blogspot.com/2014/10/hdr-low-light-and-high-dynamic-range.html
I think that most of the innovation on software side will come from smartphone world, especially with much more powerful camera api in newest Android release. And this is another point for smartphones. Not satisfied with the capabilities or user experience of standard camera app? Then try one of many third party apps. Need something very specific? There’s probably an app for that, too.
And with more powerful devices, and more people treating their smarthones more seriously as photographic devices (limited, sure, but still very capable in their own right), there’s gonna be even more photocentric software. So, not only can we take a photo with a phone, we could also develop the RAW file, do some more fancy processing, share it with the world, and archive it in the cloud. All that without turning on your PC.
Olympus puts interesting stuff into their cameras (a myriad of live modes for example) and that’s nice. But that’s nowhere near as powerful as having the ability to add new functionality to the camera by software extensions. What I would like to see is an open api for accessing camera functionality, so people could write extensions for it. But that’s not going to happen. Corporations are control freaks.
Sony has their play memories apps thing, but I’m not sure how that works, but they’re probably on to something. And maybe Samsung, as a major smartphone player, will decide to do something like that one day (I was hoping for that when they released their Android based cameras, but that was just a useless gimmick).
I am dissappointed that the Olympus patent clerk did not get any coloured pencils for Xmas -perhaps they will have birthday soon and the world will be more colourful.
Well I can’t read kanji etc but I suspect they would not be able to obtain a patent for that here under English Law due to prior use, and as English Law sets International Law precedence for patent rights … Edison fell foul of that one with the light bulb because it had been invented in 1850 and developed through to its 1880 patent by a chap called Joseph Swan and Thomas lost a couple of shirts in lawsuit.
Many companies will make patent applications like this because patent offices cannot check everything these days, and will hold the pat.pending until challenged.
interesting if they do this in camera, automatically, with decent results. So: very little effort for a higher quality image OOC.
Otherwise it’s like reinventing the wheel – image stacking is as old as photoshop – it just takes some time and skill.
This is exactly thing that brings higher quality image Out Of the Camera.
Image stacking is old, but you can’t make higher resolution in photoshop with stacking.
Ah, but you can with PhotoAcute :-). Superresolution isn’t the same thing as stacking, although they can be combined.
Neither with these tricks. You need a proper sensor, which is the thorn in the backside of this poor dying system.
Finally. Its not new technique (you can use even basic Point and Shoot for this). Very classical multi image NR. And that really could help (paired with appropriate bit-depth .. read 14+ bits dear Olympus) image quality..
You get more signal data with each sample shot but you also get more e-noise accumulated. There seems to be a trade-off on signal to noise optimum around 6 shots (statistics n=6 v=5) any more and the sample population spreads out, which in camera terms would smear or blur the picture.
The Olympus wobbly-sensor system already does a form of dithering which actually looks great (an accidental benefit perhaps) but if you examine down at pixel level you can see that each pixel has a little data from the ones next to it, not a sharp cut-off like a fixed sensor.
Hm, actually true, but its same issue with any sensor based stabilisation (reason why Panasonic never implemented it, apart GX7 experiment).
It will be interesting to see how practical this is in real-life shooting situations.
Don’t put much hopes / expectations on it cause the disappointment will very hurt .
A new higher MP sensor is still the right way to go rather than this multiple shots .
Let’s wait & see how things turn out .
Your glass is half empty, as usual.
Higher resolution/smaller pixels will make diffraction an even bigger problem for those of us that sometimes actually need a wider DOF. That’s why I’m (cautiously) very eager to see what these “new” tricks have to offer.
Don’t worry, smaller pixels don’t affect dof or diffraction, as long as the print size stays the same as with larger pixels (and sensor size stays the same too).
But then what’s the point of a higher resolution sensor if you can’t get more detail/make bigger prints from it (unless you go for a more shallow DOF which also, in a sense, robs you of detail)?
More pixels allow for crisper downscaled pics and for some cropping if required. Bigger prints will not show different dof or diffraction if observed from farther away (which is OK, do you want to go there with a loupe?)
I guess that you’re not into landscape photography? 😉
In my quest to become the best photographer that I can be, I find it bad practice to compensate bad composing skills by spending money on higher resolution cameras so that I can “crop later”.
And yes, I sometimes actually appreciate being able to take in both the whole picture and study interesting details.
I also like to be able to hang big prints in places other than galleries, like in regular rooms, in actual homes, without having the viewer step back into an awkward corner, to enjoy a crisp visual experience.
But maybe I’m just odd that way…?
Not sure how they can patent this because it is not a new idea (it would not withstand any challenge) and as such very similar techniques such as stacking have been used throughout the scientific community for past several years. Generally speaking although one obtains more data but because the e-noise is different for each successive shot, you get an overall increase in noise level too.
Its not about the idea, it is about implementation. You can patent a implementation but not the idea.
You’re right. My Nexus 5 phone does the same thing as does my Sony a6000. But it works really well on my phone and Sony, so its a smart feature to have on Olympus and Panasonic cameras. Tech is awesome.
As a lab tech i understand that they increase noise to signal ratio with stacking like this (signal is fixed, static relatively random, which questions the black frame subtraction) though I assume the em1 can do this already by simply stacking a HDR +_0,5 ev. But I do not see the unblur typical stacking creates more blur with moving subjects (waterfall) not less!
But would I want it yes, combined with the 40mp output it might give a 3x center crop of 16mp in decent quality
The static isn’t relatively random, it is actually very specific on each sensor uniquely and that can be eliminated very effectively with dark frame substraction, as long we have the same temperature sample shot (dark frame).
Sony does this in their sensors itself, each sensor has own algorithms that does it hardware level the on each pixel itself, before it gets to second A/D conversion and then processed by software.
The sensor has temperature sensor and then it just check that and exposure time and check from table what kind signal processing is required.
That’s why Sony sensors are great and why example E-M5 has lower noise in long exposures than E-M1 (use Panasonic sensor) because E-M5 does substraction to each exposure and E-M1 requires it to do separately. When done separately or taken sample shot before/after shooting, the E-M1 results same or slightly lower noise.
I am pretty sure my E-M5 took a second dark frame exposure on long exposures the same way my E-M1 does.
You can still enable it to let it to get a another one. But that has been Sony patented technology to subtract noise before second A/D conversion.
Do you have link to where this tech is detailed by Sony?
There are patents and Sony own presentation.
Sony Exmor and Exmor R CMOS Camera Sensors Explai…: http://youtu.be/AXp0u1XsrZw
I own one of those and they were so ahead everyone.
This one shows it better: https://pro.sony.com/bbsccms/assets/files/cat/camsec/solutions/E_CMOS_Sensor_WP_110427.pdf Check page 5.
Page 5: https://pro.sony.com/bbsccms/assets/files/cat/camsec/solutions/E_CMOS_Sensor_WP_110427.pdf
With dark frame subtraction, as I understand it, you are better determining what the inherent noise is for each pixel and subtracting that from each pixels signal via a separate exposure as long as the original
What I think Olympus is doing is sampling each virtual pixel location in the image with multiple physical pixels on the sensor which allows them to better differentiate the signal from the noise component as the signal, or lack thereof, should be a constant but the noise wouldn’t be, so if you have a dark part of the image and shifting the sensor around yields most pixels reporting it dark but one disagreeing due to noise, the minority results could be discarded and so noise reduced.
This of course could be done very rapidly with the IBIS and it likely won’t need you to be using a tripod to accomplish as not only will the IBIS jiggle the sensor, it will do so in reference to the camera motion sensing.
The E-M1 IBIS is already good for 2+ second exposures and it is possible Olympus may well have improved upon that in the interim so doing all this hand-held should be well within the systems capability over a wide range of shutter speeds.
This all might yield very impressive results for anyone except Yun.
Lol Yun
Well…getting clean, sharp 40MP (I would be happy if it is variable size capture. RAW at 25MP would be great!) from our “little” MFT cameras would be fantastic if even only in some shooting conditions. I don’t know that I would buy this camera or wait until the E-M2 comes out with hopefully this tech. ability as part of the package.
Will have to see the product thoroughly review and in use anyway before I would buy.
I like the variable size capture of 25MP or some variant!
My only doubt is: how will the RAW file be?
Or, you could just use an already-existing Sony FF sensor…..
The one with no lenses, you mean?
Giant slow lenses for the most part.
Remember that the fastest m43 lenses are like F2 on FF, and not smaller. Plus, the horrid sensor performance forces you to fast and expensive lenses, making for very bad deals indeed.
No thanks, I like having RAW support. Besides I already have an FF camera which can’t hold a candle to my m4/3 gear when it comes to IQ per pound.
Olympus doesn’t (in my opinion at least) rush their innovations out the door; they’re usually well implemented regardless of whether or not they were the first to implement the feature. From the first waterproof tough cameras to sensor vibration dust removal, to IBIS and weatherproofing, they have always been quality features when introduced to the Olympus line. For this reason I have faith that the sensor shift will be quite good when introduced. Not necessarily the best it will ever be, but a satisfactory introduction both functional and effective at the time of introduction. Not half baked like a GoPro.
I agree, I would put my money on a good implementation by Olympus.
Gee, I hope so. They better make a reak in-camera panaroma option rock. They’re only 6 or 7 years behind competitors.
I am surprised Panasonic is not offering HDR with it’s 4k photo mode. At Panasonic’s current 30fps for 4k you could capture some slight movements and still get a sharp image. Once we get to 60fps with 4k then this becomes even more usable for HDR and other stacking capabilities.
Aren’t there already a lot of cameras that do that, for example Samsung cell phones and Sony point and shoots both already have a low light mode that takes multiple short exposures and stacks them to reduce noise and blur. Probably others do as well.
I believe some people here are overanalising this issue.
NR by stacking works by canceling out the randomness in different but equal timed exposures.
Since this technique involves different exposures, I can’t see anything more logical and working better than taking the shadows from the longest exposures the highlights from the shortest ones, and so on…
Much like in a HDR stack but without applying curves, just applying a regular normalization instead (ex: 2x the -1EV shot and 1/2x the +1EV shot).
Not rocket science.
The Starlight Scene mode on the E-m1 does this (or similar) as a hand held feature using a fixed high ISO setting. Try it!