If Sony is going to make this for real it will change the way we used to take photo! What you see here is a Sony sensor that can change the shutter time of each single pixel on a sensor! It means areas of the sensor gathering more light will have a short exposure time and darker areas will get a longer exposure!
Unlike HDR shots where you take multiple and in time separated pictures on the new Sony sensor all this happens simultaneously:
You will no more have to struggle with extreme highlights or too dark areas. Everything on the image will get the exact light that is needed. There may be one single limit on that. If you take a picture of a fast moving object the image can result in a mess. But Sony is working on that too with the implementation of a special HDR Blending/Blur correcting unit:
Of course this sensor would work perfectly fine for sports photographer too if they turn off the asynchronous per pixel shutter option. Really smart from Sony!
Full PDF file of the patent at SonyAlphaForum.
To not miss any upcoming news and rumors subscribe our social network:
RSS feed: http://www.sonyalpharumors.com/feed/
Facebook: https://www.facebook.com/sonyalpharumors
Twitter: https://twitter.com/SonyAlphaRumors
GooglePlus: https://plus.google.com/104216500156717934391/posts
0 Responses
With such complex imaging system, seems like they will need 4 Bionz X to make things snappy~
“Moore’s law” is the observation that, over the history of computing hardware, the number of transistors in a dense integrated circuit doubles approximately every two years.
1st blood!
HAH~!!!
First useless off topic comment. Or just first idiot.
We will see this in a new Nikon camera?
Would greatly reduce its usefulness since it only really works with a continuous sensor read out (live view). And Nikon ILC’s aren’t exactly live view wonders…. (mins the 1 series, but they have other requirements, met by other sensor manufacturers).
OK
Well, the camera will simply show you the longest exposure. You need to hold it steady for that length anyway, so the shorter exposure wouldn’t matter. Or, am I missing something?
I wonder how would this work on the photographer’s end in terms of choosing what should be exposed for how long. If it is just darker vs lighter, it would only replace exposure bracketing on some occasions. It’d be nice to have some selection modes.
The illustrations show somethin that looks rather like a fixed pattern. Andrea, are you sure the patent says the sensor “can change the shutter time of each single pixel on a sensor”?
How about a link to the patent?
Forgot to add it! Will do it now…
It must lead to a very strange motion blur in the pictures…
But if it is thought as a replacement of multiple exposure HDR photos this will be better…
I think i’m not mistaking by saying that this will reduce the resolution of the camera. Because let’s say, in a dark area of your picture, the “bright” pixels will be useless, and in a “bright” zone the dark ones will be. And interpolation will never replace “real” pixels… 😉
I thought of exactly this.
I’ll take dynamic range over resolution every day 🙂
Unless you’re a Canon or M43 user. (in the m43 case less of both)
This is where some of those extra megapixels will go!
It’s a software/processing (chips) patent, which means that if materialized, you’ll be given the option to use it. Altough the patent also mentions that processing could be done after the fact on another device (RAW on a PC). In which case you obviously can’t revert back to a normally read exposure.
You’re mistaken.
Very cool, I’m curious as to the implementation and usage beyond auto mode or when using strobe /flash. Hopefully it will be more useful than simply an auto HDR mode.
canon has a patent for this for years.
Sony has had a few for years too.
Honestly it really looks unrealistic to me…
It already exists for video purposes.
Well, reference please? To me 14 bits RAW Images can do this as well…
Think of the Red Dragons, getting about 20 stops DR by DSC standards. (The 16.5+ from the spec sheet are to be read from a cinematographers frame of reference)
Exactly this is what I was hoping. In my photography, I have to trade between either overexposing the hightlights or exposing them right, but ending up with too dark image. Obviously, in post processing this can be somehow corrected, but at the cost of increased noise, thus degrading the IQ. In this case we would have a situation where the shadows would be exposed slightly longer and highlights slightly shorter to have a more balanced picture.
This does not necessarily mean that you get a blurred image – thats why the exposure starts with the shadows first. Your exposure will be basically the duration of how long you want shadows to be exposed. Sony could actually make this configurable simmilary to the EV value. Right now you set EV to -0,3 or -0,6 which means the overall image gets a slightly shorter exposure. With the new sensor, it would be just the highlights which would get a shorter exposure.
The issue here seems to be that you would need quite a complex circuitry which would allow you to signal to each pixel when to start the exposure. I do believe Sony actually made a fixed pattern here and that is simmilar to the human eye which has rods and cones, which actually is the reason the human eye can see well in bright and dark. This for now means that you kind of process two separate images and then process them, to create an image which either has a much wider dynamic range or better noise – or both actually.
How about two photos and some masking? There was a dude giving a small talk at B&H, I think, on how he works his interior photography jobs. Very good technique, excellent outcome.
This is indeed a very important tech, makes me happy to invest in Sony lenses ! I know that I’ll have the best sensors in the years to come !
Just make sure you choose the right mount.
Some of the comments are not so well thought out. If the user/camera can independently set exposure time for short and long exposure area, there are surely ways to use that to significantly reduce shadow noise and/or improve highlight recovery. It’s not like you need to have half the pixels blown out in “long exposure channel” and half black in “short exposure”. Perhaps the DR increase is less than in bracketed HDR, but I’ll take 50% (as in number of stops) improvement any day.
Re. motion blur, it’s not like you’re going to shoot bracketed HDR with moving subjects either. What this allows is hand-held single-exposure HDR, and might even work with slow moving subjects.
The Idea with the two different shutter speed surprises me. Why not just use a “different ISO” and the “same shutter speed” with those pixels?
Simply mix e.g. ISO 200 and ISO 400 pixels and you already got 1 stop more dynamic range and no motion blur problem. That’s really no rocket science 😀
How does Sony do it with their HDR video sensors on smartphones? Any description of that tech?
Yes great idea, if you mix ISO 100 and 200 you would actually get a picture exposed at ISO 150 ! You should make a patent on that 😛
haha yes, but you get the shadow details of the ISO 200 and the highlight details of ISO100. Of course you could stretch the ISO range much more. I mean this is just a matter of two different sensor circuits and some in camera post processing, if any!
Yes you can do that and I’m pretty sure magic lantern did a similar thing for canon cameras but most people want to shoot at ISO 100 and not get noise by having sections of the image with higher ISO.
Anadrol88, you understood it wrong.
Yeah, but on those modern sensors ISO 100 or 800 doesn’t really matter much. Some smart in camera processing could reduce the noise a lot. The big bright areas use the ISO 100 pixels and the shadows use the ISO 800 pixel. Sony’s “Dynamic Range Optimizer” engine should already be able to do that.
For some that would be okay but if you are a landscape photographer and do large prints, you only want to shoot at the base ISO to get the best DR and the lowest noise.
make it 72 megapixels then instead of 36, problem solved.
My smallest concern about this tech would be the resolution 🙂
If only it was that easy. My concern isn’t resolution but IQ.
Yeah, magic lantern firmware for Canon is doing this already. About noise: espacially with the Sony sensors there is not much to complain from 100-1600 ISO. Actually, with Sony sensors you don’t need the magic lantern trick. The files give you a lot room to do single shot HDR conversion in post processing. I’m not sure that the proposed tech will be really usefull: Will the benefits really be worth the investment (for Sony but also for the buyer of the final product)? The challenge for Sony sensor is that the current ones are already that good that it is questionable if further improvments will have any real impact on photos.
Hans wrote:
“The Idea with the two different shutter speed surprises me. Why not
just use a “different ISO” and the “same shutter speed” with those
pixels?”
Because that will CERTAINLY NOT be the same thing, not at all.
Let me slightly rephrase your question: Why not just use a “different ISO” and “the same shutter speed” (and the same aperture) on our cameras, to cover all different exposure situations?
A specific sensor design will give best IQ and largest dynamic range for one base ISO value only. To go beyond or below base ISO value, will give gradually more worse IQ and lower DR. You will be able to choose different shutter speeds with basically any interchangeable lens camera which will cover more than about 23 EV steps exposure using the same base ISO, i.e. with NO degradations of IQ or DR. Of course you can increase the exposure range by changing the aperture too, but that is this is outside of this discussion. And in emergency you can deviate from base ISO too ! 🙂
I am certainly sure that this will have its time.
In order to expose single pixels the sensor needs to know exactly the values which are coming from the metering module. Am I wrong or would this module need to be far more complex than it is now?
And the processing unit would need to be much faster than the ones we have in current cameras, wouldn’t it?
Interesting to know would be the research schedule meaning when would a commercial release be realistic.
Its Sony – They have no metering module, just a readout from the main sensor and some software that emulates the traditional behavior.
Have the lifeview feed take an image thats just below clipping and andjust the exposure depending on how much the individual pixels where saturated some milliseconds ago.
Admin
If this feature is turned off then will each pixel work as a normal pixel yielding the the highest resolution?
And is this a pattern that is determined by firmware, hardware, or by the image itself?
How can this pattern be determined by the image itself when it’s not taken yet? You mean the pixel sees a shadow and says “oh, I have to expose longer”? Maybe possible but maybe one step further… This patent looks like it’s a simple pattern.
For your first question: Yes, of course you could expose all the pixel the same and get full resolution. Anything else wouldn’t make sense.
Yes.
DR has for me always been the glaring weakness of digital. i would take a sensitive, extremely high DR 10mp sensor over an insensitive, low DR 30mp sensor, any day.
And yes, huge mp and huge DR and huge sensitivity (all selectable of course) would be great.
Welcome to 2004, we now (2014) have high mp sensor with huge dynamic range. Your glaring weakness is talking out of your a$$.
Finaly i though about a sensor exactly like this 2 years ago…
I though they would never make it…
Wow, the future of digital photography is going to be really exciting. Imagine this on a z-shift sensor.
vaseline and valium for Canon, pls.
Does this also mean global shutter? Would be amazing to get rid of the mechanical one and thus of the sound, vibrations and a moving part..
If you actually read the patent, it’s talking mostly about blending of two images (long and short exposures) whlie trying to avoid “a drop in image quality due to occurrence of blurring caused by camera
shake or subject movement and the occurrence of color loss and fake
colors for a moving subject”. It is not describing a sensor architecture that supports per-pixel shutter speeds. The different shutter speeds for different pixels appear to be in a fixed pattern and not really any different conceptually from existing ideas dating back to at least Fuji’s old SR CCDs.
Now that is some cool news.
This plus foveon tech? 😛 One day maybe, perhaps my grandkids will buy me one lol.
Hope to have a selection of modes, diferent iso or diferent shutter speed, with diferent iso this could be used for sports.
It’s similar to what Magic Lantern recently developed for some Canon cameras, where you are given the option of extra DR in a single image by alternating exposure times between pixels. The main differences being in the pattern (horizontal lines in the case of Canon, giving more aliasing and moiré problems or requiring extra blurring to hide those) vs a suggested example (Sony suggested several different methods in another recently published patent, see link below*) of diagonal lines. Plus the processing implementation suggested in this patent seems quite a bit “smarter”, with regards to surpression of motion blur etc.
In theory this technique could give you a HDR image in LV already, similar to what certain cellphones and Sony cameras already allow you to do for HDR videos.
* One of the previously related patents: http://patentimages.storage.googleapis.com/pdfs/US8803985.pdf
Isn’t ML just changing the amplification? Same exposure time, but different ISO.
That makes sense for Canons cams because it shifts the dark parts out of the noise floor everybody else has removed instead – coincidently making that implemention very brand specific.
On the sad side: If I read that correctly they start the exposure at a variable time, but end it together – thats a electronic first curtain, but not an indicator for a fully electronic shutter.
Yes, I should have added that part.The ML implementation using different analogue gain (ISO if you will), basically bypasses its weakness (high read noise at low ISO) for shadow areas at the cost of resolution and the mentioned artifacts. Which in theory isn’t nearly as beneficial, as different exposure times for different pixels.
how is this “groundbreaking” when canon is already selling cameras that do “dual iso”?
It’s not groundbreaking since it’s been used for video purposes for quite some time. And Canon doesn’t sell cameras equipped with this option for stills, ML hacked firmware can add some of the functionality similar to what is described in the patent, to some Canon cameras. Just less polished.
Put it in the a900II.
You have some mental ideas. 😉
Is this like the old fujifilm “super CCD” that had small and big pixels which gave it amazing dynamic range for its time?
From a users POV: Same basic idea, but this time you could change the ratio between high and low at will.
This has some excellent potential…particularly if the blurring and noise canceling firmware…as described in the patent…work well.
Obviosly this would be a global electronic shutter.
This is still a bayer array..but since thes images require interpolation in order to blend the image types this is not as serious a disadvantage as it is in a normal bayer sensor..a usefull way to bring new life to an old technology.
Just remember its Sony so unless it works out of the box your chances of any firmware improvement are 50/50 at best.
I saw something similar from Olympus few months ago.
http://www.43rumors.com/olympus-patent-discloses-variable-exposure-time-for-different-imaging-areas/
Great, they could add this type of sensor to their OM-Junk camera line.
I have no doubt that Sony can create the physical sensor to do this… my only doubt is in Sony’s ability to write the software needed to utilize the capability. I’ve not been too impressed with the software side of Sony’s cameras, hope they bring in some good programmers or farm it out to someone with more expertise than Sony has.
What you don’t know about software could fill a book.
And what I do know about software could fill a library so I don’t mind missing a book. You’re the typical fanboy that can’t accept the truth about their idol. Sony imaging has a piss poor record with software, they can’t even produce their own software to handle the raw files their cameras create they farmed that out to Phase One.
You don’t know jack $hit about software and couldn’t write a line of code to save your life, who do you think you’re fooling moron.
Agreed, Sony software is generally horrible. A lot of their hardware is wasted on poor software. I still love my a7r though, despite its flaws.
Software engineering and development techniques have been developed in the Western world. They usually require a lot of openness, communication and non-hierarchical decision making. I can’t imagine they work well within the constraints of Japanese culture.
I wonder how all those Japanese game developers cope?
smh
They don’t. That’s why you see distinct differences between Japanese and Western games.
Differences is not the same as saying their culture has a negative effect in comparison to others. I think you’ll have to be a bit more specific if you want me to take your comment seriously. Otherwise you just come across as another racist.
cool, it is about time someone does something really new, finally something that goes a real step further than film….
Does it mean we will finally get single shot HDR in a camera that will work even when shooting moving objects? Some smartphones already have it.
Isn’t that called DRO ?
No. DRO is software post processing.
I think he is talking about smartphones. They use DRO and call it single shot HDR.
They either apply DRO like curves (expose for highlights, lift shadows) or rapidly fire several shots which they combine still. Some of the most recent smartphones are so fast at doing the latter, that it practically seems like taking a single shot (Note 4 for example).
The main issue here will be the individual pixel light gathering efficiency and well saturation.
My understanding was that we are already very near the limits of quantum efficiency as it is and the only way to improve signal strength in dark areas is to have larger pixels (hence the A7s).
Unless Sony can find a way to have small pixels matching the total performance of large pixels I don’t know how effective it might be. Certainly, one has to wonder whether it will bring little more to the table than the current DRO in-camera processing applies.
It might almost be getting to the point of individual photon counting! Until we can do that random generated noise will be an inevitable by-product.
That said, I’m not sure I would have believed the performance of the A7s if it had just been a rumour, but seeing is believing!
So much confusion in your post. This is about maximum DR compressed in a single image, not low light capabilities. So larger pixels aren’t going to help in that regard, see A7S vs A7R.
And basically this describes different exposures in one image, so to speak, you’re just talking about one single and the same exposure for all pixels. The latter with DRO will have a lot more noise in shadows than getting a longer exposure for shadows and shorter for highlights in the same image.
There is no confusion. To maximise DR you need to ‘know’ exactly what light is present at each point, so that black will be black and any shade up to pure white.
However, owing to limitiations in quantum efficiency, random noise is generated in the system at low light levels. This doesn’t have to be in dark photographs, but simply dark areas of normal light; the same thing applies.
If you allow certain pixels to have longer exposure, more random noise will be generated by them (as happens with low light pictures). Unless you can remove the random generated noise, you won’t see an advantage.
I’m not saying it won’t happen, but I appreciate just how difficult it will be, given the limits of physics.
More confusion. Longer exposures are there to reduce noise, not increase noise. LV already allows for all the information needed. The dark areas get exposed until saturated enough, same goes for highlights.
The advantages are like combining 2 separate shots, one exposed for the shadows, one exposed for the highlights and blending the parts that come out best in each. The results in terms of lower noise in shadows are readily available, repeatable on current cameras and easily explainable. A benefit of the method described here, is just one overall exposure required (less time and user effort) and less motion blur due to smart algorithms and the fact it is a single overall exposure. Downside being less resolution in certain areas.
At low ISO (again, that’s what this is about), low read noise is not a benefit of large pixels. Proven time after time. So this is not the same as comparing low light high ISO shots.
As usual you don’t have a clue.
You are an idiot who clearly knows nothing.
What your wrote have pretty much nothing to do with the patent that Sony is describing. You are talking about the Gain of each pixel or what you call the light gathering efficiency of each pixel. Pixel size is not the main determining factor. It is actually pixel pitch (the center to center distance) that affects this efficiency. Obviously, the larger the pixel, the larger the pitch in most cases, which is why some people confuse the 2. You can get great light gathering ability with small pixels spaced at large pitches.
What Sony is showing here is independent of light gathering ability (i.e. they can implement this on a point and shoot size sensor or they can implement this on a medium format sensor).
“You can get great light gathering ability with small pixels spaced at large pitches”
No, relatively, you can’t. The amount of light that can be gathered is completely dependent upon the size.How efficient that gathering is in each site, is something entirely different. As I pointed out, though, we are already very high in quantum efficiency in that regard. To make this sensor do as suggested would mean surpassing even those levels.
Yes, you can. The only thing a pixel does is convert the light it receives into an electrical signal. Pretend a beam of red light hits a site, it doesn’t matter if the pixel is large or small, it’ll still register a red light. With small pixels at large pitches, the circuitry for each site has higher path to path isolation. Higher isolation means there’s less noise from each circuit affecting the adjacent path, which means you have a cleaner pixel to pixel noise isolation.
Brilliant. I wish I’d thought of this first.
Looks like a Bayer version of this: http://finepix.com/exr_cmos/en/ What’s different?
the difference is in exr-dr mode, you can’t control individual pixels, you can only control a full line of pixels, and you have to put same color together in diagonal lines, so your color resolution has to be reduced even in normal exposure mode.
panasonic has similar patent for bayer version, also under line-by-line mode.
Seems sony’s version is pixel-by-pixel mode.
That’s really cool! If they make a camera with that, I’ll probably want it! Question: will the patterns of exposure affect IQ, or was this pattern just an example and could change according to the scene’s needs?
Same question here, are there “low” and “high” pixels in a particular pattern, or can every pixel react as the light canges?
If each pixel compensates for different brightness in the scene by changing the exposure time, the result will be that each pixel gets the same amount of light. As a result, you get a uniform grey image.
Look at the diagram again.
I can see a problem with moving subjects
Hm… Maybe they should start the pixels wells at the same time and count the time until time until they fill instead of counting the electrons disrupted by light in each well over a fixed time.
We have just been given the Zebra feature and now they will be removing it. 🙂
I thought Olympus was the first one who came up with this technology.
good, but not great…
while per pixel exposure its great for HDR and the like, i see still the bayer crap,
also that per pixel esposure its a good idea but bad implemented
i guess that 16:9 non bayer sensor i was dreaming is not coming out afterall (included per pixel esposure (a diferent one from that patent), new raw/image/video file, global shutter, and more+)
Wasn’t film a good way to take a photographs ? I wonder what Ansel Adams will say … naaaa he will be on location shooting.
Hey, this is exactly (well, more or less) what old Ansel DID printing his giant negatives: he’d light every small piece of his printing paper differently! Took him ages to get a perfect print, but perfect they were…
Was A.A. photography about quantity or quality is the question
Nah, Ansel Adams will be busy post processing his pictures.
I think Adams would be all for it. When he was told Kodak was inventing the digital camera back in the seventies, he wanted to get his hands on one and try it out. Too bad he past away before they became available.
This sounds very much like the Pixim sensors for secuity cameras: http://www.pixim.com/products-and-technology/technology
Let’s just assume that it works as they state. In the end you still get the crappy in camera HDR. But I suppose that is just fine for the amateurs out there. Oh, wait! No one here is an amateur! We are all accomplished pros that can declaim on every aspect of the gear we never buy!
The patent describes the processing as possible on an external device (such as PC) too, which means HDR RAW output.
…fearful old guys. Afraid of the future.
Old guys crying like babies.
Why don’t they just make use of a splitter, like one in SLT camera, then use two sensors with different sensitivity. Or same sensitivity but receive different light intensities. Cons will be cost increase(2 sensor), A-mount only(with no IBIS), no dedicated AF module etc etc but it’s an interesting idea to try.
Shutter. You forgot about shutter.
Oh yess, the shutter. Damn.
I thought organic CMOS sensors were the future? What happened?
If you dropped this sensor it would be what would break and not the ground.
Thinking helps, dear admin. Quite stupid idea, pictures of moving objects taken at longer exposure would look horribly.