Sony keeps bringing many innovation on the sensor world. Today they announced the worls’s first Stacked Sensor with Image Plane Phase Detection! Here is the press text and the explanation of how this works:
Sony Announces the Exmor RS™, the Industry’s First*1 Stacked CMOS Image Sensor with an Image Plane Phase Detection Signal Processing Function for High-Speed AF
Exmor RS IMX230 stacked CMOS image sensor
| Model name | Shipment date (planned) | Sample price |
|---|---|---|
| Type 1/2.4 21 effective megapixels*2 stacked CMOS image sensor IMX230 |
April, 2015 | 2,100 yen |
Rather than the traditional back-illuminated CMOS image sensor’s support substrate, the Exmor RS™ uses a chip consisting of signal processing circuits, on top of which is stacked a pixel section consisting of back-illuminated pixels for an original stacked construction. This kind of stacked CMOS imaging sensor, which delivers superior image quality and high functionality in a compact size, was first commercialized by Sony in 2012.
The new IMX230 is a type 1/2.4 stacked CMOS image sensor with a significantly improved 21 effective megapixels, and it is also equipped with a newly developed signal processing function. These features fulfill the growing needs in smartphone photography for high-speed autofocus (image plane phase detection AF) and clear, high-quality capture of bright and dark areas even in backlit scenes (HDR imaging). Image plane phase detection AF is a technology used in mirrorless interchangeable-lens cameras, and HDR imaging now supports not only 4K (4096 x 2160) high-resolution videos but also still images. Sony also plans to extend the lineup by adding a 16 effective megapixels stacked CMOS image sensor equipped with image plane phase detection AF and HDR imaging functions by the end of the next fiscal year.
IMX230 Main Features
- Image plane phase detection AF
- This function enables the camera to capture quick-moving subjects and makes accurate, high-speed autofocus tracking possible when shooting still images and videos, such as kids and animals constantly on the move and other fast subjects including athletes at sporting events. Dedicated image plane phase detection AF pixels are discretely incorporated into the screen of the image sensor, and the distance to the subject (range) and lens position for focusing are calculated based on the information obtained from these dedicated pixels. Up to 192 AF points can be used. All of this is instantaneously carried out by the stacked CMOS image sensor’s internal image processing circuits, making it easy to enjoy shooting photos and videos of fast-moving subjects on a smartphone or other device.
Image plane phase detection AF function using 192 AF points
- High Dynamic Range (HDR) imaging
- This function enables the camera to record still images and videos that reproduce details and rich gradations in the highlights and shadows of high-contrast scenes, such as those found in backlit environments. This is made possible by setting two different exposure conditions and applying the appropriate signal processing to the image information obtained from each condition. Through improved pixel placement and signal processing, this new HDR imaging function is now compatible with both video recording and, by popular demand, still image capture, the latter a feature that was not available previously*3. The enhanced resolution and generation of images with a wide dynamic range enable a high level of visibility and the production of images with vivid backgrounds and subjects, even in backlit conditions. This function is also compatible with 4K (4096 x 2160) high-resolution video recording.
- *3 The HDR movie function announced on January 23, 2012 supported video recording only.
HDR imaging sample (right) with the IMX230 type 1/2.4 21 effective megapixels*2 stacked CMOS image sensor
-
Traditional HDR movieNew HDR imaging
Sample using the IMX135 3.2 effective megapixels*4Sample using the IMX230 21 effective megapixels
Notes:
are trademarks of Sony Corporation.via Image Sensor World.
0 Responses
I am really curious, what is Canon coming up with…
Nothing. Vaporware. Whatever they will come up with, will be overpriced, under featured, it won’t compete with their cine cameras therefore it won’t have 4K video. It will still use their current fabrication process and will have terrible DR, and that wonderful shadow noise which is now a trademark characteristic of Canon cameras. To put it bluntly, a rip-off to be avoided at all costs. Search elsewhere for decent cameras.
Counting Canon out is a bit like betting against the house in Vegas. The history of these companies goes back further than 10 years. Read it. Learn from it.
@Mardock, I’m fully aware of it, having lived through FD to EF transition. I’m also fully aware Canon is a camera company and that semiconductors are an expensive industry. Samsung can afford to spend some billion USd every couple of years modernizing their fabs and shrinking process since they do everything from CPUs to NAND flash memory. Sony as well since it has the largest slice of the sensor market. But Canon? Their expenditures in the last 2, 3 years don’t seem to cover the amount of money involved in a process shrink. I very much doubt they will be revolutionizing anything in terms of IQ. Their best move could be perhaps going fabless or buying sensors from an external supplier. I just don’t see Canon doing what it takes to keep at the forefront of sensor technology and judging by the lack of updates in their sensor tech in the last 10 years it would seem their management board doesn’t likes the cost of it either.
You need to expose properly and to process properly. Also, avoid using dxo, it does not work well with Canon raws. You will quickly see that the Canon DR legends are just that: legends.
Thank you Mr.DebunkingAnotherMyth. After over 30 years of photography yes, i still don’t know how to expose properly, that must be it.
With all this Kodachrome, medium format film, large format, Ektachrome, B&W, custom made emulsions, early digital, MF CCD backs, and so on, i must’ve become confused and admit my ignorance on such banalities as “exposure”.
Sadly for Canonites, such “myths” as you refer to the lack of DR of Canon sensors, are quite common in the real world, specially if you happen to dear me, do landscape photography, architectural photography, and have no control over lighting conditions. It must be blind as well, since my 5D3 is gathering dust while my D810 is now my workhorse. It couldn’t have anything to do with the “myths” you are referring to. It must be incompetence, ignorance, and lack of faith in Canon of my part. A functional pair of eyeballs couldn’t have anything to do with that could it? Thank you for pointing me in the right direction, i shall now amend my ways. Thank you. Bless you.
And Sony sensors suck, are greatly overestimated, and that shadow noise you see in Canon sensors, and the banding, and the color splotches, those aren’t really there. And if they are, it is really my fault for not knowing how to ETTR properly, or use MagicLantern’s dualiso and Auto-ETTR module. And the inevitable tradeoff for using dualISO isn’t really a trade off and the 7D sensor is more than perfect for landscapes, cityscapes, product photography, portrait, architectural photography. Right?
Get a grip, you Canon fanboys are sick. These are corporations, like any other manufacturer, Nikon, Sony, Mamiya, Leica, they want your money and have no loyalty, not even to their shareholders and they have a fiduciary duty to do so. Get a grip on yourself…
Mr brick wall shooter, there is no need to get personal. In case you are not the usual paid shill, here are some tips.
If you are sure you know how to expose (to the right!), try using a proper camera profile in the raw processing. Use Xrite. Also, avoid dxo, which works poorly with Canon – their own tests say so. It’s part of it, you need to learn it. If that still fails, you are doing something wrong.
Why?
I did not know Canon does mobile phone sensors…
I agree,(St Pauli) how can canon ignore such, they have billions in sales and $$$$ in R&D, I will bet that canon in the new year is going to crush the completion and show what they can design. The Japaneses philosophy of move slow and design right will pay off
Canon is like a large ship that is very slow in changing direction…A bit like Kodak..
That charge could be leveled at almost all the traditional Japanese camera manufacturers, save perhaps Fuji. They’re all quite conservative, at the end of the day.
Yeah because Canon’s track record on sensor design speaks for itself…
….oh wait…
Sony is Japanese too…
I’d have wished they moved slow and designed right with VAIO in the last decade so they wouldn’t have had to spin it off.
Sony is Japanese. Nikon is Japanese. Fuji, Olympus, Panasonic… As to Canon, after the EOS-M fiasco and the 7DII, I wouldn’t hold my breath.
Now, what’s wrong with the 7DII? Have you tried it? Has it disappointed you?
It is too expensive for what it is and it is the same old tired stuff. Otherwise, a perfectly fine camera.
So you haven’t tried it. Good to know you say it’s “perfectly fine” (LOL).
What is there to try precisely? It is a warmed up 7D after all. So, yes, it is a perfectly fine camera that is too expensive when compared to competing products and doesn’t bring anything new to the table. What is your point anyway?
For clarification of doubt, a “stacked” sensor in the way that Sony uses the term does not mean a Foveon-like design with stacked photodiodes for colour sampling.
Anyway, unlike the rumoured “active pixel colour sampling” sensor, with its mixture of weird technology and infeasible specs (and almost certainly a hoax), this is clearly set in the real world. It’s also the right size for a camera phone unlike that APCS sensor.
The intriguing thing is image place phase detection. There is image plane phase detection on the hybrid AF systems on several systems cameras. However, progress is clearly being made in this area and it surely has the potential to kill off one of the two key capabilities propping up the DSLR compared to the mirrorless world. The other is, of course, the optical viewfinder. Once people get used to the EVF, and all the extra features it offers, this is yet another sign that DSLRs are on a gradual, but relentless decline to being a niche product.
nb. the in-built HDR technology is interesting. If/when very fast electronic shutters become feasible on large format sensors, it clearly has great potential. It won’t suit all circumstances, but it would work well with (moderately) static subjects and would have a wider range of applications that using current HDR techniques.
I agree. I’m looking forward to seeing how well this coplanar PD succeeds as it gets deployed in larger chips / systems.
You did a good job at debunking that apcs rumor.
That’s possible, but it’s also possible that everything he said is rubbish. It wouldn’t be the first time.
I’d wager it’s not a hoax (not on account of the content but the fact that most of the “rumors” on SAR come straight from Sony’s PR dept.) My guess is the specs have been taken out of context and don’t mean what people think they mean.
The APCS rumour was from a Chinese site, not direct from Sony’s PR department.
Placing image processing circuitry beneath the photocells might enable a global shutter. Instead of having to send the analog signal off the chip to be converted to a binary value the analog signal could be processed by a larger number of ADC underneath the photocells. The sensor would generate more heat but it sounds like the advantages outweigh the disadvantages.
Rather unlikely. ADCs are moderately large circuits. One thing to note as that the analogue signal is not, on Sony CMOS sensors , sent off-chip, although it will be sent to the edge of the chip. Sony sensors also perform parallel ADC conversion at the column level. I also don’t think that putting a heat-generating circuit behind every photosite is desirable from a thermal noise generating issue.
However, that doesn’t mean a “stacked” approach won’t potentially make a global shutter easier to implement. There are such CMOS sensors, but they dump the charge into a temporary store. On standard sensors, this takes up silicon “real estate” which compromises things like saturation levels. Perhaps if this can be implemented in “3D”, then this might be less of an issue. With a global shutter such as this, then it should allow for the elimination of a physical one.
I’m a little more leery of the PD-on-sensor-kills-DSLRs claim. The problem to date has been the math. You have far less data discrimination in the on-sensor variations due to the small size. That’s fine for small sensor cameras, as the DOF covers small errors. The larger the sensor, though, the more challenge there is to implementing PD on sensor and getting the full performance and accuracy we currently have with DSLRs.
Just to be clear, I’m still on the record as saying mirrorless will replace DSLRs. But the reason is more simple than “equal performance”: lower cost. This is one of those “good enough” versus “improves profit margin” things.
Bear in mind these are hybrid AF systems. Unlike a DSLR, the camera can use both contrast and phase detection in combination. CAF is actually more accurate (for most things) than is PAF.
Also, and possibly more importantly, a hybrid sensor offers the option of combining information intelligently. The story of modern cameras has included a strong thread of adding processing power for such things as autofocus, colour correction, geometric correction, facial recognition, movement detection and focus tracking. In some ways comparable what the visual cortex does for the eye. The eye and visual cortex manage all this on the image plane, albeit with the advantage of stereoscopic vision.
Of course this all requires the hardware and the software to be complementary, but things are moving that way. Already the camera/lens interface is very complex. I think we’ll see more progress made as the economics make a lot of sense and the hybrid approach will be able to do things dedicated PAF alone cannot.
I’m aware of that. But the minute you switch to contrast detect, you’re a slave to refresh rate of the sensor data. That’s been going up over time, but even a 120 fps move rate introduces a lag higher than DSLRs produce. Your lenses also have to be tuned differently for a hybrid system.
It’s necessary to read the entire frame for AF purposes (at least with CMOS). After all, the eye doesn’t. Only a small part of the retina has full acuity. There might also be the possibility of integrating some signal processing on the sensor itself, rather in the way that some things, like movement detection, are semi hard-wired into the eye and visual cortex.
I’m not suggesting all this is coming soon, or even that it’s all possible the same way, but I’m equally sure that tight sensor/processing is going to be an area where there will be more major advances.
Uh, not really. If I’ve told the camera to do 9-point CAF at the upper left third point, you don’t need to read the whole sensor continuously, though there are some benefits to do so if you’re integrating color information into the focus calculations (pattern recognition). If you can get by with smaller data transfers, in theory you can do them faster ;~).
The “whole sensor” read types of focus are attempting to do the automagicfocus type of thing, where the user doesn’t know what they’re trying to do, but the results come out perfect. In general I’d argue, just as with our brains controlling our eyes, control is better than no control. Assuming, of course, you know what you’re doing. In baseball, for example, hitters learn to tune to the hand/ball configuration as it comes off the pitcher’s hand if they can. Waiting to see where the ball ends up is waiting too long and they miss.
One of the things I’ve wondered is – Is it possible to fabricate a phased-array of pellicle micro-mirrors on sub-sets of the photo-diodes. Actually, I know it could be fab’d, I suppose I am wondering if it would be enough to useful for focusing. What do you think?
Tom
I would like to hear your thoughts on this subject
I believe it is necessary for Sony to make the new mirrorless pro cameras they plan to produce fully high speed focus compatable with older lenses including Canikon via adaptors.
You have talked about dificulties in achieving this.
Can you explain exactly what these difficulties are, what dslr lenses are lacking to make them compatable and how you think they can be made compatable.
More than just ..it cant be done…I would like to know how it can be done and how the obstacles can be overcome.
The A6000, for instance, is very good at focus speed. Almost DSLR-like, in many cases. Unfortunately, it’s also just a little unpredictable in how it follows focus on fast moving objects. Sometimes it’s fine, sometimes it misses and has a difficult time catching back up, in my experience. More problematic, however, is that most of Sony’s AF systems on the mirrorless cameras simply don’t have enough direct user control over where and how the focus is performed. See my comment to SteveGJ, below.
The difficulty of on-sensor PD is partially due to this: the short distance between the lenses that split the signal and the detector. That’s especially true of BSI type designs. You can get a good sense of which way to move and a reasonable notion of how much, but you run out of math precision before you get to dead on results. Again, DOF can help you, but if you then have to do another run via contrast detect, then you’re running at the speed of the sensor data speed. Note that Olympus DOESN’T do PD on their PD-equipped E-M1 except with older 4/3 lenses that require it. Instead, they’ve opted to just speed up the sensor data transfer and go contrast detect all the way.
Panasonic gets around the contrast detect issue with a novel approach: they actually focus and defocus the lens in between two contrast detect data transfers. The differences in the two data sequences tell them a lot of the same information PD does, and apparently with excellent accuracy. At least if you know what lens is on the camera ;~).
So there are ways to get to high speed autofocus on mirrorless cameras. In all likelihood, a total hybrid approach that picks the best from all of the technologies will win out.
What you are describing are some general issues regarding mirrorless AF accuracy and speed.
My more specific question revolves around cross platform compatability issues.
E mount lenses focus on E mount cameras and fully utilze the cameras capabilities.
Non native E mount lenses with adapters focus at reduced speed and accuracy..if at all.
So exactly what do these lenses lack to enable them to perform at the same level as native E mount lense..and can this be built into an adapter.
Alternatively..can the mirrorless AF system be made to funtion equally well with non native E mount lenses..and how would that be done.
My question is specifically about lens compatability…
I would like to be able to choose the camera I want and the lens I want
Won’t happen. Or maybe I should say “not likely to happen.”
When you design a lens in the modern era, you design it to be tuned to the needs of your focus system. With m4/3 that means 120Hz or 240Hz response with a particular “tuned” motor movement set (lens moves focus certain amount in certain known time). With 4/3 the lenses were designed for PD, and they have different “timings” and “movements.” That’s why the E-M1 uses different technology when those lenses are mounted. No one publishes these things, and it seems that virtually every company has a different set of how they’ve tuned lenses.
So the question then becomes: is it a good idea to enable a competitor’s lenses? The answer in one sense is “yes, if you’re not Canon or Nikon then you could perhaps attract users by supporting those lenses directly.” But the answer in another sense is “no, we’ll get screwed by trying to come up with all variations that exist, and Canon/Nikon might simply change parameters on us at their whim.”
In other words, there are no standards. Sigma, for instance, has to reverse engineer a lot of things with Nikon lenses. And they get burned on that from time to time. I can point you to Sigma users who aren’t happy with SIGMA because a new Nikon camera changed something and now has an incompatibility, which Sigma would then need to change firmware/hardware for. The track record for Sigma is mixed on that. Sometimes they do, sometimes they don’t.
So you say parameters are deliberately made differrent by the manufactureres primarily to keep thier systems proprietary…and the only thing missing for full compatability is cooperation?
Sigma has a usb dock for thier new global vision lenses.
Can this via firmware alter the parameters of thier lenses to deal with this compatability issue?
Yes, I believe we have evidence that parameters have been changed by some camera makers to keep things proprietary. In particular, I believe that Nikon has done this, though we have no absolute proof. Thing is, think of it this way: Nikon sells 30% of all the lenses sold. Why would they want to enable a competitor in any way? That might cut down their market share and sales of their own lenses.
Yes, Sigma has programmable firmware now and a way to deal with users doing the updates. As long as the parameters that change are controlled by electronics in the lens, that can suffice to deal with compatibility issues. For example, one such issue is how the lens identifies itself to the camera. Nikon has a tendency to step on Sigma ID numbers. ID numbers can be changed in firmware. But what if the voltage signals change on one of the pins? This isn’t a moot point, because it has happened several times, most recently because of DBS (dead battery syndrome). I’m not sure that’s a parameter that’s in the Sigma firmware. It might be hard-wired in a lens.
“no, we’ll get screwed by trying to come up with all variations that exist, and Canon/Nikon might simply change parameters on us at their whim.”
But once a lens exists, it’s timing and movement won’t change, will it? They may change it for future lenses, but surely existing lenses wouldn’t change. I think it would take a camera manufacturer that didn’t have their own line of lenses to sell for this to happen though. Maybe one of the open-source efforts?
How many samples of an existing lens are you going to test? Because tolerances also play a factor in whether what you want to do will work or not. One of the things that we’ve found with AF Fine Tune testing on Nikon lenses is that they often don’t fall in a nice bell curve, as we’d expect. Indeed, the f/2.8 lenses are all over the place (the f/4 lenses seem to follow a good bell curve, though).
Moreover, I’ve lost count of how many variations of lenses even Nikon has made. It’s in the many hundreds at this point, including a few “unannounced” iterations. I’d think that trying to please even the majority of folk who have a Nikon lens trying to use it on another camera and expecting everything to work would be an exercise in futility.
These days, lenses even have firmware updates associated with them, which complicates things further. If the lens maker updates the firmware and changes something, you’ve got to retest.
Thing is, every camera maker is perfectly happy with the quasi-solution that appeared: third-party adapters. This moves the problem out of their hands into someone elses’, yet it makes it appear that they support use of existing legacy lenses ;~).
True, but performance is already on par for a majority of usages and when considering entry-level gear, actually better. We are discussing follow focus, which is a drop in the sea.
It’s not a drop in the sea unless you are a brick wall shooter…
Landscape, travel, studio, family, portrait, street – pretty much none of those need follow focus; and it is a safe bet that these represent the vast majority of shooters. So, yes, sports and wildlife photographers are a drop in the sea, comparatively speaking.
LOL half of the things you mentioned move. And the closer you get, the worse it gets. However, I agree that brick walls don’t generally move.
Again a “LOL”. Well, that’s a great argument. Call me when you start thinking.
“Entry-level gear.” That would be smartphones. We already have some pretty good focus systems in those devices, and they’re about to get far better. And that’s the problem. If you want to sell a dedicated camera, you now have to do better than “good enough.”
One of the things I pointed out back in 2008 was that it’s unclear how much “middle” there will be to the camera market. It was clear that smartphones were going to gobble up the bottom, and that pro-level DSLRs had a strong lock at the top. Exactly how much middle will be left in a few years? I thought I was being conservative when I made my presentation to Nikon in 2010 about this problem, but in retrospect the middle I envisioned is already smaller than I thought it would be at this point.
Just in time for the Xperia Z4…
Judging from the shipment date it would just miss the Z4 by a couple of months, except if the shipment date is actually meant for others and Sony would get internal use before that.
i like this bullshitshow….
and small sensors = sharp image…
NX1 price is going to drop. Sony better hurry up with that BSI sensored cams.
Comes out tomorrow. I bet we a comparison review on here within two days.
Hopefully they will implement it into their larger sensors
Pros sell all your gear and beat the wave. All you’ll need is a Sony cellphone. ;^)
Well, video quality will probably trash any 5 year old DSLR except for shallow DOF.
“” *4 When using the HDR movie function, the effective pixel count is reduced to 1/4 of its original number.””
Clearly means when using HDR in movie – the pixel count of 4K reduces to 1080p.
Well, the note is under traditional sensor part, also if what you say is true, Sony doesn’t need to say it support HDR in 4k mode.
So, APCS mentioned before wasn’t IMX230?
no, he said that it would probably come first on the Sony Z4 smartphone, Sony always delivers there first, as its cheaper to build per item, faster to reach more targets and is a place where they have difficulty earning money.
Well, guess it’s time to put the hassleblad up for auction before this bad boy comes out. Have to beat everyone to the party.
Hulk smash 5DM3, sensor sucks and D70 won’t focus.
This is probably the same sensor family the iPhone 6 is using, which offers on-sensor PD AF and one-shot HDR.
This is most likely just the wholesale version of the image sensor used in the iPhone 6, which is a Sony Exmor with PDAF.
How can it be “world’s first” when Samsung S5 and iPhone 6 already use OSPDAF?
But they is not stacked sensor. and they is not full image plane PDAF.
i6 use Sony sensor too …. oops !!
I am sure both are BSI, meaning their electronics is not on the lighted surface – what else do you mean by “stacked”?
if they just announced it today ..and its gonna be available only in 2015 than why people keep “hoping” their iphone has it lol .
this chip , if you took the time to read all the article is a serious leap ..and not like that idiotic..”oh lets put more mega pixels” leap .
if they keep follow that line we could be seriously start thinking about comparing a smartphone shot to a dslr’s one . well not currently . but in a few years it may be so .
would love to see hard data on the actual DR ,iso and color richness of this sensor .
now! thats a proper sensor chart
with square pixels
standard sensor size
comercial number of pixels
doable fps
an a simple name
not like all the crap from others “rumors” charts we get once in a while
with rectangle pixels
3 times bigger sensors
4 megapixels that somehow could make 8 megapixels video
16,000 fps cause they resist to say infinity fps
and coded name with lots of random numbers and letters