VLSI symposium published the PDF program of their June event in Kyoto. The program discloses a new Olympus sensor that will be presented on that event:
—-
A 3D Stacked CMOS Image Sensor with 16Mpixel Global-Shutter Mode and 2Mpixel 10000fps Mode Using 4 Million Interconnections
T. Kondo, Y. Takemoto, K. Kobayashi, M. Tsukimura, N. Takazawa, H. Kato, S. Suzuki, J. Aoki, H. Saito, Y. Gomi, S. Matsuda and Y. Tadaki, Olympus, Japan
A 16Mpixel 3D stacked CMOS image sensor with pixel level interconnections using 4,008,960 micro bumps at a 7.6μm pitch, which set no layout restriction and causes no harm to sensor characteristics, was developed to achieve both a 16Mpixel global-shutter mode with a -180dB PLS and 2Mpixel 10000fps high speed image capturing mode.
—-
There is no other detail unveiled about the new sensor. But it certainly sounds like a hell of a sensor! Wish this one could find his way in the new E-M1II !!!
Found via Image Sensor World.
78 Responses
I thought Olympus doesn’t make their own sensors and buys them from Sony. If this is a sign of things to come then I am all for it. An affordable global shutter will revolutionize consumer cameras as we know it.
Even if they buy sensors from Panasonic and Sony, it doesn’t mean that Olympus aren’t involved in designing them. nVidia designs their grahics, but never fabricate them.
Intresting news I have been waiting long time specially global shutter while they have steady 5stepilization.
Sounds nice just like the EM5 Mk II did on paper. If they execute as they did for the EM5 Mk II it will again fall into the realms of could have been great : (
STFU troll.
STFU Troll
Not really understanding what the micro bumps mean, but if the pitch of 7.6μm is also the pixel pitch, then for this to be a 4/3 proportion sensor, its dimensions would be approximately 35mm*26mm, i.e. pretty much a full frame sensor. In full frame territory this would not be attractive for photography unless it has super high sensitivity. I bet this is not really for Olympus’ camera division, but rather for their medical or industrial divisions. I don’t see them going into video with a new sensor and new mount at once.
My thoughts exactly.
The sensor is stacked. So instead of the color sub-pixels (1 blue, 1 red and 2 green?) laying beside each other forming one pixel, they are stacked under each other. Then the pixels can be up to 4 times as big. I think.
I found some information about this technology. I Think 🙂 Perhaps something to read for those who know more about sensors than I do, and care to explain for the rest of us.
https://projects.imec.be/3sis/sites/projects.imec.be.3sis/files/3SIS_Overview.pdf
No, this makes perfect sense as a 4/3 sensor.
4 million microbumps (I’m assuming these are the stacked photodiodes) = 4MP
4MP @ 4:3 = ~2300×1725 resolution
2300 * 7.6 um = 17.5mm
1725 * 7.6 um = 13.1mm
That’s a 4/3″ sensor.
Ah, I see. This definitely seems reasonable. Thanks! But why would they call it a 16 MP sensor then? An RGBG Bayer 16 MP sensor has 4 MP red, 8 MP green and 4 MP blue photo sites, so I understand that this sensor with 4 M sites in each layer gives the same resolution of the final image. Perhaps marketers would call it a 16 MP sensor, but not people in an academic setting. (This tradition of comparing to “the old standard” instead of giving the actual physical characteristic gives us stupid things like “1 inch” sensors that are 0.7 inches, when nobody has been using vidicon tubes for decades.)
“with pixel level interconnections using 4,008,960 micro bumps at a 7.6μm pitch”
The micro bumps have to do with the connections between pixels, they aren’t the pixels themselves. The underlying sensor is a 16 MP Bayer. This is not a Foveon-style layered sensor, but a stacked sensor. Read about Sony Exmor RS, the stacked variant of Sony Exmor R, to learn what stacked might mean.
It’s a 16 MP Bayer sensor. Stacked sensor is not the same as layered sensor (like the Foveon).
The 4 million microbumps sit on top of the 16 MP sensor. Your calculation of sensor size is correct, though.
Not exactly. Everyone is getting this wrong in different ways. “3D Stacked” means something different in sensor development than everyone is guessing. This is not post-Bayer where you have R/G/B layers in the sensor instead of Bayer’s side-by-side layout. 3D Stack in sensor-speak refers to an optical/sensing integration. Think Lytro. Or Apple’s acquisition of LinX.
The 4m bumps sit atop a 16mp sensor, I’d guess. Given 4m and 7 micron, it is not full frame, it’s close to 4/3.
Could some explain this to me. Does this mean it can fire off 4 shots at a moving subject in a blink of an eye then merge together for hi ISO and resolution?
Sensor for the medical and/or high speed camera business.
Well it looks like Oly beats Panasonic to global shutter
who made this sensor,
unknow new sensor for new lumix MFT but possible new generation image sensor
Probably Olympus. Why would Olympus present it, if it is a Panasonic sensor?
maybe 3rd factory,if they can do,image-sensor company all around the world,nor sony or pany but CMOSIS can make best video sensor better than Sony,and had long time Global Shutter ,why do not talk about them(CMOSIS)
Lol m43 making toys….meanwhile on the otherside Sony rocks with very small, cheap and excelent FF body’s with superb lenses to go.
Ah, no they don’t. Every other manufacturer does more with Sony’s own sensors.
Started drooling when I read global shutter, but then read… 2mpxels? So many creative things one can do with higher sync speeds!! Hope we don’t have to wait forever to see them in the next OMDs
Read further — 2Mpixels is only for the crazy-fast mode (10,000 frames/sec). Using rough numbers, let’s say 2 bytes per pixel — that sensor will pump out image data at 40 GBytes/sec. This is about 400 times faster than current SD cards marketed for 4K video — so a camera using this sensor will need a LOT of new technology in other areas in order to keep up.
Definitely doesn’t sound like a consumer camera. But that’s a really fascinating sensor, regardless.
It’s like concept car from an auto show. Really cool in an engineering sense, but it’s the answer to the question that no one asked.
I’d love to have a 1,000fps HD camera!!!
global shutter at 16megapixels and 10000fps HD? Sign me up!
Isn’t this just an academic paper?
No, Olympus will present the tech at a Image Sensor Symposium in June 2015. They worked on this tech in 2013 till now. I assume this means they fixed the bugs they were having and ready for production for 2016. Olympus is not alone on this tech and other chip manufacturers have similar tech for 2016 and beyond.
Interesting. Wonder if they plan on commercialising this tech on consumer market, or will it just be for medical/industrial applications.
Sounds great, either way.
Go Olympus!
Yeah and Olympus doesn’t even use the full capability of the existing E-M1 sensor! Wake me up when Olympus says they will put actual specs to use on the friggen existing E-M1.
Current E-M1 specs: 1080/60p, 720/60p, high speed mode: 240fps/481fps, Full scan any size cropping 4k2k 30fps. Electronic shutter, Output still frame rate 22.5fps 10bit, 14.7fps 12bit.
This is supposed to be the holy grail sensor that removes the rolling shutter jello effect if I recall and also increase sensitivity but could have issues in bright light. This is not new, Olympus has been working this tech for some time and apparently they must of resolved some backlight shielding to reduce the effect of bright light conditions. This would also greatly increase sensitivity. This tech effectively reduces and moves circuitry so that you get more light exposure to the sensor.
Apparently this type of sensor could be capable of 8k. It could also do multiple exposures all at the same time to extend DR with no processing time. This would effectively be the sensor along with any new processor that would do hand-held high res.
This is Olympus sensor tech that they worked on from 2013. However, the sensor may be produced by Sony.
Also note that Apple submitted a patent for a 3D stacked sensor with Global Shutter likely for the next iPhone.
why sony only,CMOSIS , TowerJazz made global shutter sensor use on hi-end professional camcorder,that sony can’t compare. or your brain full fill Sony.
Sony still owns a majority share stake in Olympus, sits on its Board and they have an agreement with Olympus still to force sharing tech and there would be an obvious push for Sony to have Olympus use them for CMOS fabrication. JPMorgan own’s the other Olympus stake that Sony just sold to them and Sony and JPM have an agreement in regard to Olympus.
So either way it would be in Sony’s interest. Not sure why Sony allow Olympus to go with a third party CMOS? This 3D stacked technology is not new or exclusive to Olympus and Sony also has it also going back many years. Olympus simply has a new technique. I mentioned above Apple as well.
So no, I don’t have Sony on the brain, Olympus has a Sony shackle on their neck. Sony just made major investments to increase manufacturing of stacked CMOS technology since that is where the technology is headed and they explicitly say this investment will occur in 2015+ which is the same type of technology Olympus is announcing today.
The manufacturer you listed is Panasonic and they sold 3 of their manufacturing plants to TowerJazz and Panasonic has 49% stake. I just don’t see Sony allowing Olympus to again use Panasonic tech and then letting someone else manufacturer chips for Olympus. Sony and Panasonic and their manufacturing agents are competitors.
http://www.sony.net/SonyInfo/News/Press/201407/14-070E/
https://www.semiconportal.com/en/archive/news/news-by-sin/140107-sin-wafer-fab-reorganization.html
Time will tell if we’ll have an E-M1 Mark II with a Sony or Panasonic chip. My bet is on Sony chips.
Well if 5% is a majority share, then you are right.
😀
If you are a purist then let me rephrase as “major influencer” and “minority” shareholder. Any individual or entity that owns 5% of shares has greater shareholder rights legally and a bigger influence to drive outcomes with the controlling board by being able to more directly voice opinion. In the case of the medical business, Sony owned 51% and Olympus 49% which is a majority which was in exchange for 11.5% of Olympus stock. Sony sold 6.5% and retained 5%. In that sense Sony has a “minority” stake in Olympus stock. Also note that Sony sold its shares to its “partner” JP Morgan, which means Sony can exert pressure on Olympus when it wants.
With all that in mind, I labeled Sony as a “Majority shareholder” because their influence is greater than those owning more or less stock.
With the sale of Olympus stock, there was nothing in the announcement I saw that dissolved the medical business partnership which spills over to the consumer imaging side as well with Sony being able to access Olympus tech and force Olympus to use its sensors.
“Source Engadget: The share switch doesn’t give Sony enough influence to dictate Olympus’ day-to-day affairs, but Sony won’t have to do as much to rally support if it wants action. ”
Largest shareholders:
Nippon Life Insurance (8.4%), JP Morgan (5%+), Sony (5%) Bank of Tokyo-Mitsubishi (4.98%), and Sumitomo Mitsui Banking (3.13%), and the Government of Singapore Investment Corporation (2.55%).
yeah,wait to see that,i sure no long time for SoNy,they
will lose again like walkman,PS & mobile phone. 🙂
The E-M1 sensor is the same as the GH4 (but with PD-AF pixels) & are under a 2 year contract with Panasonic for the use of this sensor which obviously limits the use for video. The contract should expire around the end of Sept (I think) so let’s see if we get any more video capabilities added in firmware updates after then.
Handheld Hi Res, handheld HDR, ultra high fps and flash sync speeds, etc
This is most likely for their high speed video cameras for use in science/medicine.
No doubt, it’s probably super expensive to manufacture for now
Olympus HIghspeed/4K camera department was separated to totally own company, about year or two ago. Olympus has been manufacturing and selling 4K cameras since 2003 and high FPS (up to 2 million frames per second) cameras to semi-conduction manufacturers for over decade. Many doesn’t know that Olympus has high expertise on high speed cameras and ultra high resolution cameras, but used only in industrial and medical markets where Olympus is the leader with over 70% market share (making few billions dollars profit every year).
Global shutter, wish it finally appears in consumer cameras, at least top models like e-m1. But probably have to wait at least several more years 🙁
By specs that sensor would have 10 000fps slow motion capture capability at 2Mpix (so Full HD video), global shutter to totally eliminate banding problem in moving subjects (or rotating camera), and about 17 stop dynamic range.
And because it is “3D stacked”, it means each pixel has RGB sub-pixels, so no more bayer pattern.
“And because it is “3D stacked”, it means each pixel has RGB sub-pixels, so no more bayer pattern.”
Not necessarily, lower levels can be just transistors and interconnects (no photodiodes), like in Sony Exmor RS sensors.
This sensor is male? Says “he” right there. I thought German sensors were male, while English sensors were neuters.
Like. Mechanical shutters should have stayed in 20th century.
Yeah well, currently electronic shutters don’t get as good results as mechanical ones. They still suffer from roll, And I have noticed that the images produced by my GM1 and GH4 when using mechanical shutter have noticeably less dynamic range. That may be something to do with how Panasonic has implemented it, but it’s a fact. I never use the electronic shutter if I can avoid it.
I’m sure you meant that the electronic shutter limits DR, not mechanical one 🙂
That is basically correct. Electronic shutter has some effect on the recorded signal. There is no cure. The only “solution” is to make sensors so good that they will still be easily acceptable without mechanical shutter.
Best implementations of consumer Sony sensors offer around 15 stops of DR at base ISO. Even if electronic shutter limits this to 14 or 13 – it is still enough even for basic professional work.
As far as pro gear goes – proper cinema gear (ARRI, Red) is in a different league, but it is not exactly pocketable nor affordable. 🙂
+1 Piotr – yes I corrected that – my bad. It’s the Electronic shutter I meant. It has less DR. And that IS a fact. It becomes most apparent when attempting to make HDR images from a sequence taken with electronic shutters. Smooth gradients suddenly have banding issues. I beleive it’s because there is less color/lighting data to combine, and despite using several different exposures they contain overlapping areas that contain identical pixel data.
“Electronic shutter has some effect on the recorded signal. There is no cure.”
BS. It is just the choice of camera manufacturers to use 8-bit or 10-bit readout mode instead of 12-bit because the bus to the sensor is too narrow, to speed up the process.
They suck because they are not global shutters. They are unable to really stop exposure, they “stop” it by reading out each pixel sequentially, one by one, with a very low number of ADCs. In a more reasonable architecture, they would have a personal ADC behind each pixel, or a very small group of pixels. Or even photon counters. Each would not have to work as fast and as hard and overheat in the process, leading to read noise too. But the ancient 180nm (Sony) and even 500nm technologies are not any good for that with modern pixel counts of course.
180nm in VLSI is a good standard and not ancient…
I meant VLSI analog circuits…
Digital you can easily go smaller
Come on, it is not like wires around photodiodes require large production nodes. It is just a single modern electronics factory costs more than the whole Sony or Nikon, so they HAVE to use the old junk nobody else needs.
180 nm process (or rather products made on that process) first appeared 16 years ago. For electronics, it is ancient.
Yes I agree they are not global But I was replying to your comment “Mechanical shutters should have stayed in 20th century” and Since you didn’t specify that you were talking about future global shutters I figured I might point out that current electronic shutters have issues, and still cannot compete with mechanical ones, at least not in Panasonic’s cameras. Hope I’m making myself clear enough for everyone now that I’ve said everything twice? By the way Peevee, I do agree with you, I’m just wondering why you’re disagreeing with me and why “teesquare” thinks he’s discovered something by simply repeating what I said 🙂
Actually, even regal non-global shutters, done right (like in some Nikon 1 cameras with fast Aptina sensors) can compete with specifically shitty Panasonic mechanical shutters in GM1, GM5 and GF7. But of course these are sensors with fastest readouts vs shittiest shutters out there. But, sensor makers could have designed they sensor for faster readout. Sony makes a lot of sensors with 8-channel bus, why not 80 channels for example? Even 24 to 32 channels would beat the best mechanical shutters out there, at least on 16 mpix.
“may be something to do with how Panasonic has implemented it, but it’s a fact”
Sounds like a “fact” based on your experience with ONE brand.. Let’s not presume that Oly or any other company for that suffers the same mis-steps.
Each of them are creative enough to screw up in their own individual manner.
Yes it’s a fact with that one brand. You understood exactly what I said. Have a cigar 😉
Would be interesting to see if Olympus has the same issue though. It’s not easy to spot. I didn’t notice until I tried using images shot with electronic shutter for HDR. That’s when the poor DR became obvious.
compact camera sensor or M43? and is Oly now designing sensors?
The huge “wow” in comments show how little some people know about current sensor technology.:/
Sensors like this one exist today. You can go to a shop and buy a camera (Phantom Flex4K is over $100k :)). It is very nice that Olympus wants to join this segment, but this is not a “revolutionary” idea or product.
For people who expect similar sensors in a OMD-like camera: good luck in life. 🙂
1) high speed cameras need a lot of energy. You usually plug them to mains or a mobile generator.
Batteries? Sure. High speed cameras consume around 200-300W. Battery for E-M5 has around 10Wh. Do the math. 🙂
2) high speed cameras produce huge amount of data – it will take decades for small flash cards to offer necessary transfer and capacity. Phantom Flex has 64 GB of RAM – it can hold few seconds of footage.
3) even if you would put a 10k fps sensor into a E-M1 and add everything else (power, video output) the camera would burst into flames because of the generated temperature.
So in general: sensor good or bad – that’s to be checked when they make it. But we are talking about video rigs, not compact cameras. Grow up.
You have missed the point. The hope is that some of this technology may find its way into the next generation cameras, like global shutter.
You have forgotten that a computer with a a few K of core occupied a whole data centre with tons of power generator etc. Now anymobile has millions more powerful processing storage, Cpu cache, etc.
The significance is that a small camera company bothered to present this paper, which should mean some progress from what you know and described in your post.
This has nothing to do with consumer cameras. Olympus have a division that make high speed cameras for industrial/scientific use. That would be the related relevance of this, not the consumer camera division.
Nope, Olympus sold their high speed camera division late last year.
Irrelevant. Modern computers are a fairly young invention.
Comparing a PC from 2015 to a e.g. ENIAC is pointless, because the technology was different (vacuum tubes vs transistors).
Compare a PC from 2015 to a PC from 1995. The size is basically the same.
The processing power is bigger, but so is power consumption and created heat. 20 years ago you could play all games and use photoshop on a computer which needed 70W and was passively cooled by tiny radiators.
A “functional” equivalent today will use 700W and cooling system will be enormous.
As you can see – we are reaching physical limits. A computer which looks more or less like a computer 10 years ago will have similar processing power. Efficiency is growing very slowly.
The 10k fps sensor Olympus develops is based on current technology. We can expect that it will have similar requirements to existing 10k fps sensors. There is no way that it could work in a pocketable, autonomous camera. Seriously.
Maybe one day a revolution in sensors will happen similar to that of semiconductor transistors. Maybe small cameras will be able to record at 10k fps. But this moment is really far away and this will not be a CMOS how we know it today. Possibly not a CMOS at all.
Forgetting about electronics – think about “personal transport” – we call it a “car”, “bike” etc. This is a great example of how we form our expectations based on past observation (instead of proper forecasting).
120 years ago cars were something new and usually slower than bicycles (or pedestrians…).
40 years later they could easily reach 100 kph. At that point people expected that in 2000 everyone will have something traveling with speed of sound, fly, teleport or whatever. But actually the cars are not much faster than 50 years ago. Physical limits. They’re just more affordable; easier to use.
“Compare a PC from 2015 to a PC from 1995. The size is basically the same.”
Any phone today is tens of times more powerful than a PC from 1995. Pentium 100MHz? Gimme a break, 2.3GHz 4-core in the pocket is now the norm. No, the size of that phone is not the same as PC from 1995. Neither is the size of Mac Mini etc.
But, PC processors have pretty much peaked a few years ago (not in 1995, more like 2005 for single core speed, and 2010 for practical home core count), Intel won the competition and now invests into built-in graphics instead of CPUs, or into number of cores for Xeons.
Mobile CPUs though have progressed quickly all that time and only now starting to show sighs of peaking.
CPUs in cameras, with the exception of Samsung, are 10+ years behind the curve though, they are still on 65nm process, sensors on 180-500nm, both planar and not 3D or at least stacked (only tiny Sony sensors became stacked recently). A sensor stacked on a modern-tech 14nm processor with something like 2048-line DDR bus between them can be read out instantly for all practical purposes, there is no even need for global shutter. Japanese just don’t have the production lines and know-how, and unfortunately they have monopolized camera industry so cameras are stuck in the stone age.
I said: compare a PC to a PC.
That’s how a top GPU card looked in 1997 (Riva 128):
http://www.techpowerup.com/gpudb/images/1307.jpg
in 2005 (GeForce 8600):
http://www.techpowerup.com/gpudb/images/12.jpg
in 2015 (Geforce GTX 980):
http://www.techpowerup.com/gpudb/images/2621.jpg
All this cards offered the same thing when they were released: you could play existing games in very high quality.
You see what I mean?
As a result in 1997 you could put a fully featured GPU into a laptop, because it was small and didn’t need much cooling (and the laptops were larger).
The GPUs you get in laptops today are very limited compared to desktop models (even if they are similarly called, have similar clock rates and amount of memory).
As for the “2.3 GHz 4-core” ARM chips you get in phones – you’d be surprised how shitty they are in absolute terms. They are optimized for typical smartphone operations, but it is very difficult to make them do things PCs can perform very efficiently. Try compressing files on your smartphone – you’ll see what I’m talking about. 🙂
By equivalence I mean e.g. playing a very demanding game today compared to a demanding game from 1995 on 1995 PC. A computer which could do that 20 years ago was smaller, used less power and (of course) produced less heat.
Processors in cameras is a totally different subject, but also quite interesting.
They do different things than those in computers or phones. I mean: if you want your phone to run games, download apps etc, you will be disappointed by the performance and possibilities.
Electronics in cameras are very limited to be very efficient. They have to process and compress high resolution still and video pretty much on the fly.
Simple example: a modern high-end image processor will read the signal coming from the sensor, create a RAW file, compress it into JPG and write both files to the buffer in shorter time, than a typical PC needs to open the resulting JPG. Seriously. 🙂
But what you seemed to have missed are the current advances in the mass market semi conductor industry. All leading semiconductor manuufacturers have adapted some form of 3d stacking and are now using things like through silicon via’s. this should enable olympus to be able to produce these sensors. Actually this would only be like a 2 layer system, while they already can achieve 48 layers currently.
If you don’t know what i’m talking about just do a search on 3d nand flash. you might be suprised to find out that your current smartphone is already might be using this technology. Also an OMD could use this sensor for it’s global shutter and use the high speed mode on a very low resolution or limit the rate somewhat, thus lowering the processing power required.
The power of these kind of professional systems is mostly used for the processing and storing of all the data. just do the math it you would try to compress FHD footage on 10000fps. but something like 96×72 might be fun already for consumer and achievable and options like 200fps at FHD should be feasable and all kind of intermediates.
The global shutter and reduced electronic shutter doesn’t require noticable more power actually. The power is needed for processing the huge amount of data, which is only required if you want to do very high speed full resolution capturing. Also this amounts for the bigest part of the production cost. Video processing capability determines the bulk of the cost and power needs of a camera and as explained you can easily keep them the same as current and still benefit hugely of such a sensor. As the removal of a high quality mechanical shutter also lowers cost, noise and space for a camera.
Next up, a digital iris and liquid lens system and we can truly make high performance and extremely compact camera’s. If nobody is thinking and working on real innovations and just keep screaming big changes will never happen we would still be in the stone age. revolutions DO happen in every industry and it seems you have missed the major change that HAS already happened in the semiconductor industry.
I read an article about a similar sensor design that may explain this. The stacked sensor can have the top layer of micro bumps with each covering 4 pixels, 16MP imager layer, analog to digital converter layer that will work on a pixel groups (2×2 pixels for 4M interconnections?) to implement the global shutter, and a possible digital layer to process the image. The micro bumps may act as the lens for the camera.
Oh don’t get me started on the bad mech. shutter in the GM/GF cameras! I read about it and thought “electronic shutter takes over? OK” only to realize it’s not good enough at all. There should be 2 large warning labels all over those cameras.
*electronic shutter reduces dynamic range
**mechanical shutter is disabled if a manual lens is attached
For most people their go-to PC today is their phone. Tablet at most.
That’s more or less true, but not what I meant.
Some people moved from PC to smartphones/tablets not because the latter are as good, but because the former were or became too much. E.g. some people (my mother) only need a computer to check mail and weather. So a big desktop is not the optimal solution (in the past you didn’t have choice).
In other words: people move from PCs to tablets because they didn’t need a proper computer in the first place or their needs shrunk.
Simple example: most teenagers 10 years ago were playing proper, big, complicated games. You get older and for many gaming becomes less important. So at 15 y.o. they played Starcraft. At 30 they’re fine with simple mobile time killers.
This is not what I was talking about.
I’ve shown you the pictures of high end GPU from different periods. This is “equivalence” I’m talking about.