Sony develops first image sensor combined with AI chip
Sony has announced it has developed the world's first image sensor equipped with artificial intelligence (AI) processing.
The new image sensor has a configuration consisting of a pixel chip and logic chip. The logic chip, called Sony's digital signal processor (DSP), is dedicated to AI signal processing, along with memory for the AI model.
According to Sony, the DSP eliminates the need for additional hardware, such as high-performance processors or external memory, making it "ideal for edge AI systems".
By using AI, Sony said the sensor is also able to output metadata -- semantic information belonging to image data -- instead of image information, which reduces data volume and addresses privacy concerns. For practical applications, Sony said the logic chip within the image sensor allows it to send data for each individual output image frame for AI processing within a single video frame.
"This design makes it possible to deliver high-precision, real-time tracking of objects while recording video," Sony said.
Users can also write their own AI models into the embedded memory according to their requirements or the conditions of the location where the system is being used.
"For example, when multiple cameras employing this product are installed in a retail location, a single type of camera can be used with versatility across different locations, circumstances, times, or purposes," Sony said.
"When installed at the entrance to the facility it can be used to count the number of visitors entering the facility; when installed on the shelf of a store it can be used to detect stock shortages; when on the ceiling it can be used for heat mapping store visitors (detecting locations where many people gather), and the like."
The pixel chip itself is back-illuminated, packs 12.3 effective megapixels, and is 1.55μmx1.55μm. It can shoot at 60 frames per second (fps) in full frame, as well as 4k in 60fps and 1080p in 240fps for videos.
The AI-equipped image sensor will come in two form factors, being available as a bare chip -- called the IMX500 -- or with ceramic packaging as the IMX501.
The sample version of the bare chip became available in April and the sample packaged version is expected to be available in June.
They are priced at ¥10,000 and ¥20,000, respectively.
Despite the novel coronavirus outbreak affecting supply chains around the world, Sony said in March that its imaging and sensing solutions segment had not experienced any "material impact".
It did note, however, that there was a potential risk for sensor sales to be impacted by an eventual slowdown in the smartphone market from COVID-19 and that its electronics business had been hampered by supply chain issues arising from the outbreak.
Related Coverage
Sony's electronics supply chain impacted by COVID-19 instability
Its manufacturing plants in Malaysia and the UK will be shut down until late April.
Sony establishes $100m global relief fund to support COVID-19 affected individuals
This includes frontline medical workers, children and educators, and those in the entertainment industry.
Best aerial photography drones for business in 2020: DJI, Freefly, Skydio, and more
A lineup of aerial hardware stacks to fit a variety of enterprise photography and video use cases.
Sony and Intel team up for global forum on new communication infrastructure (TechRepublic)
The companies are banding together to build a system capable of handling the world's new digital needs.