Raspberry Pi Foundation Announces Libcamera-Based Open Source Camera Stack
"The new Raspberry Pi Open Source Camera System is providing something distinctly novel," says engineer David Plowman.
Hot on the heels of the Raspberry Pi High Quality Camera Module's release last week, the Raspberry Pi Foundation has announced the release of an open source camera stack that is designed to replace the previously heavily-proprietary GPU-driven version.
The Raspberry Pi family of single-board computers, and the low-cost CSI-connected Raspberry Pi Camera Modules, have proven popular — but they're not without their drawbacks. At launch, the project was criticized for relying on proprietary technologies, firmware, and software — a consequence of the system-on-chip having originally been developed at Broadcom for use in set-top boxes and other multimedia systems.
In the years since, the Raspberry Pi Foundation has been working to open as much of the platform as it can — and has now released an open source stack for the Raspberry Pi Camera Module family.
"We’ve had the building blocks for connecting other sensors and providing lower-level access to the image processing for a while, but Linux has been missing a convenient way for applications to take advantage of this," says engineer David Plowman of the Foundation's work. "In late 2018 a group of Linux developers started a project called libcamera to address that. We’ve been working with them since then, and we’re pleased now to announce a camera stack that operates within this new framework.
"We’ve supplied a Pipeline Handler that glues together our drivers and control algorithms, and presents them to libcamera with the API it expects."
The released stack provides Video for Linux 2 (V4L2) drivers for the Raspberry Pi Camera Module v1 and v2 - based on the Omnivision OV5647 and Sony IMX219 sensor modules respectively — but not, yet, the recently-released Raspberry Pi High Quality Camera Module. The release also includes control algorithms for features including automatic exposure control and white balance.
"Previously these functions were proprietary and closed source, and ran on the Broadcom GPU," Plowman explains. "Now, the GPU just shovels pixels through the ISP hardware block and notifies us when it’s done; practically all the configuration is computed and supplied from open source Raspberry Pi code on the Arm processor. A shim layer still exists on the GPU, and turns Raspberry Pi's own image processing configuration into the proprietary functions of the Broadcom SoC.
"So far as we know, there are no similar camera systems where large parts, including at least the control (3A) algorithms and possibly driver code, are not closed and proprietary. Indeed, for anyone wishing to customise a camera system — perhaps with their own choice of sensor — or to develop their own algorithms, there would seem to be very few options — unless perhaps you happen to be an extremely large corporation.
"In this respect," Plowman concludes, "the new Raspberry Pi Open Source Camera System is providing something distinctly novel. For some users and applications, we expect its accessible and non-secretive nature may even prove quite game-changing."
More details on the release can be found on the Raspberry Pi blog, while instructions for switching from the standard driver to libcamera and the newly opened stack are available on GitHub.