Multi-Input Recording
High-bandwidth data capture for validation and AI training
Description
Modern cars comprise multiple cameras as sensors in the visual and infrared spectra with increasing resolutions and frames-per-second requirements. Likewise, robotics and industrial applications are propelled by AI-based classification and tracking also utilizing cameras as sensors. Increasing resolutions, increasing frames per second (FPS) and an increasing number of cameras all have an impact on throughput and recording bandwidth for scenes applicable for validation or AI training. PLC2 has gained a lot of video-recording experience in multiple development service projects and also by developing own products.
Methodology
Automotive applications have pushed the simplification of camera interfaces and manufacturers of integrated circuits have responded with serial protocols and 2-wire cables that can handle DC power supply, video data transfers, configuration channels and even discrete GPIO signals by clever modulation techniques. The required serializer / deserializer chips convert from and to MIPI-CSI and an FPGA first has to convert from MIPI to an internal protocol such as AXI4-Stream. Data formatting so that recorded scenes can be stored efficiently is supported by PLC2’s L5 compression algorithm that compresses video linewise with very low latency. Images are typically stored in local DDR-RAM to decouple input from output timing. Data can then be transferred over PCIe or Multi-10 Gbps Ethernet in suitable protocols to logging devices such as PLC2’s ADL-1000 data logger which is also optimized to handle frame-based video data but can also record from other sources such as CAN to Ethernet bridges or Radar and Lidar data.
When 2-wire interfaces are not absolutely required for cameras, then industrial cameras with GigE Vision interfaces can be the proper choice. Their output can be directly recorded by ADL-1000 over 1-10 Gbps Ethernet interfaces. AI applications can significantly enhance the recording device, since they can classify image content and serve as smart trigger engines to record only those valuable scenes that are really worth recording.
The block diagram depicts a typical video and data recording solution from robotics or automotive validation. High speed data is acquired from diverse sources. ADL-1000 is used as a data recorder that records with high bandwidth including metadata such as timestamps to solid state disks (SSDs).
Conclusion
PLC2 has gained profound experience with vision-based applications involving serial protocols (GSMLx, FPDx, …), MIPI-CSI, compression, GigE Vision, RTSP, AI-based classification, recording in .mdf4 or rosbag formats, PTP timestamping, etc. PLC2 supports customers across a wide range of development and integration aspects.
