By integrating these smart cameras into cars, we are looking at a future with fewer pedestrian accidents and safer streets for everyone.
In recent times, we have all become increasingly concerned about pedestrian safety at night. It is a topic that is making headlines worldwide, with a New York Times article shedding light on the alarming rise in nighttime pedestrian fatalities, a trend that has been on the increase since the 2010s. Shockingly, we are talking about approximately 700,000 pedestrian deaths each year, and a sizable portion of these tragedies happen when it is dark outside.
This growing concern is leading international regulatory bodies, not just in the United States but also in the EU and Asia-Pacific regions, to consider strict rules for car makers. These rules are all about making sure vehicles have advanced systems that can detect pedestrians, even in low-light conditions. The urgency of this issue is further underlined by Forbes magazine and Vox, which report on innovative solutions like thermal imaging, and the sobering fact that pedestrian deaths in the US have reached their highest point in 40 years.
So, what is at the core of these solutions? It is the complex technology of vehicle sensor suites. Modern vehicles are equipped with advanced driver assist systems (ADAS) that rely on a range of sensors to detect and classify objects, helping to make split-second decisions. These sensor systems include various types of cameras and radar, but they face a big challenge: performing effectively when it is dark, which is a significant factor in the recent rise in nighttime incidents. We are all in this together, working on solutions to keep our streets safer, especially when the sun goes down.
In a recent chat, Nidhi Agarwal – co-author of this article, spoke to Wade Appelman, Chief Business Officer at Owl Autonomous Imaging, about how thermal imaging technology can make a big difference in nighttime pedestrian safety within the automotive industry.
| Car companies adapting to new safety rules |
| To all car enthusiasts and concerned citizens, we are at a pivotal moment in the automotive industry. Faced with strict government safety mandates and the pressing need to protect pedestrians, automakers are navigating a challenging path. Here is a brief look at this evolving landscape: Cost concerns. The industry, traditionally cautious about costs, is being pushed to innovate while keeping production viable. Safety mandates. Global governments are stepping up with regulations that focus not just on passenger safety but also on protecting pedestrians. Historical lessons. Remember how seatbelts and airbags became standard? They were also born from regulatory pressures. Technological leap. Automakers are turning to technologies like thermal imaging to meet these new safety standards—a potential game-changer for night-time pedestrian safety. Future outlook. By 2028, we might see a significant shift with many vehicles equipped with this life-saving technology. As stakeholders in road safety, whether as drivers, pedestrians, or enthusiasts, we are witnessing a crucial transition. Automakers, balancing costs and compliance, are embracing innovation to make our roads safer for everyone. Let’s stay tuned to see how this journey unfolds, driving us toward a safer future. |
How smart cameras make cars safer?
In the discussion, they concluded that it was all about the perfect blend of hardware and software that was driving some amazing breakthroughs in this field. At the heart of this innovation is a thermal camera that works hand in hand with advanced software modules known as convolutional neural networks (CNN). These components are up and running smoothly on Nvidia Jetson Orin processors, making it a practical and efficient solution for real-world applications. Let’s dive into the important components and capabilities:
Smart neural networks
The software in these cars can do some pretty amazing stuff like classifying images, segmenting them, and even determining distances. It is like having a super-smart co-pilot that understands everything around you in real time.
ROS integration
These cars talk in a language called ‘ROS topics,’ making it easier to turn complex data into actionable insights. Think of it as the car translating its surroundings into helpful advice for safer driving.
Versatile outputs
Imagine your car having multiple vision modes—from basic thermal views to 3D object listing, and even combining regular and thermal imaging. It’s like having night vision goggles, giving you a clear view in low light and detailed colour information.
Efficient data management
These cars are not just smart, they’re efficient too! They focus on processing only the important stuff, which means quicker decisions and less power use. It’s all about being fast and efficient without draining the battery.
Monocular vision in imaging technology
Imagine having just one eye but still being able to judge distances and depths accurately. That is essentially what monocular vision technology does. It takes a single image from a camera and, using some clever neural network algorithms, figures out how far away things are in that image.
Now, you might be thinking ‘how does a car learn this stuff?’ It is all about training these algorithms with loads of correct distance data, a method known as ‘ground truth.’ This training is crucial; almost like giving the car a crash course in depth perception.
Tesla is doing this brilliantly. They use this one-eye wonder to navigate roads and avoid obstacles. It is a big deal because it shows that even with just one camera, cars can get a good sense of depth and distance, making our rides safer and the future of driving super exciting. So next time you see a Tesla or any autonomous car, remember, it has got this cool monocular vision tech watching the road!
How are safety cameras built?
The advanced technology involved in thermal cameras is incredible, particularly with respect to its potential to enhance pedestrian safety and withstand harsh environments. Let’s delve into the core components and assembly process:
Readout IC and microbolometer
It all starts with the readout IC, a tiny yet powerful semiconductor. Picture a tiny chip housing a million converters, each working to interpret temperature data from the microbolometer. The microbolometer, which is key for reading temperatures, consists of MEMS structures finely aligned with the readout IC.
Chip design and wafer scale packaging
Designing this chip was no small feat. It took years due to its complexity. Built in a semiconductor foundry, this chip is a marvel of modern engineering, combining those million converters and MEMS structures into what we call the focal plane array.
Size and scalability
Despite its high-tech innards, this camera is incredibly compact, about the size of a US quarter. But don’t let its size fool you. To get this chip up and running, it needs to be integrated with other electronics on a board, leading to an efficient and cost-effective camera core.
Final product assembly
The final step is placing this core on a PCB board, adding a lens, and getting it ready to process signals.
The challenge? Packing all this technology into a tiny space without driving up power use and costs.
The end goal of all this tech? To bring pedestrian fatalities down to zero. Big names in the automotive world, like Toyota with their ‘Vision Zero’ initiative, are on board, aiming for ultra-safe vehicles by 2030.
The key to success? Making this tech affordable and widespread. That is why spreading the word among car makers, suppliers, and even everyday drivers is crucial.
| Cars of tomorrow |
| Envision a future where driving is safer than ever, thanks to a network of advanced sensors on our vehicles. Here’s a quick look at how thermal cameras, a key part of this network, will transform our driving experience: No more blinding lights. You know how challenging it can be to see with bright headlights in your mirrors? Thermal cameras will cut through that glare, giving you a clear view of motorcycles, cars, and other objects approaching from the rear or sides. Safer urban drives. In city driving, thermal cameras will be your extra set of eyes, spotting pedestrians, cyclists, and oncoming vehicles, helping you navigate crowded streets with ease. All-around safety ‘cocoon.’ Imagine your car surrounded by a protective ‘cocoon’ made of thermal imaging, RGB cameras, and radar. This combination will drastically improve safety for everyone, inside and outside the car. This vision of enhanced safety through technology isn’t just a dream—it is on the horizon. As these advancements unfold, we’re moving towards a world where driving is more secure and accidents are significantly reduced. |
Road safety camera test: A case study
The recent demonstration in Detroit, Michigan, was a significant milestone in showcasing the effectiveness of a state-of-the-art camera system integrated with neural network software. This real-life test, conducted at a test track, was aligned with the expected standards of the National Highway Transportation Safety Authority (NHTSA) in the United States.
Picture this
A car equipped with this high-tech camera system, a test driver, and a pedestrian dummy, all on a test track at night with headlights coming at them.
The big challenge? To see if the camera system could spot the pedestrian and stop the car on its own.
And guess what? It worked! The car detected a child-sized dummy and stopped a safe 6.8 metres away, even in the dark, cold, and windy conditions.
But that’s not all. Over in Las Vegas, with its bright lights, another test compared this thermal imaging technology with regular RGB (visible light) imaging. The goal was to see which was better at spotting pedestrians, especially in tricky light conditions.
Turns out, thermal imaging was the clear winner, especially in spotting pedestrians that might be hidden or missed by regular cameras due to backlighting. It even outperformed other tech like lidar and radar in certain situations, like detecting someone near a reflective surface or wearing dark clothing.
So, what does this mean for us? This technology could be a game-changer in making our roads safer, especially in busy urban areas where it’s hard to see pedestrians. By integrating these smart cameras into cars, we are looking at a future with fewer pedestrian accidents and safer streets for everyone.
The recent advancements in imaging technology, particularly thermal imaging combined with neural network software, bring us a step closer to a safer world. Through engaging discussions and eye-opening demonstrations in Detroit and Las Vegas, we’ve seen firsthand the power of this technology to pierce through the veil of night, offering a beacon of hope in low-visibility conditions. These tests are not just technical achievements; they represent a significant leap over traditional imaging and sensor systems in critical safety scenarios. The integration of this advanced technology into vehicles marks the beginning of a transformative era in automotive safety, one that aligns with the goals of global leaders and regulatory bodies. This journey, fuelled by innovation and a collective dedication to saving lives, promises a future where our roads would be safer for pedestrians and drivers alike, especially during the night. It is paving the way to significantly reduce nighttime pedestrian fatalities, making our streets a safer place for everyone.
The Co-author Nidhi Agarwal is a journalist at EFY. She is an Electronics and Communication Engineer with over five years of academic experience. Her expertise lies in working with development boards and IoT cloud. She enjoys writing as it enables her to share her knowledge and insights related to electronics with like-minded techies.
