The Self-Driving Car’s Bicycle Problem

6210901187_dc5fc87af0_b

Human error plays a role in 94% of U.S. traffic fatalities. Image: William Murphy via Flickr

Robotic cars are great at monitoring other cars, and they’re getting better at noticing pedestrians, squirrels, and birds. The main challenge, though, is posed by the lightest, quietest, swerviest vehicles on the road.

“Bicycles are probably the most difficult detection problem that autonomous vehicle systems face,” says UC Berkeley research engineer Steven Shladover.

Nuno Vasconcelos, a visual computing expert at the University of California, San Diego, says bikes pose a complex detection problem because they are relatively small, fast and heterogenous. “A car is basically a big block of stuff. A bicycle has much less mass and also there can be more variation in appearance — there are more shapes and colors and people hang stuff on them.”

That’s why the detection rate for cars has outstripped that for bicycles in recent years. Most of the improvement has come from techniques whereby systems train themselves by studying thousands of images in which known objects are labeled. One reason for this is that most of the training has concentrated on images featuring cars, with far fewer bikes.

Consider the Deep3DBox algorithm presented recently by researchers at George Mason University and stealth-mode robotic taxi developer Zoox, based in Menlo Park, Calif. On an industry-recognized benchmark test, which challenges vision systems with 2D road images, Deep3DBox identifies 89 percent of cars. Sub-70-percent car-spotting scores prevailed just a few years ago.

Deep3DBox further excels at a tougher task: predicting which way vehicles are facing and inferring a 3D box around each object spotted on a 2D image. “Deep learning is typically used for just detecting pixel patterns. We figured out an effective way to use the same techniques to estimate geometrical quantities,” explains Deep3DBox contributor Jana Košecká, a computer scientist at George Mason University in Fairfax, Virginia.

However, when it comes to spotting and orienting bikes and bicyclists, performance drops significantly. Deep3DBox is among the best, yet it spots only 74 percent of bikes in the benchmarking test. And though it can orient over 88 percent of the cars in the test images, it scores just 59 percent for the bikes.

Košecká says commercial systems are delivering better results as developers gather massive proprietary datasets of road images with which to train their systems. And she says most demonstration vehicles augment their visual processing with laser-scanning (ie lidar) imagery and radar sensing, which help recognize bikes and their relative position even if they can’t help determine their orientation.

Further strides, meanwhile, are coming via high-definition maps such as Israel-based Mobileye’s Road Experience Management system. These maps offer computer vision algorithms a head start in identifying bikes, which stand out as anomalies from pre-recorded street views. Ford Motor says “highly detailed 3D maps” are at the core of the 70 self-driving test cars that it plans to have driving on roads this year.

Put all of these elements together, and one can observe some pretty impressive results, such as the bike spotting demonstrated last year by Google’s vehicles. Waymo, Google’s autonomous vehicle spinoff, unveiled proprietary sensor technology with further upgraded bike-recognition capabilities at this month’s Detroit Auto Show.

Vasconcelos doubts that today’s sensing and automation technology is good enough to replace human drivers, but he believes they can already help human drivers avoid accidents. Automated cyclist detection is seeing its first commercial applications in automated emergency braking systems (AEB) for conventional vehicles, which are expanding to respond to pedestrians and cyclists in addition to cars.

Volvo began offering the first cyclist-aware AEB in 2013, crunching camera and radar data to predict potential collisions; it is rolling out similar tech for European buses this year. More automakers are expected to follow suit as European auto safety regulators begin scoring AEB systems for cyclist detection next year.

That said, AEB systems still suffer from a severe limitation that points to the next grand challenge that AV developers are struggling with: predicting where moving objects will go. Squeezing more value from cyclist-AEB systems will be an especially tall order, says Olaf Op den Camp, a senior consultant at the Dutch Organization for Applied Scientific Research (TNO). Op den Camp, who led the design of Europe’s cyclist-AEB benchmarking test, says that it’s because cyclists movements are especially hard to predict.

Košecká agrees: “Bicycles are much less predictable than cars because it’s easier for them to make sudden turns or jump out of nowhere.”

That means it may be a while before cyclists escape the threat of human error, which contributes to 94 percent of traffic fatalities, according to U.S. regulators. “Everybody who bikes is excited about the promise of eliminating that,” says Brian Wiedenmeier, executive director of the San Francisco Bicycle Coalition. But he says it is right to wait for automation technology to mature.

In December, Wiedenmeier warned that self-driving taxis deployed by Uber Technologies were violating California driving rules designed to protect cyclists from cars and trucks crossing designated bike lanes. He applauded when California officials pulled the vehicles’ registrations, citing the ridesharing firm’s refusal to secure state permits for them. (Uber is still testing its self-driving cars in Arizona and Pittsburgh, and it recently got permission to put some back on San Francisco streets strictly as mapping machines, provided that human drivers are at the wheel.)

Wiedenmeier says Uber’s “rush to market” is the wrong way to go. As he puts it: “Like any new technology this needs to be tested very carefully.”

This post was created for Cars That Think, IEEE Spectrum’s blog about the sensors, software, and systems that are making cars smarter, more entertaining, and ultimately, autonomous.

Censors Take On China’s Silent Spring Moment

Jinhua skyline 2005c

Jinhua skyline, 2005

Chinese censors took down a hugely popular documentary on China’s air pollution crisis this past weekend, according to reports by the Wall Street Journal and the New York Times. Under the Dome, a polished, 104-minute report by Chinese broadcast journalist Chai Jing [embedded below], had gone viral after its release last week, attracting several hundred million views in China before censors restricted domestic access to the video and squelched news coverage of it.

The film is a damning account of China’s declining air quality, the sources of its pollution, and the toothlessness of environmental agencies charged with controlling it. It’s a wide-ranging production that tries to explain the price China has paid for its industrialization and wealth generation, as well as a passionate call to action.

For me, the film’s visceral portrayal of contemporary life amidst smog—and the movie’s historic sweep—sparked flashbacks to my own discomfort breathing in Chinese air during visits in 1991, 2005, and 2006.

In 1991, my eyes burned as the aging cruise liner I’d taken over from Japan motored up the Huangpu River, past the petrochemical plants then lining the river’s eastern banks, on its way into Shanghai. But the historic city across the river was clean. Aside from a few buses, it was a city that still moved on pollution-free pedal power, its streets a flood of bicycles. And as I traveled inland for several weeks, the pollution faded further, revealing China’s natural beauty.

When I flew into Shanghai 14 years later to report on China’s rising tide of electric bicycles for IEEE Spectrum, Shanghai itself seemed still cleaner than I’d recalled. While cars and trucks were on the rise, the East-bank industry had been cleared to make way for gleaming skyscrapers.

But China was clearly changing. I visited smaller cities where smog nearly blocked out the sun. Continue reading

Renewables to Dethrone Nuclear Under French Energy Plan

After months of negotiation, the French government has unveiled a long-awaited energy plan that is remarkably true to its election promises. The legislation’s cornerstone is the one-third reduction in the role of nuclear power that President François Hollande proposed on the campaign trail in 2012.

Under the plan, nuclear’s share of the nation’s power generation is to drop from 75 percent to 50 percent by 2025, as renewable energy’s role rises from 15 percent today to 40 percent to make up the difference. That is a dramatic statement for France, which is the world’s second largest generator of nuclear energy, after the United States. France has a globally-competitive nuclear industry led by state-owned utility Electricité de France (EDF) and nuclear technology and services giant Areva. Continue reading

How Canada Should Return Obama’s Oil Pipeline Punt

Late last week President Barack Obama deferred consideration of the Keystone XL oil pipeline, designed to ship Alberta petroleum to the Gulf Coast, until after next year’s U.S. elections. Obama’s move immediately sparked vows in Canada to redirect crude exports to Asian markets less angst-ridden by the environmental impacts associated with tapping Alberta’s tough, tarry petroleum. A smarter strategy would be to reduce those impacts, starting with the black mark that brought Keystone XL to national attention: oil sands crude’s bloated carbon footprint. Continue reading

Chinese Bullet Trains’ Worrisome “Black-box” Controls

In August we brought you disquieting news that Hollysys Automation — the supplier of a control system implicated in China’s deadly bullet-train collision this summer — also provides controls for China’s nuclear reactors (which are multiplying just as fast as its high speed rail lines). The Hollysys story now looks darker after informed speculation reported in the Wall Street Journal that the company may not fully comprehend how the control systems work. Continue reading

Nuclear Safety Implications in China’s Bullet Train Wreck?

The hand-wringing over China’s high-speed train wreck last month may have just begun if the government’s current explanation for the crash proves out. At present official fingers are pointing to a failure in the trains’ signaling system. The firm that installed them, it now appears, provides similar equipment for the nuclear reactors that China is building just as fast as it is adding rail lines. Continue reading

Compressed-air Car Proponents Losing Faith

Licensees of the much-hyped AirPOD minicar are pressing for results from Motor Development International, the Luxembourg-registered firm behind the compressed-air-powered vehicle. In recent postings to their websites and coverage by European news sources, some of MDI’s partners are now openly questioning the technology and MDI’s capacity to develop it — questions that Spectrum raised in November 2009 in the investigative feature, “Deflating the Air Car.”

When Spectrum’s feature went to print, MDI was guaranteeing mass-production of AirPODs within a few months at its development base on France’s Cote d’Azur. A year and a half later there is no sign of the promised minicars and their advertised 140-kilometer range, and outspoken licensees are blaming MDI. Continue reading