UNL phenotyping system going autonomous

Each year, the concrete track at the Nebraska Tractor Test Lab is used to test the latest model tractors — the lab has tested over 2,000 since it was founded in 1920. However, last year was the first time the track was used to test an autonomous machine.

This autonomous machine, nicknamed Flex Ro (for "Flexible Robot") is still in its early stages. But Santosh Pitla, University of Nebraska-Lincoln assistant professor of advanced machinery systems, hopes to have it in corn and soybean fields collecting data in the spring of 2019. Pitla has been working on the project for the last three years.

This high-clearance machine will be used not only for remote phenotyping (similar to UNL's Field Phenotyping Facility), but also for reacting to issues in the field in real-time — specifically, weeds. While no sensors are equipped on the machine yet, Pitla intends to use the same sensors found on the Spidercam at the one-acre Field Phenotyping Facility — including hyperspectral imagery, lidar, and normalized difference vegetation index (NDVI).

"We're talking about 20 machines out scouting, not just collecting imagery, but managing those issues. The idea is high-resolution management," he says.

Often, when field applications are made, pounds or gallons per acre are used. "I want to go down to milliliters or milligrams per plant – using high-resolution imagery to go to that small, per plant scale," Pitla says.

Equipped with a 46-horsepower Kubota industrial engine, Pitla notes the machine is capable of pulling a two-row planter unit and mounting a 100-to 150-gallon sprayer tank and four-nozzle boom. With an adjustable high-clearance platform, the machine can be used in crops up to 5 feet tall. Because it's mobile, it can cover hundreds of acres — and it isn't limited to a one-acre plot.

Size matters
Pitla and his fellow researchers in UNL's Department of Biological Systems Engineering designed the machine with a hydrostatic transmission. This drivetrain uses a hydraulic pump operated by the gas engine that's controlled through the CAN-bus. While hydraulic components are more easily integrated into the CAN (Controller Area Network), hydraulics are also five-to-ten times lighter than an electric motor. Because it's so compact, it has a more appropriate power density — or the power-to-weight ratio — for tasks such as phenotyping an entire field.

The flow from the hydraulic pump controls four 10-horsepower electric motors — one mounted on each of the machine's wheels. This way, each wheel can be programmed and controlled individually, with different steering modes — including four-wheel (crab) steering, differential steering (similar to a bulldozer) and normal tractor steering.

While the machine weighs around 3,800 pounds, it's still smaller than some of the modern 15- to 20-ton tractors found in fields today. So, it's easier to traverse fields of varying shapes.

Because of its smaller size, Pitla notes it will be easier to target weeds on a smaller scale with just four rows of spray nozzles operating at once.

"Machinery is getting bigger and bigger. Now we have section and individual nozzle control. It's because we're covering such a wide area, and we're varying the spray rate across the boom," Pitla says. "With only four nozzles, you don't need all of those electronics to vary the spray rate."

And, this machine is designed to be modular. Additional sensors and functions can be added on, and multiple machines can be used for different operations — including tillage.

It could even be used in tandem with UAVs.

"This machine has enough power you could use this as a mobile launch pad for UAVs," Pitla says. "There could be a UAV and a ground robot working in collaboration. This machine could get high-resolution imagery, while the UAV gets a bigger picture of the whole field."

Work in progress
Currently, the machine is operated manually by remote, but Pitla hopes to have it fully autonomous sometime in the next two to three years. However, this will take some fine-tuning.

"The big gap is on-board computing. You can put on cameras and lidar, but the big challenge is you need to compute data collected in real-time," Pitla says. "Similar to UAVs, the biggest hold-up is going from images to useful data."

One challenge is identifying objects — including stationary and moving obstacles, as well as large, slow-moving objects such as center pivots. Certain stationary obstacles can be pre-mapped for various weights and dimensions, but others will take machine learning algorithms — a predictive modeling technique.

Pitla is also working with students in UNL's Computer Science and Engineering Department and Agronomy and Horticulture Department to develop graphical processing units that can be used to identify different objects in the field. This includes different species of weeds, and mobile obstacles such as center pivots.

"We can take the model so that based on imagery, with a good amount of accuracy, we can say: this is the tower, this is the drive train and this is the truss," Pitla says. "So far, we've collected 2,000 images labeling those components. To train the model, we need images of objects from different angles."

In addition to identifying obstacles, what's referred to as local intelligence, each machine needs to have global intelligence — the ability to work in sync with other autonomous machines or UAVs. This includes algorithms for machine-to-machine communications under different scenarios where multiple machines are involved. This includes situations where one or two machines are offline for maintenance — the other machines will have to be rerouted to complete the task.

"If we're going to be truly autonomous, that's the system we're talking about," Pitla says. "All of this intelligence needs to be programmed into the machine."