Agriculture-Vision

Agriculture-Vision Dataset, Challenge and Workshop (CVPR 2020)

A joint effort with many great collaborators to bring Agriculture and Computer Vision / AI communities together to benefit humanity!

Update:

The 2nd Agriculture-Vision Prize Challenge has started and new deadline is June 5, 2021. Total Prize: 20,000$.

Content

• Papers
• Dataset
  • Agriculture-Vision Challenge dataset
  • Download
• Evaluation
  • Evaluation Metric
  • Submission
• Challenge
  • Teams and Methods
  • Challenge leaderboard
• Workshop
  • Schedule
  • Paper Track
  • Accepted Papers
  • Accepted Posters

Papers:

CVPR paper on Agriculture-Vision
ArXiv, CVPR 2020 open access

@article{chiu2020agriculture,
  title={Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis},
  author={Chiu, Mang Tik and Xu, Xingqian and Wei, Yunchao and Huang, Zilong and Schwing, Alexander and Brunner, Robert and Khachatrian, Hrant and Karapetyan, Hovnatan and Dozier, Ivan and Rose, Greg and others},
  journal={arXiv preprint arXiv:2001.01306},
  year={2020}
}

@InProceedings{Chiu_2020_CVPR,
author = {Chiu, Mang Tik and Xu, Xingqian and Wei, Yunchao and Huang, Zilong and Schwing, Alexander G. and Brunner, Robert and Khachatrian, Hrant and Karapetyan, Hovnatan and Dozier, Ivan and Rose, Greg and Wilson, David and Tudor, Adrian and Hovakimyan, Naira and Huang, Thomas S. and Shi, Honghui},
title = {Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

CVPR Workshop paper on the 1st Agriculture-Vision Challenge (challenge dataset, methods and results)
ArXiv, CVPR 2020 open access

@article{chiu20201st,
  title={The 1st Agriculture-Vision Challenge: Methods and Results},
  author={Chiu, Mang Tik and Xu, Xingqian and Wang, Kai and Hobbs, Jennifer and Hovakimyan, Naira and Huang, Thomas S. and Shi, Honghui and others},
  journal={arXiv preprint arXiv:2004.09754},
  year={2020}
}

@InProceedings{Chiu_2020_CVPR_Workshops,
author = {Chiu, Mang Tik and Xu, Xingqian and Wang, Kai and Hobbs, Jennifer and Hovakimyan, Naira and Huang, Thomas S. and Shi, Honghui},
title = {The 1st Agriculture-Vision Challenge: Methods and Results},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2020}
}

Dataset

Agriculture-Vision Challenge Dataset

The dataset used in this challenge is a subset of the Agriculture-Vision dataset. The challenge dataset contains 21,061 aerial farmland images captured throughout 2019 across the US. Each image consists of four 512x512 color channels, which are RGB and Near Infra-red (NIR). Each image also has a boundary map and a mask. The boundary map indicates the region of the farmland, and the mask indicates valid pixels in the image. Regions outside of either the boundary map or the mask are not evaluated.

This dataset contains six types of annotations: Cloud shadow, Double plant, Planter skip, Standing Water, Waterway and Weed cluster. These types of field anomalies have great impacts on the potential yield of farmlands, therefore it is extremely important to accurately locate them. In the Agriculture-Vision dataset, these six patterns are stored separately as binary masks due to potential overlaps between patterns. Users are free to decide how to use these annotations.

Each field image has a file name in the format of (field id)_(x1)-(y1)-(x2)-(y2).(jpg/png). Each field id uniquely identifies the farmland that the image is cropped from, and (x1, y1, x2, y2) is a 4-tuple indicating the position in which the image is cropped. Please refer to our paper for more details regarding how we construct the dataset.

Download

The challenge dataset contains images, boundaries and masks for train, val and test set. It also contains labels for the train and val set only. The 2021 new dataset .tar.gz file is around 20 GB. Please visit this page to get access. By downloading or using the dataset, user signifies its agreement to Agriculture-Vision Workshop Terms and Conditions.

Evaluation

Evaluation metric

We use mean Intersection-over-Union (mIoU) as our main quantitative evaluation metric, which is one of the most commonly used measures in semantic segmentation datasets. The mIoU is computed as:

Where c is the number of annotation types (c = 7 in our dataset, with 6 patterns + background), P_c and T_c are the predicted mask and ground truth mask of class c respectively.

Since our annotations may overlap, we modify the canonical mIoU metric to accommodate this property. For pixels with multiple labels, a prediction of either label will be counted as a correct pixel classification for that label, and a prediction that does not contain any ground truth labels will be counted as an incorrect classification for all ground truth labels.

Concretely, we construct the confusion matrix M^c×c with the following rules:

For each prediction x and label set Y:

1. If x ∈ Y, then M_y,y = M_y,y + 1 for each y in Y
   
2. Otherwise, M_x,y = M_x,y + 1 for each y in Y
   

The mIoU is finally computed by (true_positive) / (prediction + target - true_positive), averaged across all classes.

Submission

Update:

• The 1st Agriculture-Vision Prize Challenge has ended on April 20, 2020 (10:00 AM PDT). The final results as of the end date of the challenge can be found in these links (png, csv).
• The Codalab evaluation server will remain open to encourage further research in Agriculture-Vision. No registration is required.

Registration

We are now hosting our evaluation/competition on Codalab. The competition page can be found here. Each participating team is required to register for the challenge. To register your team, fill out the registration form here, then register on the competition page.

*Make sure your Codalab account email matches one of the member emails in the registration form. Each team can only register once.

Codalab submission

All registered teams can evaluate their results on Codalab and publish their results on the leaderboard. The submission file should be a compressed .zip file that contains all prediction images. All prediction images should be in png format and the file names and image sizes should match the input images exactly. The prediction images will be converted to a 2D numpy array with the following code:

numpy.array(PIL.Image.open(‘field-id_x1-y1-x2-y2.png’))

In the loaded numpy array, only 0-6 integer labels are allowed, and they represent the annotations in the following way:

0 - background
1 - cloud_shadow
2 - double_plant
3 - planter_skip
4 - standing_water
5 - waterway
6 - weed_cluster

This label order will be strictly followed during evaluation.

All teams can have 2 submissions per day and 20 submissions in total.

Challenge

We hosted the 1st Agriculture-Vision Prize Challenge with the following terms:

• Model size will be limited below 150M parameters in total.
• The mIoU derived from the "results/" folder in the final submission should match the mIoU on the leaderboard.
• Predictions in "results/" in the final submission can be reproduced with the resources in "code/" and "models/".
• The training process of the method can be reproduced and the retrained model should have a similar performance.
• The test set is off-limits in training.
• For fairness, teams need to specify what public datasets are used for training/pre-training their models in their challenge_ report.pdf. Results that are generated from models using private datasets, and results without such details will be excluded from prize evaluation. (Results using private datasets can still be included in the report.)

Teams and Methods

Please refer to our workshop summary paper for more details regarding notable methods. Selected teams & methods:

Team Hyunseong: Hyunseong Park, Junhee Kim, Sungho Kim (Agency for Defense Development, South Korea)

• Method: Residual DenseNet with Expert Network for Semantic Segmentation

Team SCG Vision: Qinghui Liu, Michael C. Kampffmeyer, Robert Jenssen, Arnt B. Salberg (Norwegian Computing Center, UiT The Arctic University of Norway)

• Method: Multi-view Self-Constructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentation

Team AGR: Alexandre Barbosa, Rodrigo Trevisan (University of Illinois at Urbana Champaign)

• Method: Techniques for Improving Aerial Agricultural Image Semantic Segmentation

Team TJU: Bingchen Zhao, Shaozuo Yu, Siwei Yang, Yin Wang (Tongji University)

• Method: Reducing the feature divergence of RGB and near-infrared images using Switchable Normalization

Team Haossr: Hao Sheng, Xiao Chen, Jingyi Su, Ram Rajagopal, Andrew Ng (Stanford University, Chegg, Inc.)

• Method: Effective Data Fusion with Generalized Vegetation Index: Evidence from Land Cover Segmentation in Agriculture

Team CNUPR_TH2L: Van Thong Huynh, Soo-Hyung Kim, In-Seop Na (Chonnam National University, Chosun University)

• Method: Multi-spectra Attention in Aerial Agricultural Segmentation

Team TeamTiger: Ujjwal Baid, Shubham Innani, Prasad Dutande, Bhakti Baheti, Sanjay Talbar (SGGS Institute of Engineering and Technology)

• Method: Eff-PN: A Novel Deep Learning Approach for Anomaly Pattern Segmentation in Aerial Agricultural Images

Challenge leaderboard

Workshop

Schedule

(Sunday, Jun 14 PDT)

08:30 – 08:40 Opening Remarks and Welcome

Humphrey Shi (University of Oregon)

Naira Hovakimyan (UIUC)

08:40 – 09:00 Invited Talk 1: Scaling Spatio-Temporal Analytics: A Case in Agricultural Insights (Talk video here)

Sharath Pankanti (IBM T.J. Watson Research Center)

09:00 – 09:20 Invited Talk 2: Multi-modality Remote Sensing in High Throughput Phenotyping: Opportunities for Machine Learning (Talk video here)

Melba M. Crawford, Edward J. Delp (Purdue)

09:20 – 09:40 Invited Talk 3: Bridging Application and Research. Is agriculture just another application? (Talk video here)

Jennifer Hobbs (IntelinAir Inc.)

09:40 – 10:00 Invited Talk 4: Improving Visual and Speech Recognition on Out-Domain Data (Talk video here)

Liangliang Cao (Google Inc. and Umass)

10:00 – 10:20 Invited Talk 5: Learning to Anticipate (Talk video here)

Alex Schwing (UIUC)

10:20 – 11:10 Poster Session 1

11:10 – 11:20 Oral 1: MSCG-Net with Adaptive Class Weighting Loss for Semantic Segmentation (Talk video here)
11:20 – 11:30 Oral 2: Finding Berries: Segmentation and Counting of Cranberries using Point Supervision and Shape Priors (Talk video here)
11:30 – 11:40 Oral 3: Visual 3D Reconstruction and Dynamic Simulation of Fruit Trees for Robotic Manipulation (Talk video here)
11:40 – 11:50 Oral 4: Cross-Regional Oil Palm Tree Detection (Talk video here)
11:50 – 11:00 Oral 5: Effective Data Fusion with Generalized Vegetation Index (Talk video here)
11:00 – 12:10 Oral 6: Weakly Supervised Learning Guided by Activation Mapping Applied to a Novel Citrus Pest Benchmark (Talk video here)
12:10 – 12:20 Oral 7: Fine-Grained Recognition in High-throughput Phenotyping (Talk video here)
12:20 – 12:30 Oral 8: Climate Adaptation: Reliably Predicting from Imbalanced Satellite Data (Talk video here)

12:30 – 13:30 Lunch Break

13:30 – 13:50 Invited Talk 6: Towards Understanding Agricultural Systems at Scale using Machine Learning (Talk video here)

Stefano Ermon (Stanford)

13:50 – 14:10 Invited Talk 7: Interactive Object Segmentation (Live*) (Talk video here)

Yunchao Wei (University of Technology Sydney)

14:10 – 14:30 Invited Talk 8: Project FarmBeats: Aerial Mapping forAgricultural Farms and Beyond (Live*) (Talk video here)

Ranveer Chandra, Sudipta Sinha (Microsoft)

14:30 – 14:50 Invited Talk 9: Enabling the African Farmer (Live*) (Talk video here)

Munther Dahleh (MIT)

14:50 – 15:10 Invited Talk 10: (Live*)

Zsolt Kira (Georgia Tech)

15:10 – 15:55 Poster Session 2

15:55 – 16:10 Agriculture-Vision Prize Challenge Results Summary (Talk video here)

16:10 – 16:20 Challenge Presentation 1: Residual DenseNet with Expert Network for Semantic Segmentation

16:20 – 16:30 Challenge Presentation 2: MSCG-Net Models for The 1st Agriculture-Vision Challenge

16:30 – 17:20 Panel Discussion: Challenge & Opportunities for Computer Vision in Agriculture (video here)

Host:
Naira Hovakimyan (UIUC)

Panelists:
Sharath Pankanti (IBM T. J. Watson Research Center)

Liangliang Cao (Google Research)

Ranveer Chandra (Microsoft Azure Global)

Sudipta Sinha (Microsoft)

Munther Dahleh (MIT)

Ed Delp (Purdue)

Melba Crawford (Purdue)

Jennifer Hobbs (Intelinair Inc.)

Yunchao Wei (University of Technology Sydney)

Alexander Schwing (UIUC)

Zsolt Kira (Georgia Tech)

David Wilson (Intelinair Inc.)

Jim Yuan (Google X)

Humphrey Shi (U of Oregon)

17:20 – 17:30 Closing Remarks

Humphrey Shi (University of Oregon)

Naira Hovakimyan (UIUC)

Paper Track

• We accept papers related to Agriculture-Vision with a rigorous peer-review process with program committee members from multiple research communities including computer vision, machine learning, image processing, remote sensing, agriculture etc.
• All accepted paper will be published in IEEE CVPR 2020 Workshop Proceedings.
• Several top accepted papers will be given oral presentation opportunity at our workshop, the rest accepted papers will be given poster presentation slots during the workshop.

Topic Description

The Workshop will be open to the broader community addressing various technical and application aspects of challenges and opportunities in computer vision research for agriculture. We aim to provide a venue to both show current relevant efforts in interdisciplinary areas between computer vision and agriculture, and to encourage further research and conversations within the computer vision community to tackle impactful agriculture-vision problems. We will solicit papers from the following wide range of topics:

• Computer vision research for agricultural imagery in general
• Resources and benchmarks for agricultural imagery based pattern analysis
• Farmland pattern classification and segmentation from LSHR aerial imagery
• Efficient data sampling methods for effective training on agricultural imageries
• Effective data fusion of multi-spectral image data such as RGB and Near-Infrared (RGB-NIR)
• Robust and stable pattern recognition on sparse and imbalanced annotations
• Other novel vision applications in agriculture

Accepted Papers

• Multi-view Self-Constructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentation (Oral),
  Qinghui Liu, Michael C. Kampffmeyer, Robert Jenssen, Arnt B. Salberg (Norwegian Computing Center, UiT The Arctic University of Norway)

• Reducing the feature divergence of RGB and near-infrared images using Switchable Normalization (Poster),
  Siwei Yang*, Shaozuo Yu*, Bingchen Zhao*, Yin Wang (* equal contribution)(Tongji University)

• Finding Berries: Segmentation and Counting of Cranberries using Point Supervision and Shape Priors (Oral),
  Peri Akiva, Kristin Dana, Peter Oudemans, Michael Mars (Rutgers University)

• Leaf Spot Attention Network for Apple Leaf Disease Identification (Poster),
  Hee-Jin Yu, Chang-Hwan Son (Kunsan National University)

• Visual 3D Reconstruction and Dynamic Simulation of Fruit Trees for Robotic Manipulation (Oral),
  Francisco J Yandun, Abhisesh Silwal; George Kantor (Carnegie Mellon University)

• Cross-Regional Oil Palm Tree Detection (Oral),
  Wenzhao Wu, Juepeng Zheng, Haohuan Fu, Weijia Li, Le Yu (Tsinghua University, The Chinese University of Hong Kong)

• Multi-Stream CNN for Spatial Resource Allocation: a Crop Management Application (Poster),
  Alexandre Barbosa, Thiago Marinho, Nicolas Martin, Naira Hovakimyan (University of Illinois at Urbana-Champaign)

• Effective Data Fusion with Generalized Vegetation Index: Evidence from Land Cover Segmentation in Agriculture (Oral),
  Hao Sheng, Xiao Chen, Jingyi Su, Ram Rajagopal, Andrew Ng (Stanford University)

• Deep Transfer Learning For Plant Center Localization (Poster),
  Enyu Cai, Sriram Baireddy, Changye Yang, Melba Crawford, Edward Delp (Purdue University)

• Segmentation and detection from organised 3D point clouds: a case study in broccoli head detection (Poster),
  Justin Le Louëdec, Hector A. Montes, Tom Duckett, Grzegorz Cielniak (University of Lincoln)

• Deep Learning based Corn Kernel Classification (Poster),
  Henry Velesaca, Raúl Mira, Patricia L Suarez, Christian Larrea, Angel Sappa (ESPOL, Computer Vision Center, Spain)

• Improving In-field Cassava Whitefly Pest Surveillance with Machine Learning (Poster),
  Jeremy Tusubira, Solomon nsumba, Flavia Ninsiima, Benjamin Akera, Guy Acellam, Joyce Nakatumba-Nabende, Ernest Mwebaze, John A Quinn, Tonny Oyana (Makerere University, Google)

• Weakly Supervised Learning Guided by Activation Mapping Applied to a Novel Citrus Pest Benchmark (Oral),
  Edson Riberto Bollis, Helio Pedrini, Sandra Avila (UNICAMP)

• Fine-Grained Recognition in High-throughput Phenotyping (Oral),
  Beichen Lyu, Stuart Smith, Keith Cherkauer (Purdue University)

• A Novel Technique Combining Image Processing, Plant Development Properties, and the Hungarian Algorithm, to Improve Leaf Detection in Maize (Poster),
  Nazifa Khan, Ian McQuillan, Mark Eramian, Oliver A.S. Lyon (University of Saskatchewan, Queen's University)

• Farm Parcel Delineation Using Spatio-temporal Convolutional Networks (Poster),
  Han Lin Aung, Burak Uzkent, Marshall Burke, David Lobell, Stefano Ermon (Stanford University)

• Climate Adaptation: Reliably Predicting from Imbalanced Satellite Data (Oral),
  Ruchit Rawal, Prabhu Pradhan (NSIT, Max Planck Institute for Intelligent Systems)

Accepted Posters

• Scoring Root Necrosis in Cassava Using Semantic Segmentation,
  Joyce Nakatumba-Nabende, Jeremy Tusubira, Benjamin Akera, Ernest Mwebaze (Makerere University, Google)

• Active Learning Sampling Strategies For Weed Segmentation,
  Andrei Polzounov, Anshul Tibrewal, Ben Cline, Olgert Denas, Chris Padwick, Jim Ostrowski (Blue River Technology)

• Convolutional Neural Networks for Image-based Corn Kernel Detection and Counting,
  Saeed Khaki, Hieu Pham, Ye Han, Andy Kuhl, Wade Kent, Lizhi Wang (Iowa State University, Syngenta)

• Eff-PN: A Novel Deep Learning Approach for Anomaly Pattern Segmentation in Aerial Agricultural Images,
  Shubham Innani, Prasad Dutande, Ujjwal R Baid, Bhakti Vilas Baheti, Sanjay Talbar (SGGSIET Nanded)

• Deep Unsupervised Deblurring Approach for Improving Crops Disease Classification,
  Javed Ahmad, Fahad Shamshad, Junaid Maqbool, Ali Ahmed (Information Technology University)

• NDVI Vegetation Index Estimation Through an\ Unsupervised Deep Learning based Approach,
  Patricia L Suarez, Angel Sappa, Boris Vintimilla (ESPOL POLYTECHNIC UNIVERSITY)

• Plants Disease Detection in Low Data Regime,
  Shruti Jadon (Brown University (Rhode Island Hospital))

• Semi-Supervised Crop Detection in UAV Orthophotographs,
  Jana Wieme, Sam Leroux, Simon Cool, Ruben Van De Vijver, Wouter H. Maes, Jan Pieters (Ghent University)

• Counting Corn Stands using Drone Images from Cloud to Edge –A Deep Learning-based Object Detection Approach,
  Zhiyi Li, Kshitiz Dhakal, Song Li (Virginia Tech)