Revolutionizing Remote Sensing Image Annotation: LabelRS Tool in ArcGIS

Introduction

Remote sensing, a pivotal technology in acquiring information about the Earth's surface, has witnessed substantial advancements. These strides are largely attributable to integrating deep learning techniques, revolutionizing how we process and analyze remote sensing images. Despite these advancements, creating robust tools for annotating remote sensing images with deep learning samples remains a significant challenge.

The emergence of LabelRS, an innovative tool integrated into the ArcGIS environment, signifies a breakthrough in simplifying the creation of deep learning samples for remote-sensing images. This tool aims to automate and streamline the process of generating samples for tasks such as semantic segmentation, object detection, and image classification.

The Need for LabelRS

Conventional annotation tools in computer vision, while effective for natural images, often fall short when handling multispectral remote sensing images. The complexities inherent in remote sensing data, characterised by diverse spatial resolutions and vast data volumes, render standard annotation tools inadequate for fundamental functions like image loading and spatial referencing.

Public datasets available for remote sensing, while accessible, present limitations in terms of categories, image sources, spatial-temporal resolutions, and annotation quality. Consequently, a pressing need exists for a universal annotation tool specifically tailored to address the intricacies of remote sensing images.

LabelRS: A Game-Changing Solution

LabelRS, designed as an add-in for ArcGIS 10.2, emerges as a powerful annotation tool adept at managing the complexities of remote sensing images. The tool encompasses a comprehensive framework featuring three primary sub-tools: Semantic Segmentation, Object Detection, and Image Classification.

Semantic Segmentation Tool

LabelRS' Semantic Segmentation module employs a feature merging strategy capable of adapting to both sparse and densely distributed scenarios in remote sensing images.

It adeptly handles the diverse sizes of targets by utilizing sliding windows and employs multiple band stretching and gray level transformation algorithms for enhanced precision.

Object Detection Tool

The Object Detection module within LabelRS enables the accurate recording of sample ranges and target ranges for each labeled object. Leveraging sliding window segmentation, this tool minimizes the truncation of large targets and supports popular metadata formats such as PASCAL VOC, YOLO, and KITTI.

Image Classification Tool

LabelRS' Image Classification module efficiently manages the task by segmenting large targets and resampling smaller objects. It offers flexibility in choosing interpolation methods and band combinations, facilitating the creation of samples tailored for image-level classification tasks.

                    Figure: Split the large target in the image classification tool.

Implementation Details

Based on ArcGIS 10.2, developed LabelRS, an annotation tool for remote sensing images using Python 2.7, ensuring upward compatibility. LabelRS leverages essential Python libraries such as ArcPy, OpenCV, and Pillow. It encompasses two versions: one as an ArcGIS toolbox featuring a visual graphical interface, allowing convenient parameter input and seamless integration into ArcGIS. The other version consists of Python scripts, prioritizing flexibility and suitability for batch data processing, thereby facilitating code debugging and integration.

The tool comprises three sub-modules: Semantic Segmentation, Object Detection, and Image Classification. Each module presents a dialog box for parameter input and offers specific functionalities:

Semantic Segmentation Tool: The dialog box (Figure 1a) exhibits 14 input parameters, with four being mandatory. These parameters enable users to conduct semantic segmentation tasks, delineating meanings detailed in Table 1.

Object Detection Tool: This tool, represented by its dialog box (Figure 1b), comprises 11 input parameters, with four being required for operation. Users employ these parameters to conduct object detection tasks, with their meanings outlined in Table 2.

Image Classification Tool: The dialog box (Figure 1c) for image classification contains a total of 11 input parameters, with four being obligatory. Users utilize these parameters to perform image classification tasks, as explained in Table 3.

     Figure 1 is below: Dialog Boxes of LabelRS (a) Semantic Segmentation Tool           (b) Object

These tools cater to distinct functionalities within remote sensing image analysis, offering flexibility and ease of use through both graphical interface-based and script-based approaches, ensuring efficiency across various workflows and processing environments.

Case Studies Demonstrating LabelRS' Efficacy

1. Water Samples Generation

Water bodies are vital natural features, and employing deep learning methods to identify and extract them from remote-sensing images is a common goal. However, the intricate boundaries of water bodies make manual labeling a time-consuming process. To tackle this challenge, NDWI (Normalized Difference Water Index) is combined with LabelRS, offering an automated procedure for generating water body samples.

Methodology

• NDWI calculation using red and near-infrared bands.
• Application of the OTSU algorithm to determine water/non-water segmentation thresholds.
• Filtering out non-water objects using area thresholds.
• Utilization of the obtained vector of water bodies with LabelRS' semantic segmentation tool to create water body samples.

Experimental Setup

• GF-2 satellite images with four bands: red, green, blue, and near-infrared.
• Selection of diverse regions, such as Beijing-Tianjin-Hebei and Zhenjiang, Jiangsu Province, known for differing climate types and land covers.
• Output wavebands for samples: near-infrared, red, and green bands of GF-2.
• Sample size set to 256, employing Maximum Contrast for gray level transformation and PTS for band stretching.
• Desktop computer specifications: Intel Core i7-6700 3.40 GHz CPU, 32 GB RAM.

Results and Observations

• The combination of LabelRS and NDWI accurately segments water body areas, yielding detailed and smooth boundaries in the generated labels.
• Below figure 2 showcases samples created using the semantic segmentation tool of LabelRS.

                         Figure 2: Water Samples

• Table 4 demonstrates the processing time for different tasks, highlighting LabelRS' efficiency with an average time of 1–2 seconds per multispectral remote sensing sample.

                                       Table 4: Processing time of each sub-tool.

2. Making Dam Samples

Dams play a crucial role in water resource management, requiring accurate identification for various purposes like flood control and hydroelectric power generation. Leveraging the same experimental areas and data sources as the previous case study, LabelRS was employed to create samples for identifying dams, differentiating them from similar structures like bridges.

Approach

• Manual marking of dam and bridge locations in ArcGIS, saved as a vector file.
• Utilization of the object detection tool within LabelRS for sample creation.
• Setting the sample size to 512 and employing the PASCAL VOC metadata format.
• Generation of true-color and false-color composite samples for data augmentation.

Visual Representation

• Figure 3 displays samples showcasing dams and bridges, emphasizing the distinct bounding boxes of these structures.

Figure 3: Samples of dams and bridges, the green and red rectangle represent the bounding box of the dam and bridge respectively.

• Figure 4 illustrates PASCAL VOC annotations, offering a visual representation of the labeling effect.

                                Figure 4: Annotation of PASCAL VOC

3. Making Land Cover Classification Samples

Classifying images at an image level rather than a pixel level is a valuable approach, particularly in land use classification tasks. LabelRS was used in conjunction with remote sensing images of Huairou District and Changping District in Beijing to create samples for land cover classification.

Procedure

• Manual delineation of different features in ArcGIS.
• Utilization of LabelRS' image classification tool to generate classification samples.
• Setting the sample size to 128.

Results

• Figure 5 showcases primary land cover types in specific regions, including forests, water bodies, buildings, and farmland.

• Figure 6 exhibits land cover classification samples, precisely delineating various land use types.

These case studies exemplify LabelRS' effectiveness in diverse applications, from accurately identifying water bodies and dams to facilitating land cover classification tasks. The tool's adaptability and efficiency in handling different remote sensing challenges underscore its significance in streamlining image annotation and sample creation processes.

Conclusion

LabelRS emerges as a groundbreaking tool, empowering researchers and practitioners in the field of remote sensing. Its innovative capabilities within the ArcGIS environment significantly enhance the process of generating deep learning samples for remote sensing images. LabelRS effectively addresses critical challenges, unlocking new possibilities in image annotation and analysis within the realm of remote sensing.

Researchers and professionals engaged in remote sensing and deep learning tasks can leverage LabelRS to streamline the sample creation process. The open-source nature of LabelRS encourages collaboration, customization, and further advancements in remote sensing image annotation.

LabelRS stands as a pivotal tool driving efficiency, accuracy, and innovation in remote sensing image annotation, paving the way for enhanced applications across various domains.

Frequently Asked Questions

1. What is remote sensing in ArcGIS?

Remote sensing within ArcGIS encapsulates "The Science of Where," employing a suite of tools, functions, and capabilities to extract both spatial and spectral information from imagery through scientific and image processing methodologies.

This advanced functionality in ArcGIS enables the analysis of remotely sensed data, allowing for the extraction of detailed spatial and spectral insights from various imagery sources. It involves a range of scientific techniques and image processing methodologies within the ArcGIS platform, facilitating the interpretation, manipulation, and extraction of valuable information embedded within remote sensing imagery.

2.  Why is image annotation important in deep learning?

Image annotation is pivotal in deep learning, specifically for tasks like computer vision. It involves labeling objects in images, enabling neural networks to learn patterns and make accurate predictions.

This process is essential as it helps train models by associating features with labels, improving their ability to identify objects in new, unlabeled data. Quality and quantity of annotated data directly impact model performance, making image annotation fundamental for training accurate and effective deep learning models in tasks like object detection and classification.

3. What is ArcGIS Pro stereo mapping?

ArcGIS Pro's Image Analyst in stereo mapping enables the viewing and manipulation of stereo pairs, facilitating the collection of 3D features. It allows the gathering of 3D point, line, and polygon features for diverse tasks like parcel editing, constructing basic 3D structures, and measuring distances and elevations. The anaglyph stereo image pair provided is credited to Vexcel.