[Instance segmentation] Mask R-CNN/Detectron2 모델 파일 분석

mask-rcnn

• .circleci configuration of the environment to run

  • config.yml: environments to run the jobs in

• .github arrangement on the github environment

  • ISSUE_TEMPLATE
  • workflows
  • CODE_OF_CONDUCT.md
  • CONTRIBUTING.md
  • ISSUE_TEMPLATE.md
  • pull_request_template.md

• build options that is built on my macosx environment

  • lib.macosx-10.9-x86_64-3.7

• configs structure of the whole model(-common-models-mask_rcnn_fpn.py, Base-RCNN-FPN.yaml)

  • Cityscapes ~ PascalVOC-Detection configuration of each dataset mask-rcnn
    • Cityscapes
    • COCO-Detection
    • COCO-InstanceSegmentation
    • COCO-Keypoints
    • COCO-PanopticSegmentation
    • Detectron1-Comparisons
    • LVISv0.5-InstanceSegmentation
    • LVISv1-InstanceSegmentation
    • Misc
    • new_baselines
    • PascalVOC-Detection
    • quick_schedules
  • common
    • data options about dataloader
    • models → mask_rcnn_fpn.py structure of the whole model
  • Base-RCNN-FPN.yaml & other yaml files
    • Base-RCNN-C4.yaml
    • Base-RCNN-DilatedC5.yaml
    • Base-RCNN-FPN.yaml
    • Base-RetinaNet.yaml

• datasets structure of datasets(json, train, test)

  • prepare_ade20k_sem_seg.py
  • prepare_cocofied_lvis.py
  • prepare_for_tests.sh
  • prepare_panoptic_fpn.py
  • README.md

• demo Run model by cfg

  • demo.py
  • predictor.py
  • README.md

• detectron2

  • checkpoint don't know the usage
    • Checkpointer
    • PreiodicCheckpointer
    • DetectionCheckpointer
  • config
    • cfgNode
    • get_cfg
    • global_cfg
    • set_global_cfg
    • downgrade_config
    • configurable
    • instantiate
    • LazyCall
    • LazyConfig
  • data

• engine

• evaluation

• export

• layers

• model_zoo

• modeling

  • backbone FPN

    • backbone.py
      • Backbone abstract base class for network backbone
    • build.py: build_backbone
    • fpn.py
      • FPN(Backbone)
        • → build_resnet_fpn_backbone
        • → build_retinanet_resnet_fpn_backbone
    • regnet.py
      • AnyNet, RegNet, ResStem, SimpleStem, VanillaBlock, ResBasicBlock, ResBottleneckBlock
    • resnet.py
      • ResNetBlockBase, BasicBlock, BottleneckBlock, BasicStem, ResNet, make_stage, build_resnet_backbone

  • meta_arch meta-architectures of the whole model GeneralizedRCNN

    • build.py
      • build_model: build the whole model architecture
    • dense_detector.py
      • DenseDetector: base class for dense detector ( a dense detector as a fully-convolutional model that makes per-pixel prediction) → FCOS
    • fcos.py
      • → FCOS(DenseDetector), FCOSHead(RetinaNetHead): Fully convolutional One-stage object detection
    • panoptic_fpn.py
      • PanopticFPN(GeneralizedRCNN)
    • rcnn.py GeneralizedRCNN, ProposalNetworks
      • GeneralizedRCNN
        1. Per-image feature extraction (aka backbone): backbone
        2. Region proposal generation: proposal_generator
        3. Per-region feature extraction and prediction: roi_heads
      • ProposalNetwork: a meta architecture that only predicts object proposals
    • retinanet.py
    • semantic_seg.py

  • proposal_generator StandardRPNHead, RPN

    • build.py
      • build_proposal_generator
    • proposal_utils.py
      • find_top_rpn_proposals for each feature map, select the 'pre_nms_topk' highest scoring proposals, apply NMS, clip proposals, and remove small boxes. return the 'post_nms_topk' highest scoring proposals among all the feature maps for each image.
      • add_ground_truth_to_proposals_single_image
        • → add_ground_truth_to_proposals
    • rpn.py StandardRPNHead
      • build_rpn_head.py
      • StandardRPNHead standard RPN classification and regression heads described in FasterRCNN
      • RPN Region Proposal Network, introduced by Faster RCNN
    • rrpn.py
      • find_top_rrpn_proposals
      • RRPN(RPN)

  • roi_heads StandardROIHeads FastRCNNConvFCHead FastRCNNOutputLayers MaskRCNNConvUpsampleHead

    • roi_heads.py StandardROIHeads

      • build_roi_heads
      • ROIHeads
        1. (in training only) match proposals with ground truth and sample them
        2. crop the regions and extract per-region features using proposals
        3. make per-region predictions with different heads
      • Res5RoIHeads(ROIHeads)
      • StandardROIHeads(ROIHeads)
      • select_foreground_proposals return a list of instances that contain only instances with gt_classes ≠ -1 & gt_classes ≠ bg_label
      • select_proposals_with_visible_keypoints

    • box_heads.py FastRCNNConvFCHead

      • build_box_head
      • FastRCNNConvFCHead a head with several 3x3 conv layers and then several fc layers

    • cascade_rcnn.py

      • CascadeROIHeads(StandardROIHeads)

    • fast_rcnn.py FastRCNNOutputLayers

      • fast_rcnn_inference Call 'fast_rcnn_inference_single_image' for all images

      • _log_classification_stats log the classification metrics to EvnetStorage

      • FastRCNNOutputLayers

        Two linear layers for predicting Fast R-CNN outputs

        1. proposal-to-detection box regression deltas
        2. classification scores

    • keypoint_head.py

      • build_keypoint_head
      • keypoint_rcnn_loss
      • keypoint_rcnn_inference
      • BaseKeypointRCNNHead
      • KRCNNConvDeconvUpsampleHead

    • mask_head.py mask_rcnn_loss, MaskRCNNConvUpsampleHead

      • build_mask_head
      • mask_rcnn_loss
      • mask_rcnn_inference
      • BaseMaskRCNNHead implement the basic mask R-CNN losses and inference logic
      • MaskRCNNConvUpsampleHead(BaseMaskRCNNHead)

    • roatated_fast_rcnn.py

      • fast_rcnn_inference_rotated
      • fast_rcnn_inference_single_image_rotated
      • RotatedFastRCNNOutputLayers
      • RROIHeads

  • anchor_generator.py

    • build_anchor_generator
    • _create_grid_offsets
    • _broadcast_params
    • DefaultAnchorGenerator Compute anchors in the standard ways described in Faster R-CNN
    • RotatedAnchorGenerator

  • box_regression.py Box2BoxTransform

    • Box2BoxTransform box-to-box transform defined in R-CNN
    • Box2BoxTransformRotated
    • Box2BoxTransformLinear

  • matcher.py Matcher

    • Matcher assigns to each predicted element a ground truth element → returns a vector of length N containing the index of the ground-truth element m in [0, M) that matches to predictions in [0, N), a vector of length N containing the labels for each predictions

  • mmdet_wrapper.py

    • MMDetBackbone
    • MMDetDetector
    • _convert_mmdet_result
    • parselosses

  • poolers.py

    • assign_boxes_to_levels
    • convert_boxes_to_pooler_format
    • ROIPooler ROI feature map pooler that supports pooling from one or more feature maps

  • postprocessing.py detector_postprocess

    • detector_postprocess resize the output instances
    • sem_seg_postprocess

  • sampling.py

    • subsample_labels return num_samples(random samples from labels which is a mixture of positives and negatives

  • test_time_augmentation.py

    • DatasetMapperTTA
    • GeneralizedRCNNWithTTA

• projects → went to projects file

• solver LR scheduler 정의하기

  • init.py
  • build.py
  • lr_scheduler.py

• structures

  • boxes.py
    • BoxMode: different ways to represent a box
    • Boxes: a list of boxes as a Nx4 tensor
    • calculate IoU
  • Image_list.py
    • ImageList: structure that holds a list of images as a single tensor
  • instances.py
    • Instances: a list of instances (boxes, masks, labels, scores) in an image
  • keypoints.py
    • Keypoints: keypoint annotation data (x,y location and visibility flag of each keypoint)
  • masks.py
    • BitMasks: segmentation masks for all objects in one image, in the form of bitmap(NxHxW)
    • PolygonMasks: segmentation masks for all objects in one image, in the form of polygon(float64 vector)
    • RoIMasks: Represent masks by N smaller masks defined in some ROIs. Once ROI boxes are given,
      full-image bitmask can be obtained by "pasting" the mask on the region defined by the corresponding ROI box.
  • rotated_boxes.py
    • RotatedBoxes: a list of rotated boxes as a Nx5(+angle) tensor

• utils

  • analysis.py
  • collect_env.py
  • colormap.py
  • comm.py
  • env.py
  • events.py
  • file_io.py
  • logger.py
  • memory.py
  • README.md
  • registry.py
  • serialize.py
  • testing.py
  • video_visualizer.py
  • visualizer.py

• init.py

• detectron2.egg-info detectron2 information(PKG-INFO etc.)

  • dependency_links.txt, PKG-INFO, requires.txt, SOURCES.txt, top_level.txt

• dev scripts for develops to use (GPU, log, commit)

  • packaging
  • linter.sh
  • parse_results.sh
  • README.md
  • run_inference_tests.sh
  • run_instant_tests.sh

• docker docker

  • deploy.Dockerfile
  • docker-compose.yml
  • Dockerfile
  • README.md

• docs documentation built for this directory

  • _static
  • modules
  • notes
  • tutorials

• output output(config, log history) of the trained model

  • config.yaml config file of this detectron2 model
  • log.txt log history of the training procedure

• projects few projects that are built on detectron2

  • DeepLab
  • DensePose
  • Panoptic-DeepLab
  • PointRend
  • PointSup
  • Rethinking-BatchNorm
  • TensorMask
  • TridentNet

• tools Train network(train_net.py, plain_train_net.py) / visualize data results / analyze model

  • deploy
  • init.py
  • analyze_model.py analyze model(flop, activation, parameter, structure)
  • benchmark.py a script to benchmark builtin models
  • convert-torchvision-to-d2.py
  • lazyconfig_train_net.py training script using the new "LazyConfig" python config files
  • lightning_train_net.py beta version
  • plain_train_net.py training script with a plain training loop, fewer default features but an example to how to use the library
  • README.md
  • train_net.py main training script
  • visualize_data.py
  • visualize_json_results.py

• tests file for testing, but more simple than detectron2. help to set detectron2 model

  • config
    • dir1
    • root_cfg.py
    • test_instantiate_config.py
    • test_lazy_config.py
    • test_yacs_config.py
  • data
    • init.py
    • test_coco.py
    • test_coco_evaluation.py
    • test_dataset.py
    • test_detection_utils.py
    • test_rotation_transform.py
    • test_sampler.py
    • test_transforms.py
  • layers
    • init.py
    • test_blocks.py
    • test_deformable.py
    • test_losses.py
    • test_mask_ops.py
    • test_nms.py
    • test_nms_rotated.py
    • test_roi_align.py
    • test_roi_align_rotated.py
  • modeling
    • init.py
    • test_anchor_generator.py
    • test_backbone.py
    • test_box2box_transform.py
    • test_fast_rcnn.py
    • test_matcher.py
    • test_mmdet.py
    • test_model_e2e.py
    • test_roi_heads.py
    • test_roi_pooler.py
    • test_rpn.py
  • structures
    • init.py
    • test_boxes.py
    • test_imagelist.py
    • test_instances.py
    • test_keypoints.py
    • test_masks.py
    • test_rotated_boxes.py