Initial commit

Author: tohruk

README.md

@@ -0,0 +1,107 @@
+# Human Pose Estimation usign Deep Neural Network
+
+
+This demo shows how to train and test a human pose estimation using deep neural network. In R2019b, Deep Learning Toolbox(TM) supports low-level APIs to customize training loops and it enables us to train flexible deep neural networks. GPU Coder(TM) also enables us to deploy the trained model to an NVIDIA(R) Jetson(TM) devices. Once deployed, the human pose estimator will be running as a standalone.
+
+
+
+
+This repository includes a pretrained network using the COCO dataset, which was collected by the COCO Consortium (cocodataset.org).
+
+
+  
+# Required Toolboxes
+
+
+Pose Estimation Toolbox requires the following products:
+
+
+
+   -  MATLAB(R) R2019b or later (due to the R2019b new feature) 
+   -  [Deep Learning Toolbox(TM)](https://mathworks.com/products/deep-learning.html) 
+   -  [Image Processing Toolbox(TM)](https://mathworks.com/products/image.html) 
+   -  [Computer Vision Toolbox(TM)](https://www.mathworks.com/products/computer-vision.html) 
+   -  [Parallel Computing Toolbox(TM)](https://mathworks.com/products/parallel-computing.html) (Required for using GPU computation) 
+
+
+
+To deploy the model to a NVIDIA Jetson or Drive platforms, you'll also need the following products.
+
+
+
+   -  [MATLAB Coder(TM)](https://mathworks.com/products/matlab-coder.html) 
+   -  [GPU Cder(TM)](https://mathworks.com/products/gpu-coder.html) 
+   -  [GPU Coder Interface for Deep Learning Libraries support package](https://www.mathworks.com/matlabcentral/fileexchange/68642-gpu-coder-interface-for-deep-learning-libraries) (Addon package for GPU Coder) 
+   -  [GPU Coder Support Package for NVIDIA GPUs](https://mathworks.com/help/supportpkg/nvidia/index.html) (Addon package for GPU Coder) 
+
+# Getting Started
+
+
+Load a pose estimator model.
+
+
+
+```matlab:Code
+detector = posenet.PoseEstimator;
+```
+
+
+
+First, read a test image, then crop a person and resize it to fit to the network input.
+
+
+
+```matlab:Code
+I = imread('visionteam1.jpg');
+bbox = [182 74 303 404];
+Iin = imresize(imcrop(I,bbox),detector.InputSize(1:2));
+```
+
+
+
+Then, perform the pose estimation on the image. To visualise the results we can superimpose the detected keypoints on the original image.
+
+
+
+```matlab:Code
+keypoints = detectPose(detector,Iin);
+J = detector.visualizeKeyPoints(Iin,keypoints);
+imshow(J);
+```
+
+
+![figure_0_png.jpg](doc/README_images/figure_0_png.jpg)
+
+
+
+[Click here](doc/GettingStarted.md) for a complete example.
+
+
+# Examples
+
+   -  [Estimate Human Pose For Multiple Person Using Pretrained Network](doc/SimplePoseNetForMultiPerson.md) 
+   -  [Train Deep Neural Network for Human Pose Estimation](doc/SimplePoseNetTrainingExample.md) 
+   -  [Human Pose Estimation With Webcam Images Using Deep Learning](doc/SimplePoseNetEstimationWithWebcamExample.md) 
+   -  [Deploy Simple Pose Estimation on NVIDIA(R) Jetson(TM) Using GPU Coder(TM)](doc/SimplePoseNetEstimationOnJetson.md) 
+
+# About the Model
+
+
+The network architecture is based on Xiao's  pose estimation network[1] which combines upsampling and convolutional parameters into transposed convolutional layers in a much simpler way, without using skip layer connections. We also use COCO dataset[2] which is one of the well known large public human pose dataset.
+
+
+
+
+[1] Xiao, Bin, Haiping Wu, and Yichen Wei. “Simple baselines for human pose estimation and tracking.” Proceedings of the European Conference on Computer Vision (ECCV). 2018.
+
+
+
+
+[2] Lin, T., et al. "Microsoft COCO: Common objects in context. arXiv 2014." *arXiv preprint arXiv:1405.0312*.
+
+
+
+
+*Copyright 2020 The MathWorks, Inc.*
+
+

README.mlx

@@ -0,0 +1,6 @@
+# Reporting Security Vulnerabilities 
+
+If you believe you have discovered a security vulnerability, please report it to 
+[security@mathworks.com](mailto:security@mathworks.com). Please see 
+[MathWorks Vulnerability Disclosure Policy for Security Researchers](https://www.mathworks.com/company/aboutus/policies_statements/vulnerability-disclosure-policy.html) 
+for additional information.  

SECURITY.md

@@ -0,0 +1,54 @@
+# Getting Started with Simple Pose Estimation
+## Load pretrained pose estimator model
+
+```matlab:Code
+detector = posenet.PoseEstimator;
+```
+
+## Read an image
+
+
+First, read a test image.
+
+
+
+```matlab:Code
+I = imread('visionteam1.jpg');
+imshow(I);
+```
+
+
+![figure_0_png.jpg](GettingStarted_images/figure_0_png.jpg)
+
+## Crop a person and resize it to fit to the network input.
+
+```matlab:Code
+bbox = [182 74 303 404];
+Iin = imresize(imcrop(I,bbox),detector.InputSize(1:2));
+imshow(Iin);
+```
+
+
+![figure_1_png.jpg](GettingStarted_images/figure_1_png.jpg)
+
+## Detect pose and visulaze on the original image
+
+
+To visualise the results we can superimpose the detected keypoints on the original image.
+
+
+
+```matlab:Code
+keypoints = detectPose(detector,Iin);
+J = detector.visualizeKeyPoints(Iin,keypoints);
+imshow(J);
+```
+
+
+![figure_2_png.jpg](GettingStarted_images/figure_2_png.jpg)
+
+
+
+*Copyright 2020 The MathWorks, Inc.*
+
+

doc/GettingStarted.md

@@ -0,0 +1,107 @@
+# Deploy Simple Pose Estimation on NVIDIA® Jetson™ Using GPU Coder™
+
+
+![image_0_png.jpg](SimplePoseNetEstimationOnJetson_images/image_0_png.jpg)
+
+
+
+
+Running on a Jetson Nano.
+
+
+
+
+![image_1_png.jpg](SimplePoseNetEstimationOnJetson_images/image_1_png.jpg)
+
+
+
+
+Running on a Jetson AGX Xavier.
+
+
+# **System Requirements**
+
+
+[https://www.mathworks.com/help/supportpkg/nvidia/ug/install-and-setup-prerequisites.html](https://www.mathworks.com/help/supportpkg/nvidia/ug/install-and-setup-prerequisites.html)
+
+
+# **Connect to the NVIDIA Hardware**
+
+```matlab:Code
+clear;
+hwobj = jetson('192.168.55.1', 'nvidia', 'nvidia');
+```
+
+# **Verify the GPU Environment**
+
+
+Use the [coder.checkGpuInstall](matlab:doc('coder.checkGpuInstall')) function and verify that the compilers and libraries needed for running this example are set up correctly.
+
+
+
+```matlab:Code
+envCfg = coder.gpuEnvConfig('jetson');
+envCfg.DeepLibTarget = 'tensorrt';
+envCfg.DeepCodegen = 1;
+envCfg.HardwareObject = hwobj;
+coder.checkGpuInstall(envCfg);
+```
+
+# **Prepare Pose Estimation for Stand-Alone Deployment**
+
+
+Include the webcam and display interfaces inside the Sobel edge detection application
+
+
+
+```matlab:Code
+w = webcam(hwobj);
+img = snapshot(w);
+d = imageDisplay(hwobj);
+image(d,img);
+```
+
+# Test Pose Estimation Function on MATLAB
+
+```matlab:Code
+clear hwobj d w envCfg;
+figure;
+set(gcf,'Visible','on');
+simplePoseEstimation
+% Need to Ctrl+C to stop the running program.
+```
+
+# **Generate CUDA Code for the Target Using GPU Coder**
+
+```matlab:Code
+cfg = coder.gpuConfig('exe');
+cfg.Hardware = coder.hardware('NVIDIA Jetson');
+cfg.Hardware.BuildDir = '~/remoteBuildDir';
+cfg.DeepLearningConfig = coder.DeepLearningConfig('tensorrt');
+cfg.GenerateExampleMain = 'GenerateCodeAndCompile';
+codegen('-config ',cfg,'simplePoseEstimation.m','-report');
+```
+
+# Run the Pose Estimation on the Target
+
+
+To run the generated executable on the target, use the MATLAB® runApplication function.
+
+
+
+```matlab:Code
+clear hwobj;
+hwobj = jetson
+pid = runApplication(hwobj,'simplePoseEstimation');
+```
+
+
+```matlab:Code
+%killApplication(hwobj,'simplePoseEstimation');
+```
+
+
+
+*Copyright 2020 The MathWorks, Inc.*
+
+

doc/GettingStarted_images/figure_0_png.jpg

@@ -0,0 +1,62 @@
+# Human Pose Estimation With Webcam Images Using Deep Learning
+
+
+This example shows how to predict human poses using a pretrained mode with a web camera.
+
+
+
+
+![image_0_png.jpg](SimplePoseNetEstimationWithWebcamExample_images/image_0_png.jpg)
+
+
+# Load Camera and Pretrained Network
+
+
+Connect to the camera and load a pretrained GoogLeNet network. You can use any pretrained network at this step. The example requires MATLAB Support Package for USB Webcams, and Deep Learning Toolbox™ Model *for GoogLeNet Network*. If you do not have the required support packages installed, then the software provides a download link.
+
+
+
+```matlab:Code
+detector = posenet.PoseEstimator;
+cam = webcam;
+```
+
+# **Continuously Estimate Human Pose**
+
+```matlab:Code
+player = vision.DeployableVideoPlayer;
+I = zeros(256,192,3,'uint8');
+player(I);
+
+while player.isOpen
+    
+    % Read an image from web camera 
+    I = snapshot(cam);
+    
+    % Crop the image fitting the network input size of 256x192 
+    Iinresize = imresize(I,[256 nan]);
+    Itmp = Iinresize(:,(size(Iinresize,2)-192)/2:(size(Iinresize,2)-192)/2+192-1,:);
+    Icrop = Itmp(1:256,1:192,1:3);
+    
+    % Predict pose estimation
+    heatmaps = detector.predict(Icrop);
+    keypoints = detector.heatmaps2Keypoints(heatmaps);
+
+    % Visualize key points
+    Iout = detector.visualizeKeyPoints(Icrop,keypoints);
+    player(Iout);
+    
+    if ~isOpen(player)
+        break
+    end
+end
+imshow(Iout);
+clear cam
+release(player)
+```
+
+
+
+*Copyright 2020 MathWorks, Inc.*
+
+