Skip to content

Commit

Permalink
Add face_following package and demo (#60)
Browse files Browse the repository at this point in the history
* Added audio and video mute control from robot GUI and frontend

* Addition of volume control and microphone from toggle to slider

* Remove whitespaces and unused include

* Addition of battery statistics and indicators for network and battery

* Addition of battery and network icons

* remove useless files

* Add network and system to statistics page

* Changes statistics style and adds network speeds

* Visual improvments and addition of upload and download speed to status

* Remove local files

* Change to static_cast for Axis types

* Add face-following package

* format face follower code

* Add model download in cmake and gpu utilization

* Add face-following demo

* Add face_following readme, format code and remove unused import

* Update opentera-webrtc submodule and clean code

* Add cv_camera to README and pull request workflow

* Rename "face_following" to "face_cropping" to prevent conflict in ttop

* Face cropper code corrections

* switch from opencv dnn face detection to haar or lbp

* Change behaviour when no faces are detected, add parameters and known issues to readme and format code

* Add test results to readme

* Grammar error correction in readme

* Add the option to disable face detection in the robot gui and bug correction

* Clarify FaceCropper.cpp with comments

* Remove reminder comment

* Improve code readability and general corrections

* Change c++ standard to c++17 and use std::gcd

* General corrections and code cleanup

* Use const for color scalar instead of constexpr

* Move color scalar to cpp file and rename them

* Renamed color scalars and models download script

Co-authored-by: philippewarren <[email protected]>
  • Loading branch information
joli-1801 and philippewarren authored Dec 6, 2022
1 parent 6bf39ca commit 758b2c4
Show file tree
Hide file tree
Showing 26 changed files with 1,453 additions and 35 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/pull-request-audit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install nodejs ros-noetic-turtlebot3 ros-noetic-turtlebot3-gazebo ros-noetic-dwa-local-planner ros-noetic-rtabmap-ros
sudo apt-get install nodejs ros-noetic-turtlebot3 ros-noetic-turtlebot3-gazebo ros-noetic-cv-camera ros-noetic-dwa-local-planner ros-noetic-rtabmap-ros
sudo apt-get install libglib2.0-dev libgtk-3-dev libpulse-dev libasound2-dev
sudo apt-get install python3-pip portaudio19-dev
sudo apt-get install nodejs npm
Expand Down
48 changes: 24 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,25 +45,25 @@ The procedure is written for Ubuntu 20.04 using ROS noetic. We assume ROS is alr

```bash
# opentera-webrtc-ros packages
$sudo apt-get install nodejs ros-noetic-turtlebot3 ros-noetic-turtlebot3-gazebo $ros-noetic-dwa-local-planner ros-noetic-rtabmap-ros
sudo apt-get install nodejs ros-noetic-turtlebot3 ros-noetic-turtlebot3-gazebo ros-noetic-cv-camera ros-noetic-dwa-local-planner ros-noetic-rtabmap-ros

# protobuf
$sudo apt-get install libprotobuf-dev protobuf-compiler python3-protobuf
sudo apt-get install libprotobuf-dev protobuf-compiler python3-protobuf

# python dependencies
$sudo apt-get install python3-pip portaudio19-dev
sudo apt-get install python3-pip portaudio19-dev

# nodejs dependencies
$sudo apt-get install nodejs npm
sudo apt-get install nodejs npm

# audio_utils packages
$sudo apt-get install cmake build-essential gfortran texinfo libasound2-dev libpulse-dev libgfortran-*-dev
sudo apt-get install cmake build-essential gfortran texinfo libasound2-dev libpulse-dev libgfortran-*-dev

# odas_ros packages
$sudo apt-get install libfftw3-dev libconfig-dev
sudo apt-get install libfftw3-dev libconfig-dev

# qt submodules
$sudo apt-get install libqt5charts5-dev
sudo apt-get install libqt5charts5-dev
```

## Installation
Expand All @@ -72,42 +72,42 @@ $sudo apt-get install libqt5charts5-dev

```bash
# Make sure ROS is installed first.
$source /opt/ros/noetic/setup.bash
source /opt/ros/noetic/setup.bash
# Create the workspace and initial build files
$mkdir -p ~/teleop_ws/src
$cd ~/teleop_ws/
$catkin_make
mkdir -p ~/teleop_ws/src
cd ~/teleop_ws/
catkin_make
```

### 2 - Get all the required ROS packages

```bash
$cd ~/teleop_ws/src
cd ~/teleop_ws/src
# audio_utils
$git clone https://github.com/introlab/audio_utils.git --recurse-submodules
git clone https://github.com/introlab/audio_utils.git --recurse-submodules
# odas_ros
$git clone https://github.com/introlab/odas_ros.git --recurse-submodules
git clone https://github.com/introlab/odas_ros.git --recurse-submodules
# opentera-webrtc-ros
$git clone https://github.com/introlab/opentera-webrtc-ros.git --recurse-submodules
git clone https://github.com/introlab/opentera-webrtc-ros.git --recurse-submodules
```

### 3 - Install the Python requirements

```bash
$cd ~/teleop_ws/src/opentera-webrtc-ros/opentera_client_ros
$python3 -m pip install -r requirements.txt
$cd ~/teleop_ws/src/opentera-webrtc-ros/opentera_webrtc_ros
$python3 -m pip install -r requirements.txt
$cd ~/teleop_ws/src/opentera-webrtc-ros/opentera_webrtc_ros/opentera-webrtc
$python3 -m pip install -r requirements.txt
cd ~/teleop_ws/src/opentera-webrtc-ros/opentera_client_ros
python3 -m pip install -r requirements.txt
cd ~/teleop_ws/src/opentera-webrtc-ros/opentera_webrtc_ros
python3 -m pip install -r requirements.txt
cd ~/teleop_ws/src/opentera-webrtc-ros/opentera_webrtc_ros/opentera-webrtc
python3 -m pip install -r requirements.txt
```

### 4 - Build all the ROS packages

```bash
$cd ~/teleop_ws
$source devel/setup.bash
$catkin_make
cd ~/teleop_ws
source devel/setup.bash
catkin_make
```

## Running the demos
Expand Down
1 change: 1 addition & 0 deletions face_cropping/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
models/
229 changes: 229 additions & 0 deletions face_cropping/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
cmake_minimum_required(VERSION 3.16.3)
project(face_cropping)

## Compile as C++11, supported in ROS Kinetic and newer
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

## Find catkin macros and libraries
## if COMPONENTS list like find_package(catkin REQUIRED COMPONENTS xyz)
## is used, also find other catkin packages
find_package(catkin REQUIRED COMPONENTS
cv_bridge
opentera_webrtc_ros_msgs
roscpp
image_transport
sensor_msgs
)

## System dependencies are found with CMake's conventions
# find_package(Boost REQUIRED COMPONENTS system)


## Uncomment this if the package has a setup.py. This macro ensures
## modules and global scripts declared therein get installed
## See http://ros.org/doc/api/catkin/html/user_guide/setup_dot_py.html
# catkin_python_setup()

################################################
## Declare ROS messages, services and actions ##
################################################

## To declare and build messages, services or actions from within this
## package, follow these steps:
## * Let MSG_DEP_SET be the set of packages whose message types you use in
## your messages/services/actions (e.g. std_msgs, actionlib_msgs, ...).
## * In the file package.xml:
## * add a build_depend tag for "message_generation"
## * add a build_depend and a exec_depend tag for each package in MSG_DEP_SET
## * If MSG_DEP_SET isn't empty the following dependency has been pulled in
## but can be declared for certainty nonetheless:
## * add a exec_depend tag for "message_runtime"
## * In this file (CMakeLists.txt):
## * add "message_generation" and every package in MSG_DEP_SET to
## find_package(catkin REQUIRED COMPONENTS ...)
## * add "message_runtime" and every package in MSG_DEP_SET to
## catkin_package(CATKIN_DEPENDS ...)
## * uncomment the add_*_files sections below as needed
## and list every .msg/.srv/.action file to be processed
## * uncomment the generate_messages entry below
## * add every package in MSG_DEP_SET to generate_messages(DEPENDENCIES ...)

## Generate messages in the 'msg' folder
# add_message_files(
# FILES
# Message1.msg
# Message2.msg
# )

## Generate services in the 'srv' folder
# add_service_files(
# FILES
# Service1.srv
# Service2.srv
# )

## Generate actions in the 'action' folder
# add_action_files(
# FILES
# Action1.action
# Action2.action
# )

## Generate added messages and services with any dependencies listed here
# generate_messages(
# DEPENDENCIES
# opentera_webrtc_ros_msgs# sensor_msgs
# )

################################################
## Declare ROS dynamic reconfigure parameters ##
################################################

## To declare and build dynamic reconfigure parameters within this
## package, follow these steps:
## * In the file package.xml:
## * add a build_depend and a exec_depend tag for "dynamic_reconfigure"
## * In this file (CMakeLists.txt):
## * add "dynamic_reconfigure" to
## find_package(catkin REQUIRED COMPONENTS ...)
## * uncomment the "generate_dynamic_reconfigure_options" section below
## and list every .cfg file to be processed

## Generate dynamic reconfigure parameters in the 'cfg' folder
# generate_dynamic_reconfigure_options(
# cfg/DynReconf1.cfg
# cfg/DynReconf2.cfg
# )

###################################
## catkin specific configuration ##
###################################
## The catkin_package macro generates cmake config files for your package
## Declare things to be passed to dependent projects
## INCLUDE_DIRS: uncomment this if your package contains header files
## LIBRARIES: libraries you create in this project that dependent projects also need
## CATKIN_DEPENDS: catkin_packages dependent projects also need
## DEPENDS: system dependencies of this project that dependent projects also need
catkin_package(
# INCLUDE_DIRS include
# LIBRARIES face_cropping
# CATKIN_DEPENDS cv_bridge opentera_webrtc_ros_msgs roscpp image_transport sensor_msgs
# DEPENDS system_lib
)

###########
## Build ##
###########

## Specify additional locations of header files
## Your package locations should be listed before other locations
include_directories(
include
${catkin_INCLUDE_DIRS}
)

add_executable(face_cropping
include/face_cropping/FaceCropper.h
include/face_cropping/Parameters.h
src/main.cpp
src/FaceCropper.cpp
src/Parameters.cpp
)

target_link_libraries(face_cropping
${catkin_LIBRARIES}
${OpenCV_LIBRARIES}
)

## Declare a C++ library
# add_library(${PROJECT_NAME}
# src/${PROJECT_NAME}/face_cropping.cpp
# )

## Add cmake target dependencies of the library
## as an example, code may need to be generated before libraries
## either from message generation or dynamic reconfigure
# add_dependencies(${PROJECT_NAME} ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})

## Declare a C++ executable
## With catkin_make all packages are built within a single CMake context
## The recommended prefix ensures that target names across packages don't collide
# add_executable(${PROJECT_NAME}_node src/face_cropping_node.cpp)

## Rename C++ executable without prefix
## The above recommended prefix causes long target names, the following renames the
## target back to the shorter version for ease of user use
## e.g. "rosrun someones_pkg node" instead of "rosrun someones_pkg someones_pkg_node"
# set_target_properties(${PROJECT_NAME}_node PROPERTIES OUTPUT_NAME node PREFIX "")

## Add cmake target dependencies of the executable
## same as for the library above
# add_dependencies(${PROJECT_NAME}_node ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})

## Specify libraries to link a library or executable target against
# target_link_libraries(${PROJECT_NAME}_node
# ${catkin_LIBRARIES}
# )

#############
## Install ##
#############

# all install targets should use catkin DESTINATION variables
# See http://ros.org/doc/api/catkin/html/adv_user_guide/variables.html

## Mark executable scripts (Python etc.) for installation
## in contrast to setup.py, you can choose the destination
# catkin_install_python(PROGRAMS
# scripts/my_python_script
# DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )

## Mark executables for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_executables.html
# install(TARGETS ${PROJECT_NAME}_node
# RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )

## Mark libraries for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_libraries.html
# install(TARGETS ${PROJECT_NAME}
# ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
# LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
# RUNTIME DESTINATION ${CATKIN_GLOBAL_BIN_DESTINATION}
# )

## Mark cpp header files for installation
# install(DIRECTORY include/${PROJECT_NAME}/
# DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
# FILES_MATCHING PATTERN "*.h"
# PATTERN ".svn" EXCLUDE
# )

## Mark other files for installation (e.g. launch and bag files, etc.)
# install(FILES
# # myfile1
# # myfile2
# DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
# )

#############
## Testing ##
#############

## Add gtest based cpp test target and link libraries
# catkin_add_gtest(${PROJECT_NAME}-test test/test_face_cropping.cpp)
# if(TARGET ${PROJECT_NAME}-test)
# target_link_libraries(${PROJECT_NAME}-test ${PROJECT_NAME})
# endif()

## Add folders to be run by python nosetests
# catkin_add_nosetests(test)

#Download face detection dnn model
add_custom_target(
download_face_detection_models ALL
COMMAND scripts/download_models.bash
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)
Loading

0 comments on commit 758b2c4

Please sign in to comment.