2023-09-13 Alarm Watchdog Documentation

Drone operation

Drone operation is based on ROS-1 architecture. It consist of topics, like speed, battery, odometry that are constantly emitted by drone and ROS actions like startMapping, flyToPOI, land etc. which have continous control in between nodes. “Exit node” currently is rosbridge that exposes ROS infrastructure to websocket for use in TypeScript based react-native app.

Current App architecture

Green nodes - ASYA team Red nodes - Alarm Watchdog team Blue nodes - Alarm app team

Proposed app architecture - A

Green nodes - ASYA team Red nodes - Alarm Watchdog team Blue nodes - Alarm app team

Need to resolve interrupts for schefulling service - wake on LAN, time, sound

Proposed app architecture - B

Advantage: AI processing client-side, more control over drone, better battery life Disadvantage: More cost to client-side

Green nodes - ASYA team Red nodes - Alarm Watchdog team Blue nodes - Alarm app team

Github / Figma

React-Native mobile app https://github.com/asya-ai/alarm_watchdog_react_native_mobile_app

AI backend processing server (FastAPI Swagger UI REST API) https://github.com/asya-ai/alarm_flight_model

Figma app designs (whole history of UX experiments and iterations) https://www.figma.com/file/NQicYun9EL9JCDF2yTO4rg/Alarm?type=design&node-id=1479-47358&mode=design&t=bIzK3oygncZTw1JX-0

Backend map processing

LIDAR on drone and SLAM produces very low quality data, often with large missing pieces and in arbritary rotation and scaling. AI based generative models allow to fix point cloud and transformation, it also produces enhanced drone path, walls as vectors, rooms as polygons, transformation matrices Notice significant differences between map orientation and detail between offline and online version

Offline mode

Offline mode can be set in Settings, it allows to use app directly with drone without any internet connection. This setting most likely would not be used for end-user experience, but just as backup for testing. When in offline mode map will not be processed in backend server and will just operate with raw point cloud. Beaware that whole point cloud should not be drawn on screen as it contains too many points and will cause rendering to be slow and app to be unresponsive.

App operation - Mapping

Sepecific procedure to walk tgrough the rooms to collect point-cloud data.

Afterwards sent to AI backend server for processing.

Must be able to store multiple maps and different missions for each map.

After processing it should be possible to distinguish rooms in each map as polygons

App operation - Control

Functions to control drone: Take off, Land in base, Emergency land, Rotate in flight, as well as add POIs and fly to POIs. See camera feed in real time and other features.

App operation - Missions

Functions to drag and drop POIs in sequence, set different settings like sleep in each POI, loops etc. To automatically execute flight starting from flight to landing and collect photos from stream.

App operation - Scheduled Missions

Not yet decided on implementation. Will need service on drone or comapnion server.

Testing without drone

We have deployed in-cloud ROS node for testing purposes to simulate drone environment. In order to run it you must be sure that rosbridge screen is open and running ( ⚠️ do not close any other screens because on the backend map processing nodes are running )

SSH connection h: 144.24.250.72 u: ubuntu key: [will be sent by Evalds]

Running bag file (for mapping or control screen simulation)

Choose bag recording from


1
cd ~/Documents/bags
2
ls

Run bag recording in replay mode


x
1
rosbag play 2023-07-14-V31-WH.bag

^ Beaware that bags files contain only part of ROS topics and usually they are incomplete, it might be necessary to run different bag files to test all functions

Troubleshooting

Sometimes if bag file data is not reaching app it might be that rosbridge node is not working properly then you need to restart it. In order to do it you must first open screen, kill node and then restart it

Connect to screen (if you are not familiar how to work with screens, read carefully below section “Linux Screens”)


x
1
screen -rd rosbridge

Kill rosbridge


xxxxxxxxxx
1
1
ctrl+C

Restart


xxxxxxxxxx
1
39
3
1
cd ~/Documents/alarm
2
source ./devel/setup.bash
3
roslaunch rosbridge_server rosbridge_websocket.launch

Testing with drone

To test with drone, first connect to local router where drone is connected then in Settings enter ROS IP to local IP address of drone. Local router should also have access to internet and should work with Backend processing server which is deployed on ROS testing node. If no internet connectivity then Offline mode should be used.

Linux Screen

create with name screen -S newname

List screens: screen -ls

Attach to existing screen -rd newname

Detach from screen ctrl+a => d

Kill screen ctrl+a => k

Scroll mode

Screen shortcuts

Send commands to screen:


xxxxxxxxxx
2
1
 screen -dmS server
2
 screen -r server -X stuff "/media/hdd/Document/server.sh\n"