How to use ================ .. toctree:: :maxdepth: 3 :caption: Contents: Introduction ------------ This repo contains all the log processing tools centralized. This tools can use user generated protobuf mavlink content. They are composed of cpp buildable files and python scripts. Shell and bat scripts are also provided to build the cpp files for windows or unix platforms. - The logging mechanism is described in the stream logger library: `libDM_stream_logger <../libdm_stream_logger/index.html>`_. - The protobuf mechanism used is described in the protobuf library: `libDM_protobuf <../libdm_protobuf/index.html>`_. The proposed logging mechanism allows unique file logging for: - protobuf events - mavlink events - binary logs - text logs ... parse_binDM ----------- This part of the repo contains the tools to parse the binary logs logged inside the protolog. As this kind of logs are often high frequency with a lots of variables, the tools are optimized to be fast and efficient. The main script used is located in `/parse_binDM/src/parse_log.cpp` It can be recompiled with the provided build script located at `/parse_binDM/build_unix.sh` or `/parse_binDM/build_windows.bat` The executable files are located in `/parse_binDM/builds/xxx` The script can generates outputs in `csv`or `.mat` format. From a binary file. It can be used with the following command line: .. code-block:: bash ./parse_log_[platform] -i [input_file] -o [output_file] -f [format] with : .. hint:: - input_file: input file path - output_file: output file path - output_format: mat or csv - debug: enable debug printf parse_protolog ----------------- A decoding script is provided to decode completely the protobuf logs, including, events, binary logs, mavlink messages, etc. The script is located in the folder parse_protolog/tools/decode_protobuf.py. The script can be used as follow: - Install the requirements: .. code-block:: bash pip install -r requirements.txt - Run the script: .. code-block:: bash python decode_protobuf_log.py --input_file log.binDM Full options with python3 decode_protobuf_log.py -h: .. code-block:: bash usage: python3 decode_protobuf_log.py --input_file ../test/log_test.binDM --format csv --do_zip True --parse_protolog_custom_binary decode_protobufDM_linux --parse_binary_custom_binary parse_log_linux Extract protobuf encoded logs options: -h, --help show this help message and exit --input_file INPUT_FILE Input file path --format FORMAT Output format for binary data (mat or csv) --do_zip DO_ZIP Zip the extracted file? --parse_protolog_custom_binary PARSE_PROTOLOG_CUSTOM_BINARY Custom compiled protolog extractor --parse_binary_custom_binary PARSE_BINARY_CUSTOM_BINARY Custom compiled binary log extractor The script will create a new folder containing the raw extracted data and decoded files with the indicator **_decoded**. .. note:: If the protobuf/mavlink built files are not present or not up to date, they can be automatically imported with : .. code-block:: python python3 get_messaging.py The files will be imported either from the user workspace, directly taking the files from the protobuf/mavlink repos. If these folders are not present, the script will clone the current repos. The extraction results should looks like this: .. image:: images/command_extraction.png :height: 300px :alt: Protobuf decoded The extracted data will be stored in the `output` folder: .. image:: images/result_extraction.png :height: 300px :alt: Protobuf decoded The content of the different files extracted should looks like: - The binary log: .. image:: images/binary_extracted.png :height: 300px :alt: Protobuf decoded - The protobuf events: .. image:: images/quaternion_extracted.png :height: 300px :alt: Protobuf events - The mavlink messages: .. image:: images/mavlink_extracted.png :height: 300px :alt: Mavlink decoded - The text logs: .. image:: images/ulog_extracted.png :height: 300px :alt: Text logs