decode os1_node/imu_packets and os1_node/lidar_packets

farid

Given a file.bag with following info: $ rosbag info file.bag --freq

path:        file.bag
version:     2.0
duration:    1:04s (64s)
start:       May 07 2019 21:39:57.03 (1557254397.03)
end:         May 07 2019 21:41:01.58 (1557254461.58)
size:        500.1 MB
messages:    47768
compression: none [656/656 chunks]
types:       ouster_ros/PacketMsg [4f7b5949e76f86d01e96b0e33ba9b5e3]
topics:      /os1_node/imu_packets      6456 msgs    : ouster_ros/PacketMsg
             /os1_node/lidar_packets   41312 msgs    : ouster_ros/PacketMsg

I would like to decode it into: /os1_cloud_node/imu and /os1_cloud_node/points with the following python script:

import rospy
from ouster_ros.msg import PacketMsg

def cb(msg):
    rospy.loginfo('len_msg: {}, type_msg: {}'.format(len(msg.buf), type(msg.buf))) <<-- output: len_msg: 49, type_msg: <type 'str'>
    rospy.loginfo('msg: {}'.format(msg.buf)) <-- output: �)�[�:�}����}�@�ʼ@�?|�@�=

def main():
    rospy.init_node("my_sub")
    rospy.Subscriber('/os1_node/imu_packets', PacketMsg,cb)
    rospy.spin()

if __name__ == "__main__":
    main()

The PacketMsg in ouster_ros, however, is

$ rosmsg show ouster_ros/PacketMsg
uint8[] buf

Can anyone help me on what the saved data in uint8[] buf actually contains? Is there any documentation available?

$ rostopic echo /os1_node/imu_packets
buf: [53, 141, 234, .......... 192, 128, 227, 160, 192, 0, 220, 130, 191, 0]

Which looks like a list but as in the above code, I get <type 'str'>!

PS. I have read Software Manual in here about IMU and lidar data format!

OusterNewbie

This is my first reply, I am also struggling with this issue. I was hoping to manipulate raw data in Python easily that I recorded to .bag files.

For my setup, I am using python 2.7.17 (because of ROS dependencies), I import "rosbag" and "rospy", and am able to load my prerecorded .bag file easily using the command myBag = rosbag.Bag('Filename.bag'). I can get it to list what topics are in the bag, and then I pick the LIDAR topic using myBag.read_messages('topics=[/os1_node/lidar_packets']) I can get the timestamp and the msg output for a single packet - I can also iterate thru all the packets in the ROS bag file. All seems ok.

So, if I select the LIDAR topic, and then use a for loop to iterate thru the file, for each message (msg) and timestamp (t), the msg is 12609 bytes in length, coming in 8 bit chunks.

Everyone in the forums says "refer to the manual" but, and on page 20 of V1.13.0 of the OS1 Software User Guide it says that each LIDAR data packet should be 12608 bytes. What is up with the extra byte. I was worried I did the math wrong:

[Timestamp (2 words = 8 bytes) + Measurement ID (0.5 words = 2 bytes)+ Frame ID (0.5 words = 2 bytes) + Encoder Count (1 word = 4 bytes) + 64 Data Blocks (3 words*64 = 768 bytes) + Azimuth Status (1 word = 4 bytes)] * 16 azimuth positions = 12608 bytes

Looks like the math is ok, still all my "msg"s from my .bag files are 12609 bytes. If I look at the data stream from "msg" carefully, I have an extra "0" byte at the end of very message (right after my 0xFFFF "the azimuth is good" 32-bit word). I am throwing the "0" byte at the end of every message away, so, that gets us to 12608 bytes, just like the manual.

Q1) What is that extra "0" byte at the end of every ROS .bag message? OK to discard it?

Well, I guess it is time to pull the message out and make it relevant using pg 20-21 of the manual. This leads me to my next issue:

Q2) Why aren't there public functions to pull relevant data out of the ROS .bag object? It looks like we only can get a timestamp and a message from the bag? Is there a ROS .bag message parser routine for Python already written or some function that I missed?

So, I started to write one, and right off the bat, I can't get the timestamp to match with the myBag time object. Not even close. I suspect the issue is I don't know how the timestamp word is formed.

Q3) If you take the first 8 bytes from the message to get your timestamp, how are these ordered to form the uint64 for the timestamp of that azimuth block? I know it won't match exactly, since I have 16 azimuth blocks, and they will all have a slightly different timestamps, but I can't get it right to even get close. I did however, manage to pull the measurement ID, the frame ID, and the encoder count A-OK, and the values seem correct as I watch the bag get parsed.

So, if anyone can point me to a ROS / Ouster Python library so I can get access to the data (range, reflectivity, etc) in the bag message easily without having to bit shift on my own, I would be grateful. I suspect it would answer (or bypass) all three of my q's above.

eldorado

A1. I'm also disregarding that extra 0 with no apparent impact on the resulting data. I.e., I'm able to reproduce the point cloud with the extracted data.

A2a. We ran into the same problem. It would be nice to have a Python library for reading the UDP packet msg from rosbag, but I couldn't find anything out there.

A2b. As for the timestamp, our experience is that the rosbag and raw UDP times won't match. I don't know what the rosbag timestamp corresponds to, and it's something we're looking into right now. The azimuth blocks (in the UPD packet contained in the rosbag msg) have the raw data and most accurate time. Plus, you'll have 16 azimuth blocks (i.e., 16 timestamps) per rosbag message (i.e., UDP packet) for the /os1_node/lidar_packets topic.

A3. Again, our experience is that the times won't match up, not even close. The UDP timestamp values look most reasonable based on our observations. We're currently disregarding the rosbag time. Separately, we're also running pcl_ros and trying to find the relationship between the PCD time (PointCloud2 time?) and the UDP time.

For our Python code, we couldn't find any standard ROS OS1 Python library, so we rolled our own going down to the byte and bit level. It's not the most efficient but it's a start. We may move over to C/C++ in the future, taking advantage of what Ouster's done already, once we get more proficient.

As a side note, we noticed that the range is 20-bit but there's no standard NumPy datatype for 20 bits. Since the bit numbering is little-endian, we went with uint64 or u8, assuming the extra unused digits would have no impact due to the bit numbering. It worked for the most part; we encountered a small number of extreme outliers that we immediately threw out.