IMU Data Ingestion and TCP Telemetry Forwarding

This recipe demonstrates how to ingest, parse, and forward high-frequency vibration data from an STMicroelectronics LSM6DSR IMU. The system captures 64-sample batches at 6664 Hz via an ESP32, transmits them as binary frames over a serial link, and uses CycBox to decode the signals into engineering units while forwarding the raw telemetry to a downstream TCP server.

What this recipe does

• Synchronizes binary frames using the AA 55 prefix and validates 780-byte packets using a CRC16-CCITT checksum.
• Parses 64-sample batches consisting of 3-axis accelerometer and 3-axis gyroscope data (12 bytes per sample).
• Corrects microsecond jitter by anchoring the MCU's internal clock against the CycBox engine timestamp.
• Converts raw LSB values to physical units: m/s² for acceleration and degrees per second (dps) for angular velocity.
• Forwards raw binary frames to a remote TCP server at 127.0.0.1:8080 for archival or further processing.

End-to-end data flow

Device and wire protocol

The LSM6DSR is a high-performance 6-axis IMU. In this configuration, it is operated at its maximum Output Data Rate (ODR) of 6664 Hz. To handle the high data throughput, the ESP32 firmware reads the internal hardware FIFO and aggregates 64 samples into a single packet.

The table below defines the 780-byte binary frame layout used by the device.

Offset	Size	Field	Description
0	2	Sync	Fixed AA 55 prefix.
2	1	Type	Packet type ID (0x01 for IMU batches).
3	1	Count	Number of samples in batch (fixed at 64).
4	2	Seq	16-bit Little-Endian sequence counter.
6	4	ts_us	32-bit Little-Endian MCU microsecond timestamp.
10	768	Data	64 samples of 12 bytes each: {ax, ay, az, gx, gy, gz} (int16 LE).
778	2	CRC	CRC16-CCITT (False) checksum.

CycBox configuration

The setup utilizes two connections: a serial transport for device ingestion and a TCP client for data egress.

Connection 0: Device Ingestion

This connection handles the serial communication with the ESP32 and performs the initial frame synchronization and checksum verification.

{
  "app": {
    "app_transport": "serial_port_transport",
    "app_codec": "frame_codec"
  },
  "serial_port_transport": {
    "serial_port_transport_baud_rate": 115200,
    "serial_port_transport_port": "/dev/ttyACM0"
  },
  "frame_codec": {
    "frame_codec_length_mode": "fixed",
    "frame_codec_checksum_algo": "crc16_ccitt",
    "frame_codec_prefix": "AA 55",
    "frame_codec_fixed_payload_size": 776
  }
}

Connection 1: TCP Forwarding

This connection acts as the downstream sink, receiving the original raw frames forwarded by the Lua script.

{
  "app": {
    "app_transport": "tcp_client_transport",
    "app_codec": "passthrough_codec"
  },
  "tcp_client_transport": {
    "tcp_client_transport_host": "127.0.0.1",
    "tcp_client_transport_port": 8080
  }
}

Lua pipeline walkthrough

The Lua script is the core of the integration. It filters for messages from the serial port, validates the data, performs the batch-to-sample expansion, and routes the raw frame to the TCP connection.

-- LSM6DSR FIFO IMU Burst Packet Parser and Forwarder
-- Frame Codec over Serial Port (Conn 0): parses 776-byte batched IMU payloads from an LSM6DSR sensor.
-- TCP Client (Conn 1): Passthrough Codec to forward the raw frames.
-- Decodes 64 ACC/GYR samples per batch (6664 Hz) and aligns MCU microsecond timestamps to the receive time.
-- Incoming raw frames from the serial port are forwarded out via the TCP connection.
--
-- Packet Layout (Payload only - 776 bytes):
--   [1]     type
--   [2]     count: 64
--   [3-4]   seq  (LE uint16)
--   [5-8]   ts_us (LE uint32, MCU µs clock)
--   [9-776] 64 × {ax, ay, az, gx, gy, gz} (int16 LE, 12 bytes/sample)
--
-- Scales (LSM6DSR):
--   Accel ±2 g   → 0.061 mg/LSB → ~0.000598 m/s²
--   Gyro  ±2000 dps → 70 mdps/LSB → 0.07 dps

local PAYLOAD_SIZE = 776
local SAMPLE_SIZE  = 12

local INTERVAL_US  = 1000000 / 6664

-- LSM6DSR: 0.061 mg/LSB at ±2 g
local ACCEL_SCALE  = 0.061e-3 * 9.80665
-- LSM6DSR: 70 mdps/LSB at ±2000 dps
local GYRO_SCALE   = 0.07

local anchor_recv_us = nil
local anchor_mcu_us  = nil

local function mcu_delta(current, anchor)
    local d = current - anchor
    if d < 0 then
        d = d + 4294967296
    end
    return d
end

function on_receive()
    -- Only process and forward messages originating from the Serial connection (ID 0)
    if message.connection_id ~= 0 then
        return false
    end

    local p = message.payload

    if p == nil or #p ~= PAYLOAD_SIZE then
        return false
    end

    if message.checksum_valid == false then
        log("warn", "IMU: checksum invalid.")
        return false
    end

    -- Forward the original complete wire frame to the TCP connection (ID 1)
    if message.frame then
        send_after(message.frame, 0, 1)
    end

    local ptype = read_u8(p, 1)
    local count  = read_u8(p, 2)
    local seq    = read_u16_le(p, 3)
    local ts_us  = read_u32_le(p, 5)

    local recv_us = tonumber(message.timestamp)
    local newest_mcu_us = (ts_us + (count - 1) * INTERVAL_US) % 4294967296

    if anchor_recv_us == nil then
        anchor_recv_us = recv_us
        anchor_mcu_us  = newest_mcu_us
        log("info", string.format("IMU: anchor set — recv=%s mcu=%u", tostring(recv_us), newest_mcu_us))
    end

    local oldest_us = anchor_recv_us + mcu_delta(ts_us, anchor_mcu_us)

    for i = 0, count - 1 do
        local off = 9 + i * SAMPLE_SIZE

        local ax = read_i16_le(p, off)
        local ay = read_i16_le(p, off + 2)
        local az = read_i16_le(p, off + 4)
        local gx = read_i16_le(p, off + 6)
        local gy = read_i16_le(p, off + 8)
        local gz = read_i16_le(p, off + 10)

        local ts = math.floor(oldest_us + i * INTERVAL_US)

        message:add_float_value("ax", ax * ACCEL_SCALE, ts)
        message:add_float_value("ay", ay * ACCEL_SCALE, ts)
        message:add_float_value("az", az * ACCEL_SCALE, ts)
        
        message:add_float_value("gx", gx * GYRO_SCALE, ts)
        message:add_float_value("gy", gy * GYRO_SCALE, ts)
        message:add_float_value("gz", gz * GYRO_SCALE, ts)
    end

    message:add_int_value("count", count)
    message:add_int_value("seq", seq)
    message:add_int_value("mcu_ts_us", ts_us)

    return true
end

Downstream service contracts

TCP Egress

• Endpoint: 127.0.0.1:8080
• Protocol: Raw binary (passthrough).
• Frame Schema: The server receives the exact 780-byte frames as defined in the protocol section, including the AA 55 header and trailing CRC.

Operational concerns

• Baud Rate Constraints: At 6664 Hz with 12 bytes per sample, the raw data rate is approximately 640 kbps. The configured 115200 baud is insufficient for real-time streaming of full 6.6 kHz data. For full-rate operation, the serial baud rate must be increased to at least 921600 or use a high-speed transport like USB Serial JTAG as indicated in the ESP32 source code.
• Timestamp Drifting: The Lua script anchors the first packet to the system clock. While mcu_delta handles microsecond rollovers, long-term drift between the ESP32 crystal and the CycBox host clock may occur. For long-duration captures, the anchor should be periodically resynchronized.
• TCP Backpressure: If the TCP server at 127.0.0.1:8080 becomes unresponsive, the passthrough_codec will drop packets once internal buffers are full to prioritize live engine stability.

Reproducing this recipe

1. Hardware: ESP32 connected to an LSM6DSR sensor via SPI.
2. Device Firmware: Compile and flash the ESP32 with the provided lsm6dsr_fifo.c source, ensuring LSM6DSR_MODE_FIFO is selected.
3. Downstream: Start a TCP listener (e.g., nc -l 8080 > imu_data.bin).
4. CycBox Setup: Apply the JSON configuration to create the Serial and TCP connections, then paste the Lua script into the pipeline editor.
5. Validation: Monitor the Device Message History; you should see RX frames on Connection 0 and immediate TX echoes on Connection 1.

Gotchas and recommendations

• CRC Validation: The frame_codec validates the CRC16-CCITT before the Lua script runs. If message.checksum_valid is false, the script logs a warning and exits, preventing corrupted data from being forwarded.
• Scaling Factors: The ACCEL_SCALE and GYRO_SCALE are specific to the ±2g and ±2000 dps ranges. If the sensor sensitivity is changed in the ESP32 firmware, these constants in the Lua script must be updated accordingly.
• Resource Usage: Parsing 64 samples per frame at high frequency is computationally intensive. Ensure the host environment has sufficient CPU cycles to avoid processing lag.