iceoryx 整体框架与模块划分

iceoryx 整体框架与模块划分

源码根:ros2_humble/src/eclipse-iceoryx/iceoryx · 版本 2.0.6(见 VERSION
官方文档:doc/mkdocs.yml、各模块 README.md


1. 产品定位

iceoryx(Eclipse iceoryx™)是一个 进程间零拷贝共享内存通信中间件(IPC middleware),起源于 Bosch 车载领域,现由 Apex.AI 等维护。在 ROS 2 Humble 中它是 CycloneDDS / rmw_iceoryx 的零拷贝(Shared Memory)后端。

设计目标:

目标 实现手段
真零拷贝 payload 从生产到消费始终在同一块共享内存 Chunk 中,只传递”指针”(相对偏移)
无锁 数据面全部使用 lock-free 队列/free-list(LockFreeQueueLoFFLiSoFi
实时安全 关键路径无系统调用、无阻塞;最坏执行时间可预测
无动态内存分配 运行期不调用 new/malloc;一切容器定容量(cxx::vector<T, Capacity> 等),共享内存在 RouDi 启动时一次性划分
无异常 全库 noexcept,错误经 cxx::expected 返回

2. 分层架构(自上而下)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
┌──────────────────────────────────────────────────────────────┐
│ 应用层:ROS 2 (rmw_iceoryx / CycloneDDS-SHM) / 原生 iceoryx │
└──────────────┬───────────────────────────┬───────────────────┘
│ C++ API │ C API
┌──────────────▼──────────────┐ ┌─────────▼─────────────────┐
│ iceoryx_posh(POSIX SHM) │◄─┤ iceoryx_binding_c │
│ Publisher/Subscriber │ │ iox_pub_* / iox_sub_* │
│ RouDi · Runtime · WaitSet │ └───────────────────────────┘
│ mepoo(MemPool/Chunk)·CaPro │
└──────────────┬──────────────┘

┌──────────────▼──────────────────────────────────────────────┐
│ iceoryx_hoofs — 无异常/无堆分配 C++ 基础库 │
│ cxx 容器 · concurrent 无锁原语 · posix 封装 · relocatable ptr│
└──────────────┬──────────────────────────────────────────────┘

┌──────────────▼──────────────────────────────────────────────┐
│ platform 抽象层(linux/mac/qnx/unix/win) │
└──────────────────────────────────────────────────────────────┘

数据面主路径publisher.loan() → MemPool 取 Chunk(共享内存)→ 用户写入 → publish() → 把 Chunk 的相对指针推入各 Subscriber 的无锁队列 → subscriber.take() 直接读同一块内存。
控制面主路径:应用 Runtime 经 IPC channel(Unix Domain Socket) 向 RouDi 注册进程/创建端口 → RouDi 在共享内存管理段中分配端口数据结构并做 CaPro 服务发现/连接。


3. 模块总览

模块 路径 产物 职责
iceoryx_hoofs iceoryx_hoofs/ libiceoryx_hoofs “Handy Objects For Utilizing Files and Streams”:STL 替代容器、无锁并发原语、POSIX 封装、相对指针、platform 层
iceoryx_posh iceoryx_posh/ libiceoryx_posh + iox-roudi 可执行 POSIX SHM 通信核心:RouDi、Runtime、Pub/Sub、Client/Server(RPC)、mepoo、CaPro、WaitSet/Listener、gateway 框架
iceoryx_binding_c iceoryx_binding_c/ libiceoryx_binding_c C 语言绑定(iox_runtime_initiox_pub_loan_chunk 等)
iceoryx_dds iceoryx_dds/ iox-dds-gateway(本仓带 COLCON_IGNORE,ROS 构建中不编译) iceoryx ↔ DDS(Cyclone DDS)网关,跨主机桥接
iceoryx_meta iceoryx_meta/ CMake 元构建 统一构建入口、编译期常量配置(build_options.cmake 定义 IOX_MAX_PUBLISHERS 等)
tools tools/ iox-introspection-client 内省客户端(tools/introspection/)、构建/CI 脚本、docker
(示例/测试) iceoryx_examples/iceoryx_integrationtest/ icedelivery、waitset、iceperf 等示例与集成测试

4. 模块依赖关系

应用网关rmw_iceoryx / CycloneDDS-SHMC 应用iceoryx_dds gatewayiceoryx_binding_ciceoryx_posh<br/>RouDi · Runtime · Pub/Sub · mepooiceoryx_hoofs<br/>cxx · concurrent · posix · rpplatform 抽象层<br/>linux/qnx/mac/winiceoryx_meta<br/>编译期配置
  • posh 依赖 hoofs:所有端口数据结构建立在 cxx 容器 + rp::RelativePointer + concurrent 无锁原语之上
  • binding_c 只包一层 posh,不引入新机制
  • iceoryx_meta 不产生运行时代码,只负责把编译期常量(如 IOX_MAX_PUBLISHERS)传给 posh:
1
2
3
4
5
6
7
8
9
10
11
//--------- Communication Resources Start---------------------
// Publisher
constexpr uint32_t MAX_PUBLISHERS = build::IOX_MAX_PUBLISHERS;
constexpr uint32_t MAX_SUBSCRIBERS_PER_PUBLISHER = build::IOX_MAX_SUBSCRIBERS_PER_PUBLISHER;
constexpr uint32_t MAX_CHUNKS_ALLOCATED_PER_PUBLISHER_SIMULTANEOUSLY =
build::IOX_MAX_CHUNKS_ALLOCATED_PER_PUBLISHER_SIMULTANEOUSLY;
constexpr uint64_t MAX_PUBLISHER_HISTORY = build::IOX_MAX_PUBLISHER_HISTORY;
// Subscriber
constexpr uint32_t MAX_SUBSCRIBERS = build::IOX_MAX_SUBSCRIBERS;
constexpr uint32_t MAX_CHUNKS_HELD_PER_SUBSCRIBER_SIMULTANEOUSLY =
build::IOX_MAX_CHUNKS_HELD_PER_SUBSCRIBER_SIMULTANEOUSLY;

一切资源上限在编译期锁死——这正是”无动态内存分配”目标的体现(另有 MAX_PROCESS_NUMBER = 300 等,见同文件)。


5. 核心概念速览

概念 定义位置(相对 iceoryx 根) 一句话解释
RouDi iceoryx_posh/source/roudi/roudi.cppport_manager.cppprocess_manager.cpp Routing and Discovery 守护进程:创建共享内存段、管理进程注册、创建端口、撮合 Pub/Sub 连接。不在数据路径上
Runtime iceoryx_posh/include/iceoryx_posh/runtime/posh_runtime.hpp 每个应用进程一个单例,负责与 RouDi 的 IPC 会话,代理创建 Publisher/Subscriber 等端口
Publisher / Subscriber iceoryx_posh/include/iceoryx_posh/popo/publisher.hpp / subscriber.hpp(typed),untyped_*.hpp(untyped) 用户 API 层;底层是 internal/popo/ports/ 里的 PublisherPort/SubscriberPort
Chunk iceoryx_posh/include/iceoryx_posh/mepoo/chunk_header.hpp 一次消息的载体:ChunkHeader + 可选 user-header + user-payload,位于共享内存
MemPool iceoryx_posh/include/iceoryx_posh/internal/mepoo/mem_pool.hpp 同尺寸 Chunk 的定长池,free-list 用无锁 LoFFLi 管理
Segment iceoryx_posh/include/iceoryx_posh/internal/mepoo/mepoo_segment.hppsegment_manager.hpp 一个 POSIX 共享内存段 = 一组 MemPool + 读写访问组(按用户组授权)
CaPro iceoryx_posh/include/iceoryx_posh/capro/service_description.hppinternal/capro/capro_message.hpp Canonical Protocol 服务模型:(Service, Instance, Event) 三元组标识一个通信通道,RouDi 据此撮合
WaitSet / Listener iceoryx_posh/include/iceoryx_posh/popo/wait_set.hpp / listener.hpp 事件多路等待:WaitSet 是同步阻塞式(用户线程 wait),Listener 是异步回调式(自带线程)

CaPro 的服务三元组(本质是三个定容字符串 + 128 位类型哈希):

1
2
3
4
5
6
7
8
9
10
11
12
13
private:
/// @brief string representation of the service
IdString_t m_serviceString;
/// @brief string representation of the instance
IdString_t m_instanceString;
/// @brief string representation of the event
IdString_t m_eventString;

/// @brief 128-Bit class hash (32-Bit * 4)
ClassHash m_classHash{0, 0, 0, 0};

/// @brief How far this service should be propagated
Scope m_scope{Scope::WORLDWIDE};

ChunkHeader 是 Chunk 的头部元数据,含版本号以支持兼容性检测与 record&replay:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
struct ChunkHeader
{
using UserPayloadOffset_t = uint32_t;

/// @brief constructs and initializes a ChunkHeader
/// @param[in] chunkSize is the size of the chunk the ChunkHeader is constructed
/// @param[in] chunkSettings are the settings like user-payload size and user-header alignment
ChunkHeader(const uint32_t chunkSize, const ChunkSettings& chunkSettings) noexcept;

// copy/move ctors/assignment operators are deleted since the calculations for the user-header and user-payload
// alignment are dependent on the address of the this pointer
ChunkHeader(const ChunkHeader&) = delete;
ChunkHeader(ChunkHeader&&) = delete;

ChunkHeader& operator=(const ChunkHeader&) = delete;
ChunkHeader& operator=(ChunkHeader&&) = delete;

/// @brief From the 1.0 release onward, this must be incremented for each incompatible change, e.g.
/// - data width of members changes
/// - members are rearranged
/// - semantic meaning of a member changes
static constexpr uint8_t CHUNK_HEADER_VERSION{1U};

MemPool 的实现完全体现”共享内存 + 无锁”两大主题(成员是相对指针 + LoFFLi 无锁 free-list):

1
2
3
4
5
6
7
8
9
10
class MemPool
{
public:
using freeList_t = concurrent::LoFFLi;
static constexpr uint64_t CHUNK_MEMORY_ALIGNMENT = 8U; // default alignment for 64 bit

MemPool(const cxx::greater_or_equal<uint32_t, CHUNK_MEMORY_ALIGNMENT> chunkSize,
const cxx::greater_or_equal<uint32_t, 1> numberOfChunks,
posix::Allocator& managementAllocator,
posix::Allocator& chunkMemoryAllocator) noexcept;

6. 进程模型:RouDi 守护进程 + 应用进程 + IPC channel

RouDi 守护进程 iox-roudi应用进程 A应用进程 BProcessManagerPortManager<br/>CaPro 撮合RouDiMemoryManager<br/>创建共享内存段PoshRuntimePublisherPoshRuntimeSubscriber(共享内存<br/>管理段 + 用户段)

要点:

  1. RouDi 必须先启动:它 mmap 创建两类段——管理段(端口数据、队列、内省数据)和用户 payload 段(按 POSIX 用户组划分访问权限,见 roudi/memory/mepoo/segment_manager.hpp)。
  2. 应用启动即注册PoshRuntime::initRuntime("app_name") 通过 Unix Domain Socket 发送 REG,RouDi 回 REG_ACK 并附上共享内存布局信息;之后应用 mmap 同样的段。
  3. IPC channel 只走控制面,消息类型一目了然:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
enum class IpcMessageType : int32_t
{
BEGIN = -1,
NOTYPE = 0,
REG, // register app
REG_ACK,
CREATE_PUBLISHER,
CREATE_PUBLISHER_ACK,
CREATE_SUBSCRIBER,
CREATE_SUBSCRIBER_ACK,
CREATE_CLIENT,
CREATE_CLIENT_ACK,
CREATE_SERVER,
CREATE_SERVER_ACK,
CREATE_INTERFACE,
CREATE_INTERFACE_ACK,
CREATE_CONDITION_VARIABLE,
CREATE_CONDITION_VARIABLE_ACK,
CREATE_NODE,
CREATE_NODE_ACK,
KEEPALIVE,
TERMINATION,
TERMINATION_ACK,
PREPARE_APP_TERMINATION,
PREPARE_APP_TERMINATION_ACK,
ERROR,
APP_WAIT,
WAKEUP_TRIGGER,
REPLAY,
MESSAGE_NOT_SUPPORTED,
// etc..
END,
};
  1. Runtime 是应用侧的”RouDi 客户端”,所有端口创建都是向 RouDi 发请求:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
/// @brief The runtime that is needed for each application to communicate with the RouDi daemon
class PoshRuntime
{
public:
PoshRuntime(const PoshRuntime&) = delete;
PoshRuntime& operator=(const PoshRuntime&) = delete;
PoshRuntime(PoshRuntime&&) = delete;
PoshRuntime& operator=(PoshRuntime&&) = delete;
virtual ~PoshRuntime() noexcept = default;

/// @brief returns active runtime
///
/// @return active runtime
static PoshRuntime& getInstance() noexcept;

/// @brief creates the runtime with given name
///
/// @param[in] name used for registering the process with the RouDi daemon
///
/// @return active runtime
static PoshRuntime& initRuntime(const RuntimeName_t& name) noexcept;
  1. KEEPALIVE 心跳:RouDi 周期性检查应用存活,应用崩溃后回收其 Chunk 与端口(roudi/process_manager.cpp)。

7. 一次发布-订阅数据流总览

RouDiSubscriber(进程B)Subscriber无锁队列(共享内存)MemPool(共享内存)Publisher(进程A)RouDiSubscriber(进程B)Subscriber无锁队列(共享内存)MemPool(共享内存)Publisher(进程A)建链(控制面,仅一次)数据面(每条消息,RouDi 不参与)CREATE_PUBLISHER (CaPro: Service/Instance/Event)CREATE_SUBSCRIBER (同一 CaPro)PortManager 撮合,把 S 的队列挂到 P 的端口loan() → LoFFLi.pop() 取空闲 Chunk直接在 Chunk 上构造/写入数据publish() → Chunk 相对指针入队(引用计数++)take() → 弹出相对指针就地读 payload(零拷贝)释放 → 引用计数--,归零则 LoFFLi.push() 回池

关键性质:

  • 传的是”相对指针”不是数据——多订阅者时同一 Chunk 被引用计数共享,全程 0 次 memcpy。
  • 队列溢出策略可配:默认丢最旧(SoFi/LockFreeQueue 的 overflow push 语义),也可配置阻塞 publisher(ConsumerTooSlowPolicy::WAIT_FOR_CONSUMER)。
  • 指针跨进程有效的根基是 hoofs 的 RelativePointer(详见 01-hoofs基础库.md 第 6 节)。

8. 目录结构对照表

路径(相对 iceoryx 根) 内容
iceoryx_hoofs/include/iceoryx_hoofs/cxx/ STL 替代:expected/optional/variant/vector/string/function_ref
iceoryx_hoofs/include/iceoryx_hoofs/concurrent/ 公开无锁队列:lockfree_queue.hppresizeable_lockfree_queue.hpp
iceoryx_hoofs/include/iceoryx_hoofs/internal/concurrent/ sofi.hpploffli.hppsmart_lock.hpptrigger_queue.hpp
iceoryx_hoofs/include/iceoryx_hoofs/posix_wrapper/ + internal/posix_wrapper/ 信号量、互斥量、共享内存、UDS、ACL 等 POSIX 封装
iceoryx_hoofs/include/iceoryx_hoofs/internal/relocatable_pointer/ RelativePointer/relocatable_ptr(零拷贝的基石)
iceoryx_hoofs/platform/{linux,mac,qnx,unix,win}/ 平台抽象层头文件
iceoryx_posh/include/iceoryx_posh/popo/ 用户 API:Publisher/Subscriber/Client/Server/WaitSet/Listener
iceoryx_posh/include/iceoryx_posh/mepoo/ + internal/mepoo/ Memory Pool:ChunkHeader、MemPool、Segment、MemoryManager
iceoryx_posh/include/iceoryx_posh/capro/ CaPro 服务描述
iceoryx_posh/include/iceoryx_posh/roudi/ + source/roudi/ RouDi 应用框架、PortManager、ProcessManager、内存编排
iceoryx_posh/include/iceoryx_posh/runtime/ PoshRuntime、Node、ServiceDiscovery
iceoryx_posh/include/iceoryx_posh/gateway/ 网关基类(供 iceoryx_dds 等复用)
iceoryx_binding_c/include/iceoryx_binding_c/ C API 头文件
iceoryx_dds/ DDS 网关(本仓 COLCON_IGNORE)
iceoryx_meta/ 元构建 + build_options.cmake(编译期上限配置)
tools/introspection/ iox-introspection-client(ncurses 内省界面)
iceoryx_examples/ icedelivery / iceoptions / waitset / callbacks / iceperf 等示例

9. 文档阅读路线

  1. 本文 — 模块划分、进程模型与数据流
  2. 01-hoofs基础库.md — cxx 容器、无锁原语、POSIX 封装、相对指针(理解零拷贝的关键)

下一篇01-hoofs基础库.md

文章互动

阅读 --

留言

0 条留言

正在加载留言…