Zeromq 基础
about zeromq
大神已经走了,留下的才更显弥足珍贵.
why zeromq
raw socket 在面临海量数据和并行时的一些问题,正好是 zeromq 这类 message oriented middleware 要解决的:
- block IO or non-block IO? 前者低效,后者复杂;
- 传统 socket 的 cs 定义,在实际中 server client 随时可能掉线,特别是 server 挂掉很要命;
- socket 传输的数据有大有小,五花八门,raw socket 能 handle?
- 有些复杂的应用场景,比如某些消息不能马上 deliver,要等待 client 上线,怎么处理处理这些消息,database ? or queue?
- message queue 怎么存储?
- 传输信息丢失了怎么办,网络错误,是重发还是丢弃?
- ipv4 ipv6 互转
- message route for multicast peers
- 多语言支持
- 数据的编码管理
解决网络中复杂的连接与数据交换场景;
易用,轻量,快;
强调并行,跨线程,跨进程;
模糊掉 client 和 server 的概念,万物连接更像人的大脑,只有节点,没有绝对的中心;相比 http 为何不好?
it arguably makes the problem worse by encouraging developers and architects to think in terms of big servers and thin, stupid clients.
总的来说:
-
It delivers blobs of data (messages) to nodes, quickly and efficiently.
-
You can map nodes to threads, processes, or nodes. ZeroMQ gives your applications a single socket API to work with, no matter what the actual transport (like in-process, inter-process, TCP, or multicast).
-
It automatically reconnects to peers as they come and go.
-
It queues messages at both sender and receiver, as needed.
-
It limits these queues to guard processes against running out of memory.
-
It handles socket errors.
-
It does all I/O in background threads.
-
It uses lock-free techniques for talking between nodes, so there are never locks, waits, semaphores, or deadlocks.
从应用上看,消息中间件经常解决的问题:
- 网络通信(看成1个网络库?)
- 应用解耦
- 应用消息通讯
- 流量削峰( 消息缓存)
- 分布式消息通信
zeromq example
https://www.ibm.com/developerworks/library/se-distributed-apps-zeromq-part1/index.html
client 访问某个 url 从别的机器的 python machine learning 处理中心要结果,web server 是 node server, 如预测值.
client <--http--> node.js server <--zeromq--> python
https://medium.com/@amit.kulkarni/consuming-services-for-web-using-zeromq-5d9817fec2cd