Recent Forum Posts
From categories:
page 1123...next »
Me Me (guest) 03 Jan 2020 21:13
in discussion Hidden / Per page discussions » Broker vs. Brokerless

Nine years!

by Me Me (guest), 03 Jan 2020 21:13

For those interested, I figured out my issue. I forgot to add the identity and empty frames at the beginning of the reply message.

The identity frame is necessary for the DealerSocket to figure out which caller's ReceiveReady event to invoke, so without those frames it will not invoke ANY RecieveReady events, as if the message wasn't even received. I am a little surprised that this socket didn't raise an exception indicating that the socket received a reply message that did not match up with a previously sent message (via the identity value).

I'm pretty sure I had the identity frame and empty frame at the beginning of the reply message, but I was switching back and forth between the Request/Response and Dealer/Router patterns and along the way I removed them thinking I was simplifying the reply message and forgot that they were necessary.

I am trying to use zmq with C# and Python and have run into an issue that I hoping someone can help me with. My DealerSocket's ReceiveReady event is not being invoked!

I am trying to implement an asynchronous client where a C#/NetMQ client sends a message to the Python/pyzmq 'server' and later gets a reply via a callback method.

On the C# side, I have a DealerSocket and I subscribe the method MyReceiveReady to the socket's ReceiveReady event. I also create a poller and add the client to the poller and I am calling the poller.RunAsync() method to start the poller. The client assembles a NetMQMessage and sends the message via the client.SendMultipartMessage(msg) method.

On the Python side, I have a RouterSocket that receives a message with the server.recv_multipart() method, performs some work with the contents of the message, then assembles a return message and replies to the client with the server.send_multipart(msg) method.

When I run the Python code and then the C# code, the C# client sends the message, the Python server receives the message and sends the reply, but the client's ReceiveReady event never invokes, so the client never receives the reply! If I implement the server in C# the client's event IS invoked, but, I have a requirement that the server be in Python and the client in C#.

Am I doing something wrong? Or, is this some deficiency or bug between NetMQ and pyzmq? Please help!

Here is what I've learned in dealing with this issue:

  • Do not use blocking receive in sockets that are created in a separate thread
  • DO set linger to 0 in sockets that send, but make sure not to close the socket until you are ready to exit or the socket has successfully sent all it's messages
  • Close the socket in the thread that created it, not in the main thread
  • Close the context after all the threads have been closed

Finally, see the various answers that address this issue of "infinite wait":
stackoverflow[dot]com/questions/7538988/zeromq-how-to-prevent-infinite-wait

JohnDoe (guest) 21 Oct 2019 07:17
in discussion Hidden / Per page discussions » ØMQ is just Sockets!

Agreed.

by JohnDoe (guest), 21 Oct 2019 07:17
myself (guest) 18 Oct 2019 11:14
in discussion Hidden / Per page discussions » ØMQ is just Sockets!

I have to watch Life of Brian again because of this comment =D

by myself (guest), 18 Oct 2019 11:14

I'm using the majordomo pattern for my current setup. I'm using it to spread small requests out amongst a large number of workers. The nature of the services requires that everything be synchronous. With this in mind, I don't think I'm getting the peek performance out of ZMQ. Using the clients in the ZMQ Guide, I am unable to get the relative performance that the guide suggests. Instead of having round trip messages flowing at the rate of 9000+ per second, I seem to only be able to get 1000-2000 per second. So I'm unsure if this is related to the .net components that I'm using or if this is what the expected round trip overheard actually is.

My architecture is also spread over multiple pc's and VM's, so I'm sure that plays some role in it, but I'm not seeing the sub-millisecond response times I was expecting. Roundtrip message times seem to be in the 20-25ms range using the examples provided. Adding my own logic to these provided examples obviously adds more latency, but I'm wondering if these numbers that I'm seeing are expected or not.

running perf/*_lat on multi core machine latencies are worst

Below are the figures

16 cpu machine
./remote_lat tcp://127.0.0.1:5555 30 1000

message size: 30 [B]
roundtrip count: 1000
average latency: 97.219 [us]

single cpu machine
/remote_lat tcp://127.0.0.1:5555 30 1000

message size: 30 [B]
roundtrip count: 1000
average latency: 27.195 [us]

how is number of cpu making difference here?This is repeating in ubuntu ,red hat,centos
using libzmq 4.2.5

Hello

we have a difficult to find problem with a Python script, where an CNN for license plate recognition runs using Intel OpenVino and the communication to other programs is done using ZeroMQ. We are not sure what causes the problem, but as it causes so much network traffic that other PCs cannot communicate anymore, a problem with ZeroMQ seems possible.

Problem description:

We have a Python script, which runs 3 different CNNs using the Inference Engine from OpenVino 2018R5 on images from Ethernet cameras, which are retrieved with OpenCV VideoCapture. In addition ZeroMQ is used to pass results to other programs (via tcp). The used hardware is either an Intel NUC7BNH, an NUC7DNH or an NUC8BEH (on the NUC8 no freeze was observed until now). The OS is an Ubuntu 16.04 (with patched kernel 4.7.0.intel.r5.0 or kernel 4.15.0-15-generic (freezes happen less frequent with kernel 4.15). The script is running multiple times in separated Docker containers together with programs in other docker containers.

What happens is that the Linux freezes randomly after some time (sometimes after a few minutes, sometimes after a few hours but also two are now running for many days without a problem). When it freezes no ACPI shutdown works, the screen freezes and even the Magic SysRq keys have no effect. A strange side effect is that a lot of network traffic is created (so much traffic that the network dies and no PC on the switch can communicate). The logs (kern.log, syslog) show nothing special.

If anyone observed a similar problem or has an idea, what can cause this behavior, please let me know.

Greetings,

Thomas

aziule (guest) 30 Jan 2019 16:40
in discussion Hidden / Per page discussions » Broker vs. Brokerless

Busy for 8 years!

by aziule (guest), 30 Jan 2019 16:40

Are there any links or examples of using ZMQ with VB or VB.net?
Maybe hit a button & start subscribing & showing a stream of published messages

VB install/setup & examples by ZMQjediZMQjedi, 10 Jan 2019 16:53

Hello everyone,

I am trying out 0mq for Python <-> C++ IPC. What is interesting, whenever I send a request shorter than 5 characters from Python client to C++ server I get interesting result:
H+�t
instead of just
H

But if I have at least 5 characters the message is parsed correctly… For Hello and longer I get what I send.
I don't define the buffer length anywhere so I am wondering.

In C++ server code looks like:

        zmq::message_t request;

        //  Wait for next request from client
        socket_.recv (&request);
        std::istringstream iss(static_cast<char*>(request.data()));
        std::string message;
        iss >> message;
        std::cout << "Received " << message << std::endl; // here I get strange output!

        //  Do some 'work'
        sleep(1);

        //  Send reply back to client
    ...

and Python
        print("Sending request to C++")
        self.socket.send_string("H")
        #  Get the reply.
        message = self.socket.recv()
        print("Received reply [ {} ]".format(message.decode('utf-8')))

all the code is taking from the examples hwserver and hwclient

If I send short message from C++ to Python it is parsed correctly though… Interesting.
I am sending long messages so actually it shouldn't be that terrible atm, but it may become problematic at some point.

Thank you for any insight!

Minimum lenght of request by JendkerJendker, 10 Jan 2019 11:54
Tal (guest) 17 Dec 2018 01:40
in discussion Hidden / Per page discussions » Multithreading Magic

Hi,
I enjoyed reading your article and 0mq sounds great.

Yet, there are cases where you need these working threads to maintain a common control state that rarly changes.
How would you suggest to approach this issue?

Regards,
Tal.

by Tal (guest), 17 Dec 2018 01:40
uuuu (guest) 25 Oct 2018 08:43
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

水电费

by uuuu (guest), 25 Oct 2018 08:43
hello (guest) 21 Oct 2018 09:32
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

12312312343113121

by hello (guest), 21 Oct 2018 09:32
hello (guest) 21 Oct 2018 09:32
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

sdf

by hello (guest), 21 Oct 2018 09:32

Call stack of crash is as below

00 KERNELBASE!RaiseException+0x68
01 zmq::zmq_abort(char * errmsg_ = <Value unavailable error>)+0x21 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\err.cpp @ 89]
02 zmq::tcp_read(unsigned int64 s_ = <Value unavailable error>, void * data_ = <Value unavailable error>, unsigned int64 size_ = <Value unavailable error>)+0xa9 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\tcp.cpp @ 276]
03 zmq::stream_engine_t::in_event(void)+0x132 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\stream_engine.cpp @ 318]
04 zmq::select_t::trigger_events(class std::vector<zmq::select_t::fd_entry_t,std::allocator<zmq::select_t::fd_entry_t> > * fd_entries_ = 0x0000019d`323400c8, struct zmq::select_t::fds_set_t * local_fds_set_ = 0x0000006f`e3f9fb90, int event_count_ = 0n2)+0x74 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\select.cpp @ 122]
05 zmq::select_t::select_family_entry(struct zmq::select_t::family_entry_t * family_entry_ = 0x0000019d`323400c8, int max_fd_ = 0n0, bool use_timeout_ = <Value unavailable error>, struct timeval * tv_ = 0x0000006f`e3fffc98)+0xe9 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\select.cpp @ 404]
06 zmq::select_t::loop(void)+0x352 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\select.cpp @ 360]
07 thread_routine(void * arg_ = <Value unavailable error>)+0xd [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\thread.cpp @ 47]
08 invoke_thread_procedure+0xe (Inline Function @ 00007ff7`50b99cf9) [d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 91]

Can anyone please help how to catch this exception ?

zmq raise exception in I/O thread by vibkvibk, 28 Sep 2018 18:28
zenva (guest) 07 Sep 2018 02:38
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

h呵呵呵

by zenva (guest), 07 Sep 2018 02:38
georgy (guest) 16 Aug 2018 19:47
in discussion Hidden / Per page discussions » ØMQ is just Sockets!

Get it! the guy is only saying that ZMQ is just layer over sockets!

by georgy (guest), 16 Aug 2018 19:47
Valerio (guest) 14 Jun 2018 07:37
in discussion Hidden / Per page discussions » Measuring jitter

Is the raw data and the code available somewhere, so that I can run the same experiment ?

by Valerio (guest), 14 Jun 2018 07:37
page 1123...next »