Recent Forum Posts
From categories:
page 1123...next »

I'm using the majordomo pattern for my current setup. I'm using it to spread small requests out amongst a large number of workers. The nature of the services requires that everything be synchronous. With this in mind, I don't think I'm getting the peek performance out of ZMQ. Using the clients in the ZMQ Guide, I am unable to get the relative performance that the guide suggests. Instead of having round trip messages flowing at the rate of 9000+ per second, I seem to only be able to get 1000-2000 per second. So I'm unsure if this is related to the .net components that I'm using or if this is what the expected round trip overheard actually is.

My architecture is also spread over multiple pc's and VM's, so I'm sure that plays some role in it, but I'm not seeing the sub-millisecond response times I was expecting. Roundtrip message times seem to be in the 20-25ms range using the examples provided. Adding my own logic to these provided examples obviously adds more latency, but I'm wondering if these numbers that I'm seeing are expected or not.

running perf/*_lat on multi core machine latencies are worst

Below are the figures

16 cpu machine
./remote_lat tcp://127.0.0.1:5555 30 1000

message size: 30 [B]
roundtrip count: 1000
average latency: 97.219 [us]

single cpu machine
/remote_lat tcp://127.0.0.1:5555 30 1000

message size: 30 [B]
roundtrip count: 1000
average latency: 27.195 [us]

how is number of cpu making difference here?This is repeating in ubuntu ,red hat,centos
using libzmq 4.2.5

Hello

we have a difficult to find problem with a Python script, where an CNN for license plate recognition runs using Intel OpenVino and the communication to other programs is done using ZeroMQ. We are not sure what causes the problem, but as it causes so much network traffic that other PCs cannot communicate anymore, a problem with ZeroMQ seems possible.

Problem description:

We have a Python script, which runs 3 different CNNs using the Inference Engine from OpenVino 2018R5 on images from Ethernet cameras, which are retrieved with OpenCV VideoCapture. In addition ZeroMQ is used to pass results to other programs (via tcp). The used hardware is either an Intel NUC7BNH, an NUC7DNH or an NUC8BEH (on the NUC8 no freeze was observed until now). The OS is an Ubuntu 16.04 (with patched kernel 4.7.0.intel.r5.0 or kernel 4.15.0-15-generic (freezes happen less frequent with kernel 4.15). The script is running multiple times in separated Docker containers together with programs in other docker containers.

What happens is that the Linux freezes randomly after some time (sometimes after a few minutes, sometimes after a few hours but also two are now running for many days without a problem). When it freezes no ACPI shutdown works, the screen freezes and even the Magic SysRq keys have no effect. A strange side effect is that a lot of network traffic is created (so much traffic that the network dies and no PC on the switch can communicate). The logs (kern.log, syslog) show nothing special.

If anyone observed a similar problem or has an idea, what can cause this behavior, please let me know.

Greetings,

Thomas

aziule (guest) 30 Jan 2019 16:40
in discussion Hidden / Per page discussions » Broker vs. Brokerless

Busy for 8 years!

by aziule (guest), 30 Jan 2019 16:40

Are there any links or examples of using ZMQ with VB or VB.net?
Maybe hit a button & start subscribing & showing a stream of published messages

VB install/setup & examples by ZMQjediZMQjedi, 10 Jan 2019 16:53

Hello everyone,

I am trying out 0mq for Python <-> C++ IPC. What is interesting, whenever I send a request shorter than 5 characters from Python client to C++ server I get interesting result:
H+�t
instead of just
H

But if I have at least 5 characters the message is parsed correctly… For Hello and longer I get what I send.
I don't define the buffer length anywhere so I am wondering.

In C++ server code looks like:

        zmq::message_t request;

        //  Wait for next request from client
        socket_.recv (&request);
        std::istringstream iss(static_cast<char*>(request.data()));
        std::string message;
        iss >> message;
        std::cout << "Received " << message << std::endl; // here I get strange output!

        //  Do some 'work'
        sleep(1);

        //  Send reply back to client
    ...

and Python
        print("Sending request to C++")
        self.socket.send_string("H")
        #  Get the reply.
        message = self.socket.recv()
        print("Received reply [ {} ]".format(message.decode('utf-8')))

all the code is taking from the examples hwserver and hwclient

If I send short message from C++ to Python it is parsed correctly though… Interesting.
I am sending long messages so actually it shouldn't be that terrible atm, but it may become problematic at some point.

Thank you for any insight!

Minimum lenght of request by JendkerJendker, 10 Jan 2019 11:54
Tal (guest) 17 Dec 2018 01:40
in discussion Hidden / Per page discussions » Multithreading Magic

Hi,
I enjoyed reading your article and 0mq sounds great.

Yet, there are cases where you need these working threads to maintain a common control state that rarly changes.
How would you suggest to approach this issue?

Regards,
Tal.

by Tal (guest), 17 Dec 2018 01:40
uuuu (guest) 25 Oct 2018 08:43
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

水电费

by uuuu (guest), 25 Oct 2018 08:43
hello (guest) 21 Oct 2018 09:32
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

12312312343113121

by hello (guest), 21 Oct 2018 09:32
hello (guest) 21 Oct 2018 09:32
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

sdf

by hello (guest), 21 Oct 2018 09:32

Call stack of crash is as below

00 KERNELBASE!RaiseException+0x68
01 zmq::zmq_abort(char * errmsg_ = <Value unavailable error>)+0x21 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\err.cpp @ 89]
02 zmq::tcp_read(unsigned int64 s_ = <Value unavailable error>, void * data_ = <Value unavailable error>, unsigned int64 size_ = <Value unavailable error>)+0xa9 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\tcp.cpp @ 276]
03 zmq::stream_engine_t::in_event(void)+0x132 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\stream_engine.cpp @ 318]
04 zmq::select_t::trigger_events(class std::vector<zmq::select_t::fd_entry_t,std::allocator<zmq::select_t::fd_entry_t> > * fd_entries_ = 0x0000019d`323400c8, struct zmq::select_t::fds_set_t * local_fds_set_ = 0x0000006f`e3f9fb90, int event_count_ = 0n2)+0x74 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\select.cpp @ 122]
05 zmq::select_t::select_family_entry(struct zmq::select_t::family_entry_t * family_entry_ = 0x0000019d`323400c8, int max_fd_ = 0n0, bool use_timeout_ = <Value unavailable error>, struct timeval * tv_ = 0x0000006f`e3fffc98)+0xe9 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\select.cpp @ 404]
06 zmq::select_t::loop(void)+0x352 [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\select.cpp @ 360]
07 thread_routine(void * arg_ = <Value unavailable error>)+0xd [objdir-windows-msvc-14.0.x86_64\build\libzmq\src\thread.cpp @ 47]
08 invoke_thread_procedure+0xe (Inline Function @ 00007ff7`50b99cf9) [d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 91]

Can anyone please help how to catch this exception ?

zmq raise exception in I/O thread by vibkvibk, 28 Sep 2018 18:28
zenva (guest) 07 Sep 2018 02:38
in discussion Hidden / Per page discussions » Using ZeroMQ In c#

h呵呵呵

by zenva (guest), 07 Sep 2018 02:38
georgy (guest) 16 Aug 2018 19:47
in discussion Hidden / Per page discussions » ØMQ is just Sockets!

Get it! the guy is only saying that ZMQ is just layer over sockets!

by georgy (guest), 16 Aug 2018 19:47
Valerio (guest) 14 Jun 2018 07:37
in discussion Hidden / Per page discussions » Measuring jitter

Is the raw data and the code available somewhere, so that I can run the same experiment ?

by Valerio (guest), 14 Jun 2018 07:37
tnavda (guest) 09 Jun 2018 20:35
in discussion Hidden / Per page discussions » 0MQ Termination

I had the same problem and came here to say, I don't think you can. Ultimately I used a Proxy_Steerable, and then I said up another PUB/SUB as they did on their sample. Then I sent the TERMINATE to it, and proxy returned and shutdown. Then I closed other connections, shutdown the context. At this point its a clean termination and you can restart a new context or exit your app.

by tnavda (guest), 09 Jun 2018 20:35
Said Sryheni (guest) 08 Jun 2018 14:07
in discussion Hidden / Per page discussions » NORM Transport Notes

I'm using ZMQ with NORM but I'm stuck at an error. Here is my C++ code:

PUB Sender :

string sendHost = "norm:2,127.0.0.1:5556"; <NormNodeId>,<addr:port>
string tag = "MyTag";
string sentMessage = "HelloWorld";
string fullMessage = tag + sentMessage;

zmq::context_t *context = new zmq::context_t( 20 );

zmq::socket_t publisher( *context, ZMQ_PUB );
zmq_connect( publisher, sendHost.c_str() );

zmq_send( publisher,
fullMessage.c_str(),
fullMessage.size(),
0
);

SUB receiver:

char message[256];
string receiveHost = "norm:1,127.0.0.1:5556"; <NormNodeId>,<addr:port>
string tag = "MyTag";

zmq::context_t *context = new zmq::context_t( 20 );

zmq::socket_t subscriber( *context, ZMQ_SUB );
zmq_bind( subscriber, receiveHost.c_str() );
zmq_setsockopt( subscriber, ZMQ_SUBSCRIBE, tag.c_str(), tag.size() );

zmq_recv( subscriber,
message,
256,
0
);

cout « bytesReceived « endl;
cout « message « endl;

When I run the above code on a multi-threaded environment I get the following error printed to my console:

Proto Error: ProtoSocket::Bind() bind() error: Address already in use
Proto Fatal: NormSession::Open() error: rx_socket.Bind() error
Address already in use (src/session_base.cpp:692)
Aborted (core dumped)

I added some debugging statements. First I ran the receiver which binded successfully. After that I ran the sender which failed after the connect statement. I don't why is the error that bind failed while it was supposed to be just a connect request, not a bind one. Any help?

by Said Sryheni (guest), 08 Jun 2018 14:07
chao.wang (guest) 28 Apr 2018 03:06
in discussion Hidden / Per page discussions » ZMQ + Maven

Jzmq.dll and libzmq.dll are required in the jdk\bin directory

by chao.wang (guest), 28 Apr 2018 03:06
Giordano (guest) 16 Apr 2018 15:30
in discussion Hidden / Per page discussions » 0MQ Termination

Bump. I do have the exact same problem and I hope someone will answer. Thanks.

by Giordano (guest), 16 Apr 2018 15:30
Alexey (guest) 23 Mar 2018 17:28
in discussion Hidden / Per page discussions » 0MQ Termination

Anyone?
How do I terminate zmq_proxy?

by Alexey (guest), 23 Mar 2018 17:28

I have written a demo Windows service using POCO library.
As per POCO API the function waitForTerminationRequest() waits for a service termination request.

Now, in this POCO based windows service, I want to start a ZeroMQ library based Message Queue proxy to implement an XSUB/XPUB message queue.
From this service I will be running a xsub-xpub proxy.

More can be learnt here zguide.zeromq.org page:all.

For this I wrote another class ZeroMQProxy, which starts the proxy in the service's main function.

class ZeroMQProxy
{
private:
    zmq::context_t context;
    zmq::socket_t xsub;
    zmq::socket_t xpub;

public:
    ZeroMQProxy()
        : context(1),
        xsub(context, ZMQ_XSUB),        // Publisher End Proxy Sockets
        xpub(context, ZMQ_XPUB)         // Subscriber End Proxy Sockets
    {
    }

    ~ZeroMQProxy()
    {
    }

    void proxyopen()
    {
        xsub.bind("tcp://*:5559");
        xpub.bind("tcp://*:5560");

        zmq::proxy(xsub, xpub, nullptr);
    }

    void proxyclose()
    {
    }
};

class demopocoservice : public ServerApplication
{
private:
    bool _helpRequested;

public:
    demopocoservice() : _helpRequested(false)
    {
    }

    ~demopocoservice()
    {
    }

protected:
    void initialize(Application& self)
    {
        loadConfiguration();
        ServerApplication::initialize(self);
    }

    void uninitialize()
    {
        ServerApplication::uninitialize();
    }

    void defineOptions(OptionSet& options)
    {
        ServerApplication::defineOptions(options);

        options.addOption(
            Option("help", "h", "poco display help")
            .required(false)
            .repeatable(false)
            .callback(OptionCallback<demopocoservice>(
                this, &demopocoservice::handleHelp)));
    }

    void handleHelp(const std::string& name,
        const std::string& value)
    {
        _helpRequested = true;
        displayHelp();
        stopOptionsProcessing();
    }

    void displayHelp()
    {
        HelpFormatter helpFormatter(options());
        helpFormatter.setCommand(commandName());
        helpFormatter.setUsage("OPTIONS");
        helpFormatter.setHeader("poco: Zero message Queue.");
        helpFormatter.format(std::cout);
    }

    int main(const ArgVec& args)
    {
        if (!_helpRequested)
        {
            ZeroMQProxy zmqproxyObj;
            zmqproxyObj.proxyopen();

            waitForTerminationRequest();

            zmqproxyObj.proxyclose();
        }
        return Application::EXIT_OK;
    }
};

int main(int argc, char** argv)
{
    cout << "Hello Poco ZMQ\n";

    demopocoservice pobj;
    pobj.run(argc, argv);

    return 0;
}

My aim was when I start the service then the zmq::proxy() should have started the proxy and when I stop the service then proxy should be closed along with the sockets.

Problem is zmq::proxy() does not return back. So I am not able to stop the service.
Even if I do net stop <service name> the waitForTerminationRequest() does not receive termination request because of zmq::proxy().

I had tried calling the proxy() in a poco runnable thread. But it did not help.

class ZMQThread : public Poco::Runnable
{
private:
    std::string name;
    int value;

public:

    ZMQThread(const std::string& arg1, int arg2) : name(arg1), value(arg2)
    {
    }

    ~ZMQThread()
    {
    }

    void run()
    {
           ZeroMQProxy zmqproxyObj;
           zmqproxyObj.proxyopen();
    }
};

Then called the thread as

            MQThread mqOne("Poco One", 4);
            Poco::Thread threadOne;
            threadOne.start(mqOne);
            threadOne.join();

What should I do to stop/close the proxy when I stop the service?

page 1123...next »