multithreaded epoll

 

I am creating a multithreaded server using epoll (edge-triggered) and non-blocking sockets. Currently I'm creating an event loop on the main thread and waiting for notifications and it works correctly
I have to choose between two approaches to make it multithreaded:

  • Create an event loop for each thread and add the server socket's file descriptor to look for notifications on each thread. (is that possible? I mean: is epoll thread-safe?)
  • Create a single event loop and wait for notifications. Whenever a notification is received, spawn a thread to handle it.

    • If I use the first method, is there a chance for multiple threads to get notified with the same event? how can I handle this situation?

    What could be the best approach? Thank you.

    I think option 1 is more popular since the primary purpose of non-blocking IO is to avoid the overhead of create & destroy threads.

    take the popular web server nginx as an example, it create multiple processes (not threads) to handle incoming events on a handle, and the process the events in the subprocess. all of them share the same listening socket. it's quite similar to option 1.

    I'm also writing a server using epoll , and I have considered the same model as you attached.

    It is possible to use option 1, but it may cause "thundering herd" effect, you can read the source of nginx to find the solution. As for option 2, I deem that it is better to use thread pool instead of spawning a new thread each time.

    And you can also the following model：

    Main thread/process: accept incoming connection with blocking IO, and send the fd to the other threads using BlockingList or to the other processes using PIPE .

    Sub threads/process: create an instance of epoll respectively, and add the incoming fd to the epoll , then processing them with non-blocking IO.

    epoll是线程安全的，一个好的解决方案是你的主进程停留在accept（2）中，一旦你得到文件描述符，在目标线程的epoll fd中注册一个文件描述符，这意味着你为每个线程都有一个epoll队列，一旦创建了线程，就可以在调用pthread_create（3）中共享epoll文件描述符作为参数，因此当新连接到达时，使用目标线程的epoll fd执行epoll_ctl（... EPOLL_CTL_ADD ..）接受（2）后创建的新套接字，是否合理？

    链接地址: http://www.djcxy.com/p/61348.html

    上一篇: epoll并发送滞后

    下一篇: 多线程epoll