RDMA tranfer without mapping memory
I have supplementary processes which should exchange files over IB network. Then the files will be used by other processes.
The workflow is following:
It turned out that the bottleneck in my scheme is ib_reg_mr (I measured separately registering 3Gb memory takes 1.78 seconds). It seems that it triggers mapping of the memory region to the processor's address space. Unfortunately, this operation is not needed, because the receiving processor does not use this memory. The memory should be mapped and used by other processes later.
So I thought it would be wise to circumvent altering page tables for the receiving process, but I was not able to find if it is possible at all.
Could you advise me, if there is a way to initiate a data transfer on a memory region without mapping it to process' address space?
As far as I know there is no way to register a memory region without having it mapped to the process page tables. During registration the pages are pinned so that the mapping doesn't change while the device is accessing the pages, and this requires mapping them to the process. With on-demand-paging, you could delay the mapping of the pages into the process until the HCA actually uses them, but eventually they will be mapped.
Please try FRWR (Fast memory Registration Work Request) instead. It is an API that does memory registration via ibv_post_send verb - much faster way:
http://lists.openfabrics.org/pipermail/general/2008-May/050235.html
http://lxr.free-electrons.com/source/include/rdma/ib_verbs.h?v=3.2#L734
链接地址: http://www.djcxy.com/p/64628.html下一篇: RDMA传输无需映射内存