Parallel Foreach slow creation of the threads

I have an IEnumerable with a lot of items that needs to be processed in parallel. The items are not CPU intensive. Ideally these items should be executed simultaneouslyon 100 threads or more.

I've tried to do this with Parallel.ForEach(). That works, but the problem is that new threads are spawn too slowly. It takes a (too) long time before the Parallel.Foreach() reaches the 100 threads. I know there is a MaxDegreeOfParallelism property, but that's a maximum, not mimimum.

Is there a way to execute the foreach immediatelly on 100 threads? ThreadPool.SetMinThreads is something that we prefer to avoid, because it has an impact on the whole process.

Is there a solution possible with a custom partitioner?

---

I'm pinging a lot of devices with a timeout of 5 seconds. How would you do that as quick as possible with only 4 threads (4cores)?

I'm going to assume your pinging devices on a LAN and each one is identifiable and reachable by an IP address.

namespace PingManyDevices {

    public class DeviceChecker {                

        public async Task<PingReply[]> CheckAllDevices(IEnumerable<IPAddress> devices) {
            var pings = devices.Select(address => new Ping().SendPingAsync(address, 5000));
            return await Task.WhenAll(pings);
        }
        /***
        * Maybe push it a little further
        ***/ 
        public async Task<PingReply[]> CheckAllDevices(IEnumerable<IPAddress> devices) {
            var pings = devices.AsParallel().Select(address => new Ping().SendPingAsync(address, 5000));
            return await Task.WhenAll(pings);
        }          
    }
} 

---

I've had success using ThreadPool instead of Parallel:

public static void ThreadForEach<T>(this IEnumerable<T> items, Action<T> action)
{
    var mres = new List<ManualResetEvent>();

    foreach (var item in items)
    {
        var mre = new ManualResetEvent(false);

        ThreadPool.QueueUserWorkItem((i) =>
        {
            action((T)i);
            mre.Set();
        }, item);

        mres.Add(mre);
    }

    mres.ForEach(mre => mre.WaitOne());
}

In cases where I've had to use this, it ran faster than attempts using Parallel.ForEach. I can only speculate that it is because it attempts to use already existing threads (instead of taking the overhead to create new ones).

链接地址: http://www.djcxy.com/p/50174.html

上一篇: 如何在使用Ninject的Web应用程序中使用线程作用域进行并行操作

下一篇: 并行Foreach缓慢创建线程