How can I limit Parallel.ForEach?
I have a Parallel.ForEach() async loop with which I download some webpages. My bandwidth is limited so I can download only x pages per time but Parallel.ForEach executes whole list of desired webpages.
Is there a way to limit thread number or any other limiter while running Parallel.ForEach?
Demo code:
Parallel.ForEach(listOfWebpages, webpage => {
Download(webpage);
});
The real task has nothing to do with webpages, so creative web crawling solutions won't help.
You can specify a MaxDegreeOfParallelism in a ParallelOptions parameter:
Parallel.ForEach(
listOfWebpages,
new ParallelOptions { MaxDegreeOfParallelism = 4 },
webpage => { Download(webpage); }
);
MSDN: Parallel.ForEach
MSDN: ParallelOptions.MaxDegreeOfParallelism
您可以使用ParallelOptions并设置MaxDegreeOfParallelism来限制并发线程的数量：
Parallel.ForEach(listOfwebpages, new ParallelOptions{MaxDegreeOfParallelism=2}, webpage => {Download(webpage);});
使用另一个接受ParallelOptions实例的Parallel.Foreach重载，并设置MaxDegreeOfParallelism以限制并行执行多少个实例。