Limit number of Threads in Task Parallel Library

I have few hundreds of files i need to upload to Azure Blob Storage.
I want to use parallel task library.
But instead of running all the 100 threads to upload in a foreach on list of files, how can i put a limit on max number of threads that it can use and finish the job in parallel. or does it balance the things automatically?

---

You should not be using threads for this at all. There's a Task -based API for this, which is naturally asynchronous: CloudBlockBlob.UploadFromFileAsync. Use it with async/await and SemaphoreSlim to throttle the number of parallel uploads.

Example (untested):

const MAX_PARALLEL_UPLOADS = 5;

async Task UploadFiles()
{
    var files = new List<string>();
    // ... add files to the list

    // init the blob block and
    // upload files asynchronously
    using (var blobBlock = new CloudBlockBlob(url, credentials))
    using (var semaphore = new SemaphoreSlim(MAX_PARALLEL_UPLOADS))
    {
        var tasks = files.Select(async(filename) => 
        {
            await semaphore.WaitAsync();
            try
            {
                await blobBlock.UploadFromFileAsync(filename, FileMode.Create);
            }
            finally
            {
                semaphore.Release();
            }
        }).ToArray();

        await Task.WhenAll(tasks);
    }
}

---

Did you try use MaxDegreeOfParallelism? Like this:

System.Threading.Tasks.Parallel.Invoke(
new Tasks.ParallelOptions {MaxDegreeOfParallelism =  5 }, actionsArray)

---

You can find out by running this:

class Program
{
    static void Main(string[] args)
    {
        var list = new List<int>();

        for (int i = 0; i < 100; i++)
        {
            list.Add(i);
        }

        var runningIndex = 0;

        Task.Factory.StartNew(() => Action(ref runningIndex));

        Parallel.ForEach(list, i =>
        {
            runningIndex ++;
            Console.WriteLine(i);
            Thread.Sleep(3000);
        });

        Console.ReadKey();
    }

    private static void Action(ref int number)
    {
        while (true)
        {
            Console.WriteLine("worked through {0}", number);
            Thread.Sleep(2900);
        }
    }
}

As you can see the number of parallelism is smaller at the start, gets bigger, and grows smaller towards the end. So there definitely is some sort of automatic optimization going on.

链接地址: http://www.djcxy.com/p/50168.html

上一篇: 如何在单独的CPU内核上并行执行任务

下一篇: 限制任务并行库中的线程数