Tfs Build – bug with Queued Builds, MaxConcurrentBuilds and Tags

On a recent TFS consultancy job, I was asked to monitor how long some builds spent waiting in the Build queue before Starting.

My plan was to use the TFS API  to query all builds with a status of ‘Queued’ and monitor the wait times.

I wrote the code and everything seemed to work fine.  However, after capturing a number of wait times and comparing them to the overall build times I noticed that the times did not match together.

In fact a build that was estimated to complete in 2 hours; took more than  6 hours and did not spend any time ‘Queued’

How?

The Build Controller distributes builds across multiple agents  and will start one build per build agent. (given you haven’t changed the default MaxConcurrentBuilds setting ).  i.e If you have 3 build agents and you start 3 builds then the controller will set 3 builds into ‘InProgress.’  If you start a 4th build the this build will be ‘Queued’

This works fine given that any build agent can run any build definition

Unfortunately it does not take into account ‘Tags’ that may force certain builds onto specific agents.

Given the same conditions of 3 build agents:-
If you tag  a build agent  so only certain builds can use it and then start 3 builds that should only run on this tagged build agent. –> well you would expect that only one build would be set ‘InProgress’ and the other 2 builds would remain ‘Queued’ until the build agent finished the 1st build.

However the actual behaviour is that all 3 builds change to ‘InProgress’ at the same time; one per the MaxConcurrentBuilds setting on the build controller);  but only the first build is actually doing anything. The second two builds are stuck waiting to be allocated an agent .

image

Side Affects

You look at your dashboard and see a list of builds ‘In Progress’ that are actually blocked waiting for a build agent.

image

On the above screen-shot, only 213 is actually running on Build Agent 1. (214 and 215 are blocked waiting for agent 1 to become available)

Worse than that is 217; that can run on any build agent; is blocked in a ‘queued’ state when there are 2 idle build agents that could be running this build. However, It cannot start because the MaxConcurrentBuilds value of 3 has been reach.

Conclusion

Be very careful with the use of Tags. In future I will try and avoid tags when it could introduce the above bottleneck.

Additionally when attempting to use the TFS Api  to capture metrics on wait times then you cannot rely on Queued build only.  Instead I’ll query all builds assigned to a controller; and then filter out the list of builds that have been assigned a build agent –> This will give me the accurate list of builds pending.

73 Replies to “Tfs Build – bug with Queued Builds, MaxConcurrentBuilds and Tags”

  1. Hello I am so glad I found your site, I really found you by mistake, while I was researching on Digg for something else, Anyhow I am here now and would just like to say kudos for a fantastic post and a all round entertaining blog (I also love the theme/design), I don't have time to read it all at the moment but I have bookmarked it and also included your RSS feeds, so when I have time I will be back to read a lot more, Please do keep up the great work. Visit my web site: Michael Kors Outlet ( Ethel – http://www.avlogbook.com/michael-kors-handbags.html )

  2. Opposers would say that consumers get trapped into using these financing options frequently so as to pay them off my website – http://www.total7otfake2.com/ absence associated with a collateral and then for any lengthy documentation procedure makes this financial help much faster and faster.

  3. I've recently started a site, the info you provide on this website has helped me greatly. Thank you for all of your time & work. "The achievements of an organization are the results of the combined effort of each individual." by Vince Lombardi.

  4. Today, I went to the beach front with my kids. I found a sea shell and gave it to my 4 year old daughter and said "You can hear the ocean if you put this to your ear." She put the shell to her ear and screamed. There was a hermit crab inside and it pinched her ear. She never wants to go back! LoL I know this is completely off topic but I had to tell someone! Also visit my web blog; ohio mud runs – http://alasbarricadas.org/ateneo/index.php/Usuario:BerndSelby

  5. The other day, while I was at work, my cousin stole my iPad and tested to see if it can survive a twenty five foot drop, just so she can be a youtube sensation. My iPad is now broken and she has 83 views. I know this is totally off topic but I had to share it with someone! Also visit my webpage :: gyms around perrysburg ohio – http://www.bks-campus.ch/author/LanceWigi

  6. Please let me know if you're looking for a article author for your site. You have some really great posts and I believe I would be a good asset. If you ever want to take some of the load off, I'd really like to write some material for your blog in exchange for a link back to mine. Please blast me an e-mail if interested. Regards! Here is my website Mud run ohio – http://catania.spacespa.it/author/AnjaRockw

  7. Hi! I know this is sort of off-topic but I needed to ask. Does running a well-established website such as yours take a large amount of work? I am brand new to blogging but I do write in my diary every day. I'd like to start a blog so I can share my experience and views online. Please let me know if you have any ideas or tips for brand new aspiring blog owners. Thankyou! Also visit my blog mud run ohio – http://www.hvacsimulator.net/index.php/Nourishment_Recommendation_For_Ohio_Mud_Runners

  8. Its like you read my mind! You appear to know so much about this, like you wrote the book in it or something. I think that you could do with a few pics to drive the message home a little bit, but instead of that, this is great blog. A great read. I'll definitely be back. Also visit my weblog – haarausfall was tun frau – http://www.nepalvista.com/2011/topics/naturlich-nachwachsen-sie-verloren-haar-haarausfall-stoppen-regrow-haar/

  9. An interesting discussion is worth comment. I do believe that you should write more on this topic, it may not be a taboo subject but generally people do not talk about these issues. To the next! All the best!! Feel free to surf to my homepage … dentist in san antonio – http://Dentist-Sanantonio.info/

  10. Have you ever considered publishing an ebook or guest authoring on other websites? I have a blog centered on the same subjects you discuss and would really like to have you share some stories/information. I know my subscribers would value your work. If you're even remotely interested, feel free to shoot me an e mail.

  11. hey there and thank you for your information – I've certainly picked up something new from right here. I did however expertise several technical issues using this site, as I experienced to reload the website a lot of times previous to I could get it to load correctly. I had been wondering if your web host is OK? Not that I'm complaining, but slow loading instances times will often affect your placement in google and could damage your high-quality score if advertising and marketing with Adwords. Well I am adding this RSS to my e-mail and could look out for a lot more of your respective fascinating content. Make sure you update this again soon. Feel free to surf to my page: ohio mud runs – http://www.wolfandassociates.com/author/QQRSven

  12. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  13. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  14. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  15. Should you are using WordPress, to about arrive spam difficulty just set up plug-in contact Akismet.

  16. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  17. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  18. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  19. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  20. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  21. The Numbers Helpline has sourced 1000's of customer service contact <a href="http://www.numbershelpline.co.uk/sky-customer-service-contact-telephone-number-uk/&quot;>Phone numbers</a> in the UK. We know how hard it can be to find those <a href="http://www.numbershelpline.co.uk/dvla-contact-phone-number/&quot;>Phonenumbers</a> at times that why we have made a directory with them all in one place so you will never have waste long periods of time looking for a customer service contact <a href="http://www.numbershelpline.co.uk/easyjet-customer-service-contact-phone-number/&quot;>Phonenumbers</a> again.

  22. [url=http://www.softassembly.com/Screenshot.html]cheap Screenshot++[/url]. Caputure image from your Desktop

  23. Wow! This can be one particular of the most beneficial blogs We have ever arrive across on this subject. Basically Fantastic. I'm also a specialist in this topic therefore I can understand your effort.

  24. My family members all the time say that I am wasting my time here at web, however I know I am getting experience daily by reading thes pleasant posts.|

  25. seo optimization company […]Every once inside a whilst we pick blogs that we study. Listed below would be the latest sites that we pick […]

Leave a Reply

Your email address will not be published. Required fields are marked *