• Roslyn Based Attribute Remover

    I’m a big fan of XUnit as a replacement for MSTest and use it extensively in my home projects, but I’m still struggling to find a way to integrate it into my work projects.

    This post looks at one of the obstacles I had to overcome, namely the use of [TestCategory("Atomic")] on all tests that are run on TFS as part of the build. The use of this attribute came about because the MSTest test runner did not support a concept of “run all tests without a category”, so we came up with an explicit category called “Atomic” - probably not the best decision in hindsight. The XUnit test runner does not support test categories, so I needed to find a way to remove the TestCategory attribute with the value of Atomic from any method. I’m sure I could have used regex to solve this, and I’m sure that would have caused more problems:

    To generate #1 albums, 'jay --help' recommends the -z flag.

    via xkcd

    Instead I created a Linqpad script and used the syntactic analyser from the Microsoft.CodeAnalysis package.

    PM> Install-Package Microsoft.CodeAnalysis
    

    I found that the syntactic analyser allowed me to input some C# source code, and by writing my own CSharpSyntaxRewriter, remove any attributes I didn’t want.

    I started by creating some C# that had the TestCategory attribute applied in as many different ways as possible:

    namespace P
    {
        class Program
        {
            public void NoAttributes() { }
    
            [TestMethod, TestCategory("Atomic")]
            public void OnOneLine() { }
    
            [TestMethod]
            [TestCategory("Atomic")]
            public void SeparateAttribute() { }
            
            //snip...
            //And so on down to, right down to...
                    
            [TestMethod, TestCategory("Atomic"), TestCategory("Atomic")]
            public void TwoAttributesOneLineAndOneThatDoesntMatch() { }
        }
    }

    You can see all the examples I tested against in the Gist.

    The CSharpSyntaxRewriter took a lot of messing around with to get right, but I eventually figured that by overriding the VisitAttributeList method I could remove attributes from the syntax tree as they were visited.

    To get some C# code into a syntax tree, there is the obviously named CSharpSyntaxTree.ParseText(String) method. You can then get a CSharpSyntaxRewriter (in my case my own AttributeRemoverRewriter class) to visit everything by calling Visit(). Because this is all immutable, you need to grab the result, which can now be converted into a string and dumped out.

    var tree = CSharpSyntaxTree.ParseText(code);
    var rewriter = new AttributeRemoverRewriter(
        attributeName: "TestCategory", 
        attributeValue: "Atomic");
    
    var rewrittenRoot = rewriter.Visit(tree.GetRoot());
    
    rewrittenRoot.GetText().ToString().Dump();

    The interesting part of the AttributeRemoverRewriter class is the VisitAttributeList method which finds and removes attribute nodes that are not needed:

    public override SyntaxNode VisitAttributeList(AttributeListSyntax attributeList)
    {
        var nodesToRemove = 
            attributeList
            .Attributes
            .Where(
                attribute => 
                    AttributeNameMatches(attribute)
                    &&
                    HasMatchingAttributeValue(attribute))
            .ToArray();
    
        if (nodesToRemove.Length == 1 && attributeList.Attributes.Count == 1)
        {
            //Remove the entire attribute
            return 
                attributeList.RemoveNode(attributeList, SyntaxRemoveOptions.KeepNoTrivia);
        }
        else
        {
            //Remove just the matching ones recursively
            foreach (var node in nodesToRemove)
                return
                    VisitAttributeList(attributeList.RemoveNode(node, SyntaxRemoveOptions.KeepNoTrivia));
        }
        
        return 
            base.VisitAttributeList(attributeList);
    }

    The AttributeNameMatches method is implemented to find an attribute that starts with TestCategory, this is because attributes in .NET have Attribute at the end of their name e.g. TestCategoryAttribute, but most people never type it. I figured in this case it was more likley to exist than to have another attribute starting with TestCategory. I don’t think there is an elegant way to avoid using StartsWith in the syntactic analyser, I would have had to switch to the sematic analyser and that would have made this a much more complicated solution.

    The HasMatchingAttributeValue pretty much does what it says, it looks for the value of the attribute been just Atomic and nothing else.

    Once the nodes that match are found, there were 2 choices:

    Remove the whole attribute

    If all the attributes match, remove the entire attribute node. For example:

    [TestMethod]
    [TestCategory("Atomic")]
    public void SeparateAttribute() { }

    When the visitor reaches the [TestCategory("Atomic")] attribute, the entire attribute node should be removed, if not then the attribute is removed but the [] remains.

    Remove just the matching attributes

    If there are some attributes that do not need removing, then just the matching one should be removed. For example:

    [TestMethod, TestCategory("Atomic")]
    public void OnOneLine() { }

    When the visitor reaches the attribute on this method, it should only remove the attribute that matches, the TestMethod should be left alone. It then recursively checks the attribute again incase there is more than one, and removes any others that match.

    Conclusion

    Using Roslyn was a bit of a steep learning curve to start with, but once I found out what I was doing, I knew I could rely on the Roslyn team to have dealt with all the different ways of implementing attributes in C#. If I were to try and use regex to find and remove some of the more complicated ones, and deal with the other edge cases I’d have gone mad by now.

    • You can get the full Gist here.

    If you paste this into a Linqpad “program” and then just install the NuGet Package you should be able to try it out. Note this was built against the 1.0.0 version of the package.

  • Automating the Deployment of TFS Global Lists

    The TFS Global List is a Team Project Collection wide entity and, to the best of my knowledge, requires someone to be a member of the Collection Administrators group to be able to update it – there is no explicit group or permission for “Upload Global List”. This can be quite a problem if there are a number of Lists within your Global List that are updated frequently by the users of your Collection.

    Your current options are either:

    1. Ask the Collection Administrators for every little change (and complain if they take too long, they have a holiday, etc.)
    2. Keep adding people/groups to the Collection Administrators group (and hand out way too much power to people who don’t need it).

    We went for option #1, then option #2, until neither became sustainable.

    The solution I came up with is based on post Deploying Process Template Changes Using TFS 2010 Build by Ed Blankenship, but instead of deploying the whole process template, we just deploy the Global List. (N.B. our TFSBuild account is a Collection Administrator).

    Building the Template

    To build the template I started by copying the DefaultTemplate.11.1.xaml file that ships with TFS 2012 and stripped out all of the activities and process parameters that were no longer required then added a new activity to invoke the witadmin command line tool to import the Global List.

    I won’t go into detail of the process of how I changed the activities because there were quite a lot of steps. It is quite straight forward. However, a quick overview is: remove anything to do with compiling code, running tests or gated checkins, then add a new activity to invoke the witadmin command line. It will probably be easier understood by looking at the finished template - available to download at the end. I may write a follow up post with the exact details.

    Using the template

    • To use the tempalte you need to have the Global Lists file checked into Version Control, you can follow the advice in the Wrox Professional Team Foundation Server 2013 book to create a Team Project for your all your Process artefacts, or if you just want to keep it simple:
      • Use witadmin to export the global list file:
      • witadmin exportgloballist /collection:http://tfs:8080/tfs/DefaultCollection /f:GlobalList.xml
      • Check that file into its own folder somewhere in souce control, in this example we will use $/TFS/GlobalList/GlobalList.xml (having it in its own folder helps).
    • Once you have the template downloaded, you need to check it into Version Control, usually $/MyTeamProject/BuildProcessTemplates/.
    • Create a new build definition.
    • Fill in the General tab however you like.
    • In the Trigger tab select Continous Integration.
    • In the Source Settings tab select the folder with your GlobalList.xml as Active ($/TFS/GlobalList/)
    • In the Build Defaults tab, select “This build does not copy output files to a drop folder”.
    • In the Process tab we need to do a few steps:
      • To install the template, click Show Details:
      • Show details
      • Click New… and browse to the template we checked in ($/MyTeamProject/BuildProcessTemplates).
      • Fill in the sections as follows:
      • Process Parameters
      • I didn’t know the best way to get the URI of the Team Project collection, so I made it a argument you need enter.
      • If you are not using VS2012 on your build server, you will need to find a way to get witadmin.exe on there and then update the path to the location.

    Once the above has been completed you should be able to the queue a new build using the new defintion and check the output to see if the global list has been successfully uploaded. Just open the build and check the summary, if everything went well you should see the following:

    Build Summary

    If there were any problem, check the “View Log”, the build is using Detailed logging which should include enough information to figure out what went wrong.

    Conclusion

    I’ve now stopped worrying about having to update the global list for everyone who needs something new adding and I no longer am affraid of lots of people been Collection Administrators who really shouldn’t have been. I can just grant check-in permissions to the folder that contains our global list and leave people to it.

    Download

    I’m keeping this on my GitHub:

    If have any improvements (to the post / template), feel free to send me a PR.

  • Moving from WordPress

    This is my last post on WordPress and first post on Jekyll GitHub Pages.

    I’ve decided to abandon WordPress running on Azure Web Apps for a simpler static blog using Jekyll to convert Markdown to static content hosted on GitHub pages. I’ll go into the process I went though in a future post.

    This posts is here as a marker of when I moved everything over. I’ve tried to get the permalinks in Jekyll to match the ones in WordPress - but breaking any that were from my brief stint on DasBlog. As far as I know everything should just be the same - including the RSS feed on /feed.

    Shoot me a mail if there is a problem.

    Everything from this point on will be on the new format.

    The actual migration will occur at some point next week.

  • Fixing a broken Kanban Board

    Today I came across a team who had completely broken their TFS Kanban Board (Backlog Board). All they could see was a generic “there has been a problem” pink popup instead of their cards.

    When presented with this, the usual fix is to ensure background agent job is running, which it was. So I took a look in the Windows event logs for more detail and found this error (most of the details are removed for brevity, this is from the middle):

    Detailed Message: TF30065: An unhandled exception occurred.
     
    Exception Message: The given key was not present in the dictionary. (type KeyNotFoundException)
    Exception Stack Trace:    at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
       at Microsoft.TeamFoundation.Server.WebAccess.Agile.Models.WorkItemSource.<>c__DisplayClass18.<GetProposedInProgressWorkItemData>b__13(IDataRecord dataRecord)
       at Microsoft.TeamFoundation.Server.WebAccess.Agile.Utility.WorkItemServiceUtils.<GetWorkItems>d__c.MoveNext()
       at Microsoft.TeamFoundation.Server.WebAccess.Agile.Models.WorkItemSource.GetProposedInProgressWorkItemData(ICollection`1 rowData, ICollection`1 hierarchy, ISet`1 parentIds)
    

    With this little information, all I could assume was that, somehow the configuration had become corrupted. My solution was to find the board settings for that team and delete them from the TFS Collection database.

    NOTE: Neither I nor Microsoft support you making changes directly to your TFS database. You do so at you own risk, and probably best with a backup. This SQL worked against our TFS 2012.3 Database, I cannot guarantee other versions have the same schema.

    First step is to find your TeamId from the Collection Database. Team Ids can be found in the ADObjects table.

    select * from ADObjects
    where SamAccountName like '%MyTeamName%';

    The TeamFoundationId GUID in this table is the value we are interest in.

    You can find the Board and Columns in the tbl_Board and tbl_BoardColumn tables using the following SQL:

    select * from tbl_Board b
    join tbl_BoardColumn bc on b.Id = bc.BoardId
    where TeamId = 'YouTeamId';

    Once you are happy that you have the found the rows for the team, you can then delete them from those two tables. You should probably copy the results into Excel just in case things go wrong.

    To delete you can use the following SQL Queries:

    delete bc
    from tbl_Board b
    join tbl_BoardColumn bc on b.Id = bc.BoardId
    where TeamId = 'YouTeamId';
    
    delete tbl_Board
    where TeamId = 'YouTeamId';

    Now if you refresh the board it should report that there is no configuration and needs to be setup again from scratch.

    I’ve no idea what caused this problem, or if it is fixed in a future update, but this got things working again for me.

  • Working in Visual Studio behind the Firewall

    Working in an “Enterprise” type environment means lots of fun obstacles getting in the way of your day to day work – the corporate proxy is one of my challenges.

    Since giving up on CNTLM Proxy, I haven’t been able to connect to nuget.org from the package manager, view the Visual Studio Extension Gallery or even get any extension/product updates from Visual Studio.

    This is a quick post with the changes I needed to get Visual Studio 2013 Update 4, NuGet 2.8 and Web Platform (Web PI) 5 to see past the corporate Squid proxy.

    NuGet

    Configuring NuGet based on this Stack Overflow answer by arcain.

    Running nuget.exe with the following switches will allow NuGet to use and authenticate with the proxy:

    nuget.exe config -set http_proxy=http://proxy-server:3128
    nuget.exe config -set http_proxy.user=DOMAIN\Dave
    nuget.exe config -set http_proxy.password=MyPassword
    

    It will put the values into your nuget.config file (with the password encrypted)

    <configuration>
        <config>
            <add key="http_proxy" value="http://proxy-server:3128" />
            <add key="http_proxy.user" value="DOMAIN\Dave" />
            <add key="http_proxy.password" value="base64encodedHopefullyEncryptedPassword" />
        </config>
    </configuration>

    Once Visual Studio is restarted, it should be able to see through the proxy.

    As per the comments on the answer some people might have success without the password – sadly, not in my case. Also, remember if you have to change your password (as I have to every month or so) you will need to re-enter your password.

    Visual Studio

    Setting up Visual Studio based on this blog post by Raffael Herrmann.

    • Open the devenv.exe.config file. I find it by right clicking the Visual Studio shortcut, selecting Properties and then “Open File Location”. If you have UAC enabled you will need to open it in a program running as Administrator.
    • Scroll to the end of the file and find the system.net section:
    <!-- More -->
    </system.data>
    <system.net>
        <settings>
            <ipv6 enabled="true"/>
        </settings>
    </system.net>
    <appSettings>
    <!-- More -->
    • Add the following below </settings>:
    <defaultProxy useDefaultCredentials="true" enabled="true">
        <proxy bypassonlocal="true" proxyaddress="http://proxy-server:3128" />
    </defaultProxy>
    • The final version will look something like this:
    <system.net>
        <settings>
            <ipv6 enabled="true"/>
        </settings>
        <defaultProxy useDefaultCredentials="true" enabled="true">
            <proxy bypassonlocal="true" proxyaddress="http://proxy-server:3128" />
        </defaultProxy>
    </system.net>

    Web Platform Installer

    This was the same set of changes needed for Visual Studio, except with the WebPlatformInstaller.exe.config file, which I again obtained from the shortcut properties using “Open File Location”.

    Thanks

    Big thanks to Eric Cain and Raffael Herrmann for enabling me to connect to the internet again :).

  • UNC to URI Path Converter

    Yesterday, at work, I was trying to help someone enter a UNC path (\\server\share\file.txt) into a hyperlink control on our application, but it was rejecting it because it wasn’t a valid hyperlink. I discovered that you could enter a URI path (file://server/share/file.txt) and it worked fine. Problem solved? Not exactly, otherwise I wouldn’t have anything to write about.</p>

    The issue was that the users are non-technical folk and they can’t just fire up Linqpad and run the following code like I did to test if it worked:

    new Uri(@"\\server\share\file.txt").AbsoluteUri
    

    The rules around converting UNC paths to URI’s gets a little tricky when you have paths with spaces and special characters in them, so I thought I would Google/Bing for an online UNC to URI Path Converter… turns out I couldn’t find one, so I did what every software developer does, writes one.

    Path Converter

    This took a little over an evening, mostly due to me been unfamiliar with CSS, JQuery (I know, but it’s quicker than Vanilla JS) and TypeScript (first time I’ve ever used it).

    The entire website was setup in my VS Online account and linked to my Azure site as part of the new “one ASP.NET” setup template, and I even got to add Application Insights as part of the setup template. A few clicks on the Azure Portal and I had a Continuous Delivery Build Definition setup in VS Online. All I had to do then was push changes from my local git repository to VS Online and TFS would build the code and if it succeeded the site was updated within a few minutes.

    The site works by making a quick AJAX HTTP GET when you click the Convert button to a ASP.NET MVC site to use the URI class in .NET. That’s about it.

    Here’s the link to anyone who want’s to use it: http://pathconverter.azurewebsites.net/

  • The Future Looks Roslyn

    I’ve been keeping an eye on the development of Roslyn ever since Anders Hejlsberg announced Microsoft were open sourcing it at Build 2014. For those of you who don’t know what Roslyn is, it is Microsoft’s new C# and VB.NET compiler platform. There are many aspects to it, and if you want a nice overview Schabse Laks has a couple of blog posts on the subject.

    Most developers seem to be excited by the new C# 6.0 language and the future of the language going forward – I can see why, C# hasn’t really changed that much recently. Whilst I am interested in the language, another thing that has got me really excited as a C# developer is the Language Service.

    Language Service

    The language service is what Visual Studio uses to talk to the compiler to understand your code. Previously this was implemented as a series of “hacks” between Visual Studio and the compilers. This made it hard for the Visual Studio team to implement many decent refactoring experiences. The last time anything new really appeared for the “coders” was in VS2010 when the existing refactoring tools were released. Two full versions of VS later, and we still have the same refactoring features. Whilst the C# team have been busy working on Roslyn, products like Jet Brains ReSharper have come along with loads of features of their own, adding refactoring tools and code analysis. But I think times are changing…

    For those of you who have installed “Visual Studio 14” or the Roslyn End User Preview, you are seeing the beginning, of what I think, will be a revolution to Visual Studio’s coding experience for C# (as well as VB.NET).

    New Features

    Here’s a quick overview of some of the features added already:

    Visualising using directives that are not needed.

    Usings

    Here you can see that I don’t need three of my five using directives so they are coloured grey.</span>

    Ctrl + . to Resolve

    Taking the previous “issue” with my code I can just place the caret on any of the lines with an issue and press Ctrl + . (full stop) to fix it.

    Remove Usings

    You even get a preview of what your class will look like when “fixed”.

    Ctrl + . has been used to Resolve issues with your code before (e.g. add using directives), but now it has previews and can suggest fixes to new problems. For example, it could suggest not to use Hungarian Notation and remove the prefix, this would be a lot better experience than the existing Code Analysis CA1704 rule.

    Alt + . and Refactoring

    Since VS2010, the basic refactoring was done via a series of keyboard combination shortcuts. Ctrl + R, M to Extract Method is the only one I remember, the others I know are there but I can’t remember the keyboard shortcuts. Now everything is under Alt + . and is context sensitive.

    So if I select a few statements in a block of code and do Alt + . get the option to Extract Method, select a field and do Alt + . I get Encapsulate Field, select the class name, Extract Interface, you get the idea.

    The interface for renaming has changed too to include live preview and errors.

    New Method

    Above you can see a New Method I’ve extracted and the Live Preview of renaming. If I decided to call “NewMethod” “Foo” I’d get this error because I already have method called Foo.

    New Method Error

    Code Lens

    Code Lens is a heads up display that is already available in VS2013 Ultimate and is powered by Roslyn to provide code and source control information inline in the editor. I’ve only played with this during the CTP of VS2013, but it seemed really promising – I’m now waiting on Microsoft to move it from the Ultimate SKU to Premium.

    CodeLens

    Visual Studio 14

    All the aforementioned features are in VS2013 + End User Preview, Roslyn will actually ship enabled in the next Visual Studio code named “Visual Studio 14”. This has an early preview available and already more refactoring features have appeared. For example, Soma mentions a new Introduce Local refactoring in his blog post.

    This sort of stuff isn’t really ground breaking from a developer experience point of view and it’s nothing new for people who use Visual Studio add-ins like ReSharper. But, it is proof that Visual Studio’s Editor is back in the game.

    My opinion is that from now until VS 14 RTM ships, the C# and VS team will be focusing their efforts on stabilising the Compiler aspect of Roslyn and making sure everything “just works”, we may see a few more refactoring tools appear in the next update, but I’m not expecting much.

    After the RTM

    I’m looking forward to when Visual Studio 14 has hit RTM has shipped, what happens next…? The Visual Studio Editor team have time to take a good hard look at what they want to build, the C# compiler team can go back to working on language features. I expect the VS Editor team will be working on some awesome editor experiences, some we might have seen before in other products, others brand new. I cannot see them resting on their laurels once they have the Language Services as a platform to build on, they can do anything they want.

    Community

    The other thing that really excites me is the community aspect, building your own VS Editor extensions that leverage the Language Services from the compiler is now easy. What ideas will the community build, what would you build? I’m hoping that individual extensions will appear first, and then grow into suites of common functionality, and maybe even be implemented by the VS Editor team in the future.

    It doesn’t just have to be extensions, it is also really easy to write your own diagnostics tools. Already there is a String.Format Diagnostic package on the VS Gallery. So, where does this leave products like ReSharper? In my opinion, Microsoft always implement the features most people want, there will still be room for people to create their own extensions suites and JetBrains can charge for a professional package. However, now it will be a lot easier for people to build their own, no longer do you need to write a C# Code Analyser, so there could be some awesome free open source extension suites appearing too.

    Summary

    I think the next few updates for Visual Studio will brings lots of exciting and powerful enhancements to the Visual Studio Editor for both the C# and VB.NET developers. It’s been a long time coming, but I think Visual Studio is back on the front font and ready to innovate some more.

  • Dealing with NewsBlur and Large Feeds

    I’m a premium NewsBlur member, and have been ever Google decided to shutdown Google Reader. Mostly my experiences have been very good, with great support from Samuel when I needed it.

    However, there has been one issue nagging at me for quite a while and this is, I cannot get a feed to Mark Seemann’s blog. I posted the problem on Get Satisfaction, but Samuel was unable to help due to the size of the feed. A few weeks ago a co-worker of mine mentioned Yahoo Pipes should be able to sort this, so I finally gave it a try. For those (like me until recently) who don’t know what Yahoo Pipes is it’s “a powerful composition tool to aggregate, manipulate, and mashup content from around the web”.

    After a few minutes tinkering, I had finally built a “pipe” that took the blog feed, performed a “truncate” operation to 10 posts and output the feed again. I then took the RSS feed to the pipe and handed it to NewsBlur and this time it coped perfectly with the feed.

    Piping hot

    I’m sure there’s more I can do with it, but for now that’s all I need.

    So, if anyone else needs a “Last 10 Feeds from Ploeh.dk blog” you can get the here.

  • Using Linqpad to Find Work Items with External IDs

    When you work on projects of a certain size, you will find that you begin to add custom fields to your TFS Work Items that you later want to search on. In my case it was an External Requirement ID field that we use to store the ID of requirements that our customers use to track requirements. I often use these when communicating with our customers, or even members of my team.

    For example:

    “Have you checked in A1 yet?”

    is easier to ask and understand than

    “Have you checked in 177484?”

    The problem that arises with this approach, is been able to find a work item by its External Requirement ID.

    To solve this issue, I once again turned to Linqpad and came up with a script that lets you search by a comma separated list entries against your work items. After a bit of digging I managed to find the correct API to be able to produce reliable TFS Web Access URL’s in the results:

    Results

    To use the script, just place you collection address and custom field name at the top. You can also add any other filtering into WIQL, for example, you might only want to search a certain Area Path.

    When you run the script you will be asked to “Enter External Requirement ID”, just enter the ID i.e. A1 or a list of ID’s i.e. A1, A2 and press Enter.

    I keep mine “Pinned” to the Jump List of Linqpad on my taskbar for ease of access.

    You can download the script from here:

    The script is based on my Linqpad TFS Template, if you need it for any other version of Visual Studio, download the correct template and copy and paste the body of the script between them.

  • Team Explorer Shortcuts

    Visual Studio 2013 has a re-designed Team Explorer, and I really like it, but I sometimes find some options, such as “Find Shelvesets” and “New Query”, to be buried away.

    For example, the normal path I have to take to find a shelveset is:

    • Pending Changes
    • Actions
    • Find Shelvesets

    This is really “clicky” and something I wished I could do quicker. Today I found out I could right-click on the tiles on the Team Explorer Home Page to get to some of those actions that are buried away:
    # Pending Changes From Pending Changes you can get to Find Shelvesets

    Pending Changes

    Work Items

    From Work Items you can get to “New Query”

    New Query

    My Work

    From My Work you can get to Request Review

    My Work

    Team Members

    If you have the TFS 2013 Power Tools installed, then even the Team Members gets the right-click treatment with access to “Team Utilities”

    Team Utils

subscribe via RSS