Tuesday, May 28, 2019

Be a Hero on Day 1 with ASP.Net Boilerplate

ASP.Net Boilerplate is a web framework you've likely never heard of, but probably should have. Fortunately, I just published a new episode of Code Hour where I give an overview of it.

In the episode I show how ASP.Net Boilerplate provides a huge leg up when starting new asp.net-based web projects. I show how it generates a fully functional site complete with authentication, authorization, multi-tenancy, auditing, and a ton of best practices like dependency injection, unit of work, and repository pattern.
While I only show the Angular front-end, I point out that there are four different front-ends to choose from including React, Angular, Vue, or ASP.Net MVC 5 with Razor Views.
If you've got a spare hour (or far less at chipmunk speed), watch and learn why I think it's awesome, and how it can make you a hero on day 1 of your next asp.net project.

p.s. don't hesitate to subscribe and like, it's good feng shui.



Wednesday, May 15, 2019

Code Sharing Part 2: Automatic Semantic Versioning of NuGet Deploys

How to automatically incorporate semantic version information into NuGet libraries during building, packaging, and publishing of .Net libraries to private NuGet feeds.

In my last code sharing post I covered how create and consume private NuGet feeds in Azure DevOps via a script, but only locally, and manually.  This isn't close to good enough assuming you've bought into the DevOps dream of consistently and reliably deploying code into production up to hundreds of times per day (ala The Phoenix Project, which I simply can't recommend enough).

To get a continuous integration server automatically compiling, packaging, and deploying a library to NuGet, you'll first need to solve versioning.  Ideally the versioning information will contain semantic information such as whether an API was broken.  That type of information can only be provided by a human, and yet a human shouldn't have to manually specify the version number for each build.

In this article I'll show how to incorporate semantic versioning information into libraries and yet automate it.  I'll start with incorporating versioning information into the Cake scripts from the last post and then get into version automation with GitVersion.

Hard Coded Versioning


There's a subtle bug with the NuGetPush task from my last post.  If you run it twice you'll get this error:

Response status code does not indicate success: 409 (Conflict - The feed already contains 'LibAuthenticator 1.0.0'.

That's because I didn't specify a version in dotnet pack.  Dotnet pack then tried to pull version information from the .csproj, and since no version information existed in csproj, it defaulted to 1.0.0 every time.  There are two ways to solve this.

We could put version information in the .csproj.  However, modifying a csproj during a build feels yucky, and locally would require a git revert.  Uch.

Better, the dotnet pack command documentation shows that we can manually override MSBuild properties with a syntax like "/p:Property=value".  Cake supports this inside the DotNetCorePackSettings via the strongly typed MSBuildSettings property:

Task("Pack") .IsDependentOn("Build") .Does(() => { DotNetCorePack(projectDirectory, new DotNetCorePackSettings { NoBuild = true, IncludeSymbols = true, Configuration = configuration, MSBuildSettings = new DotNetCoreMSBuildSettings().SetVersion(version) }); });
The file name when we pack will now contain the version, so we'll additionally need to update the name of the file we're pushing like:

var nupkgFile = configDirectory + File("LibAuthenticator." + version + ".nupkg");

and put that in the NuGetPush task.

Assembly Meta-Data


There's now a small problem.  If we override PackageVersion during pack and fail to specify the version during build, then the source library's dll meta-data will always be marked as 1.0.0, even though git pulls the updated version e.g. 1.0.1.    Fortunately we can pull that "/p:" trick again this time on dotnet build to override the version.  The code looks nearly identical:

Task("Build") .IsDependentOn("Clean") .IsDependentOn("Version") .Does(() => { DotNetCoreBuild(projectDirectory, new DotNetCoreBuildSettings { Configuration = configuration, MSBuildSettings = new DotNetCoreMSBuildSettings().SetVersion(version) }); });
Now we just need to define and set the version variable.  If we hard-code one like var version = "1.0.1"; toward the top of the file, we'll be able to NuGetPush up a new version of our library.  Once at least.  Then we'll run into the 409 error again.  We could make variable an argument, but we'd have to specify it on every build, and that's not very automated.  There's a better way.

Calculating Version


If you're using git, and gitflow in particular, then all of the information you need should already be in source control history.  This is especially easy for official releases.  According to gitflow author Vincent Driessen:

origin/master [should] be the main branch where the source code of HEAD always reflects a production-ready state … When the source code in the develop branch reaches a stable point and is ready to be released, all of the changes should be merged back into master somehow and then tagged with a release number.

So for official releases, you could just extract and parse the git tag. But suppose you want to publish to alpha or beta NuGet feeds from non-master branches. The good news is that if you're following git flow everything you need is still available in git, and even better you can use the GitVersion tool to extract it for you. GitVersion returns semantic versions that make sense. Definitely read the documentation, because it's a bit counter-intuitive at first and it's very customizable, but here's a quick summary of the defaults:



  1. If you tag a branch (typically on master for an official release) like v1.0.3, then GitVersion will return "1.0.3"
  2. If you're on a feature branch off of develop then GitVersion will increment the minor number and append a suffix of the branch name and a 1 up counter
  3. As you merge PR's back into develop GitVersion updates the minor version from what it found in master and appends "alpha" and a 1 up counter.
  4. When you're ready to deploy and start a release branch, GitVersion parses the version number out of the branch name and appends "beta" and a 1 up counter
  5. When you merge the release branch back into master GitVersion recognizes the version from the merged release branch even before you're tagged
If you're suspicious about strings in version numbers, they do work in assembly meta-data:


And when you publish to NuGet even without a separate alpha or beta feed they show up but only if you include prerelease versions:




Now there's just the issue of implementing it.


GitVersion and Cake


Using GitVersion in Cake is extremely easy. First include the plugin like this:

#tool "nuget:?package=GitVersion.CommandLine"
Specify a version there for bonus points. Then just set a version variable, something like:

Task("Version") .Does(() => { var symVer = GitVersion(); Information($"SemVer: {symVer.SemVer}"); version = symVer.SemVer; }); 
And finally include the new dependency:

Task("Build") .IsDependentOn("Clean") .IsDependentOn("Version") .Does(() => { ... }
To DevOps

And now you can get run this from an Azure DevOps Build via a yml step like:

steps: - powershell: .\build.ps1 -target=NuGetPush -nugetUsername='$(nugetUsername)' -nugetPassword='$(nugetPassword)'
The full version of this entire project is open sourced and the source code for the cake solution is located here.

HouseCleaning

As a quick aside since my last post DevOps changed the NuGetFeed url. If this happens to you or you ever need to manually remove or debug your NuGet credentials, they live in: %appData%\NuGet\NuGet.Config

Summary

In this post I've shown how to incorporate version info into NuGet libraries during building and packaging in Cake tasks. I've also shown how you can extract semantic versioning info from source control using GitVersion. I hope this helps you to automate the versioning and deploying of your shared .Net code. Please hit me up on twitter or the comments if this was helpful or you have any questions or suggestions.

Saturday, April 27, 2019

Share Code Like a Boss Part 1 - Private NuGet Feeds in Azure DevOps

If you've ever tried to share code like an API proxy, math calculations, or validation logic between multiple projects, you'll know there are many options.  In this post I'll summarize the most common, then dig into the most versatile for .Net projects: private NuGet feeds.  I'll cover how to:

  • Create private NuGet feeds in Azure DevOps
  • Package .Net Core libraries into NuGet packages
  • Publish NuGet packages to feeds via Cake (C# Make) including authentication
  • Consume NuGet packages from private NuGet feeds
I'll save some of the more interesting bits like automatic semantic versioning, and publishing to private NuGet feeds from an Azure DevOps Build, for a subsequent post.

Bro, Just Ctrl-C, Ctrl-V

Private NuGet feeds are pretty great for many code sharing problems, but they're a little tricky to set up.  Before jumping to the Cadillac solution, remember different problems require different approaches.

For instance, I love copy and pasting for simple code that's unlikely to change.  It's fast and easy of course, but over time the code can be customized on each project without any fear of accidentally breaking another project.  Obviously when the code requires a bug fix, update or new feature, then this approach falls apart fast.

Git SubModules seem nice at first because they're easy to set up, but if you've ever used them much you'll find they're kind of a pain to work with.  Also, they don't support versioning, particularly semantic versioning.  That means it's hard to know which version of shared code a particular client is running and thus what features or bug fixes it'll get by upgrading.
    Private NuGet Feeds are trickier to set up, but they offer an easy and powerful distribution model that clients are already familiar with, a sophisticated security model, and they offer semantic versioning so clients can know exactly what they've got and what they're getting.

    Create a Feed

    There are a variety of options for private NuGet feeds.  You could use MyGet, host your own feed, or use Azure DevOps.  If you're already using Azure DevOps for source control or continuous integration, then using it for NuGet feed keeps everything in one place, and makes authentication easy.  I'll show the Azure DevOps option in this post, just be aware the free tier limits NuGet feeds to 5 users.

    To get started click on "Artifacts" in the left-menu.



    Then New Feed


    Give it a name and permissions.  Click Create.  Then the "Connect to Feed" panel will provide all the details you'll need to push artifacts.


    Packing NuGet

    At this point you could follow the instructions and package and publish to the feed at the command line with a nuget.exe command.  However, a one-off command isn't repeatable, and can't be consumed by a continuous integration server.

    If you want the option of automating the building, packaging, and publishing of your library, then some kind of scripting is in order.  As my regular readers know I'm a big fan of Cake (C# Make) for devops solutions.  It:


    • Manages dependencies between tasks
    • Allows running or debugging locally
    • Is cross platform
    • Offers intellisense (in VS Code with the plugin)
    • Is source controlled
    • Supports migrating between any CI Servers


    To use it you'll need to first compile the library you plan to publish.  If you've watched my intro to Cake video then you'll be comfortable with the following script to clean and build your library:

    Task("Clean")
       .Does(() =>
    {
        CleanDirectory(buildOutputDirectory);
    });

    Task("Build")
       .IsDependentOn("Clean")
       .Does(() =>
    {
       DotNetCoreBuild(projectDirectory, new DotNetCoreBuildSettings {
          Configuration = configuration
       });

    });

    If that looks foreign, it's actually just a C# DSL with lambdas that declares two tasks Clean and Build.  Build is dependent on Clean, so if you run Build it will always Clean first.  Seems simple, but dependency management helps prevent devops spaghetti.

    What we need now is to package our library up into a nuget package.  Fortunately asp.net core makes this really simple with the "dotnet pack" command, and Cake exposes it with the DotNetCorePack command like this:

    Task("Pack")
       .IsDependentOn("Build")
       .Does(() =>
    {
       DotNetCorePack(projectDirectory, new DotNetCorePackSettings {
          NoBuild = true,
          IncludeSymbols = true,
          Configuration = configuration
       });

    });

    The NoBuild = true attribute is important because it keeps us from accidentally building multiple times.  Instead we let cake handle our dependencies.  We only ever want the "Build" task to perform builds.

    Running that task (e.g. .\build.ps1 -t Pack on Windows) should result in something like MyLib.1.0.0.nupkg being output to your bin\Release folder.  I'll cover how to update the version in the next post in the series.

    By the way, instead of packaging explicitly from the command line, another option is to check "Generate NuGet package on build" in your .csproj file for the Release configuration.  Personally I prefer to do it in cake so all my devops logic in one place, but either option is fine.

    Pushing Artifacts to Your Feed

    If you follow the instructions in "Connect to Feed" you'd next need to do something like:

    nuget.exe push -Source "MyNewFeed" -ApiKey AzureDevOps my_package.nupkg

    Switching that over to cake looks like this:

    const string nugetFeedUrl = "https://sirenofshame.pkgs.visualstudio.com/_packaging/CodeSharingFeed/nuget/v3/index.json";

    Task("NuGetPush")
       .IsDependentOn("Pack")
       .Does(() =>
    {
       NuGetPush(nupkgFile, new NuGetPushSettings {
          Source = nugetFeedUrl,
          ApiKey = "AzureDevOps",
       });
    });

    Easy enough.  Right up until you run it and get this error:

    Unable to load the service index for source https://sirenofshame.pkgs.visualstudio.com/_packaging/CodeSharingFeed/nuget/v3/index.json.
    Response status code does not indicate success: 401 (Unauthorized).

    Authenticating Your Feed

    The solution to authenticating is to create a Personal Access Token with Read and Write permissions for Packaging:


    Then in Cake ask for the username and password from the personal access token as parameters, add a new task to create a NuGet Source with those credentials, and set the new task as a dependency for NuGetPush:


    var nugetUsername = Argument<string>("nugetUsername", null);
    var nugetPassword = Argument<string>("nugetPassword", null);

    Task("AddNugetSource")
       .Does(() =>
    {
       if (!NuGetHasSource(nugetFeedUrl)) {
          NuGetAddSource("CodeSharingFeed", nugetFeedUrl, new NuGetSourcesSettings {
             UserName = nugetUsername,
             Password = nugetPassword
          });
       }
    });

    Task("NuGetPush")
       .IsDependentOn("AddNugetSource")
       .IsDependentOn("Pack")
       .Does(() => { ... }



    Now if you run that puppy like:

    .\build.ps1 -target=NuGetPush -nugetUsername=ShareCodeNuGetPackager -nugetPassword=MyPassword

    It should succeed and your feed page should look something like this:


    Pull Feed

    Now that you've published the feed you're ready to consume it.  You can do that in Visual Studio with 

    1. Tools -> Options
    2. Nuget Package Manager -> Package Sources
    3. +
    4. Add a Name and a Source

    Visual Studio will automatically handle Azure DevOps authentication using your signed in credentials.  Pretty nice.

    But, how does consuming that feed on a CI server work?  Not very well at first.  I'll cover how to overcome that hurdle that in my next post.

    Conclusion

    In this article I described the benefits of sharing code with private NuGet feeds, explained how to create feeds, showed how to build, package, authenticate, and push to feeds, and briefly covered how to consume them.

    In the next article I'll cover versioning, building and publishing the library on the build server, and consuming the nuget feed from a Continuous Integration server.

    I hope this was helpful.  If so please let me know in the comments or on twitter.

    Wednesday, January 2, 2019

    How to Make Printed Circuit Boards 101

    I just posted Episode 18 of Code Hour.  In this one I show how to build a Printed Circuit Board (PCB) from prototype, to design, to manufacture.


    I demonstrate how to populate a board with LED's, resistors and solder paste, and then reflow solder the components to make a finished board.

    Along the way I show a cool time-lapse video of solder paste condensing into its mercury-like liquid state.




    This is a very different code hour. Please write in in the comments or me hit up on twitter to let me know how it went and if he should do more like this or if I should really just stick to coding.