Local Development and Validation of Nuget Packages
Why is an ML company talking about nuget instead of PyTorch or transformers? Because we are C# nerds and because production ML is a whole lot more than training a model. You shouldn’t need to know anything about PyTorch, tensors, or GPUs to use ML. More on that in this short video.
At Nyckel, we use C# extensively for critical parts of our stack. We publish a set of internal .NET libraries that are then used by our tools and services. Since nuget is the preferred .NET package manager, we publish and consume these libraries as nuget packages hosted on a private nuget repo on AWS CodeArtifact.
We insist on having a local development workflow for our stack. This is for two reason:
- We want to detect problems with code changes as early as possible.
- We want to keep our change –> verify –> fix cycle time as short as possible.
Naturally, we wanted a local development experience for verifying changes to our nuget libraries against their consumers. Setting this up took a surprising amount of research and effort. This post attempts to save you the same effort by showing you where we ended up.
For the rest of the post, let’s assume we have the following:
Nyckel.CommonC# library that doesn’t depend on any other library we author.
Nyckel.HttpC# library that depends on
- Both libraries live in the same github repository called
dotnet-libraries. The repository has a
.slnsolution file that contains both projects.
- A CI/CD setup that automatically publishes the libraries on each commit to
main. The libraries are published to AWS CodeArtifact and versioned as
Nyckel.ServerC# service in a separate github repository called
server. It depends on a recent
dotnetcli and SDK 5.0 installed on the local development machine. Instructions below have only been tested on Mac and Linux, but I don’t foresee any problems following along on Windows.
Local Development Requirements
We wanted the following the following local-development workflows to work in the above setup:
- Make a local, uncommitted breaking change to
Nyckel.Http. Detect the breaking change when building the
dotnet-librariesrepo. Make the corresponding change to
Nyckel.Httpand verify that it fixes the build locally.
- Make enhancements to
Nyckel.Serverlocally using the enhanced
Nyckel.Httpbefore we commit those changes. We want to do this even though we have comprehensive unit-tests for our libraries.
- Achieve the above two with minimal cycle time and cognitive overhead. Existing local development workflows should require minimal modifications.
The solution requires setting up project dependencies for inter-library dependencies, and a local nuget repository for quick testing of library-consumer dependencies. Below I walk through those, build and package steps, and some gotchas like nuget package caching.
Nyckel.Http.csproj has the following lines it it:
<ItemGroup> <ProjectReference Include="..\Common\Nyckel.Common.csproj" /> </ItemGroup>
You’ll notice that there’s nothing special here - it’s just the usual project reference from
Nyckel.Common (remember that they are in the same git repository, so the relative path reference works). We were
initially worried that it would result in
Nyckel.Common.dll being copied over instead of creating a nuget reference.
But, as we shall see when we test it out, it actually does the right thing.
This also ensures that
Nyckel.Common is always built before
Nyckel.Http. This fulfills requirement #1 above -
a breaking change to
Nyckel.Common will be detected when building the
dotnet-libraries repo. We can make the
corresponding change to
Nyckel.Http and re-build to verify that it works.
Local Nuget Source
For requirement #2, to minimize cycle time, we set up a local nuget repository. Run the following:
dotnet nuget add source ~/nuget --name local
This sets up a local nuget repository at
~/nuget and names it
local. We will soon be pushing to it when building
Local Nuget Package Version and Nuget Caching
CI systems usually have a monotonically increasing build number that you can use to version your nuget packages and
publish them as immutable artifacts. We currently use a version number of the form
1.0.<build-number>. For local
builds, however, we don’t care about immutability and want to avoid the cognitive overhead of incrementing some version
number when publishing and consuming. Instead, we always use the version
This presents a challenge - nuget caches recently used packages so that they don’t have to be
fetched from their (usually remote) repositories each time. Unfortunately, the cache is used even for locally
published packages. Given this, and given our fixed
1.0.0-local version number, we could end up in a situation
Nyckel.Server is using an older cached version of
Nyckel.Http. To get around this, we add a step to the
.csproj files that clears the local cache before creating a new nuget package. Let’s look at this next.
Csproj File Additions
dotnet-libraries repo, we have a
Nuget.targets file that contains the following:
<Project> <PropertyGroup> <IsPackable>true</IsPackable> <NYCKEL_NUGET_VERSION Condition="'$(NYCKEL_NUGET_VERSION)' == ''">1.0.0-local</NYCKEL_NUGET_VERSION> <PackageVersion>$(NYCKEL_NUGET_VERSION)</PackageVersion> </PropertyGroup> <Target Name="DeleteLocalCache" BeforeTargets="Pack"> <RemoveDir Directories="$(NugetPackageRoot)/$(PackageId.ToLower())/1.0.0-local"/> </Target> </Project>
This does a few things:
IsPackableindicates that the project is a type (library) that can be packaged into a nuget package. See dotnet pack.
NYCKEL_NUGET_VERSIONenvironment variable is used as the nuget package version. It is set by the CI build script. If it’s not set, we assume that this is a local build and default to
DeleteLocalCachetarget runs before the
Packstep (which creates a nuget package) and deletes the cache for this package.
In each of our library
.csproj files (like
Nyckel.Proxy.csproj), we add the following
snippet to include the above file:
<Import Project="../Nuget.targets" />
To build libraries locally, we run the following from the root of the
Nothing fancy - just the normal build command. It works because we have a
.sln solution file in the repo root,
and because the project dependencies (and build order) are encoded in
.csproj files. We followed instructions
here to use the
dotnet sln command to create a solution file and add projects to it.
Local Package and Publish
To create nuget packages and publish to the local nuget repository, we run the following:
# Creates nuget packages (in .nupkg files) in ./nupkgs dotnet pack -o nupkgs # Publishes all nuget packages in ./nupkgs to the local nuget repository dotnet nuget push 'nupkgs/*.nupkg' --source local
We can then modify
Nyckel.Server.csproj to use version
Nyckel.Http, test locally, then
commit changes to
dotnet-libraries. We then change
Nyckel.Server.csproj back to using a non-local version of
Nyckel.Http. If we forget to do this for some reason, the error will be caught as a build failure in the CI system.
CI Package and Publish
On our CI system we set up
AWS CodeArtifact as a nuget repo named
nyckel/libraries. We run the following from
the root of the
export NYCKEL_NUGET_VERSION=1.0.$CI_BUILD_NUMBER dotnet pack -c Release -o nupkgs dotnet nuget push 'nupkgs/*.nupkg' --source nyckel/libraries
When we want
Nyckel.Server to use the latest version of
Nyckel.Http, we update the version in
1.0.$CI_BUILD_NUMBER for the latest CI build of the
Testing the Workflows
To make sure that it all works as expected, we ran the following steps:
- Make a change in
- Run local build. Verify that the build breaks.
- Fix the build in
- Re-run local build. Verify that the build passes.
- Run local package and publish.
Nuget Package Explorer(Web App or Windows App) to open up
.nupkgfile would have been created in the previous local package step. Go to the
Dependenciestab and make sure there is a dependency on
Nyckel.Common. Expand the folders in the
Contentstab and make sure
Nyckel.Common.dllis not included.
Nyckel.Server.csprojlocally to use version
Nyckel.Serverlocally. This will ensure that the nuget cache is populated.
- Make a change in
- Re-run local package and publish.
Nyckel.Server. Verify that the build breaks.
As you can see, getting everything to work is not trivial, even without all the research and trial-and-error it took to get here. Despite the level of effort, we are happy with our setup. If you have similar requirements, I hope you find this write-up helpful.