Using Aspire to go from a simple ILogger to a team friendly cloud ready solution, for free

When you listen to senior developers, speakers, and other subject matter experts (SMEs) talk about logging in .NET applications, it all sounds so easy. An ILogger here, an ILogger there; a package here, a config there; and voilà, you're done. And although Microsoft.Extensions.Logging and packages like Serilog do in fact make implementing advanced logging quite straightforward nowadays, there's a whole lot more to it than that.

See, writing logs is all good and fine, but as soon as you start working with a team and the application starts being used substantially by lots of users, many new factors come into play. The speed at which logs are written and the volume of those logs start to increase. It's not uncommon for applications to go through multiple stages of deployment, with the developers having little to no access to production. Production environments often have operational complexities such as clustering, scaling, and data residency requirements. The number of things that can go wrong also increases, which leads to adding more logging "just in case". And things quickly snowball into an avalanche of text that require plenty of searching.

And this is without even talking about Observability, which includes Tracing and Metrics that further increase that volume. Combined with the complexity of those production environments, it doesn't take long for teams to start looking for alternatives to simple logging such as those offered by Application Insights. Often though, it's not enough, and companies end up looking for even more specialized solutions like those offered by Datadog.

However, you'll often hear those same SMEs who advocate the use of specialized observability solutions, also complain about how large the wheelbarrow of money they need to give Datadog every month is. In a large-scale high-usage distributed cloud application, it's not uncommon for the application to produce several hundred megabytes, if not gigabytes, of logging, tracing, and metric data. All of which has historically been packaged up and delivered through custom vendor implementations that have often left companies locked in to those vendors.

Thankfully, that landscape is changing thanks to the arrival of OpenTelemetry. Developed as an open standard, it is unifying the concepts of Structured Logs, Tracing, and Metrics under a universal vendor agnostic solution that is slowly making its way into all the different software development ecosystems, including .NET. Microsoft is heavily involved in the development process as they're part of both the Governance and Technical Committees while also being maintainers.

I have been writing software for a very long time now. I started with .NET back when .NET Framework was released as a public beta in 2001. Over the years, it's been my experience that there's often this disconnect between the reality that Microsoft lives in, and the reality that exists within existing development teams. Perhaps it's the scale at which Microsoft operates (it is, after all, the largest software company in the world), or perhaps it's their privileged position as the inventors and thus maintainers of the .NET technologies we all use. I don't really know. I just know there is a gap between those two realities.

I recently joined the .NET development team at Orbis Communications, a division of Symplicity. And though my colleagues are just awesome (hey everyone! 👋), our application is having some growing pains that we're working through. Born in the days of .NET Core 1.1, it has accumulated a lot of technical debt that has made that gap even more evident.

Logging, for example, has been a pain point for our team for a few years now. Although Serilog was shoehorned in at some point, it was integrated in such a way that getting to logs essentially turns into a hunting adventure of searching through the many text files that are actively dumped to Azure Storage.

When I first read about Aspire, I was both excited and frustrated at the same time. As most things brought to us by the various .NET teams, I could see a lot of the value in what was being done and the benefits this would bring in the future. Thanks to such initiatives, I can now imagine a future where I can write software while running a full cloud stack on my laptop while I'm on my 6 hours flight to Colombia. And though that's an exciting future to look forward to, I have a reality to get back to on Monday that's not so futuristic.

My frustration mostly stems from the fact that although these are great things that will accelerate the development of new applications in the future, it doesn't do much for existing applications. I once had the privilege of exchanging thoughts with David Fowler on Discord regarding the integrated nature of Aspire as all-in-one solution, but I don't think I succeeded in communicating my thoughts as well as I would have liked to.

Aspire is an incredible developer experience if you're writing a new application. It's also possible to work an existing application into an Aspire solution, if it's not too long in the tooth. But what do you do with an application such as ours that's been around as long as it has? Our application has 7 years of custom implementations built into it to accommodate for the realities of how tumultuous the landscape for ASP.NET was in the pre-.NET 5.0 days. Some things also take time to get around to. For example, although we've managed to successfully upgrade to .NET 8.0, we're still using WebHost.CreateDefaultBuilder inside a Startup.cs file with server-side compiled Razor pages and custom-written APIs all packaged up as a single executable that only recently lost its strong dependency on Windows.

As much as I'd like to get on board the .NET Aspire developer train, I can't, in good conscience, do so carelessly. We have more than 14 million users across Canada and the United States who depend on our applications. A full rewrite is a monumental task and I can't stop the development of the product just to add, what essentially amounts to, some extra developer tools to make our lives easier.

Week after week, I began to feel the burden that had become the routine choreography that is our dance when we jump through the required hoops of searching text files to get log information when things fail in one of our environments. Application Insights wasn't of much help. So I started looking at solutions such as Seq, but it was hard to justify the price to management for something that, at this point, was nothing more than exploratory experiments in an attempt to look for a solution to our logging woes. Disappointed that I couldn't find an easy fix to our situation, I went back to our priorities and was about to relegate myself to the confines of our reality.

That is, until I discovered that the Aspire Dashboard was now available as a standalone container image. That changed everything.

Currently, our deployment pipeline is based on the older approach of building artifacts from a branch for each environment, packaging them up as ZIP files, and unzipping them on the destination server. It's been modernized slightly over the years and currently runs off some automation that was built in Azure DevOps.

I was thankfully fortunate that our repositories are hosted in GitHub and even more fortunate that I was given the flexibility of rewriting some of our deployment pipelines as GitHub Actions. Over the last few weeks, I've been using that freedom to explore how we could package and deploy our application as a Docker container, with all of the benefits that come with a modern CI/CD workflow, since I have the rare opportunity of being able to build a parallel workflow that doesn't affect the one that currently deploys to production.

Knowing that the Aspire Dashboard is essentially just a basic user interface over an OpenTelemetry collector, further confirmed by reading about the configuration of the standalone Aspire Dashboard, I started wondering if I could use this to solve some of our issues with logging. I had already looked into adding support for OpenTelemetry using the NuGet packages provided by their SDK and didn't see much of a challenge in doing so within our current application. The problem is I didn't have anywhere to send the data.

There are some really great platforms, like Honeycomb, available. But due to the need for security and auditing that must tie into our Microsoft Entra ID, combined with our use of three development and two production environments, we often find ourselves needing to access Enterprise level pricing for even the most basic usage by our application. That's a hard sell for, again, what is essentially nothing more than developer tools to make our lives easier. Those are costs that can often be justified for production environments, but they're harder to negotiate in the pre-production ones.

While investigating options for the deployment of our application as a container in Azure, I also recently ran across documentation that showed how to use the Aspire Dashboard container with Azure Container Apps. Now this is cool, I thought. That looks exactly like the kind of solution I need.

Azure Container Apps are awesome, don't get me wrong. But considering our application cannot be scaled to multiple instances as it's not designed to support such a scenario, it seemed to be a bit too much complexity and cost for what would only ever be a single container with an accompanying dashboard. Although going through deployment with Azure Container Apps is much simpler than a full Azure Kubernetes Service deployment, it's still a fully managed version of Kubernetes.

I needed something simpler and less expensive that I could use for the deployment of our various environments. Something that would also allow us to move away from our current setup based on various Azure Web Apps with deployment slots. There was the obvious choice of using Azure Web App for Containers, but I imagined the mess I could quickly get myself into if I tried to coordinate an extra floating container used to host the Aspire Dashboard. I really didn't want to start having to manage a VNet with all of the associated routing responsibilities.

And that's when I saw that the Gods of Perfect Timing seemed to have gotten together to align all of the stars for me. Turns out that a team at Azure had been working on a new feature for Azure Web App, sidecars for the Azure App Service.

If you don't know what the sidecar pattern is, it describes the concept of deploying a container that accompanies another container. Like the sidecar of a motorcycle. Sounds a lot like an accompanying Aspire Dashboard that sidecars an application, doesn't it?

As the feature is still in preview, there's a lack of documentation, and some of the tooling is sort of incomplete. It took me a lot of trial and error, but I finally figured out how to bundle up a solution that works really well. There are some things you have to be aware of to avoid some frustration and make for a reliable deployment, but once you understand what those are, they're straightforward and easy to take into consideration.

Stay tuned for the follow-up to this post where I'll help you cross the gap to get your application deployed with an Aspire Dashboard sidecar, just as I helped our team cross the gap with ours. And together, we'll go on the shared adventure of discovering the world of observability through the eyes of OpenTelemetry.

Leave me a comment below or find me online. I'd love to hear your thoughts!