First, I want credit for not calling this post "AI in .NET 9 is Mighty Fine." I'm a sucker for a bad rhyme so it took a lot of self control. Second, I'm excited for the upcoming release of .NET 9! With .NET 9, Microsoft is making a HUGE push into AI - I know, shocking... It's like AI is a big deal right now or something. Anyway, I did a bunch of investigative work to figure out what's going on in .NET 9 to help developers build more intelligent applications. There's A LOT.

Here's what I found - and I'll continue to update this as I learn more:

TensorFlow.NET Gets a Turbo Boost

TensorFlow.NET is leveling up with high performance C# bindings and automatic differentiation support. The full Keras API is now in C# too:

• New high performance C# bindings generated using cppSharp
• Automatic differentiation support for custom C# operations
• Keras API fully implemented in C#, allowing for seamless model definition and training

Example:

var model = keras.Sequential(new[] {
    keras.layers.Dense(64, activation: "relu", input_shape: new[] { 784 }),
    keras.layers.Dense(10, activation: "softmax")
});
model.compile(optimizer: "adam", loss: "categorical_crossentropy", metrics: new[] { "accuracy" });

OpenAI SDK Integration is 🔥

The OpenAI SDK gives developers direct access to OpenAI's latest public AI models, including GPT-4 and its structured output capabilities. This means you'll have full API support, both synchronous and asynchronous APIs to fit your needs - and even streaming completions for realtime processing. The SDK is built to be extensible, so you can customize it further. Plus, there's legit integration with Azure OpenAI for enterprise-level deployments. This opens up a ton of possibilities for building smarter .NET applications with conversational AI, dynamic content generation and AI-driven features like audio transcription and text-to-speech generation.

ONNX Runtime Goes Native

No more messing around with separate package installs. We're getting a dedicated namespace (Microsoft.ML.OnnxRuntime) with a sweet API to directly load and run ONNX models:

• Direct model loading: var session = new InferenceSession("model.onnx");
• Efficient memory memory management with Span<T> and Memory<T> for input/output tensors
• Hardware acceleration support (CPU, GPU, DirectML) through a unified API

ML.NET 4.0: Developer Friendly

AutoML is even smarter with multi-metric optimization and time series forecasting support:

• AutoML enhancements:
  • Multi-metric optimization for balanced model selection
  • Support for time series forecasting in AutoML
• New Infer<T> API for simplified model deployment:

var result = mlContext.Model.Infer<SentimentPrediction>(model, "This is a great product!");

• TensorFlow and ONNX model conversion to ML.NET format for improved performance

AI-Assisted Coding in .NET 9 is Sweet

AI-assisted code generation (snippets, refactoring, unit tests) is integrated into the .NET 9 SDK via the dotnet ai command and is pretty amazing:

• Generate code snippets: dotnet ai snippet "Implement a binary search algorithm"
• Refactor existing code: dotnet ai refactor --file Program.cs --description "Convert to LINQ query"
• Generate unit tests: dotnet ai test --file MyClass.cs

NLP Tools

.NET 9 offers a rich set of NLP tools for tokenization, NER*, sentiment analysis and text classification:

• Tokenization and sentence segmentation:

var tokenizer = new Tokenizer();
var tokens = tokenizer.Tokenize("Hello, world!");

• Named Entity Recognition (NER):

var ner = new NamedEntityRecognizer();
var entities = ner.RecognizeEntities("Microsoft was founded by Bill Gates.");

• Sentment analysis and text classification using pre-trained models

*Note: It has come to my attention that my attribution for NER is suspect. Still trying to confirm. Thanks to Ron Clabo for asking.

Optimized GPU Acceleration

GPU acceleration in .NET 9 is more accessible than ever:

• New Tensor<T> type for efficient multi dimensional array operations
• CUDA interop improvements:

using var gpuContext = new CudaContext();
using var gpuTensor = new CudaTensor<float>(gpuContext, new[] { 1000, 1000 });

• Integration with Nvidia's cuDNN library for deep learning primitives

Simplifed AI Model Deployment

.NET 9 simplifies AI model deployment with new ASP.NET Core integration:

• New project template: dotnet new webapi --ai -model
• Automatic OpenAPI/Swagger documentation for model endpoints
• Built in model versioning and A/B testing support
• Scalable model serving with gRPC integration

Example of defining a model endpoint:

app.MapPost("/predict", async (HttpContext context, MLModel model) =>
{
    var input = await context.Request.ReadFromJsonAsync<InputData>();
    var prediction = model.Predict(input);
    return Results.Json(prediction);
})
.WithOpenApi()
.WithName("PredictEndpoint")
.WithVersion("v1");

New Numerics APIs

.NET 9 introduces new numerics APIs for efficient tensor and matrix operations:

• System.Numerics.Tensor<T> for efficient tensor operations
• System.Numerics.Matrix<T> for matrix algebra
• SIMD accelerated linear algebra routines

Example usage:

var matrix1 = Matrix<float>.Create(100, 100);
var matrix2 = Matrix<float>.Create(100, 100);
var result = Matrix<float>.Multiply(matrix1, matrix2);

.NET 9 AI Monitoring Tools are Cool!

So when we're talking about .NET 9, it's not just about building models - we need to keep an eye on them in production too. Think of it like having a dashboard for your AI:

• Performance monitoring: We want to know how fast our models are running, how much memory they're using and fi they're handling the load. This helps us spot bottlenecks and optimize things.
• Model drift detection: Models get stale over time as the world changes. We need tools to catch when a model's perf starts slipping so we know it's time for a retraining. 
• Explainability and transparency: AI should NOT be a black box. We have to have ways to peek inside and understand how the model makes decisions. This builds trus and helps us catch any unintended biases.
• Ethical & bias monitoring: AI should be fair and unbiased. We need tools to actively check for and address any potential biases in our models.

And since Microsoft is cloud-first, there's tight integration with Azure for scalable monitoring solutions.

The Community for .NET 9 is Stronger Than Ever

I might be biased (note: I am biased) but .NET has the best community in the developer ecosystem. With .NET being open source - including all of the libraries, tools and frameworks available on GitHub, there's tons of collaboration happening all the time. Not to mention all of the cool Microsoft and community sponsored events, it's a great time to be a .NET developer. All of the AI support is just another reason it's a great place to be.

.NET 9 + AI = 😍

Bottom line, Microsoft is making MAJOR investments in AI and they're bringing the .NET and development community along. Also, I'm pretty sure I missed stuff so good news is that you'll probably find even more AI treasures as you play around with .NET 9. Haven't tried it yet? You can download it here.

Want more AI resources? Here's some stuff we've been working on at GAP:

• We created a bunch of tools to help us get a handle on AI (which you may have heard is kind of a big deal). We decided there might be benefit to others so we bottled it all up and you can download it here: GAP AI Starter Kit
• Have you tried to use ChatGPT or GitHub Copilot for code migration? We migrated an app and documented the experience.