Ollama Integration Using Semantic Kernel and C#: A Step-by-Step Guide

We’re going to explore how to integrate Ollama models using the Semantic Kernel SDK in C#. We’ll walk through building a .NET 8 console application that connects to local AI models like LLaMA, Gemma, and Phi3.

Interacting Ollama LLaMA 3.2 with Coding

Get Udemy Course with limited discounted coupon — Generative AI Architectures with LLM, Prompt, RAG, Fine-Tuning and Vector DB

Interacting with AI models requires the right coding frameworks to manage connections, prompts, and responses efficiently. For different programming languages, various tools are available:

• Python: Libraries like LlamaIndex and LangChain.
• Java: Spring AI provides robust support.
• C#/.NET: The Semantic Kernel SDK.

Why Do We Need Coding Frameworks?

• Simplify Model Interaction: Manage model connections and prompt handling without dealing with low-level API calls.
• Efficient Integration: Seamlessly incorporate AI capabilities into existing applications.
• Enhanced Functionality: Utilize built-in features for managing chat history, context, and more.

In this tutorial, we’ll focus on using the Semantic Kernel C# SDK to interact with local models running on Ollama.

Integration Overview: Architectural Diagram

Interacting Ollama LLaMA 3.2 with Coding

• Developer: Writes code using the Semantic Kernel C# SDK.
• Console App: Our .NET 8 application that will interact with the models.
• Ollama Connection: Connects to the local Ollama instance running the models.
• Models: Local AI models like LLaMA, Gemma, and Phi3.

Ensuring Ollama is Running

Before we begin, make sure that Ollama is running on your machine:

• Open your web browser and navigate to http://localhost:11434.
• You should see a confirmation page indicating that Ollama is active.

Step-by-Step Guide: Building the Console Application

Step 1: Create a .NET 8 Console App

Open your terminal or command prompt and run the following commands:

dotnet new console -n Ollama_Integration
cd Ollama_Integration

This creates a new console application named Ollama_Integration.

Step 2: Install Necessary Packages

We’ll need to install the Semantic Kernel packages that include support for Ollama.

Using the .NET CLI

dotnet add package Microsoft.SemanticKernel --version 1.24.1
dotnet add package Microsoft.SemanticKernel.Connectors.Ollama --version 1.24.1-alpha --prerelease

Updating the Project File

Open the .csproj file and add the following lines to include the packages and suppress experimental warnings:

<ItemGroup>
  <PackageReference Include="Microsoft.SemanticKernel" Version="1.24.1" />
  <PackageReference Include="Microsoft.SemanticKernel.Connectors.Ollama" Version="1.24.1-alpha" />
</ItemGroup>
<PropertyGroup>
  <NoWarn>SKEXP0070</NoWarn>
</PropertyGroup>

Step 3: Developing Program.cs

Open the Program.cs file and replace its contents with the following code:

using System;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
namespace Ollama_Integration
{
    class Program
    {
        static async System.Threading.Tasks.Task Main(string[] args)
        {
            // Create a kernel builder
            var builder = Kernel.Builder;
            // Add Ollama chat completion service
            builder.AddOllamaChatCompletion(
                model: "llama2",
                endpoint: new Uri("http://localhost:11434")
            );
            // Build the kernel
            var kernel = builder.Build();
            // Get the chat completion service
            var chatService = kernel.GetService<IChatCompletion>();
            // Initialize chat history
            var history = kernel.CreateNewChat("You are a helpful assistant.");
            while (true)
            {
                Console.Write("You: ");
                var userMessage = Console.ReadLine();
                if (string.IsNullOrWhiteSpace(userMessage))
                {
                    break;
                }
                history.AddUserMessage(userMessage);
                var response = await chatService.GenerateMessageAsync(history);
                Console.WriteLine($"\nBot: {response}\n");
                history.AddAssistantMessage(response);
            }
        }
    }
}

• Kernel Builder: Creates a new kernel instance that manages AI services.
• AddOllamaChatCompletion: Configures the kernel to use the Ollama chat completion service with the specified model (llama2) and endpoint.
• Chat Service: Retrieves the chat completion service from the kernel.
• Chat History: Manages the conversation history between the user and the assistant.
• User Interaction Loop: Continuously reads user input, sends it to the model, and displays the response.

Verifying Ollama and Running the App

Step 1: Ensure Ollama is Running

Confirm that Ollama is active by visiting http://localhost:11434 in your browser.

Step 2: Run the Console App

In your terminal, execute:

dotnet run

Step 3: Test the Interaction

Sample interaction:

You: Why is the sky blue?
Bot: The sky appears blue because molecules in the Earth's atmosphere scatter sunlight in all directions, and blue light is scattered more than other colors because it travels as shorter, smaller waves. This phenomenon is known as Rayleigh scattering.

Exploring Other Models

You can switch between different models available on Ollama by changing the model name in the AddOllamaChatCompletion method:

builder.AddOllamaChatCompletion(
    model: "gemma2:2b",
    endpoint: new Uri("http://localhost:11434")
);

Conclusion: Unlocking AI with Semantic Kernel and Ollama

By integrating Ollama models with the Semantic Kernel SDK in C#, we’ve achieved the following:

• Local AI Interaction: Communicate with AI models running entirely on your machine.
• Enhanced Privacy: Keep data processing local, ensuring sensitive information remains secure.
• Low Latency: Experience faster responses without network delays.
• Flexible Integration: Seamlessly incorporate AI capabilities into your .NET applications.

Integrating local AI models using the Semantic Kernel SDK provides a powerful way to enhance your applications with AI capabilities while maintaining control over your data and infrastructure. This approach is ideal for scenarios where privacy, performance, and customization are critical.

Get Udemy Course with limited discounted coupon — Generative AI Architectures with LLM, Prompt, RAG, Fine-Tuning and Vector DB

EShop Support App with AI-Powered LLM Capabilities

You’ll get hands-on experience designing a complete EShop Customer Support application, including LLM capabilities like Summarization, Q&A, Classification, Sentiment Analysis, Embedding Semantic Search, Code Generation by integrating LLM architectures into Enterprise applications.