Designing EShop Support with Azure Cloud AI Services: Azure OpenAI and Azure AI Search

Mehmet Ozkaya
7 min readDec 4, 2024

--

We’re going to explore how to build a cloud-native EShop Support application by leveraging the power of Azure OpenAI and Azure AI Search.

EShop Support with Azure Cloud AI Services

Get Udemy Course with limited discounted coupon — Generative AI Architectures with LLM, Prompt, RAG, Fine-Tuning and Vector DB

This architecture will transform our customer support system into an intelligent, scalable, and efficient solution, seamlessly integrated within the Azure ecosystem.

Introduction: EShop Support with Azure AI Services

As businesses evolve, customer expectations for support services are higher than ever. Customers demand fast, accurate, and context-aware responses to their inquiries. To meet these demands, we’re turning to Azure’s fully managed AI services:

  • Azure OpenAI: Provides access to advanced language models like GPT-4, enabling natural language understanding and intelligent response generation.
  • Azure AI Search: Offers vector database functionality for semantic search, allowing us to retrieve and rank documents based on their semantic relevance.
https://learn.microsoft.com/en-us/azure/search/vector-search-integrated-vectorization

By integrating these services into our EShop Support application, we can create a robust system capable of handling complex customer interactions with ease.

Application Architecture: Azure Cloud-Native Approach

Let’s dive into the architecture that brings this vision to life.

EShop Support with Azure Cloud AI Services

1. Frontend Applications

  • Web and Mobile Clients: Our customers and support agents interact with the system through user-friendly web and mobile applications.
  • Features:
  • Ticket Submission: Customers can open support tickets.
  • Q&A Chatboxes: Support agents have access to AI-powered chat interfaces.
  • Semantic Search: Enables agents to search for information using natural language queries.
  • Ticket Management: Streamlined interface for handling and tracking support tickets.

2. Azure API Gateway

  • Purpose: Acts as the secure entry point for all client requests.
  • Functionality:
  • Routing: Directs incoming requests to the appropriate backend microservices.
  • Security: Provides authentication, throttling, and monitoring features.

3. CustomerSupport Microservice

  • Development Stack: Built with .NET 8 for high performance and modern features.
  • Containerized: Packaged as a Docker container for consistent deployment.
  • Hosting: Deployed on Azure Container Apps for scalability and managed infrastructure.
  • Data Storage: Uses Azure PostgreSQL DB to store structured data like customer profiles and ticket information.

4. Azure OpenAI Services

  • GPT-4: Provides advanced language understanding and response generation. Powers the AI capabilities within our support application.
  • Text-Embedding-ADA-002: Generates semantic embeddings of text data. Essential for converting queries and documents into vectors for semantic search.

5. Azure AI Search

  • Role: Acts as a vector database.
  • Capabilities:
  • Semantic Search: Retrieves documents based on semantic similarity rather than keyword matching.
  • Similarity Search: Finds relevant information even when exact terms aren’t used.
  • Document Retrieval: Efficiently accesses large volumes of data.

6. Semantic Kernel

  • Integration Framework:
  • Acts as the glue between our microservices and Azure OpenAI.
  • Simplifies interactions with LLMs.
  • Functions:
  • Manages prompts and responses.
  • Handles embedding generation and retrieval.
  • Benefits:
  • Streamlines development by abstracting complex API interactions.
  • Ensures consistency across different AI components.

Deployment Platform

  • Azure Container Apps:
  • Provides a serverless environment for our containerized applications.
  • Automatically scales based on demand.
  • Simplifies deployment and management.
  • Unified Azure Ecosystem:
  • All components are hosted within Azure, ensuring seamless integration and communication.
  • Benefits from Azure’s security, monitoring, and compliance features.

End-to-End Request Flow: Offline and Runtime Workflows

Understanding the workflows is crucial to appreciating how our system operates.

1. Offline Workflow: Data Ingestion

This process prepares our system with the necessary data to provide intelligent responses during runtime.

Steps:

  1. Data Upload: Administrators upload product manuals, FAQs, and other support-related documents to the CustomerSupport Microservice.
  2. Data Preprocessing: The uploaded documents are split into smaller, manageable chunks.
  3. Embedding Generation: Each chunk is converted into a vector embedding using the text-embedding-ADA-002 model via Azure OpenAI. These embeddings capture the semantic essence of the text.
  4. Indexing and Storage: The embeddings are indexed and stored in Azure AI Search. This enables fast and efficient retrieval during runtime.

2. Runtime Workflow: User Query Processing

This is the real-time process that occurs when a support agent interacts with a customer query.

Steps:

  1. Query Submission: A support agent enters a query into the Q&A chatbox on the frontend application.
  2. Routing the Query: The query is sent to the CustomerSupport Microservice via the Azure API Gateway.
  3. Query Embedding: The microservice uses the text-embedding-ADA-002 model to generate an embedding of the query.
  4. Semantic Search: Azure AI Search performs a similarity search using the query embedding. Retrieves the most relevant document chunks from the indexed data.
  5. Prompt Creation: The microservice combines the retrieved context with the original query to form a comprehensive prompt.
  6. Response Generation: The prompt is sent to GPT-4 in Azure OpenAI Services. GPT-4 generates a context-aware and detailed response.
  7. Delivering the Response: The generated response is sent back to the frontend application. The support agent can review and send the response to the customer.

Technology Choices: Why Azure?

Choosing Azure for our architecture brings several strategic advantages.

EShop Support with Azure Cloud AI Services

1. Azure API Gateway

  • Managed Service: Simplifies the process of request routing, security, and monitoring.
  • Scalability: Automatically adjusts to handle high volumes of traffic.

2. Azure Container Apps

  • Serverless Containers: Run containerized applications without managing infrastructure.
  • Simplified Deployment: Easy to deploy and update microservices.
  • Scalability: Automatically scales based on load.

3. Azure PostgreSQL DB

  • Managed Database Service: Handles database management tasks like backups and patching.
  • High Availability: Ensures data is accessible when needed.

4. Azure OpenAI Services

  • Advanced AI Models: Access to powerful models like GPT-4 and embedding models.
  • Enterprise-Grade Security: Keeps data secure and complies with regulations.
  • Managed Infrastructure: No need to manage AI model hosting and scaling.

5. Azure AI Search

  • Integrated Vector Database: Combines traditional search with vector search capabilities.
  • Semantic Search: Improves search relevance by understanding context and intent.
  • Streamlined RAG Workflows: Simplifies retrieval-augmented generation processes.

6. Azure Cloud Integration

  • Unified Ecosystem: Seamless integration between services.
  • Monitoring and Diagnostics: Robust tools for tracking performance and diagnosing issues.
  • Compliance and Security: Azure meets various compliance standards, ensuring data protection.

Alternative Cloud Options

  • While we’ve chosen Azure, similar architectures can be implemented using Google Cloud or AWS services that offer equivalent AI and container management capabilities.

Benefits of Azure AI-Powered Architecture

Integrating Azure AI services into our architecture offers numerous advantages.

1. End-to-End Integration

  • Seamless Service Interaction: Azure services are designed to work together, reducing integration challenges.
  • Simplified Development: Developers can focus on business logic rather than infrastructure management.

2. Scalability

  • Automatic Scaling: Azure services scale based on demand without manual intervention.
  • Global Reach: Deploy applications in multiple regions to serve a global customer base.

3. Data Security and Compliance

  • Enterprise-Grade Security: Protects sensitive customer data.
  • Compliance Standards: Azure complies with standards like GDPR, HIPAA, and more.

4. Cost Efficiency

  • Optimized Resources: Pay only for the resources you use.
  • Reduced Operational Costs: Managed services reduce the need for in-house infrastructure management.

Challenges and Considerations

While the architecture is robust, it’s important to be aware of potential challenges.

1. Vendor Lock-In

  • Consideration: Relying heavily on Azure services may make it challenging to switch providers in the future.
  • Mitigation: Design the system with portability in mind and consider multi-cloud strategies if necessary.

2. Latency

  • Issue: Embedding generation and vector searches can introduce delays.
  • Mitigation:
  • Optimize Query Paths: Streamline processes to reduce unnecessary steps.
  • Caching: Implement caching mechanisms where appropriate.
  • Performance Monitoring: Regularly monitor and optimize performance.

3. Infrastructure Costs

  • Issue: AI services and vector databases can be resource-intensive.
  • Mitigation:
  • Cost Monitoring: Use Azure’s cost management tools to keep track of expenses.
  • Scaling Configurations: Adjust scaling settings to match actual usage patterns.
  • Budget Alerts: Set up alerts to notify when costs exceed certain thresholds.

4. Model Updates

  • Issue: AI models and embeddings require periodic updates to stay effective.
  • Mitigation:
  • Scheduled Updates: Plan regular maintenance windows for updates.
  • Version Control: Use versioning for models and embeddings.
  • Testing: Thoroughly test updates in a staging environment before production deployment.

Conclusion: A Smarter EShop Support System with Azure AI

By harnessing the capabilities of Azure OpenAI and Azure AI Search, we’ve designed an intelligent, cloud-native support system that elevates the customer experience that demonstrates how enterprises can modernize their support systems using AI-powered capabilities.

Key Takeaways

  1. LLMs as Backing Services: GPT-4 serves as the brain of our application, generating intelligent and context-aware responses. Enables our support agents to handle customer inquiries more effectively.
  2. Vector Databases as Backing Services: Azure AI Search functions as our semantic memory, storing embeddings and enabling rapid, relevant information retrieval. Enhances search capabilities beyond traditional keyword matching.
  3. Cloud-Native Architecture: Utilizing Azure’s managed services simplifies deployment and ensures scalability. Provides a robust infrastructure that can grow with our business needs.

Get Udemy Course with limited discounted coupon — Generative AI Architectures with LLM, Prompt, RAG, Fine-Tuning and Vector DB

EShop Support App with AI-Powered LLM Capabilities

You’ll get hands-on experience designing a complete EShop Customer Support application, including LLM capabilities like Summarization, Q&A, Classification, Sentiment Analysis, Embedding Semantic Search, Code Generation by integrating LLM architectures into Enterprise applications.

--

--

Mehmet Ozkaya
Mehmet Ozkaya

Written by Mehmet Ozkaya

Software Architect | Udemy Instructor | AWS Community Builder | Cloud-Native and Serverless Event-driven Microservices https://github.com/mehmetozkaya

No responses yet