# Enhanced Upstream Network Architecture Documentation

## Table of Contents
1. [Overview](#overview)
2. [Architecture](#architecture)
3. [Configuration](#configuration)
4. [Components](#components)
5. [Message Flow](#message-flow)
6. [Management & Operations](#management--operations)
7. [Health Monitoring & Circuit Breakers](#health-monitoring--circuit-breakers)
8. [Connection Management](#connection-management)
9. [Routing Rules](#routing-rules)
10. [Troubleshooting](#troubleshooting)
11. [Performance Tuning](#performance-tuning)

## Overview

The Enhanced Upstream Network Architecture manages **outbound connections FROM our ISO8583 switch TO external payment networks** (Visa, MasterCard, Amex, Discover, etc.). This system provides intelligent routing, connection pooling, health monitoring, and failover capabilities.

### Key Features
- **Multi-Network Support**: Connect to multiple payment processors simultaneously
- **Intelligent Routing**: Route messages based on BIN ranges, MTI patterns, and custom rules
- **Connection Pooling**: Efficient connection management with pooling and reuse
- **Health Monitoring**: Continuous health checks with circuit breaker patterns
- **Load Balancing**: Distribute load across multiple upstream hosts
- **Failover & Recovery**: Automatic failover and recovery mechanisms
- **Performance Monitoring**: Detailed statistics and performance metrics

## Architecture

```
Enhanced Upstream Network Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                    APPLICATION SUPERVISOR                            │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌─────────────────────────────────────────────────────────────────┐ │
│  │              EnhancedUpstreamSupervisor                         │ │
│  │                                                                 │ │
│  │  ┌─────────────────────────────────────────────────────────────┐ │ │
│  │  │                 UpstreamRouter                              │ │ │
│  │  │                (GenServer)                                  │ │ │
│  │  │  • Message routing logic                                    │ │ │
│  │  │  • BIN/MTI/Field-based routing                             │ │ │
│  │  │  • Network selection algorithms                             │ │ │
│  │  └─────────────────────────────────────────────────────────────┘ │ │
│  │                                                                 │ │
│  │  ┌─────────────────────────────────────────────────────────────┐ │ │
│  │  │            UpstreamConnectionManager                        │ │ │
│  │  │                 (GenServer)                                 │ │ │
│  │  │  • Connection lifecycle management                          │ │ │
│  │  │  • Pool management and statistics                           │ │ │
│  │  │  • Load balancing across connections                        │ │ │
│  │  └─────────────────────────────────────────────────────────────┘ │ │
│  │                                                                 │ │
│  │  ┌─────────────────────────────────────────────────────────────┐ │ │
│  │  │            UpstreamHealthMonitor                            │ │ │
│  │  │                 (GenServer)                                 │ │ │
│  │  │  • Periodic health checks                                   │ │ │
│  │  │  • Circuit breaker management                               │ │ │
│  │  │  • Failure detection and recovery                           │ │ │
│  │  └─────────────────────────────────────────────────────────────┘ │ │
│  │                                                                 │ │
│  │  ┌─────────────────────────────────────────────────────────────┐ │ │
│  │  │          NetworkConnectorSupervisor                         │ │ │
│  │  │            (DynamicSupervisor)                              │ │ │
│  │  │                                                             │ │ │
│  │  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │ │ │
│  │  │  │    Visa     │  │ MasterCard  │  │       Amex          │ │ │ │
│  │  │  │ Connector   │  │ Connector   │  │    Connector        │ │ │ │
│  │  │  │ (GenServer) │  │ (GenServer) │  │   (GenServer)       │ │ │ │
│  │  │  │             │  │             │  │                     │ │ │ │
│  │  │  │ • Pool: 10  │  │ • Pool: 10  │  │   • Pool: 5         │ │ │ │
│  │  │  │ • SSL: Yes  │  │ • SSL: Yes  │  │   • SSL: Yes        │ │ │ │
│  │  │  │ • Status: ✓ │  │ • Status: ✓ │  │   • Status: ✓       │ │ │ │
│  │  │  └─────────────┘  └─────────────┘  └─────────────────────┘ │ │ │
│  │  │                                                             │ │ │
│  │  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │ │ │
│  │  │  │  Discover   │  │    Local    │  │      Test           │ │ │ │
│  │  │  │ Connector   │  │ Processor   │  │   Upstream          │ │ │ │
│  │  │  │ (GenServer) │  │ Connector   │  │   (GenServer)       │ │ │ │
│  │  │  │             │  │ (GenServer) │  │                     │ │ │ │
│  │  │  │ • Pool: 5   │  │ • Pool: 5   │  │   • Pool: 2         │ │ │ │
│  │  │  │ • SSL: Yes  │  │ • SSL: No   │  │   • SSL: No         │ │ │ │
│  │  │  │ • Status: ✓ │  │ • Status: ✓ │  │   • Status: ✓       │ │ │ │
│  │  │  └─────────────┘  └─────────────┘  └─────────────────────┘ │ │ │
│  │  └─────────────────────────────────────────────────────────────┘ │ │
│  └─────────────────────────────────────────────────────────────────┘ │
│                                                                     │
│  ┌─────────────────────────────────────────────────────────────────┐ │
│  │                    External Networks                            │ │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐ │ │
│  │  │ visa-proc   │  │ mc-proc     │  │ amex-proc               │ │ │
│  │  │ :8583       │  │ :8583       │  │ :8583                   │ │ │
│  │  └─────────────┘  └─────────────┘  └─────────────────────────┘ │ │
│  └─────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```

### Component Hierarchy

```
EnhancedUpstreamSupervisor
├── UpstreamRouter (GenServer)
├── UpstreamConnectionManager (GenServer)
├── UpstreamHealthMonitor (GenServer)
└── NetworkConnectorSupervisor (DynamicSupervisor)
    ├── VisaConnector (GenServer)
    ├── MasterCardConnector (GenServer)
    ├── AmexConnector (GenServer)
    ├── DiscoverConnector (GenServer)
    ├── LocalProcessorConnector (GenServer)
    └── TestUpstreamConnector (GenServer)
```

## Configuration

### File Location
`config/upstream_networks.exs`

### Configuration Structure

```elixir
# Upstream Network Configuration for external payment processors
config :da_product_app, :upstream_networks, %{
  visa: %{
    host: "visa-processor.example.com",
    port: 8583,
    packager: DaProductApp.MercuryISO8583.Packagers.ISO87BPackager,
    connection_pool_size: 10,
    timeout: 30_000,
    routing_rules: [
      {:bin_range, "400000-499999"},  # Visa BIN range
      {:mti_pattern, "01??"}          # Authorization requests
    ]
  },
  mastercard: %{
    host: "mc-processor.example.com", 
    port: 8583,
    packager: DaProductApp.MercuryISO8583.Packagers.ISO87BPackager,
    connection_pool_size: 10,
    timeout: 30_000,
    routing_rules: [
      {:bin_range, "510000-599999"},  # MasterCard BIN range
      {:mti_pattern, "01??"}          # Authorization requests
    ]
  },
  amex: %{
    host: "amex-processor.example.com",
    port: 8583,
    packager: DaProductApp.MercuryISO8583.Packagers.ISO87BPackager,
    connection_pool_size: 5,
    timeout: 25_000,
    routing_rules: [
      {:bin_range, "340000-379999"},  # American Express BIN range
      {:mti_pattern, "01??"}
    ]
  }
}

# Connection Pool Configuration
config :da_product_app, :upstream_connection_pools, %{
  default_pool_size: 10,
  max_overflow: 5,
  
  # Per-network overrides
  visa: %{pool_size: 15, max_overflow: 10},
  mastercard: %{pool_size: 15, max_overflow: 10}
}

# SSL Configuration for secure connections
config :da_product_app, :upstream_ssl, %{
  enabled: false,  # Set to true in production
  
  visa: %{
    enabled: true,
    verify: :verify_peer,
    cacertfile: "/path/to/visa-ca.pem",
    certfile: "/path/to/client.pem",
    keyfile: "/path/to/client.key"
  }
}
```

### Configuration Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| `host` | string | Upstream server hostname/IP |
| `port` | integer | Upstream server port |
| `packager` | module | ISO8583 packager for this network |
| `connection_pool_size` | integer | Number of persistent connections |
| `timeout` | integer | Network timeout in milliseconds |
| `routing_rules` | list | Rules for routing messages to this network |
| `ssl_enabled` | boolean | Enable SSL/TLS encryption |
| `health_check_interval` | integer | Health check frequency (ms) |

## Components

### 1. EnhancedUpstreamSupervisor

**Purpose**: Main supervisor for all upstream network components.

**Key Functions**:
```elixir
# Get system status
DaProductApp.Switch.EnhancedUpstreamSupervisor.get_status()

# Add new upstream network
DaProductApp.Switch.EnhancedUpstreamSupervisor.add_upstream_network(:paypal, config)

# Remove upstream network
DaProductApp.Switch.EnhancedUpstreamSupervisor.remove_upstream_network(:paypal)
```

### 2. UpstreamRouter (Enhanced GenServer)

**Purpose**: Routes messages to appropriate upstream networks based on configured rules.

**Key Functions**:
```elixir
# Route a message
{:ok, response} = DaProductApp.Switch.UpstreamRouter.route_message(iso_message, context)

# Route asynchronously
task = DaProductApp.Switch.UpstreamRouter.route_message_async(iso_message, context)

# Test upstream connectivity
DaProductApp.Switch.UpstreamRouter.test_upstream_connectivity(:visa)
```

**Routing Logic**:
1. Analyze message content (BIN, MTI, fields)
2. Match against routing rules for each network
3. Select appropriate upstream network
4. Apply network-specific transformations
5. Send message and handle response

### 3. UpstreamConnectionManager

**Purpose**: Manages connection pools and lifecycle for all upstream networks.

**Key Functions**:
```elixir
# Send message via managed connection
{:ok, response} = DaProductApp.Switch.UpstreamConnectionManager.send_message(:visa, packed_bytes)

# Get pool statistics
stats = DaProductApp.Switch.UpstreamConnectionManager.get_pool_stats()

# Add/remove networks dynamically
DaProductApp.Switch.UpstreamConnectionManager.add_network(:new_network, config)
DaProductApp.Switch.UpstreamConnectionManager.remove_network(:old_network)
```

**Pool Management**:
- Maintains connection pools per network
- Handles connection borrowing/returning
- Tracks connection statistics
- Implements connection recycling

### 4. UpstreamHealthMonitor

**Purpose**: Monitors health of upstream networks and implements circuit breaker pattern.

**Key Functions**:
```elixir
# Get health status of all networks
status = DaProductApp.Switch.UpstreamHealthMonitor.get_health_status()

# Check if specific network is healthy
healthy = DaProductApp.Switch.UpstreamHealthMonitor.is_network_healthy?(:visa)

# Force health check
DaProductApp.Switch.UpstreamHealthMonitor.check_network_health(:mastercard)

# Manually set network status
DaProductApp.Switch.UpstreamHealthMonitor.set_network_status(:visa, :unhealthy)
```

**Health Check Features**:
- Periodic connectivity tests (default: 30 seconds)
- Circuit breaker with configurable thresholds
- Automatic recovery attempts
- Status change notifications

### 5. NetworkConnector (Per-Network GenServers)

**Purpose**: Individual connectors for each upstream network.

**Key Functions**:
```elixir
# Send message through specific connector
{:ok, response} = DaProductApp.Switch.NetworkConnector.send_message(:visa, packed_bytes)

# Get connector statistics
stats = DaProductApp.Switch.NetworkConnector.get_statistics(:visa)

# Update connector configuration
DaProductApp.Switch.NetworkConnector.update_config(:visa, new_config)
```

## Message Flow

### Outbound Message Processing Pipeline

```
1. ChannelManager → UpstreamRouter
                      ↓
2. UpstreamRouter → Routing Rules Analysis
                      ↓
3. Routing Analysis → Network Selection (Visa/MC/Amex/etc)
                      ↓
4. Network Selection → Network-specific Transformations
                      ↓
5. Transformations → Message Packing (per network packager)
                      ↓
6. Packed Message → NetworkConnector (specific network)
                      ↓
7. NetworkConnector → UpstreamConnectionManager
                      ↓
8. ConnectionManager → TCP Connection (to external network)
                      ↓
9. External Network → Response Processing
                      ↓
10. Response → NetworkConnector → UpstreamRouter → ChannelManager
```

### Code Flow Example

```elixir
# 1. Route message from channel
{:ok, response} = UpstreamRouter.route_message(iso_message, channel_context)

# 2. Internal routing process:
# - Determine network based on routing rules
{:ok, upstream_config} = determine_upstream_network(iso_message, context)

# - Apply network-specific transformations
{:ok, formatted_message} = format_for_upstream(iso_message, upstream_config)

# - Send via connection manager
{:ok, response_bytes} = UpstreamConnectionManager.send_message(network_name, packed_bytes)

# - Unpack response
{:ok, response_message} = unpack_upstream_response(response_bytes, upstream_config)
```

## Management & Operations

### System Status and Control

```elixir
# Get comprehensive system status
status = DaProductApp.Switch.EnhancedUpstreamSupervisor.get_status()

# Output:
%{
  supervisor: :running,
  router: :running,
  connection_manager: :running,
  health_monitor: :running,
  network_connectors: %{
    running: 6,
    not_running: 0
  }
}
```

### Dynamic Network Management

```elixir
# Add new payment processor
paypal_config = %{
  host: "paypal-processor.com",
  port: 8583,
  packager: DaProductApp.MercuryISO8583.Packagers.ISO87BPackager,
  connection_pool_size: 8,
  timeout: 20_000,
  routing_rules: [
    {:field_value, 24, "003"},  # PayPal network ID
    {:bin_range, "600000-609999"}
  ]
}

# Add to system
{:ok, _pid} = EnhancedUpstreamSupervisor.add_upstream_network(:paypal, paypal_config)

# Remove when no longer needed
:ok = EnhancedUpstreamSupervisor.remove_upstream_network(:paypal)
```

### Configuration Updates

```elixir
# Update network configuration
new_visa_config = %{
  host: "new-visa-processor.com",
  port: 8584,  # Different port
  connection_pool_size: 15,  # Increased pool
  timeout: 25_000  # Reduced timeout
}

NetworkConnector.update_config(:visa, new_visa_config)
```

### Connection Pool Management

```elixir
# Monitor pool statistics
pool_stats = UpstreamConnectionManager.get_pool_stats()

# Output:
%{
  visa: %{
    total_connections_created: 45,
    active_connections: 8,
    pool_size: 10,
    messages_sent: 1523,
    messages_successful: 1520,
    messages_failed: 3,
    last_activity: ~U[2025-09-16 10:30:45.123Z]
  },
  mastercard: %{
    # ... similar stats
  }
}
```

## Health Monitoring & Circuit Breakers

### Health Status Monitoring

```elixir
# Check overall health status
health = UpstreamHealthMonitor.get_health_status()

# Output:
%{
  visa: :healthy,
  mastercard: :healthy,
  amex: :circuit_open,  # Circuit breaker activated
  discover: :unhealthy,
  local_processor: :healthy
}
```

### Circuit Breaker States

| State | Description | Behavior |
|-------|-------------|----------|
| `:healthy` | Network is operational | All traffic allowed |
| `:unhealthy` | Network issues detected | Traffic allowed, monitoring increased |
| `:circuit_open` | Circuit breaker activated | Traffic blocked, recovery attempts scheduled |

### Circuit Breaker Configuration

```elixir
# Health check configuration
config :da_product_app, :upstream_health_check_interval, 30_000  # 30 seconds
config :da_product_app, :upstream_failure_threshold, 3           # 3 consecutive failures
config :da_product_app, :upstream_recovery_timeout, 60_000       # 1 minute recovery wait
```

### Health Check Operations

```elixir
# Force health check for specific network
{:ok, status} = UpstreamHealthMonitor.check_network_health(:visa)

# Manual intervention
UpstreamHealthMonitor.set_network_status(:amex, :healthy)  # Force recovery

# Monitor health changes via PubSub
Phoenix.PubSub.subscribe(DaProductApp.PubSub, "upstream_health")

# Receive notifications:
{:network_status_change, :visa, :unhealthy}
```

## Connection Management

### Connection Lifecycle

```
1. Connection Creation → TCP connect to upstream host
                           ↓
2. Connection Validation → Test with ping/echo message
                           ↓
3. Pool Addition → Add to available connection pool
                           ↓
4. Active Usage → Borrow connection for message sending
                           ↓
5. Connection Return → Return to pool after use
                           ↓
6. Health Monitoring → Periodic health checks
                           ↓
7. Connection Recycling → Close and recreate stale connections
```

### SSL/TLS Configuration

```elixir
# Per-network SSL configuration
config :da_product_app, :upstream_ssl, %{
  visa: %{
    enabled: true,
    verify: :verify_peer,
    cacertfile: "/etc/ssl/certs/visa-ca.pem",
    certfile: "/etc/ssl/certs/client.pem",
    keyfile: "/etc/ssl/private/client.key",
    versions: [:"tlsv1.2", :"tlsv1.3"],
    ciphers: [:strong]
  },
  mastercard: %{
    enabled: true,
    verify: :verify_peer,
    cacertfile: "/etc/ssl/certs/mc-ca.pem",
    # ... similar SSL config
  }
}
```

### Connection Pool Tuning

```elixir
# Optimize pool sizes based on load
config :da_product_app, :upstream_connection_pools, %{
  # High-volume networks
  visa: %{pool_size: 20, max_overflow: 15},
  mastercard: %{pool_size: 20, max_overflow: 15},
  
  # Medium-volume networks  
  amex: %{pool_size: 10, max_overflow: 8},
  discover: %{pool_size: 10, max_overflow: 8},
  
  # Low-volume networks
  local_processor: %{pool_size: 5, max_overflow: 3}
}
```

## Routing Rules

### Rule Types

#### 1. BIN Range Routing
```elixir
# Route based on card number BIN ranges
{:bin_range, "400000-499999"}  # All Visa cards (4xxxxx)
{:bin_range, "510000-559999"}  # MasterCard range 1
{:bin_range, "340000-379999"}  # American Express
```

#### 2. MTI Pattern Routing
```elixir
# Route based on Message Type Indicator patterns
{:mti_pattern, "01??"}  # All authorization requests (0100, 0110, etc.)
{:mti_pattern, "02??"}  # All financial requests (0200, 0210, etc.)
{:mti_pattern, "04??"}  # All reversal requests (0400, 0410, etc.)
```

#### 3. Field-Based Routing
```elixir
# Route based on specific field values
{:field_value, 24, "001"}       # Network International ID = Visa
{:field_value, 24, "002"}       # Network International ID = MasterCard
{:field_present, 63}            # Route if field 63 is present
```

#### 4. Default Routing
```elixir
# Catch-all rule (should be last)
{:default, true}  # Route all unmatched messages here
```

### Complex Routing Examples

```elixir
# Multi-rule network configuration
visa: %{
  # ... connection config ...
  routing_rules: [
    {:bin_range, "400000-499999"},     # Visa BIN range
    {:mti_pattern, "01??"},            # AND authorization requests
    {:field_value, 18, "6011"}         # AND MCC = fuel stations
  ]
}

# Priority-based routing (rules evaluated in order)
local_processor: %{
  # ... connection config ...
  routing_rules: [
    {:field_present, 63},              # High priority: special processing
    {:field_value, 24, "999"},         # Medium priority: test transactions
    {:default, true}                   # Low priority: everything else
  ]
}
```

### Custom Routing Functions

```elixir
# Extend routing rules with custom logic
defmodule CustomRoutingRules do
  def matches_rule?(iso_message, {:custom_business_logic, params}) do
    # Implement custom routing logic
    pan = ISOMsg.get_field(iso_message, 2)
    amount = ISOMsg.get_field(iso_message, 4)
    
    # Example: Route high-value transactions differently
    case {pan, amount} do
      {pan, amount} when is_binary(pan) and is_binary(amount) ->
        amount_int = String.to_integer(amount)
        amount_int > 10000  # Route transactions > $100.00 differently
      _ ->
        false
    end
  end
end
```

## Troubleshooting

### Common Issues

#### 1. Network Connection Failures
```elixir
# Check network connectivity
case UpstreamRouter.test_upstream_connectivity(:visa) do
  :ok -> 
    IO.puts("Visa network is reachable")
  {:error, reason} -> 
    IO.puts("Visa network failed: #{inspect(reason)}")
end

# Common causes and solutions:
# - DNS resolution: Check /etc/hosts or DNS server
# - Firewall: Verify ports 8583, 8584 are open
# - SSL certificates: Check certificate validity and paths
```

#### 2. Circuit Breaker Activation
```elixir
# Check why circuit breaker opened
health_status = UpstreamHealthMonitor.get_health_status()

# Force recovery (use with caution)
UpstreamHealthMonitor.set_network_status(:visa, :healthy)

# Check failure history
connector_stats = NetworkConnector.get_statistics(:visa)
# Look at messages_failed count and patterns
```

#### 3. Message Routing Issues
```elixir
# Enable routing debug mode
Logger.configure(level: :debug)

# Test routing rules manually
iso_message = create_test_message()
matching_network = determine_upstream_network(iso_message, %{})

# Check BIN routing
pan = "4111111111111111"  # Test Visa card
bin = String.slice(pan, 0, 6)  # "411111"
# Verify this matches Visa BIN range "400000-499999"
```

#### 4. Performance Issues
```elixir
# Monitor connection pool exhaustion
pool_stats = UpstreamConnectionManager.get_pool_stats()

# Look for:
# - High active_connections relative to pool_size
# - Many messages_failed
# - Long average_response_time

# Solutions:
# - Increase connection_pool_size
# - Reduce timeout values
# - Add more upstream hosts for load balancing
```

### Debug Commands

```elixir
# System diagnostics
:observer.start()  # GUI system monitor

# Process inspection
pid = Process.whereis(DaProductApp.Switch.UpstreamRouter)
:recon.proc_window(pid, 10_000)  # Monitor for 10 seconds

# Memory analysis
:recon.proc_count(:memory, 10)  # Top 10 memory consumers

# Network statistics
stats = UpstreamConnectionManager.get_pool_stats()
Enum.each(stats, fn {network, data} ->
  IO.inspect({network, data}, label: "Network Stats")
end)
```

### Log Analysis Patterns

```bash
# Monitor upstream routing
tail -f log/dev.log | grep "Routing message MTI"

# Watch connection failures
tail -f log/dev.log | grep "Failed to connect to upstream"

# Track circuit breaker events
tail -f log/dev.log | grep "Opening circuit breaker"

# Monitor health check results
tail -f log/dev.log | grep "Health check"
```

## Performance Tuning

### Connection Optimization

```elixir
# High-throughput configuration
config :da_product_app, :upstream_networks, %{
  visa: %{
    # ... other config ...
    connection_pool_size: 25,        # Increased pool
    timeout: 15_000,                 # Reduced timeout
    max_overflow: 20,                # Allow burst capacity
    
    # TCP-level optimizations
    socket_opts: [
      :binary,
      {:nodelay, true},              # Disable Nagle's algorithm
      {:keepalive, true},            # Enable keepalive
      {:reuseaddr, true},            # Reuse addresses
      {:send_timeout, 10_000},       # Send timeout
      {:send_timeout_close, true}    # Close on send timeout
    ]
  }
}
```

### Load Balancing

```elixir
# Multiple hosts per network
config :da_product_app, :upstream_networks, %{
  visa: %{
    hosts: [
      %{host: "visa-primary.com", port: 8583, weight: 70},
      %{host: "visa-backup.com", port: 8583, weight: 30}
    ],
    load_balancing: %{
      strategy: :weighted_round_robin,  # or :least_connections
      health_check_interval: 15_000
    }
  }
}
```

### Memory Management

```elixir
# Monitor process memory
:recon.proc_count(:memory, 10)

# Check ETS tables (if used)
:ets.all() |> Enum.map(&:ets.info(&1, :memory))

# Garbage collection tuning
:erlang.system_flag(:fullsweep_after, 10)
```

### Monitoring and Metrics

```elixir
# Integration with telemetry
:telemetry.attach(
  "upstream-router-metrics",
  [:upstream_router, :route_message, :stop],
  &handle_router_metrics/4,
  %{}
)

# Custom metrics collection
defmodule UpstreamMetrics do
  def record_message_sent(network, response_time) do
    :telemetry.execute([:upstream, :message, :sent], %{
      response_time: response_time
    }, %{network: network})
  end
end
```

### Performance Monitoring Dashboard

```elixir
# Phoenix LiveView dashboard for monitoring
defmodule DaProductAppWeb.UpstreamDashboard do
  use Phoenix.LiveView
  
  def mount(_params, _session, socket) do
    if connected?(socket) do
      :timer.send_interval(1000, self(), :update_stats)
    end
    
    {:ok, assign(socket, :stats, get_upstream_stats())}
  end
  
  def handle_info(:update_stats, socket) do
    {:noreply, assign(socket, :stats, get_upstream_stats())}
  end
  
  defp get_upstream_stats do
    %{
      health: UpstreamHealthMonitor.get_health_status(),
      pools: UpstreamConnectionManager.get_pool_stats(),
      system: EnhancedUpstreamSupervisor.get_status()
    }
  end
end
```