# Repository Consolidation Migration Plan

**Document Version:** 1.0  
**Date:** March 1, 2026  
**Status:** Planning Phase  
**Priority:** Medium  

## Executive Summary

This document outlines the strategy for consolidating database access through `PlatformCore.Repo` and deprecating `DaProductApp.Repo`. The migration addresses architectural concerns raised during code review regarding the use of `PlatformCore.Repo` across the umbrella application.

## Current State Assessment

### Architecture Overview

The application currently maintains two Ecto repositories:

1. **PlatformCore.Repo** - New centralized repository in `apps/platform_core`
2. **DaProductApp.Repo** - Legacy repository in `apps/da_product_app`

Both repositories:
- Connect to the same database: `shukria_transactions`
- Maintain separate connection pools
- Are started in their respective application supervisors

### What's Working Well ✅

1. **Shared Database Configuration**
   - Both repos connect to the same database in all environments (dev, test, prod)
   - Configuration is consistent in `config/dev.exs` and `config/runtime.exs`

2. **Migration Files Are Synchronized**
   - Root `/priv/repo/migrations`: 50 migration files
   - `/apps/da_product_app/priv/repo/migrations`: 50 identical migration files
   - Migrations are duplicates, not separate migration sets

3. **New Core Modules Use PlatformCore.Repo**
   - `tms_core` - All contexts use `PlatformCore.Repo` ✓
   - `settlement_core` - All contexts use `PlatformCore.Repo` ✓
   - `risk_core` - All contexts use `PlatformCore.Repo` ✓

4. **Both Repos Are Running**
   - Separate connection pools to same database
   - No immediate operational issues

### Identified Issues ⚠️

#### 1. Mixed Repo Usage in da_product_app
**Impact:** Resource inefficiency, architectural confusion

Contexts still using `DaProductApp.Repo`:
- `DaProductApp.Transactions`
- `DaProductApp.ParameterManagement`
- `DaProductApp.Software`
- `DaProductApp.Disputes`
- `DaProductApp.Providers`
- `DaProductApp.Users`
- `DaProductApp.SBOM`
- Controllers (Settlement, Transaction, etc.)

**Count:** ~50+ files using `DaProductApp.Repo`

#### 2. Dual Connection Pools
**Impact:** Wasted database connections

- Both repos maintain separate connection pools
- Current pool size: 2 connections per repo in dev (4 total)
- Production pool size: 10 connections per repo (20 total)
- After consolidation: Can use single pool of 10-15 connections

#### 3. Schema Location Confusion
**Impact:** Maintenance difficulty

Examples of duplicate/scattered schemas:
- `TmsCore.TerminalManagement.TmsTerminal` (correct location)
- `DaProductApp.TerminalManagement.TmsTerminal` (duplicate)
- Multiple schema paths: `/lib`, `/apps/da_product_app/lib`, `/apps/tms_core/lib`

#### 4. Migration Path Duplication
**Impact:** Confusion about source of truth

- Migrations exist in 2 locations (root and da_product_app)
- Both directories have identical content
- No clear indication of which is canonical

#### 5. Missing Configuration for PlatformCore.Repo
**Impact:** Mix tasks don't recognize PlatformCore.Repo

Current config:
```elixir
config :da_product_app,
  ecto_repos: [DaProductApp.Repo]
```

Missing:
- `PlatformCore.Repo` not listed in `ecto_repos`
- Mix commands (`mix ecto.migrate`, `mix ecto.rollback`) don't see it
- Migration primary key config missing for PlatformCore

## Migration Strategy

### Goals

1. ✅ Consolidate all database access through `PlatformCore.Repo`
2. ✅ Remove `DaProductApp.Repo` from production use
3. ✅ Eliminate duplicate schemas and migrations
4. ✅ Maintain backward compatibility during transition
5. ✅ Zero downtime migration

### Non-Goals

- ❌ Database schema changes
- ❌ Migration file modifications
- ❌ Performance optimization (separate effort)

### Phased Approach

---

## Phase 1: Configuration Updates (Week 1)

**Risk Level:** Low  
**Rollback Difficulty:** Easy

### Tasks

#### 1.1 Update ecto_repos Configuration

**File:** `config/config.exs`

```elixir
# Before
config :da_product_app,
  ecto_repos: [DaProductApp.Repo]

# After
config :da_product_app,
  ecto_repos: [PlatformCore.Repo, DaProductApp.Repo]
```

#### 1.2 Add Migration Configuration

**File:** `config/config.exs`

```elixir
config :platform_core, PlatformCore.Repo,
  migration_primary_key: [name: :id, type: :bigserial]
```

#### 1.3 Create Documentation

- [x] Create `REPO_USAGE.md` (this document acts as that)
- [ ] Add inline code comments where needed
- [ ] Update team wiki/confluence

### Success Criteria

- ✅ Mix commands recognize both repos
- ✅ `mix ecto.migrate` works for PlatformCore.Repo
- ✅ All tests passing
- ✅ No production impact

### Testing

```bash
# Verify config
mix ecto.repos

# Should output:
# [PlatformCore.Repo, DaProductApp.Repo]

# Test migration commands
MIX_ENV=test mix ecto.migrate
MIX_ENV=test mix ecto.rollback --step=1
```

---

## Phase 2: Migration Consolidation (Week 2)

**Risk Level:** Medium  
**Rollback Difficulty:** Medium

### Tasks

#### 2.1 Audit Migration Files

```bash
# Verify migrations are identical
diff -r /home/prem/mercurypay/tmsuat/priv/repo/migrations \
        /home/prem/mercurypay/tmsuat/apps/da_product_app/priv/repo/migrations
```

#### 2.2 Choose Canonical Migration Location

**Decision:** Keep root `/priv/repo/migrations` as canonical source

**Rationale:**
- Already used by mix commands
- Root-level is more visible
- Consistent with Phoenix conventions

#### 2.3 Remove Duplicate Migrations

```bash
# After confirming identity
rm -rf /home/prem/mercurypay/tmsuat/apps/da_product_app/priv/repo/migrations
```

#### 2.4 Update Migration Paths (if needed)

Verify that PlatformCore.Repo finds migrations:
```elixir
# In config/config.exs or PlatformCore.Repo
# Add if migrations are not auto-discovered:
config :platform_core, PlatformCore.Repo,
  priv: "priv/repo"
```

### Success Criteria

- ✅ Only one migration directory exists
- ✅ Mix commands find migrations correctly
- ✅ Migration history intact in database
- ✅ All environments updated

### Testing

```bash
# Check migration status
mix ecto.migrations

# Verify on all environments
MIX_ENV=dev mix ecto.migrations
MIX_ENV=test mix ecto.migrations
MIX_ENV=prod mix ecto.migrations
```

---

## Phase 3: Gradual Repo Migration (Weeks 3-6)

**Risk Level:** High  
**Rollback Difficulty:** High (requires code deployment)

### Strategy: Context-by-Context Migration

Migrate contexts in order of risk (lowest to highest):

#### Week 3: Non-Critical Contexts

1. **DaProductApp.Software**
   - Files: 3-4
   - Risk: Low (minimal production use)

2. **DaProductApp.Disputes**
   - Files: 2-3
   - Risk: Low (admin feature)

3. **DaProductApp.SBOM**
   - Files: 2
   - Risk: Low (reporting only)

**Migration Pattern:**
```elixir
# Before
defmodule DaProductApp.Software do
  alias DaProductApp.Repo
  # ... context functions
end

# After
defmodule DaProductApp.Software do
  alias PlatformCore.Repo
  # ... context functions
end
```

**Testing for Each Context:**
- [ ] Unit tests pass
- [ ] Integration tests pass
- [ ] Manual testing in dev environment
- [ ] Staging environment deployment
- [ ] Monitor for 1 day before next migration

#### Week 4: Core Business Logic

1. **DaProductApp.Providers**
   - Files: 2-3
   - Risk: Medium (used in transactions)

2. **DaProductApp.ParameterManagement**
   - Files: ~5
   - Risk: Medium (terminal configuration)

3. **DaProductApp.Users**
   - Files: ~5
   - Risk: Medium (authentication)

**Additional Testing:**
- [ ] Load testing
- [ ] Transaction flow testing
- [ ] User authentication testing

#### Week 5: High-Risk Contexts

1. **DaProductApp.Transactions**
   - Files: ~5
   - Risk: HIGH (core business logic)

2. **Settlement Controllers**
   - `AaniSettlementController`
   - `SettlementController`
   - `SettlementWebhookController`

3. **Transaction Controllers**
   - `TransactionPostController`

**Additional Testing:**
- [ ] End-to-end transaction testing
- [ ] Settlement flow testing
- [ ] Webhook testing
- [ ] Performance testing
- [ ] Canary deployment recommended

#### Week 6: Seeds and Test Support

1. **Seed Files**
   - `/priv/repo/seeds.exs`
   - `/priv/repo/seeds/*.exs`

2. **Test Support Files**
   - `test/support/data_case.ex`
   - Test factories/fixtures

### File Migration Tracking

**Script to Track Progress:**
```bash
#!/bin/bash
# count_repo_usage.sh

echo "Files using DaProductApp.Repo:"
grep -r "alias DaProductApp.Repo" apps/da_product_app/lib --include="*.ex" | wc -l

echo "Files using PlatformCore.Repo:"
grep -r "alias PlatformCore.Repo" apps/da_product_app/lib --include="*.ex" | wc -l

echo "\nRemaining files to migrate:"
grep -r "alias DaProductApp.Repo" apps/da_product_app/lib --include="*.ex" | cut -d: -f1 | sort | uniq
```

### Rollback Procedure

If issues are encountered:

1. **Immediate Rollback (< 1 hour)**
   ```bash
   # Revert code changes
   git revert <commit-hash>
   
   # Deploy previous version
   mix release
   ```

2. **Database State**
   - No database changes needed
   - Both repos connect to same DB
   - No migration rollback needed

3. **Connection Pool Monitoring**
   ```sql
   -- Monitor active connections
   SHOW PROCESSLIST;
   
   -- Check connection counts by application
   SELECT USER, COUNT(*) FROM information_schema.PROCESSLIST 
   GROUP BY USER;
   ```

---

## Phase 4: Cleanup (Week 7+)

**Risk Level:** Low  
**Rollback Difficulty:** Easy

### Tasks

#### 4.1 Remove Duplicate Schemas

**Audit First:**
```bash
# Find duplicate schema definitions
find . -name "*.ex" -type f -exec grep -l "schema \"tms_terminals\"" {} \;
```

**Action:**
- Keep schemas in core apps (`tms_core`, `settlement_core`, `risk_core`)
- Remove duplicates from `da_product_app`
- Update imports/aliases as needed

#### 4.2 Remove DaProductApp.Repo from Supervisor

**File:** `apps/da_product_app/lib/da_product_app/application.ex`

```elixir
# Before
children = [
  DaProductApp.Repo,
  # ...
]

# After
children = [
  # DaProductApp.Repo removed - using PlatformCore.Repo
  # ...
]
```

**Monitoring Period:** 1-2 weeks before deletion

#### 4.3 Verify No Usage

```bash
# Search entire codebase
grep -r "DaProductApp.Repo" . --include="*.ex" --include="*.exs"

# Should only find:
# - The repo module definition itself
# - Historical comments
# - This documentation
```

#### 4.4 Delete DaProductApp.Repo Module

**Only after 1-2 weeks of stable operation:**

```bash
# Archive for reference
git mv apps/da_product_app/lib/da_product_app/repo.ex \
       docs/archived/repo.ex.archived

# Or delete
rm apps/da_product_app/lib/da_product_app/repo.ex
```

#### 4.5 Remove from Configuration

**File:** `config/config.exs`

```elixir
# Before
config :da_product_app,
  ecto_repos: [PlatformCore.Repo, DaProductApp.Repo]

# After
config :da_product_app,
  ecto_repos: [PlatformCore.Repo]
```

### Success Criteria

- ✅ No references to `DaProductApp.Repo` in codebase
- ✅ Only `PlatformCore.Repo` in supervisor
- ✅ Single connection pool active
- ✅ All tests passing
- ✅ Production stable for 2+ weeks

---

## Risk Mitigation

### Database Connection Pool Management

**Current State:**
- Dev: 2 pools × 2 connections = 4 total connections
- Prod: 2 pools × 10 connections = 20 total connections

**After Migration:**
- Dev: 1 pool × 3 connections = 3 total connections
- Prod: 1 pool × 15 connections = 15 total connections

**Benefits:**
- Reduced connection overhead
- Better connection utilization
- Simpler monitoring

### Testing Requirements

#### Per-Phase Testing Checklist

**Unit Tests:**
- [ ] All module tests pass
- [ ] Coverage > 80% for migrated modules

**Integration Tests:**
- [ ] Transaction flows end-to-end
- [ ] Settlement processing
- [ ] Parameter management
- [ ] User authentication

**Performance Tests:**
- [ ] Response time < baseline + 10%
- [ ] Database query time unchanged
- [ ] Connection pool saturation < 80%

**Production Validation:**
- [ ] Staging deployment successful
- [ ] Smoke tests pass
- [ ] Error rate < 0.1%
- [ ] Monitor for 24 hours minimum

### Monitoring During Migration

**Key Metrics:**

1. **Database Connections**
   ```sql
   SELECT 
     USER, 
     COUNT(*) as connection_count,
     STATE
   FROM information_schema.PROCESSLIST
   WHERE DB = 'shukria_transactions'
   GROUP BY USER, STATE;
   ```

2. **Application Logs**
   - Watch for Ecto connection errors
   - Monitor repo checkout timeouts
   - Track query execution times

3. **Error Rates**
   - Track 500 errors
   - Monitor Ecto.Query errors
   - Watch for timeout errors

4. **Response Times**
   - API endpoints
   - Settlement processing
   - Transaction creation

**Alerting Thresholds:**
- Error rate > 1%: WARNING
- Response time > 500ms: WARNING
- Connection pool usage > 90%: CRITICAL

### Rollback Plan

#### Scenarios and Actions

| Scenario | Severity | Action | Time to Rollback |
|----------|----------|--------|------------------|
| Config change breaks dev | Low | Revert config commit | < 5 minutes |
| Migration path issues | Medium | Restore migration dir | < 10 minutes |
| Context migration errors | High | Revert code deployment | < 30 minutes |
| Production connection issues | CRITICAL | Emergency rollback | < 15 minutes |

#### Emergency Rollback Procedure

```bash
# 1. Identify issue
tail -f /var/log/app/current.log | grep "Repo"

# 2. Quick rollback via git
git log --oneline -10  # Find last good commit
git revert <bad-commit-hash>
git push origin pr-108

# 3. Deploy
ssh production-server
cd /opt/app
git pull
mix release
systemctl restart app

# 4. Verify
curl https://api.example.com/health
```

---

## Implementation Checklist

### Pre-Migration

- [ ] Review this plan with team
- [ ] Schedule migration windows
- [ ] Set up monitoring dashboards
- [ ] Create rollback runbooks
- [ ] Notify stakeholders
- [ ] Backup production database

### Phase 1: Configuration (Week 1)

- [ ] Update `config/config.exs` with both repos
- [ ] Add migration configuration for PlatformCore
- [ ] Test mix commands
- [ ] Verify in dev environment
- [ ] Deploy to staging
- [ ] All tests pass

### Phase 2: Migration Consolidation (Week 2)

- [ ] Audit migration files
- [ ] Remove duplicate migrations
- [ ] Update migration paths
- [ ] Test migration commands
- [ ] Verify in all environments
- [ ] Document changes

### Phase 3: Code Migration (Weeks 3-6)

**Week 3: Non-Critical**
- [ ] Migrate Software context
- [ ] Migrate Disputes context
- [ ] Migrate SBOM context
- [ ] Deploy to staging
- [ ] Monitor for 1 day
- [ ] Deploy to production

**Week 4: Core Logic**
- [ ] Migrate Providers context
- [ ] Migrate ParameterManagement context
- [ ] Migrate Users context
- [ ] Deploy to staging
- [ ] Load testing
- [ ] Monitor for 2 days
- [ ] Deploy to production

**Week 5: High Risk**
- [ ] Migrate Transactions context
- [ ] Migrate Settlement controllers
- [ ] Migrate Transaction controllers
- [ ] Deploy to staging
- [ ] End-to-end testing
- [ ] Performance testing
- [ ] Canary deployment (10% traffic)
- [ ] Monitor for 3 days
- [ ] Full production deployment

**Week 6: Support Files**
- [ ] Migrate seed files
- [ ] Migrate test support files
- [ ] Update documentation
- [ ] Verify all tests pass

### Phase 4: Cleanup (Week 7+)

- [ ] Monitor for 1 week (no issues)
- [ ] Remove duplicate schemas
- [ ] Remove DaProductApp.Repo from supervisor
- [ ] Monitor for 2 weeks (no issues)
- [ ] Delete DaProductApp.Repo module
- [ ] Remove from configuration
- [ ] Update documentation
- [ ] Close migration ticket

---

## Success Metrics

### Technical Metrics

| Metric | Baseline | Target | Measure |
|--------|----------|--------|---------|
| Database connections (prod) | 20 | 15 | `SHOW PROCESSLIST` |
| Repo references | 50+ to DaProductApp.Repo | 0 | `grep -r` count |
| Test pass rate | 100% | 100% | CI/CD pipeline |
| Code duplication | 2 Repos, duplicate schemas | 1 Repo, single schemas | Manual audit |

### Business Metrics

| Metric | Target | Measure |
|--------|--------|---------|
| Zero downtime | 100% uptime during migration | Pingdom/monitoring |
| Error rate | < 0.1% increase | Application logs |
| Response time | < 10% degradation | APM tools |
| Customer impact | Zero complaints | Support tickets |

---

## Timeline Summary

| Phase | Duration | Risk | Rollback |
|-------|----------|------|----------|
| Phase 1: Configuration | Week 1 | Low | Easy |
| Phase 2: Migration Files | Week 2 | Medium | Medium |
| Phase 3: Code Migration | Weeks 3-6 | High | High |
| Phase 4: Cleanup | Week 7+ | Low | Easy |

**Total Estimated Time:** 7-8 weeks

---

## Team Responsibilities

### Development Team
- Execute code migrations
- Write/update tests
- Code review for each phase
- Monitor deployments

### DevOps Team
- Set up monitoring
- Prepare rollback procedures
- Assist with deployments
- Database connection monitoring

### QA Team
- Test each migrated context
- Perform regression testing
- Load testing for high-risk phases
- Sign-off for production deployment

### Product/Business
- Approve migration windows
- Monitor business metrics
- Communicate with stakeholders
- Approve go/no-go decisions

---

## Communication Plan

### Pre-Migration
- Kickoff meeting with all teams
- Share this document
- Schedule migration windows
- Set up Slack channel: #repo-migration

### During Migration
- Daily standups during Weeks 3-6
- Post-deployment status updates
- Incident escalation procedures
- Weekly stakeholder updates

### Post-Migration
- Retrospective meeting
- Update architectural documentation
- Share lessons learned
- Close migration project

---

## References

### Related Documents
- `docs/PLATFORM_CORE_MIGRATION.md`
- `docs/PLATFORM_WEB_MIGRATION_CHECKPOINT.md`
- `docs/INTEGRATION.md`

### Code Locations
- PlatformCore.Repo: `apps/platform_core/lib/platform_core/repo.ex`
- DaProductApp.Repo: `apps/da_product_app/lib/da_product_app/repo.ex`
- Config: `config/config.exs`, `config/dev.exs`, `config/runtime.exs`
- Migrations: `/priv/repo/migrations`

### Useful Commands

```bash
# Count repo usage
grep -r "alias DaProductApp.Repo" apps/ --include="*.ex" | wc -l

# Find migration files
find . -path "*/priv/repo/migrations/*.exs" -type f

# Check database connections
mysql -u root -p -e "SHOW PROCESSLIST;"

# Run tests for specific context
mix test test/da_product_app/software --trace

# Deploy to staging
git push staging pr-108:main
```

---

## Approval and Sign-off

| Role | Name | Signature | Date |
|------|------|-----------|------|
| Lead Developer | _____________ | _____________ | _______ |
| DevOps Lead | _____________ | _____________ | _______ |
| QA Lead | _____________ | _____________ | _______ |
| Product Manager | _____________ | _____________ | _______ |
| Engineering Manager | _____________ | _____________ | _______ |

---

## Revision History

| Version | Date | Author | Changes |
|---------|------|--------|---------|
| 1.0 | 2026-03-01 | System | Initial document creation |

---

**Document Status:** ✅ Ready for Review

**Next Steps:**
1. Review with team
2. Get approvals
3. Schedule Phase 1 for next sprint
4. Create tracking tickets in project management system