Architecture

External Identifiers

Why ZineCore2 uses separate internal and external identifiers

ZineCore2 uses a dual identifier pattern where every profile model has two separate ID fields: an internal database primary key and an external human-readable identifier.

The Pattern

Every profile model follows this structure:

class Zine(models.Model):
    # Internal primary key (never exposed via API)
    id = models.BigAutoField(primary_key=True)

    # External identifier (used in API URLs and responses)
    zine_id = models.CharField(
        max_length=255,
        unique=True,
        db_index=True,
        help_text="External unique identifier (e.g., zine_mutate_3_1st)"
    )

    # Metadata fields...
    title = models.CharField(max_length=500)

Two identifiers:

id — Integer, auto-incrementing, internal use only
zine_id — String, human-readable, used externally

Why Two Identifiers?

Problem: Integer PKs vs. Readable IDs

Database best practice: Use integer primary keys for performance.

API best practice: Use human-readable IDs for usability.

These goals conflict. The dual identifier pattern satisfies both.

Integer PKs (Internal)

Advantages:

Fast indexing and lookups
Minimal storage (8 bytes for BigAutoField)
Auto-incrementing (no collision risk)
Efficient for foreign keys

Disadvantages:

Not human-readable (1842 tells you nothing)
Sequential (can leak information about record count)
Hard to remember for API users

String IDs (External)

Advantages:

Human-readable (zine_mutate_3_1st is meaningful)
Self-documenting in URLs
Stable across environments
Easy to reference in documentation

Disadvantages:

Slower indexing than integers
More storage (up to 255 bytes)
Requires manual generation or external system

How It Works

Database Layer

Foreign keys use internal integer PKs:

-- holdings_holding table
CREATE TABLE holdings_holding (
  id BIGSERIAL PRIMARY KEY,
  holding_id VARCHAR(255) UNIQUE NOT NULL,
  zine_id BIGINT REFERENCES catalog_zine(id),  -- Integer FK
  repository_id BIGINT REFERENCES repositories_repository(id)  -- Integer FK
);

Benefits:

Efficient joins
Referential integrity enforced
Fast queries

API Layer

API URLs use external string identifiers:

GET /api/zines/zine_mutate_3_1st/
GET /api/agents/agent_judith_arcana/
GET /api/repositories/repo_barnard/
GET /api/holdings/holding_barnard_mutate_3/

Benefits:

Readable URLs
Stable across environments (dev, staging, prod)
Natural keys for fixtures

Implementation Details

Model Definition

All four profile models follow the same pattern:

# catalog/models.py
class Zine(TimestampedModel):
    id = models.BigAutoField(primary_key=True)
    zine_id = models.CharField(max_length=255, unique=True, db_index=True)
    # ... fields

# agents/models.py
class Agent(TimestampedModel):
    id = models.BigAutoField(primary_key=True)
    agent_id = models.CharField(max_length=255, unique=True, db_index=True)
    # ... fields

# holdings/models.py
class Holding(TimestampedModel):
    id = models.BigAutoField(primary_key=True)
    holding_id = models.CharField(max_length=255, unique=True, db_index=True)
    zine = models.ForeignKey('catalog.Zine', on_delete=models.CASCADE)  # Uses internal id
    repository = models.ForeignKey('repositories.Repository', on_delete=models.CASCADE)
    # ... fields

# repositories/models.py
class Repository(TimestampedModel):
    id = models.BigAutoField(primary_key=True)
    repo_id = models.CharField(max_length=255, unique=True, db_index=True)
    # ... fields

Naming convention:

id — Always the internal PK
{profile}_id — External identifier (zine_id, agent_id, repo_id, holding_id)

ViewSet Lookup

ViewSets use the external ID for URL routing:

# catalog/views.py
class ZineViewSet(viewsets.ModelViewSet):
    queryset = Zine.objects.all()
    lookup_field = 'zine_id'  # Use external ID for lookups

# agents/views.py
class AgentViewSet(viewsets.ModelViewSet):
    queryset = Agent.objects.all()
    lookup_field = 'agent_id'

# holdings/views.py
class HoldingViewSet(viewsets.ModelViewSet):
    queryset = Holding.objects.all()
    lookup_field = 'holding_id'

# repositories/views.py
class RepositoryViewSet(viewsets.ModelViewSet):
    queryset = Repository.objects.all()
    lookup_field = 'repo_id'

Effect:

# These work (external ID):
GET /api/zines/zine_mutate_3_1st/
GET /api/agents/agent_judith_arcana/

# These DON'T work (internal ID not exposed):
GET /api/zines/1842/
GET /api/agents/523/

Serializer Exposure

Only the external ID is exposed via API:

# catalog/serializers.py
class ZineReadSerializer(serializers.ModelSerializer):
    class Meta:
        model = Zine
        fields = [
            'zine_id',  # External ID (exposed)
            # 'id' is NOT included (never exposed)
            'title',
            'creator',
            # ...
        ]

API response:

{
  "zine_id": "zine_mutate_3_1st",
  "title": "Mutate Zine #3",
  "creator": [...]
}

The internal id field never appears in API responses.

Naming Conventions

External identifiers follow consistent naming patterns:

Zines (zine_id)

Format: zine_{title_slug}_{issue}_{edition}

Examples:

zine_mutate_3_1st
zine_feminist_killjoy_1_2nd
zine_punk_planet_73
zine_dishwasher_18

Pattern:

Prefix: zine_
Slug: Shortened title (lowercase, underscores)
Issue: Issue number or date
Edition: Edition (1st, 2nd, etc.)

Agents (agent_id)

Format: agent_{name_slug}

Examples:

agent_judith_arcana
agent_kathleen_hanna
agent_mimi_nguyen
agent_zinester_collective
agent_microcosm_publishing

Pattern:

Prefix: agent_
Slug: Name (lowercase, underscores)
For anonymity: Use descriptive name (e.g., agent_anonymous_zinester_seattle)

Repositories (repo_id)

Format: repo_{institution_slug}

Examples:

repo_barnard_zine_library
repo_abc_no_rio
repo_queer_zine_archive_project
repo_my_personal_collection

Pattern:

Prefix: repo_
Slug: Institution name (lowercase, underscores)

Holdings (holding_id)

Format: holding_{repo_slug}_{zine_slug}_{copy_number}

Examples:

holding_barnard_mutate_3_001
holding_qzap_feminist_killjoy_1_002
holding_my_collection_punk_planet_73

Pattern:

Prefix: holding_
Repository slug
Zine slug (shortened)
Copy number (001, 002, etc.)

Benefits

1. Readable URLs

With dual identifiers:

GET /api/zines/zine_mutate_3_1st/

Without (integer PKs only):

GET /api/zines/1842/

The first is self-documenting. The second requires documentation lookup.

2. Stable Across Environments

External IDs are the same in dev, staging, and production:

# Development database
zine_mutate_3_1st → id: 5

# Production database
zine_mutate_3_1st → id: 1842

The external ID zine_mutate_3_1st is consistent. The internal id differs.

3. Natural Keys for Fixtures

Fixtures can reference external IDs:

[
  {
    "model": "catalog.zine",
    "fields": {
      "zine_id": "zine_mutate_3_1st",
      "title": "Mutate Zine #3",
      "creator": ["agent_judith_arcana"]
    }
  },
  {
    "model": "holdings.holding",
    "fields": {
      "holding_id": "holding_barnard_mutate_3",
      "zine_id": "zine_mutate_3_1st",  // Reference by external ID
      "repository_id": "repo_barnard"
    }
  }
]

No need to know internal integer IDs.

4. Changeable IDs

External IDs can be changed without breaking database integrity:

# Update external ID
zine = Zine.objects.get(zine_id='zine_old_name')
zine.zine_id = 'zine_new_name'
zine.save()

Foreign key relationships (which use internal id) are unaffected.

With integer-only PKs, changing the identifier would break all foreign keys.

5. Database Performance

Foreign keys use integers for optimal performance:

-- Fast integer join
SELECT h.*, z.title
FROM holdings_holding h
JOIN catalog_zine z ON h.zine_id = z.id  -- Integer join
WHERE h.repository_id = 42;

6. API Usability

Clients work with meaningful identifiers:

// Client code
const zineId = 'zine_mutate_3_1st';  // Readable, memorable
const response = await fetch(`/api/zines/${zineId}/`);

No need to track obscure integer IDs.

Trade-Offs

Storage Cost

Each record stores two identifiers instead of one:

id: 8 bytes (BigAutoField)
zine_id: up to 255 bytes (CharField)

Total overhead per record: ~250 bytes

For a database with 100,000 zine records: 25 MB extra storage.

Verdict: Negligible cost on modern hardware.

Index Overhead

Both fields are indexed:

id = models.BigAutoField(primary_key=True)  # Automatic index
zine_id = models.CharField(..., db_index=True)  # Explicit index

Effect: Slightly slower writes (two indexes to update).

Verdict: Minimal impact, vastly outweighed by usability benefits.

Developer Complexity

Developers must understand two identifier concepts:

Internal ID (for foreign keys)
External ID (for API)

Mitigation: Clear documentation and consistent naming conventions.

Alternative Approaches

Approach 1: Integer PKs Only

class Zine(models.Model):
    id = models.BigAutoField(primary_key=True)
    title = models.CharField(max_length=500)

URLs:

GET /api/zines/1842/

Problems:

Opaque identifiers
Unstable across environments
Hard to remember for API users

Approach 2: String PKs Only

class Zine(models.Model):
    zine_id = models.CharField(max_length=255, primary_key=True)
    title = models.CharField(max_length=500)

URLs:

GET /api/zines/zine_mutate_3_1st/

Problems:

Slower foreign key joins (string comparison vs. integer)
More storage for foreign keys
No auto-generation (must provide ID on creation)

Approach 3: UUIDs

class Zine(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4)
    title = models.CharField(max_length=500)

URLs:

GET /api/zines/a8098c1a-f86e-11da-bd1a-00112444be1e/

Problems:

Unreadable identifiers
Longer than integers (16 bytes vs. 8 bytes)
Still need separate human-readable field for usability

ZineCore2 Approach: Dual Identifiers

Best of both worlds:

Integer PKs for performance
String IDs for usability
Explicit separation of concerns

Database Schema

Complete schema showing both identifier types:

-- Zine table
CREATE TABLE catalog_zine (
  id BIGSERIAL PRIMARY KEY,                    -- Internal
  zine_id VARCHAR(255) UNIQUE NOT NULL,        -- External
  title VARCHAR(500) NOT NULL,
  creator TEXT[] NOT NULL,
  -- ... other fields
  created_at TIMESTAMP NOT NULL,
  updated_at TIMESTAMP NOT NULL
);

CREATE INDEX idx_zine_id ON catalog_zine(zine_id);  -- External ID index

-- Holding table (demonstrates FKs use internal IDs)
CREATE TABLE holdings_holding (
  id BIGSERIAL PRIMARY KEY,                    -- Internal
  holding_id VARCHAR(255) UNIQUE NOT NULL,     -- External
  zine_id BIGINT REFERENCES catalog_zine(id),  -- FK uses internal ID
  repository_id BIGINT REFERENCES repositories_repository(id),
  -- ... other fields
  created_at TIMESTAMP NOT NULL,
  updated_at TIMESTAMP NOT NULL
);

CREATE INDEX idx_holding_id ON holdings_holding(holding_id);
CREATE INDEX idx_holding_zine_repo ON holdings_holding(zine_id, repository_id);

Key points:

PKs are all BIGSERIAL (integer)
Foreign keys reference integer PKs
External IDs have UNIQUE constraints and indexes
Both identifier types are indexed for fast lookups

Querying Patterns

Query by External ID (API Pattern)

# Get zine by external ID
zine = Zine.objects.get(zine_id='zine_mutate_3_1st')

SQL:

SELECT * FROM catalog_zine
WHERE zine_id = 'zine_mutate_3_1st';

Uses the zine_id index — fast lookup.

Query by Internal ID (Internal Use)

# Get zine by internal ID (rare, internal use only)
zine = Zine.objects.get(id=1842)

SQL:

SELECT * FROM catalog_zine
WHERE id = 1842;

Uses the primary key index — even faster.

Foreign Key Lookups

# Get all holdings for a zine
holdings = Holding.objects.filter(zine_id=zine.id)  # Uses internal ID

SQL:

SELECT * FROM holdings_holding
WHERE zine_id = 1842;

Foreign key uses internal integer ID for performance.

Reverse Foreign Key Lookups

# Get all holdings for a zine (via reverse relation)
zine = Zine.objects.get(zine_id='zine_mutate_3_1st')
holdings = zine.holdings.all()  # Uses internal ID automatically

Django ORM handles the internal ID lookup automatically.

Migration Considerations

If you need to change an external ID:

# Safe to change external ID
zine = Zine.objects.get(zine_id='zine_old_id')
zine.zine_id = 'zine_new_id'
zine.save()

Foreign keys are unaffected because they reference the internal id.

However, external systems (API clients, documentation) must update their references.

Best Practices

1. Never Expose Internal IDs

Don't do this:

class ZineSerializer(serializers.ModelSerializer):
    class Meta:
        fields = ['id', 'zine_id', 'title']  # ❌ Exposes internal id

Do this:

class ZineSerializer(serializers.ModelSerializer):
    class Meta:
        fields = ['zine_id', 'title']  # ✅ Only external ID

2. Always Use External IDs in URLs

Don't do this:

lookup_field = 'id'  # ❌ Uses internal ID

Do this:

lookup_field = 'zine_id'  # ✅ Uses external ID

3. Use Internal IDs for Foreign Keys

Don't do this:

class Holding(models.Model):
    zine_id = models.CharField(max_length=255)  # ❌ String FK

Do this:

class Holding(models.Model):
    zine = models.ForeignKey(Zine, on_delete=models.CASCADE)  # ✅ Integer FK

4. Validate External IDs

Ensure external IDs follow conventions:

def validate_zine_id(self, value):
    if not value.startswith('zine_'):
        raise ValidationError("zine_id must start with 'zine_'")
    return value

5. Document Naming Conventions

Provide clear guidelines for external ID format in documentation.

Summary

Dual identifiers provide:

Performance: Integer PKs and foreign keys
Usability: Human-readable IDs in API
Stability: Consistent IDs across environments
Flexibility: External IDs can be changed without breaking relationships

Trade-off: Slightly more storage and one additional index per table.

Verdict: The benefits vastly outweigh the minimal costs.

Next Steps

Database Schema — Complete table structure and indexes
Serializers — How read/write serializers work
API Reference — Complete endpoint documentation

Understanding dual identifiers? Continue to Database Schema to see the complete table structure.

Edit this pageorReport an issue

Serializers

Read/write serializer pattern for transforming models to/from API

Database Schema

PostgreSQL table structure, indexes, and relationships