Technical

Python Module Naming

Convention for distinguishing public API modules from internal implementation using underscore prefix.

Production

Problem

Python projects need a clear convention to distinguish between public API modules (intended for external use) and internal implementation modules (private to the package).

Without a clear convention:

Users import internal modules directly, creating tight coupling
Refactoring internal code becomes risky (breaking external dependencies)
API boundaries are unclear
from package import * pollutes namespace with internals

Context

Use this pattern when:

Building a Python library/package for external consumption
Creating a modular codebase with clear API boundaries
Working on projects where internal implementation details should be hidden
Need to protect internal refactoring from breaking external code

Don't use when:

Building simple scripts or single-file programs
All code is internal (no external consumers)
Project uses alternative patterns (e.g., internal/ subdirectories)

Solution

Use underscore prefix _ to mark modules as internal/private:

Public API modules (no prefix):

package/
  models.py           # Public API - users should import from here
  client.py           # Public API - main client interface
  utils.py            # Public API - utility functions

Internal modules (with _ prefix):

package/
  _async_enrichment.py    # Internal async implementation
  _llm_helpers.py         # Internal LLM helper functions
  _internal_config.py     # Internal configuration

Structure

Basic Package Layout

src/audit_utils/
├── __init__.py              # Public API exports
├── models.py                # ✅ Public: Core data models
├── client.py                # ✅ Public: Client interface
├── _llm_helpers.py          # ❌ Private: LLM utilities
├── _async_enrichment.py     # ❌ Private: Async implementation details
└── prompts/
    ├── __init__.py
    ├── builder.py           # ✅ Public: Prompt builder API
    └── _internal_cache.py   # ❌ Private: Cache implementation

Controlling Exports with `all`

Combine with __all__ in __init__.py for explicit API:

# src/audit_utils/__init__.py
from .models import CustomerInteraction, AuditResult
from .client import AuditClient

# Explicitly define public API
__all__ = [
    "CustomerInteraction",
    "AuditResult",
    "AuditClient",
]

# Note: _llm_helpers and _async_enrichment are NOT exported

Implementation

Step 1: Identify Internal Modules

Determine which modules are:

Public API: Intended for external use, stable interface
Internal: Implementation details, subject to change

Step 2: Rename Internal Modules

# Rename internal modules with underscore prefix
git mv src/package/helpers.py src/package/_helpers.py
git mv src/package/async_core.py src/package/_async_core.py

Step 3: Update Internal Imports

# Before (importing internal module)
from audit_utils.helpers import process_data

# After (import from internal module)
from audit_utils._helpers import process_data

Step 4: Define `all` in `init.py`

# src/audit_utils/__init__.py
from .models import CustomerInteraction
from .client import AuditClient

__all__ = ["CustomerInteraction", "AuditClient"]

Step 5: Document API Boundaries

Add docstring to internal modules:

# src/audit_utils/_llm_helpers.py
"""
Internal LLM helper functions.

This module is for internal use only and is not part of the public API.
Its interface may change without notice between versions.
"""

Applies Principles

Clear Boundaries: Explicit distinction between public API and internal implementation
Encapsulation: Internal details hidden from external consumers
Maintainability: Safe to refactor internals without breaking external code
Least Surprise: Users understand what they should/shouldn't depend on

✅ Clear API boundaries: Users know what to import ✅ Safe refactoring: Change internals without breaking external code ✅ Namespace protection: from package import * excludes internal modules ✅ Documentation signal: _ prefix is universally understood in Python ✅ IDE support: Many IDEs hide _ prefixed items from autocomplete

Drawbacks

❌ Semi-standard: Not mandated by PEP 8 (unlike _variable naming) ❌ Not enforced: Users can still import _internal modules if they want ❌ Refactoring cost: Need to rename files and update imports ❌ Git history: File renames can complicate git blame/history

Trade-offs

Alternative: internal/ subdirectory

package/
  public/
    models.py
    client.py
  internal/
    helpers.py
    async_core.py

✅ More explicit structure
❌ Deeper nesting
❌ More import path changes

Examples

Real-World: audit-utils

# src/audit_utils/models/
├── customer_interaction.py      # ✅ Public model
├── audit_result.py              # ✅ Public model
├── _async_enrichment.py         # ❌ Internal async logic
└── _llm_helpers.py              # ❌ Internal LLM utilities

# Users import public API:
from audit_utils.models import CustomerInteraction  # ✅ Supported

# Internal imports (discouraged for external users):
from audit_utils.models._llm_helpers import ...  # ❌ Internal, may break

Major Python Projects Using This Pattern

Django:

django/
  conf/
    _settings.py         # Internal settings
  utils/
    _os.py              # Internal OS utilities

Flask:

flask/
  _compat.py            # Internal compatibility layer
  _internal.py          # Internal utilities

Python Standard Library:

_collections_abc.py     # Internal abstract base classes
_weakrefset.py         # Internal weak reference utilities

Variations

Variation 1: Mixed Approach

Use both _ prefix AND internal/ subdirectory:

package/
  models.py              # Public
  internal/
    _cache.py           # Very internal
    _utils.py           # Very internal

Variation 2: Explicit `public/` Directory

package/
  public/               # All public API here
    __init__.py
    models.py
    client.py
  _internal_module.py   # Internal at package level

Repository Structure - Overall project layout
Python Packaging Best Practices (future pattern)
API Versioning (future pattern)

References

Community Discussions

Examples in Popular Projects

Django: github.com/django/django - Uses _internal.py pattern
Flask: github.com/pallets/flask - Uses _compat.py pattern
Requests: github.com/psf/requests - Uses _internal_utils.py

PEP References

PEP 8 - Style Guide (covers _variable but not modules)
PEP 484 - Type Hints (mentions private conventions)

Status: ✅ Active Last Updated: 2026-01-04 Applies to: Python projects (libraries, packages)

Lambda Deploy

Pattern for deploying Lambda code from CI/CD using direct zip upload with Terraform infrastructure separation.

Python Project Structure

Standardized Python project structure with uv, Taskfile, direnv, and consistent tooling.