Rich Search on AWS Amplify

Canonical pattern for supporting sophisticated search and filtering in AWS Amplify Gen2 applications without trying to turn DynamoDB into a general-purpose query engine.

The pattern: keep DynamoDB as the source of truth, derive an OpenSearch read model kept in sync via DynamoDB Streams, and expose a single typed AppSync query as the entry point for the frontend.

Status: Emerging. First end-to-end implementation in progress in ontopix/elsa (Epic 6 — Search Infrastructure). The pattern will graduate to Production once the implementation lands and a second project adopts it.

Problem

Amplify Gen2 generates an AppSync API backed by DynamoDB by default. client.models.X.list({ filter: ... }) works fine for prototypes and small datasets, but it has hard limits that surface fast as soon as the product needs richer search:

• DynamoDB filter expressions are post-read. A filter does not reduce read cost — DynamoDB reads the page first, then discards items that do not match. Multi-dimensional filters get expensive quickly.
• A query uses one GSI at a time. Combining "samples for participant SAM in channel TEL ordered by date" requires a GSI that exactly fits that access pattern. Each new combination is another GSI to provision.
• No nested-attribute indexing. Querying by a value inside a nested object (e.g. an item in an array of audit results) cannot be done efficiently in DynamoDB at all.
• No full-text search, fuzzy matching, faceting, or aggregations. These are out of scope for DynamoDB by design.
• Multi-select with sort by an unrelated field (e.g. "filter by 5 participants, sort by date") forces either fan-out queries with merge-in-client (which breaks cursor pagination) or scanning the whole table (which is even worse).

Beyond a small dataset and a small number of filters, building search by piling more GSIs and filter expressions onto DynamoDB stops scaling — both in latency/cost and in code complexity. The fix is not to keep stretching DynamoDB; it is to add a search index where one belongs.

Context

Use this pattern when:

• The application is built on Amplify Gen2 (or any "AppSync + DynamoDB" architecture).
• A user-facing list or search screen needs multi-dimensional filtering, multi-select, range filters, sort by arbitrary fields, filtering on nested data, full-text search, faceting, or aggregations — anything beyond a single-GSI "list by tenant ordered by date".
• Data fits the document model (one logical entity per document, denormalisation is acceptable).
• Eventual consistency on the list view is acceptable (typical lag <1 s; worst case seconds when the indexer is slow).

Don't use this pattern when:

• The required reads are well-served by a few secondary indexes — adding OpenSearch is over-engineering.
• The data is genuinely relational with strict joins, transactional reporting, or SQL-style aggregations across many entities — consider PostgreSQL/Aurora connected to Amplify Data instead.
• Strong consistency is required for the list view — the read model is eventually consistent.

Decision matrix. Pick the right tool per access pattern:

Solution

Six principles that drive every other decision in this pattern.

1. DynamoDB stays the source of truth. Every write in the application code goes to DynamoDB. No application code path writes to OpenSearch directly. Dual-write from the application is an antipattern (see §Antipatterns).
2. Sync is one-way and asynchronous. A dedicated component subscribes to DynamoDB Streams and updates OpenSearch. Application code is unaware of the index.
3. The read model is reconstructible. A backfill/reindex script can rebuild the index from DynamoDB at any time. Drift is recoverable.
4. Re-compose, don't apply deltas. On any stream event, the indexer reads the current state of the relevant items from DynamoDB and writes a fully-formed document. This makes the indexer idempotent and resilient to event ordering.
5. Reads split by access pattern. Get-by-id reads go to DynamoDB (strongly consistent). List/search reads go through a single typed AppSync query backed by OpenSearch (eventually consistent).
6. The index is denormalised. Foreign-key data needed for filtering or display is denormalised into the document. There is no JOIN in OpenSearch — there is denormalisation, recomposed on every event.

Structure

                ┌─────────────────────────────────┐
                │  DynamoDB (source of truth)     │
   writes ────► │  Entity                         │
                │  RelatedEntity                  │
                └─────┬──────────────────────┬────┘
                      │ Stream               │ Stream
                      ▼                      ▼
        ┌──────────────────────────┐  ┌──────────────────────────┐
        │ Indexer (Entity)         │  │ Indexer (RelatedEntity)  │
        │ - reads truth            │  │ - reads truth            │
        │ - composes document      │  │ - composes document      │
        │ - upserts to OS          │  │ - upserts to OS          │
        └────────────┬─────────────┘  └────────────┬─────────────┘
                     └────────────┬────────────────┘
                                  ▼
                ┌─────────────────────────────────┐
                │  OpenSearch (read model)        │
                │  search-index                   │
                └─────────────────┬───────────────┘
                                  ▲
                                  │ DSL query
                ┌─────────────────────────────────┐
                │  AppSync custom query           │
                │  Lambda resolver: searchEntity  │
                └─────────────────┬───────────────┘
                                  │
                                  ▼
                              Frontend

   Detail view reads ─→ AppSync ─→ DynamoDB GetItem(id)

A single OpenSearch index per "search experience" (one document type). Foreign-key data needed in queries is denormalised into the document — there is no JOIN in OpenSearch, only re-composition on every event.

Implementation

The pattern has eight decisions. Each has a recommended option with explicit criteria for picking a different one.

1. Sync mechanism: custom Lambdas vs. OpenSearch Ingestion (zero-ETL)

Pick Custom Lambdas when:

• Cost is a concern. The fixed monthly cost of Ingestion (~175 €/month minimum) is several times the cost of a small OpenSearch domain (~25 €/month for t3.small.search). For low-to-moderate volumes the maths does not work.
• The indexer needs cross-table lookups during transformation (e.g. "when an event arrives on table B, also fetch related row from table A"). Ingestion can do this with scripting plugins, but the moment you need scripting you lose the "no-code" benefit.
• Full control over versioning (if_seq_no/if_primary_term per upsert) is required to handle out-of-order events.
• Volume is low and predictable enough that idle Ingestion capacity is wasted spend.

Pick OpenSearch Ingestion when:

• High write throughput justifies the fixed minimum.
• The transformation is straightforward (no cross-table lookups, no custom versioning logic).
• The team prefers "no code to maintain" over flexibility.
• Initial export from DynamoDB is a meaningful win (one less script to write, run, and own).

Migrating between them is a contained change (the Lambda or YAML pipeline is internal infrastructure — neither application code nor frontend cares). Start with Lambdas, switch to Ingestion only if scale or operational pressure justifies it.

2. AppSync resolver shape: typed input vs. generic filter expression

Pick typed input when — almost always. Amplify Gen2 + AppSync are designed around typed schemas; the typed client autocompletes and refactors safely; runtime errors become compile-time errors. Adding a filter dimension means adding a field to the input and a clause to the DSL builder — a one-line schema change, not a paradigm shift.

Pick generic filter when — only if the search dimensions are genuinely user-defined (e.g. a query builder where users create new filter fields at runtime). Even then, build a small server-side validator before the DSL builder, and never let the frontend send raw OpenSearch DSL.

3. Auth: resolver Lambda → OpenSearch

Pick IAM signing when — almost always. It is the AWS-native pattern, mirrors how AppSync, Lambda, and DynamoDB authenticate today, and removes secret rotation. Initial IAM setup is the only friction.

Pick master user when — only for environments where IAM is not available (e.g. self-hosted OpenSearch that does not understand AWS IAM, or migration from a non-AWS stack). Treat it as a temporary state.

4. Networking: public endpoint + IAM vs. VPC endpoint

Pick public + IAM when — most projects. Defence in depth comes from IAM + fine-grained access control + workspace filter in resolver, not from network isolation. Adding a VPC just for this introduces NAT gateways, ENI cold starts, and subnet maintenance for no security gain in the absence of compliance requirements.

Pick VPC when — explicit compliance requirement (e.g. customer contract requires no public AWS endpoints), or the application stack already runs entirely in a VPC and adding one more service to it is trivial.

5. Pagination: search_after vs. from + size

Pick search_after when — the UI exposes "Next/Previous" or infinite scroll. The frontend treats nextToken as an opaque string and passes it back; the encoding can change without breaking the contract.

Pick from + size when — the UI specifically requires "jump to page N" (numbered pagination with a visible page count). Migrating from search_after to from+size later is a localised Lambda change as long as the frontend contract (nextToken: String) does not depend on the encoding.

6. Index reconstruction: standalone script vs. Lambda invocation

Pick standalone script when — reindexing is a maintenance operation run by the technical team a handful of times a year (initial bootstrap, mapping migrations, recovery). Adds zero new infrastructure and uses the same code path as the live indexer (extracted helpers).

Pick Lambda invocation when — a team without local AWS credentials must trigger reindex (e.g. a 24/7 operations team). The 15-minute Lambda timeout requires checkpoint/resume logic for large datasets.

Pick admin UI when — non-technical operators must reindex without involving engineers. Significant additional UI work for a rarely-used feature; usually overkill.

7. Multi-tenancy: enforce tenant filter server-side

This is mandatory, not optional. The resolver Lambda always adds a term { tenantId: <caller's tenant from auth context> } filter to every OpenSearch query, regardless of input. The frontend cannot bypass it. The frontend cannot send a tenantId field in the input — derive it server-side from the authenticated identity.

This mirrors the workspace-scoped pattern in DynamoDB GSIs: the multi-tenancy boundary lives in the resolver, not in the client.

8. Mapping evolution: physical index vs. alias-with-versioning

Pick physical index when — the environment is destructible (sandbox, dev) or downtime during migration is acceptable.

Pick alias when — production. Adds five minutes of Terraform from day one, saves significant pain on every future incompatible mapping change. Worth doing prophylactically once a project enters production.

Antipatterns

These are the failure modes the pattern explicitly prevents. Mark them as "do not do" in code reviews.

Dual-write from the application

Writing to DynamoDB and OpenSearch from application code (the mutation, the Lambda handler, the controller). Half the literature on this pattern exists because of dual-write. The failure modes:

• One write succeeds, the other fails: DynamoDB and OpenSearch diverge silently.
• Concurrent writers: order of operations cannot be guaranteed across two stores; eventual outcome is non-deterministic.
• Adding a third caller (a new mutation, a new Lambda) requires every developer to remember to write to both stores.

Always funnel writes through DynamoDB only. The Streams-based indexer is the single component that reaches OpenSearch.

Frontend submits raw OpenSearch DSL

Letting the frontend build query DSL and POST it to a passthrough endpoint. This makes the search API impossible to harden — any user can run arbitrary aggregations, exfiltrate data via term filters, or kill the cluster with deeply nested queries.

The resolver Lambda translates a small, validated input into DSL. The frontend never sees DSL.

Trusting tenantId from the client

Accepting a tenantId field in the search input. Even with auth in place, this lets a malicious caller search another tenant's data by passing the wrong ID. The tenant always comes from the authenticated identity at the resolver layer.

Indexing the entire entity

Mirroring every field of a DynamoDB row to OpenSearch, including large blobs (transcripts, raw payloads, embeddings stored as JSON). The index inflates, queries get slower, and reindexing takes hours. Index a projection: only the fields used for filtering, sorting, or display in the list. The detail view fetches the full entity from DynamoDB.

Applying deltas instead of recomposing

Writing partial updates to OpenSearch based on the delta in the Stream event. Looks elegant; breaks subtly when events arrive out of order, or when a new field is added to the document, or when a related entity's update needs to bring along the parent's fields. The recompose-from-truth strategy reads current state from DynamoDB and writes a complete document; idempotent and order-independent.

Skipping the reindex script

"We never need to rebuild from DynamoDB" is the famous-last-words version. The script is the only recovery path when (a) Streams retention runs out, (b) the cluster is destroyed, (c) the mapping changes incompatibly. Build it on day one.

Letting the cluster sit unobserved

DLQs grow silently. Indexer lag grows silently. Documents fail to write silently. Without alarms on IndexerLag, IndexerErrors, and DLQ depth, the first signal of a broken pipeline is a user reporting "missing data" — by which point recovery may require a full reindex. Wire alarms on day one.

Applies Principles

• Right tool for the job. DynamoDB for transactional access by primary key; OpenSearch for sophisticated search. Resist the urge to make one tool do everything.
• Single source of truth. DynamoDB is the only writable store. The read model is derived, not authoritative.
• Idempotency. The indexer recomposes from truth; running it twice produces the same outcome. Same property guarantees safe replay and safe reindex.
• Defence in depth. Tenant isolation lives in the resolver Lambda (not just IAM), the input is typed (not raw DSL), and the network posture is least-privilege via IAM signing.
• Reversible decisions. Every implementation choice in §Implementation is intentionally easy to migrate later (Lambda↔Ingestion, public↔VPC, search_after↔from+size, physical↔alias). The architecture is fixed; the implementation is not.

Consequences

Benefits:

• Sophisticated search becomes feasible without warping the DynamoDB schema or piling on GSIs.
• Adding a new filter dimension is a localised change: add a field to the input, a clause to the DSL builder, a field to the mapping (if needed).
• Adding a new Lambda or mutation that writes to DynamoDB does not require coordination with the search team — the Stream picks it up automatically.
• The detail view stays fast and strongly consistent (DynamoDB by primary key).

Trade-offs:

• Eventual consistency on the list view. Edits propagate in seconds. Worth documenting explicitly to operators; not a bug.
• Two systems to operate. OpenSearch domain to provision, monitor, and pay for. Indexer Lambdas to monitor.
• A new failure mode: drift. Indexer downtime past Streams retention (24 h) requires running the reindex script. Wire monitoring before the first user notices.
• Mapping migration is real work. Some mapping changes require reindexing; build the script and the alias indirection accordingly.

Examples

Real-world usage:

• ontopix/elsa — Epic 6 (Search Infrastructure). DynamoDB tables CustomerInteraction + AuditResult; OpenSearch index customer-interactions; two indexer Lambdas (search-indexer-interaction, search-indexer-audit-result); AppSync resolver searchInteractions(input: SearchInteractionsInput!): SearchInteractionsResult!. Implementation in progress as of 2026-05-16. See .context/epic/20260516-search-infrastructure/spec.md in that repo.

This is the first end-to-end implementation of this pattern in Ontopix. Once the Elsa implementation lands and a second project adopts it, this pattern graduates from Emerging to Production.

Variations

Multiple source tables feeding one index

Common case (Elsa's CustomerInteraction + AuditResult is exactly this). One indexer Lambda per source table; both write to the same index, recomposing the document by reading whatever they need from any of the source tables. Versioning (if_seq_no/if_primary_term) handles concurrent writes on the same document.

One source table feeding multiple indices

Less common, but applicable when very different read shapes need very different mappings (e.g. a "search" index optimised for filtering vs. an "analytics" index optimised for aggregations on the same data). Each consumer-Lambda subscribes to the same Stream and writes its own shape. Costs scale with the number of indices.

Hybrid with PostgreSQL for true relational queries

When some screens need search (this pattern) and others need joins/SQL reporting that OpenSearch is bad at (e.g. financial reporting), it is acceptable to add PostgreSQL as a third store, also fed from DynamoDB Streams. Keep boundaries clear: each read path picks one store; never query two for the same screen.

Full-text search

OpenSearch is also a full-text engine. If transcripts, free-form notes, or document bodies need to be searchable, index them as text fields with appropriate analysers. Out of scope for this base pattern but a natural extension.

Related Patterns

• Service Callback — IAM SigV4 — Same IAM signing primitives used by the resolver Lambda to talk to OpenSearch.
• Infrastructure Layout — .infra/ Terraform conventions where the OpenSearch domain and Stream subscriptions live.
• Lambda Deploy — Deployment pipeline that ships the indexer and resolver Lambdas.

References

• AWS — OpenSearch Service IAM authentication
• AWS — DynamoDB Streams
• AWS — OpenSearch Ingestion (zero-ETL with DynamoDB)
• OpenSearch — search_after pagination
• OpenSearch — Versioning with if_seq_no / if_primary_term
• Amplify Gen2 — Custom queries with custom data sources
• Ontopix implementations:
  • ontopix/elsa — Epic 6 spec at .context/epic/20260516-search-infrastructure/spec.md (2026-05-16).