Offset vs Cursor Pagination: API Design, CI/CD Validation & Type-Safe Client Generation

Selecting the correct pagination strategy dictates API scalability, client iteration complexity, and database query performance. This guide provides a contract-first approach to designing, validating, and generating type-safe pagination clients, with explicit CI/CD enforcement and backend alignment workflows.

Architectural Trade-offs & Selection Matrix

Pagination strategy selection must align with dataset volatility, consistency SLAs, and expected query patterns. Offset pagination (LIMIT/OFFSET) relies on absolute row positioning, while cursor pagination (LIMIT/BEFORE/AFTER) uses opaque or structured tokens representing a specific record boundary.

Map architectural decisions to API contract requirements early. Document the chosen strategy in an Architecture Decision Record (ADR) and align client iteration patterns with the broader Query Patterns & Data Shaping Strategies framework to ensure predictable data shaping across service boundaries.

Spec-Driven Endpoint Design & Validation

Define strict query parameter contracts before backend implementation. Enforce deterministic sort stability and integrate Advanced Filtering Operators to prevent ambiguous result sets.

OpenAPI 3.1 Pagination Parameters

Use oneOf with a discriminator to enforce mutually exclusive pagination strategies at the schema level.

# openapi/pagination-params.yaml
components:
 parameters:
 PaginationStrategy:
 in: query
 required: true
 schema:
 oneOf:
 - $ref: '#/components/schemas/OffsetParams'
 - $ref: '#/components/schemas/CursorParams'
 discriminator:
 propertyName: strategy
 mapping:
 offset: '#/components/schemas/OffsetParams'
 cursor: '#/components/schemas/CursorParams'

 schemas:
 OffsetParams:
 type: object
 required: [strategy, offset, limit]
 properties:
 strategy: { type: string, enum: [offset] }
 offset: { type: integer, minimum: 0 }
 limit: { type: integer, minimum: 1, maximum: 100 }
 CursorParams:
 type: object
 required: [strategy, limit]
 properties:
 strategy: { type: string, enum: [cursor] }
 cursor: { type: string, format: uri, pattern: '^[A-Za-z0-9_-]+$' }
 limit: { type: integer, minimum: 1, maximum: 100 }

JSON Schema Pagination Response Contract

Strictly type response envelopes. Note that total_count is optional and discouraged for cursor strategies due to expensive COUNT(*) overhead.

{
 "type": "object",
 "required": ["items", "has_more"],
 "properties": {
 "items": { "type": "array", "items": { "type": "object" } },
 "next_cursor": { "type": "string", "nullable": true },
 "has_more": { "type": "boolean" },
 "total_count": { "type": "integer", "nullable": true }
 },
 "additionalProperties": false
}

Validate specs against mock datasets using prism or wiremock. Ensure Sorting & Multi-Field Ordering is explicitly declared in the contract to guarantee deterministic cursor generation and prevent duplicate/skipped records.

CI/CD Workflows & Contract Enforcement

Automate pagination spec validation in deployment pipelines. Gate merges on breaking schema changes and enforce consistency across staging/production.

GitHub Actions Pipeline

name: API Contract Validation
on:
pull_request:
paths: ['openapi/**', '.spectral.yaml']

jobs:
  validate-pagination:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install Spectral & OpenAPI Diff
        run: npm install -g @stoplight/spectral-cli openapi-diff
      - name: Lint Pagination Contracts
        run: spectral lint openapi/pagination-params.yaml --ruleset .spectral.yaml
      - name: Check Breaking Changes
        run: openapi-diff openapi/base.yaml openapi/pagination-params.yaml --fail-on-breaking
      - name: Run Mock Server Contract Tests
        run: |
          npx @stoplight/prism-cli mock openapi/pagination-params.yaml &
          sleep 2
          curl -f http://localhost:4010/v1/resources?strategy=cursor&limit=10

Spectral Linting Rule

Enforce cursor encoding standards and prevent unbounded limits.

# .spectral.yaml
rules:
 cursor-encoding-format:
 description: "Cursors must be URL-safe Base64 or opaque tokens."
 given: "$.components.schemas.CursorParams.properties.cursor"
 severity: error
 then:
 field: pattern
 function: pattern
 functionOptions:
 match: "^[A-Za-z0-9_-]+$"
 limit-bounds:
 description: "Limit must not exceed 100."
 given: "$.components.schemas.*.properties.limit"
 severity: error
 then:
 field: maximum
 function: schema
 functionOptions:
 schema: { maximum: 100 }

Type-Safe Client Generation & SDK Workflows

Generate strongly-typed pagination clients that abstract cursor encoding/decoding and expose consistent iteration patterns. Align SDK behavior with backend implementations like Implementing cursor-based pagination with PostgreSQL to ensure parity between contract and execution.

TypeScript: AsyncIterator SDK Wrapper

import { z } from 'zod';

const PageSchema = z.object({
 items: z.array(z.unknown()),
 next_cursor: z.string().nullable(),
 has_more: z.boolean()
});

export async function* paginate<T>(
 fetchPage: (cursor?: string) => Promise<unknown>,
 initialCursor?: string
): AsyncIterableIterator<T> {
 let cursor = initialCursor;
 while (true) {
 const raw = await fetchPage(cursor);
 const page = PageSchema.parse(raw);
 yield* page.items as T[];
 if (!page.has_more) break;
 cursor = page.next_cursor ?? undefined;
 }
}

Python: Pydantic v2 Models

from pydantic import BaseModel, computed_field
import base64, json

class CursorPage(BaseModel):
 items: list[dict]
 next_cursor: str | None = None
 has_more: bool

 @computed_field
 def has_next_page(self) -> bool:
 return self.has_more

 @classmethod
 def decode_cursor(cls, raw: str) -> dict:
 return json.loads(base64.urlsafe_b64decode(raw + "=="))

Go: io.Reader-Style Stream

type CursorIterator struct {
 client *http.Client
 baseURL string
 cursor []byte
 buffer []byte
 err error
}

func (it *CursorIterator) Read(p []byte) (n int, err error) {
 if len(it.buffer) > 0 {
 n = copy(p, it.buffer)
 it.buffer = it.buffer[n:]
 return
 }
 // Fetch next page, parse JSON, refill buffer, handle context timeouts
 // ...
}

Debugging, Observability & Performance Tuning

Instrument pagination queries at the API gateway and database layers. Track cursor drift, monitor offset degradation, and implement structured logging for client-side iteration failures.

Key Observability Signals

• Cursor Drift Rate: % of requests returning duplicate or skipped records. Correlate with missing tiebreaker keys in sort order.
• Offset Degradation: Query execution time vs. OFFSET value. Alert when EXPLAIN shows sequential scans or temporary table spills.
• Client Loop Failures: Log has_more mismatches and next_cursor parse errors with traceable request IDs.

Common Pitfalls

1. Missing deterministic sort causing cursor drift and duplicate/skipped records.
2. Offset deep-paging performance degradation due to O(N) index scans and row skipping.
3. Inconsistent cursor encoding (opaque vs structured) across microservice boundaries.
4. Client-side infinite loops from missing or incorrectly parsed has_next_page flag.
5. Breaking pagination schema changes without API versioning or deprecation headers.
6. Ignoring timezone/UTC normalization when using timestamp-based cursors.

Runtime Debugging Workflow

1. Attach APM tracing to /resources endpoints. Extract trace_id from failed iterations.
2. Run EXPLAIN (ANALYZE, BUFFERS) on the underlying query. Verify index usage matches the declared sort keys.
3. If drift occurs, append the primary key as a tiebreaker: ORDER BY created_at DESC, id DESC.
4. Validate cursor payloads in staging using synthetic load tests that simulate concurrent inserts/deletes.

Frequently Asked Questions