Output Transforms

.mapOut() applies structural transformations to the output after extraction. It supports grouping, aggregation, nesting, sorting, filtering, renaming, and more. Multiple .mapOut() calls chain sequentially.

source → mapIn → emit → mapOut → mapOut → ... → compile

Two tiers

Tier	Description	Schema Derivation
Tier 1 — Declarative	Pure data AST. Supports lineage tracing, schema derivation, source map remapping.	Full
Tier 2 — Opaque	Closure-based (apply, derive). Records input/output in lineage but is opaque to schema derivation.	Limited

Callback selectors

All path-based methods use callback selectors for type-safe path construction with full IDE autocomplete:

// The callback receives a proxy that mirrors your output type
.mapOut($ => $.sort(o => o.items, 'amount'))
//                   ^^^^^^^^^^
//                   Type-safe path selector — IDE shows available properties

The proxy records property accesses to build a structured path. No more string-based dot-paths.

groupBy

Restructure an array into keyed groups.

// Record format (default): array → { key: items[] }
.mapOut($ => $.groupBy(o => o.transactions, 'account'))

// Entries format: array → [{key, items}]
.mapOut($ => $.groupBy(o => o.transactions, 'account', { format: 'entries' }))

// Remove grouping key from items
.mapOut($ => $.groupBy(o => o.items, 'cat', { omitKey: true }))

// Named definition in derived schema
.mapOut($ => $.groupBy(o => o.transactions, 'account', { $defName: 'AccountGroup' }))

indexBy

Convert an array to a record by unique key.

.mapOut($ => $.indexBy(o => o.users, 'id'))
// { users: [{id:'a', name:'Alice'}] } → { users: { a: {id:'a', name:'Alice'} } }

.mapOut($ => $.indexBy(o => o.users, 'id', { omitKey: true, $defName: 'User' }))
// { users: { a: {name:'Alice'} } }
// Schema: users.additionalProperties → { $ref: '#/$defs/User' }

aggregate

Compute summary values from an array.

import { sum, count, avg, min, max, first, last, collect } from '@origints/core'

// Sibling placement (default): results placed as siblings of the array
.mapOut($ => $.aggregate(o => o.items, {
  operations: [
    sum('amount', 'totalAmount'),
    count('itemCount'),
    avg('amount', 'avgAmount'),
  ],
}))

// Inline placement: wrap array with aggregates
.mapOut($ => $.aggregate(o => o.items, {
  into: 'inline',
  operations: [sum('amount', 'total')],
}))

nest / unnest

Move properties into or out of sub-objects.

// Move flat properties into a sub-object
.mapOut($ => $.nest('address', ['street', 'city', 'zip'], { $defName: 'Address' }))

// Flatten nested properties up
.mapOut($ => $.unnest('address'))

// Partial unnest — move specific fields, keep the rest nested
.mapOut($ => $.unnest('address', ['street']))

sort

Sort arrays by one or more fields.

.mapOut($ => $.sort(o => o.items, 'amount'))           // ascending
.mapOut($ => $.sort(o => o.items, 'amount', 'desc'))   // descending

// Multi-field sort
.mapOut($ => $.sort(o => o.items, [
  { field: 'category', direction: 'asc' },
  { field: 'amount', direction: 'desc' },
]))

filter

Filter arrays with declarative predicates.

import { field } from '@origints/core'

.mapOut($ => $.filter(o => o.items, field('amount').gt(0)))

.mapOut($ => $.filter(o => o.items, {
  kind: 'and',
  left: field('status').equals('active'),
  right: field('amount').gte(100),
}))

rename / pick / omit

Key-level operations on the output.

.mapOut($ => $.rename({ firstName: 'first_name', lastName: 'last_name' }))
.mapOut($ => $.pick(['name', 'email']))
.mapOut($ => $.omit(['_internal', 'debug']))

drop

Remove keys at arbitrary dot-paths. Unlike omit which only works on top-level keys, drop can remove nested properties.

// Remove top-level keys
.mapOut($ => $.drop(['debug', '_internal']))

// Remove nested keys via dot-path
.mapOut($ => $.drop(['address.zip', 'metadata.internal']))

lookup

Enrich one array with matched records from another by key.

// One-to-one: attach customer to each order
.mapOut($ => $.lookup(o => o.orders, {
  from: 'customers',
  localKey: 'customerId',
  foreignKey: 'cid',
  as: 'customer',
  cardinality: 'one',
  $defName: 'Customer',  // promote source item schema to $defs
}))

// One-to-many: attach orders to each customer
.mapOut($ => $.lookup(o => o.customers, {
  from: 'orders',
  localKey: 'cid',
  foreignKey: 'customerId',
  as: 'orders',
  cardinality: 'many',
  consumeSource: true,  // remove source array after lookup
}))

// Merge: spread matched fields directly into target items
.mapOut($ => $.lookup(o => o.orders, {
  from: 'customers',
  localKey: 'customerId',
  foreignKey: 'cid',
  cardinality: 'merge',
  select: ['name'],
  consumeSource: true,
}))

// Select only specific fields from source
.mapOut($ => $.lookup(o => o.orders, {
  from: 'customers',
  localKey: 'customerId',
  foreignKey: 'cid',
  as: 'customerName',
  cardinality: 'one',
  select: ['name'],
}))

Unmatched keys produce null (cardinality one) or [] (cardinality many). For merge, unmatched items are left unchanged.

joinBy

N-way join of multiple arrays into a unified array by shared key.

// Full join: union of all keys
.mapOut($ => $.joinBy({
  sources: {
    sales: { path: 'sales', key: 'region', cardinality: 'one' },
    costs: { path: 'costs', key: 'region', cardinality: 'one' },
  },
  outputKey: 'region',
  joinType: 'full',
  outputPath: 'combined',
  consumeSources: true,
  $defName: 'RegionSummary',  // names the joined item object schema
}))

// Inner join: intersection of keys
.mapOut($ => $.joinBy({ ..., joinType: 'inner', ... }))

// Left join: keys from primary source only
.mapOut($ => $.joinBy({ ..., joinType: 'left', primarySource: 'sales', ... }))

// Three-way join with mixed cardinalities
.mapOut($ => $.joinBy({
  sources: {
    dept: { path: 'departments', key: 'deptId', cardinality: 'one' },
    employees: { path: 'employees', key: 'empDept', cardinality: 'many' },
    budget: { path: 'budgets', key: 'budgetDept', cardinality: 'one' },
  },
  outputKey: 'deptId',
  joinType: 'full',
  outputPath: 'overview',
  consumeSources: true,
}))

derive

Add computed fields to the output.

.mapOut($ => $.derive(
  o => o.fullName,
  out => `${out.firstName} ${out.lastName}`,
  'Concatenate names',
  { type: 'string' }  // optional schema hint
))

pivot

Turn rows into columns.

.mapOut($ => $.pivot(o => o.metrics, {
  rowKey: 'quarter',
  columnKey: 'metric',
  value: 'amount',
}))
// [{quarter:'Q1', metric:'revenue', amount:100}, {quarter:'Q1', metric:'cost', amount:50}]
// → [{quarter:'Q1', revenue: 100, cost: 50}]

apply

Opaque function fallback (Tier 2).

.mapOut($ => $.apply(out => customTransform(out), 'Custom business logic'))

Scoped transforms with .at() and .each()

.at() returns a scoped builder that applies transforms within a sub-path. Chain .each() to apply transforms to each element in an array or each value in a record.

.mapOut($ => $
  .groupBy(o => o.transactions, 'account')
  .at(o => o.transactions).each().sort('amount', 'desc')
  .at(o => o.transactions).each().aggregate({
    operations: [sum('amount', 'total')],
    into: 'inline',
  })
)

Full pipeline example

const plan = new Planner()
  .in(
    load({
      transactions: [
        { account: 'A', type: 'credit', amount: 100 },
        { account: 'A', type: 'debit', amount: 50 },
        { account: 'B', type: 'credit', amount: 200 },
      ],
    })
  )
  .emit((out, $) =>
    out.add(
      'transactions',
      $.get('transactions').array(tx => ({
        account: tx.get('account').string(),
        type: tx.get('type').string(),
        amount: tx.get('amount').number(),
      }))
    )
  )
  .mapOut($ =>
    $.groupBy(o => o.transactions, 'account')
      .at(o => o.transactions)
      .each()
      .aggregate({
        operations: [sum('amount', 'total')],
        into: 'inline',
      })
  )
  .compile()

Transform reference

Transform	Description	Schema Derivation
groupBy	Restructure array into keyed record or entries	Full
indexBy	Convert array to record by unique key	Full
aggregate	Compute sum, count, avg, min, max, first, last, collect	Full
nest	Move flat properties into a sub-object	Full
unnest	Flatten nested properties up to parent	Full
sort	Sort array by field(s)	Unchanged
filter	Filter array by declarative predicate	Unchanged
rename	Rename output keys	Full
pick	Keep only specified fields	Full
omit	Remove specified fields	Full
drop	Remove keys at arbitrary paths (deep removal)	Full
lookup	Enrich one array with matched records from another	Full
joinBy	N-way join of multiple arrays by shared key	Full
derive	Computed field from output (opaque function)	Type hint
pivot	Turn rows into columns	Partial
apply	Opaque function (Tier 2 fallback)	Opaque