Changing a record’s fields without breaking the records you already have

SynaptaGrid8 min read

#records#versions

In short

Changing the shape of a record is two jobs: changing the definition, and moving the records that already exist to match it. The definition is easy. The records are the risk, because a change can make some of them invalid: a text field shortened below the length of existing values, a number field that now holds words, a required field that old records leave blank. A safe approach keeps the old and the new definition side by side, checks the impact before the change goes live, moves the records in batches, sets aside any record that does not fit instead of dropping it, and can be rerun without creating duplicates. Plan the move before you publish the change, not after.

Five kinds of change, from safe to risky

Not every change carries the same risk. Sorting a planned change into one of these kinds tells you how much care it needs.

ChangeExampleRisk to existing recordsWhat it needs
Add an optional fieldA new “Notes” fieldNone: old records simply leave it emptyNothing beyond publishing
Widen a fieldA whole number becomes a number with decimalsLow: every old value still fitsA mechanical conversion
Narrow a fieldText limited to a shorter lengthMedium: some old values may not fitA check, and a place for the ones that fail
Remove or rename a fieldDrop a column or give it a new nameHigh: values can be lost or orphanedA decision about where the values go
Compute a valueFill a new field from two old onesDepends on the ruleA defined rule applied to each record

A safe path for a field change

The old definition stays live until the new one is published, and a record that does not fit is set aside for review, not dropped.

Plan the move before you publish

List what changes
Write down each field that is added, widened, narrowed, renamed or removed. A change you did not list is a change you did not check.
Test with real values
Count how many existing records would fail the new rule. If the answer is none, the change is safe; if it is some, decide what happens to them.
Decide what happens to a failure
Fix it by hand, apply a rule, or leave it behind. Dropping it silently is the one option to rule out.
Keep a way back
The old version of the definition and its records should stay until you have checked the move, so you can go back.
Make the move repeatable
If the move stops halfway, running it again should finish the job without copying records twice.

The old way and the new way

The table compares changing a column in a spreadsheet or a hand-edited table with a managed change.

Making the change
The old way: Edit the live list and see what breaks
The new way: Draft the new version while the old one stays live
Knowing the impact
The old way: Find out when someone reports a wrong value
The new way: An impact check before the change goes live
Moving records
The old way: Copy values by hand, or a one-off script
The new way: A planned move in batches
A value that does not fit
The old way: Truncated or lost without notice
The new way: Set aside for review with the reason
Running it twice
The old way: Duplicates, or a half-finished state
The new way: Safe to rerun: records already moved are not copied again
Retiring the old shape
The old way: Delete the old copy and hope
The new way: Retire the old version when you have checked the result

How we decided

In SynaptaGrid Data a record type has versions. You work on a draft, publish it, and records are stored against a published version. A change is therefore a new version, not an edit in place. The product can produce a plan that lists, for each field, what the change does and what kind of move it needs, and it refuses a plan that has gone out of date since it was made.

The move runs in batches and can be run again: records that have already been moved are not copied a second time. A record that does not fit the new definition is placed in a separate review list with the reason and the original values, and is not dropped. You can then fix it by hand or decide it will not move. When the move is checked, the old version can be retired.

The cost we accepted is that every change needs a publish step and a plan, even a small one. We preferred that to a quick edit that changes records without anyone seeing the effect.

This is built in the product, but we have not run a managed move on live customer data while preparing this guide, so we describe how it is designed to work, not a measured result. Some kinds of change, such as renaming a group of fields or moving a field between levels, were refused by the plan in the design we reviewed, so check those with us first.

Standards and accepted practice behind this

These are the practices the guide rests on. We align with them. This page does not claim a certification.

Expand, then contract
Add the new shape next to the old one, move the data, and only then remove the old shape.
Idempotent migrations
A migration that can be run again with the same result as running it once.
Dry run before change
Check the effect on real data before the change is applied.
Never lose data silently
A value that cannot be converted is kept and reported, not discarded.

Where SynaptaGrid fits

SynaptaGrid Data lets you change the shape of a record type without a maintenance window: draft a new version, check the impact, move existing records in batches, and review any that do not fit.

Check a change that renames a group of fields or moves a field between levels before you rely on it, because the move may refuse those.

Frequently asked questions

What is the difference between a schema migration and a data migration?
A schema migration changes the definition of the data: which fields exist and what type they are. A data migration moves the records that already exist so they match the new definition. A field change needs both.
How do I add a field without losing data?
Add it as an optional field. Existing records leave it empty, and nothing else changes. Make it required only after the old records have a value.
What happens to a record that does not fit the new definition?
It should be set aside and reported, with the original values kept, so you can fix it or decide it will not move. It should never be dropped silently.
Can I change a field type?
Widening is usually safe, for example a whole number to a number with decimals. Narrowing can fail for some records, so check the existing values first and decide what happens to the ones that do not fit.
Share

SynaptaGrid

Plan a field change with us

Bring a change you need to make and see which records it would affect.