Handling European Decimal Formats in Pydantic Validators

A cost of 1.234,56 on a German vendor quote means one thousand two hundred thirty-four and fifty-six hundredths. Hand that string to Python unchanged and Decimal("1.234,56") raises, or worse, a naive float parse reads it as 1.234 and a change order posts against a budget short by twelve hundred euros. This page solves one precise problem: how to normalize European and mixed-locale decimal formats to a canonical Decimal inside a Pydantic v2 field_validator that runs before coercion, so a cost is never silently truncated, while preserving the exact raw string for audit. Locale variance is not an edge case in construction procurement — international subcontractors, imported material quotes, and multi-region ERP exports all arrive with their own grouping and decimal conventions. This normalization belongs at the ingestion boundary, as one of the schema validation rules that every extracted money field clears before it reaches the ledger, immediately downstream of field extraction. It targets Python automation builders who need locale-robust cost parsing without ever guessing at a number’s magnitude.

Key Rules and Specification

Locale normalization is only safe when it is deterministic — a rule that guesses is a rule that eventually posts the wrong cost. The rules below make the conversion unambiguous and auditable. They rely on Python’s decimal module for exact arithmetic; every money value ends its journey as a Decimal, never a float.

• Normalize before coercion, not after. A field_validator(mode="before") runs on the raw input ahead of Pydantic’s type coercion. This is the only place you can rewrite 1.234,56 into 1234.56 before Pydantic tries — and fails, or worse, mis-succeeds — to interpret it.
• The last separator is the decimal separator. Across 1.234,56, 1,234.56, and 1234,56, the rightmost , or . marks the decimal point; every separator to its left is a grouping separator to be removed. This single rule resolves the ambiguity without needing to know the source locale in advance.
• Preserve the raw string. The model keeps the original input in a raw_amount field alongside the canonical Decimal. An auditor must be able to see exactly what the vendor wrote, because the normalization itself is a transformation that a dispute may need to trace.
• Strip only known noise. Remove currency symbols, ordinary spaces, and the non-breaking space (U+00A0) that European formats use as a thousands separator — but never digits, and never the last separator. Anything left that is not a digit or the resolved decimal point is a validation failure, not something to silently drop.
• Quantize to two places, reject impossible values. The canonical Decimal is quantized to two decimal places and constrained ge=0; a negative or non-numeric residue quarantines rather than coercing to something plausible-looking.
• A malformed amount routes, it does not truncate. When the string cannot be resolved to a valid Decimal, it is quarantined for review — the same disposition a low-confidence match gets — never rounded, guessed, or dropped, because a truncated cost that validates is far more dangerous than one that visibly fails.

Production Code Example

The normalizer is a pure function that resolves the decimal separator by position, strips grouping, and returns a canonical string that Decimal accepts. The Pydantic model runs it in a mode="before" validator and keeps the raw input so the audit trail is never lost.

from __future__ import annotations

import logging
import re
from decimal import Decimal, InvalidOperation

from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)-8s | %(message)s")
logger = logging.getLogger("money.normalize")

# Currency symbols and whitespace (incl. the U+00A0 non-breaking space used as a
# European thousands separator) are noise to strip; digits and separators are not.
_NOISE_RE = re.compile(r"[^\d.,-]")


def normalize_amount(raw: str) -> str:
    """Resolve a locale-formatted amount to a canonical 'digits.dd' string.

    The rightmost ',' or '.' is treated as the decimal separator; everything to its
    left is grouping and removed. Pure and deterministic, so a retry re-derives the
    identical canonical value.
    """
    cleaned = _NOISE_RE.sub("", raw.replace(" ", "").strip())
    if not cleaned:
        raise ValueError(f"no numeric content in {raw!r}")

    neg = cleaned.startswith("-")
    cleaned = cleaned.lstrip("-")

    last_comma = cleaned.rfind(",")
    last_dot = cleaned.rfind(".")
    dec_pos = max(last_comma, last_dot)

    if dec_pos == -1:                       # no separator at all -> integer amount
        digits, frac = cleaned, ""
    else:
        int_part = cleaned[:dec_pos]
        frac = cleaned[dec_pos + 1:]
        digits = re.sub(r"[.,]", "", int_part)   # drop all grouping separators
        if not frac.isdigit() or not digits.isdigit():
            raise ValueError(f"ambiguous or malformed amount {raw!r}")

    canonical = f"{'-' if neg else ''}{digits}" + (f".{frac}" if frac else "")
    return canonical


class MoneyField(BaseModel):
    """A cost that keeps both its canonical Decimal and the raw string it came from."""
    model_config = ConfigDict(frozen=True)

    raw_amount: str = Field(min_length=1, max_length=64)  # exactly what the vendor wrote
    amount: Decimal = Field(ge=0, decimal_places=2)

    @field_validator("amount", mode="before")
    @classmethod
    def _normalize(cls, v: object) -> object:
        # Only strings need locale normalization; a Decimal already through the door
        # (e.g. from a round-trip) passes untouched.
        if isinstance(v, str):
            try:
                return Decimal(normalize_amount(v)).quantize(Decimal("0.01"))
            except (ValueError, InvalidOperation) as exc:
                raise ValueError(f"cannot normalize amount {v!r}: {exc}") from exc
        return v

    @model_validator(mode="after")
    def _raw_is_consistent(self) -> "MoneyField":
        # The raw string must itself normalize to the stored amount; guards against a
        # caller passing a mismatched raw_amount/amount pair.
        if Decimal(normalize_amount(self.raw_amount)).quantize(Decimal("0.01")) != self.amount:
            raise ValueError("raw_amount does not normalize to amount")
        return self

In the ingestion path, wrap construction of the field so a malformed amount quarantines instead of aborting the batch, and log the raw string with the failure so the audit trail captures what could not be parsed.

def parse_cost(raw: str) -> MoneyField | None:
    """Build a MoneyField or quarantine the row, preserving the raw string either way."""
    try:
        return MoneyField(raw_amount=raw, amount=raw)  # 'amount' normalizes from the same raw
    except ValueError as exc:
        logger.warning("QUARANTINE | raw=%r | %s", raw, exc)
        return None


if __name__ == "__main__":
    samples = ["1.234,56", "1,234.56", "1234,56", "8 500,00", "€1.500", "1.234.56"]
    for s in samples:
        field = parse_cost(s)
        if field is not None:
            print(f"{s:<12} raw={field.raw_amount!r:<12} -> {field.amount}")
        else:
            print(f"{s:<12} -> quarantined")

Common Mistakes and Gotchas

Coercing in a mode="after" validator or not at all. By the time an after validator runs, Pydantic has already tried to coerce the string — and Decimal("1.234,56") has already raised, or a permissive float path has already read 1.234. Locale normalization has to happen in mode="before", on the raw input, or the damage is done before your code sees it. This is the single most common failure and the reason the truncation is silent.

Assuming a fixed locale. Hard-coding “comma is the thousands separator” works until a French or German quote arrives, and hard-coding the opposite fails on the US ones. A pipeline that ingests from multiple regions cannot know the locale per row, so it must resolve the decimal separator from the string’s own structure — the rightmost separator — rather than from a configuration flag that will eventually be wrong.

Discarding the raw string once normalized. Overwriting the vendor’s 1.234,56 with 1234.56 and keeping only the Decimal destroys the audit trail. When a subcontractor later disputes a change-order amount, the record must show exactly what was submitted, not just what the pipeline computed. Keep raw_amount immutable alongside the canonical value, and validate that one normalizes to the other so the pair can never drift.

Where This Fits in the Pipeline

This normalizer is one of the schema validation rules that every extracted money field clears at the ingestion boundary, immediately after field extraction lifts the raw amount string off the document. Once an amount is canonical, it can post against a real account through budget code standardization and roll up against the scope resolved by the WBS mapping strategies — both of which assume every cost is already an exact Decimal. A quarantined amount, one whose string could not be resolved, is handed on for review rather than dropped, so a locale the pipeline has not seen surfaces as a triage item instead of a truncated cost. Keeping the raw string immutable through all of this is what lets an auditor reconstruct any figure from the vendor’s original submission.

Frequently Asked Questions

Related

• Schema Validation Rules
• Field Extraction Techniques
• Budget Code Standardization
• WBS Mapping Strategies
• Error Handling Protocols

← Back to Schema Validation Rules