Parsing Primavera P6 XER Files with Python

The XER file is Primavera P6’s native interchange format, and it looks deceptively like a CSV — until a smart quote in an activity name crashes the reader, an embedded tab splits one row into two, or a new P6 version shifts every column by one. This page solves one precise problem: how to parse an XER export reliably into typed rows without silent data loss, respecting its versioned header, its %T/%F/%R/%E record markers, its tab delimiting, and its Windows-1252 encoding, so that every TASK, PROJWBS, and CALENDAR row survives the trip into a typed record. XER is not one table; it is a stack of relational tables concatenated into a single text file, each announced by a marker line. Get the decoding wrong and the failure is quiet — activity names lose characters, float lands in the wrong column, and nothing raises. This is the reader stage of Primavera P6 export parsing within schedule sync integration, and it targets Python automation builders who need every row to arrive intact and typed.

Key Rules and Specification

A robust XER reader is a small state machine that reads the file exactly once and enforces the format’s contract at every line:

• The header line is ERMHDR. The first line of every XER file is an ERMHDR record carrying the export version, date, and originating database. It is not a table; skip it, or read the version if you branch on P6 releases, but never feed it to a table parser.
• Four record markers govern state. %T <TABLE> opens a table, %F <field…> declares that table’s column headers, %R <value…> is a data row, and %E marks end of file. Any line whose first cell is not one of these is ignored by design.
• Encoding is Windows-1252, always. P6 writes XER as cp1252, so a UTF-8 read fails on the first smart quote, en dash, or degree sign in an activity name. Decode explicitly as cp1252; never suppress the error, which deletes characters silently.
• It is tab-delimited, but not tab-split. Activity names and notes routinely contain literal tabs and embedded newlines. Use the csv module with delimiter="\t" and newline="" so quoted or multi-line fields stay whole; a bare str.split("\t") fabricates phantom columns.
• Key rows by header name, not index. A newer P6 version adds columns, so positional access (cells[7]) silently reads the wrong field after an upgrade. Zip each %R row against the %F headers captured for the current table and access fields by name.
• %E is authoritative. Stop at the end marker; treat anything after it as absent rather than parsing trailing bytes.

Production Code Example

The reader below streams the file once, dispatches on the record marker, and yields (table_name, row_dict) pairs only for the scheduling tables — nothing is held in memory beyond the current headers. Pattern details for the markers follow the standard Python re module documentation, though the dispatch here needs only exact string equality.

from __future__ import annotations

import csv
import logging
from collections.abc import Iterator
from datetime import datetime, timezone
from pathlib import Path
from typing import Literal

from pydantic import BaseModel, Field, field_validator

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)-8s | %(message)s")
logger = logging.getLogger("xer_reader")

WANTED_TABLES = frozenset({"PROJWBS", "TASK", "TASKPRED", "CALENDAR"})


def read_xer_tables(path: Path) -> Iterator[tuple[str, dict[str, str]]]:
    """Stream (table_name, row) pairs from an XER export without data loss.

    XER is CP1252, tab-delimited, with ERMHDR then %T/%F/%R/%E markers. Reading
    with csv (not str.split) preserves embedded tabs and newlines inside fields.
    """
    current_table: str | None = None
    headers: list[str] = []

    with path.open("r", encoding="cp1252", newline="") as fh:
        reader = csv.reader(fh, delimiter="\t")
        for cells in reader:
            if not cells:
                continue
            marker = cells[0]
            if marker == "%T":                 # %T <TABLE_NAME>
                current_table = cells[1] if len(cells) > 1 else None
                headers = []
            elif marker == "%F":               # header row for current table
                headers = cells[1:]
            elif marker == "%R":               # data row
                if current_table in WANTED_TABLES and headers:
                    values = cells[1:]
                    if len(values) != len(headers):
                        # Never guess: a length mismatch is corruption, not a row.
                        logger.warning(
                            "row width mismatch in %s: %d headers, %d values",
                            current_table, len(headers), len(values),
                        )
                        continue
                    yield current_table, dict(zip(headers, values))
            elif marker == "%E":               # explicit end of file
                break
            # ERMHDR and unknown markers fall through untouched, by design.

With the stream in hand, building a typed record is a validation step, not more parsing. The ScheduleActivity model rejects a naive date or a bad WBS code at construction time, so a malformed TASK row raises loudly instead of poisoning a downstream schedule comparison.

Discipline = Literal["ARCH", "STR", "MEP", "CIV", "ELEC", "PLMB"]
WBS_PATTERN = r"^PROJ-\d{3}-(ARCH|STR|MEP|CIV|ELEC|PLMB)-\d{2}$"


class ScheduleActivity(BaseModel):
    project_id: str = Field(min_length=1, max_length=40)
    activity_id: str = Field(min_length=1, max_length=40)   # P6 task_code
    activity_name: str = Field(min_length=1, max_length=240)
    wbs_node: str = Field(pattern=WBS_PATTERN)
    discipline: Discipline
    start_date: datetime
    finish_date: datetime
    total_float_days: int

    @field_validator("start_date", "finish_date")
    @classmethod
    def _tz_aware(cls, v: datetime) -> datetime:
        if v.tzinfo is None:
            raise ValueError("schedule dates must be timezone-aware")
        return v.astimezone(timezone.utc)

    @property
    def is_critical(self) -> bool:
        return self.total_float_days <= 0


def _p6_dt(raw: str) -> datetime:
    # XER date form is 'YYYY-MM-DD HH:MM'; treat P6 clock time as project UTC.
    return datetime.strptime(raw, "%Y-%m-%d %H:%M").replace(tzinfo=timezone.utc)


def build_activity(task: dict[str, str], project_id: str, wbs_node: str,
                   discipline: Discipline, day_hours: float = 8.0) -> ScheduleActivity:
    float_hr = float(task.get("total_float_hr_cnt", "0") or 0.0)
    return ScheduleActivity(
        project_id=project_id,
        activity_id=task["task_code"],
        activity_name=task["task_name"],       # embedded tabs already preserved
        wbs_node=wbs_node,
        discipline=discipline,
        start_date=_p6_dt(task["target_start_date"]),
        finish_date=_p6_dt(task["target_end_date"]),
        total_float_days=int(round(float_hr / day_hours)),
    )


if __name__ == "__main__":
    export = Path("schedule.xer")
    tasks = [row for table, row in read_xer_tables(export) if table == "TASK"]
    logger.info("read %d TASK rows", len(tasks))
    if tasks:
        act = build_activity(tasks[0], project_id="TWR-014",
                             wbs_node="PROJ-014-STR-02", discipline="STR")
        print(act.model_dump_json(indent=2))

Common Mistakes and Gotchas

Opening the file as UTF-8. The default open() codec is UTF-8, and the first Windows-1252 smart quote (0x92) in an activity name raises UnicodeDecodeError. The reflex fix — errors="ignore" — is worse: it deletes the offending byte, so "Owner's steel" becomes "Owners steel" and the WBS name match downstream quietly degrades. Always decode as cp1252 and let genuinely corrupt files fail.

Splitting on tabs by hand. line.split("\t") looks equivalent to a CSV read until an activity name contains a literal tab or a note field spans two lines. Then one logical row becomes two malformed ones, every subsequent column shifts, and float lands under the wrong header — with no exception raised. The csv reader with newline="" is what keeps multi-line and tab-bearing fields intact.

Reading fields by position. Hard-coding cells[7] for total_float_hr_cnt works until the client upgrades P6 and a new column appears earlier in the %F list. Every positional index after it now points one field left, and the import reads plausible-but-wrong data. Zipping each %R row against the %F headers for its table makes the reader immune to column additions — the field is fetched by name or it is absent.

Where This Fits in the Pipeline

This reader is the front door of Primavera P6 export parsing: it turns the raw XER stack of tables into the row dicts every later stage consumes. The TASK rows it yields become typed activities and bind to a canonical WBS node; the TASKPRED rows it surfaces are handed to extracting activity relationships, which rebuilds the dependency graph from predecessor and successor links. Once activities and relationships are typed, the resolved records feed critical path delta detection, where two schedule updates are compared for float and longest-path movement. Because this stage keys every row by header name and preserves every character, the comparison downstream is trustworthy — nothing it sees was silently dropped at the boundary.

Frequently Asked Questions

Related

• Primavera P6 Export Parsing
• Extracting Activity Relationships from P6 XML Exports
• Critical Path Delta Detection
• Schedule Baseline Variance
• WBS Mapping Strategies

← Back to Primavera P6 Export Parsing