Compile Stage¶

The compile stage transforms YAML dashboard definitions into normalized, validated CompiledBoard objects ready for execution and rendering.

YAML string → parse → validate → normalize → size → CompiledBoard

Entry Points¶

compile¶

The main entry point for compiling YAML content:

compile ¶

compile(yaml_content: str, options: Optional[Dict[str, Any]] = None, base_dir: Optional[Path] = None) -> CompileResult

Compile YAML content to a CompiledBoard.

Stage: COMPILE (Full Pipeline)

This is the main entry point for compilation. It orchestrates: 1. PARSE: YAML string → Board object 2. VALIDATE: Check structure and references 3. NORMALIZE: Resolve references, add metadata 4. SIZE: Calculate layout dimensions

PARAMETER	DESCRIPTION
`yaml_content`	YAML string to compile TYPE: `str`
`options`	Optional compilation options TYPE: `Optional[Dict[str, Any]]` DEFAULT: `None`
`base_dir`	Base directory for resolving file references TYPE: `Optional[Path]` DEFAULT: `None`

RETURNS	DESCRIPTION
`CompileResult`	CompileResult with compiled board or errors

Example

yaml_content = ''' ... title: My Dashboard ... queries: ... users: SELECT * FROM users ... charts: ... user_count: ... query: users ... type: kpi ... metric: count ... rows: ... - user_count ... ''' result = compile(yaml_content) if result.success: ... board = result.board ... print(board.title) # "My Dashboard"

Source code in dataface/compile/compiler.py

def compile(
    yaml_content: str,
    options: Optional[Dict[str, Any]] = None,
    base_dir: Optional[Path] = None,
) -> CompileResult:
    """Compile YAML content to a CompiledBoard.

    Stage: COMPILE (Full Pipeline)

    This is the main entry point for compilation. It orchestrates:
    1. PARSE: YAML string → Board object
    2. VALIDATE: Check structure and references
    3. NORMALIZE: Resolve references, add metadata
    4. SIZE: Calculate layout dimensions

    Args:
        yaml_content: YAML string to compile
        options: Optional compilation options
        base_dir: Base directory for resolving file references

    Returns:
        CompileResult with compiled board or errors

    Example:
        >>> yaml_content = '''
        ... title: My Dashboard
        ... queries:
        ...   users: SELECT * FROM users
        ... charts:
        ...   user_count:
        ...     query: users
        ...     type: kpi
        ...     metric: count
        ... rows:
        ...   - user_count
        ... '''
        >>> result = compile(yaml_content)
        >>> if result.success:
        ...     board = result.board
        ...     print(board.title)  # "My Dashboard"
    """
    options = options or {}
    errors: List[CompilationError] = []
    warnings: List[str] = []

    # ════════════════════════════════════════════════════════════════════
    # STEP 1: Parse YAML
    # ════════════════════════════════════════════════════════════════════
    # Convert YAML string to Board object. Handles syntax errors.
    try:
        board = parse_yaml(yaml_content)
    except ParseError as e:
        return CompileResult(errors=[e])
    except Exception as e:
        return CompileResult(errors=[CompilationError(f"Parse error: {e}")])

    # ════════════════════════════════════════════════════════════════════
    # STEP 2: Validate Structure
    # ════════════════════════════════════════════════════════════════════
    # Check references and semantic constraints.
    validation_errors = validate_board(board)
    if validation_errors:
        # Cast to List[CompilationError] since ValidationError inherits from it
        return CompileResult(errors=list(validation_errors))

    # ════════════════════════════════════════════════════════════════════
    # STEP 3: Get Default Source
    # ════════════════════════════════════════════════════════════════════
    # Get the board's default source to pass to query normalization.
    default_source = (
        board.get_default_source() if hasattr(board, "get_default_source") else None
    )

    # ════════════════════════════════════════════════════════════════════
    # STEP 4: Build Query Registry
    # ════════════════════════════════════════════════════════════════════
    # Collect and normalize all queries before normalization.
    # This allows charts to reference queries from any part of the board.
    try:
        query_registry = _build_query_registry(
            board, base_dir, default_source=default_source
        )
    except CompilationError as e:
        return CompileResult(errors=[e])
    except Exception as e:
        return CompileResult(errors=[CompilationError(f"Query registry error: {e}")])

    # ════════════════════════════════════════════════════════════════════
    # STEP 5: Detect Circular Dependencies
    # ════════════════════════════════════════════════════════════════════
    # Check for {{ queries.* }} cycles in SQL.
    if query_registry:
        try:
            detect_query_dependencies(query_registry)
        except JinjaError as e:
            return CompileResult(errors=[e])

    # ════════════════════════════════════════════════════════════════════
    # STEP 5b: Build Chart Registry
    # ════════════════════════════════════════════════════════════════════
    # Collect all charts from the entire board tree (global namespace).
    # Charts are global like queries - any layout can reference any chart.
    try:
        chart_registry = _build_chart_registry(board, base_dir=base_dir)
    except CompilationError as e:
        return CompileResult(errors=[e])
    except Exception as e:
        return CompileResult(errors=[CompilationError(f"Chart registry error: {e}")])

    # ════════════════════════════════════════════════════════════════════
    # STEP 6: Normalize (Transform to CompiledBoard)
    # ════════════════════════════════════════════════════════════════════
    # Resolve references, add metadata, create unified layout.
    # Note: Variable registry is built by renderer at render time (not needed here)
    try:
        compiled = normalize_board(
            board,
            query_registry=query_registry,
            chart_registry=chart_registry,
            base_path=base_dir,
        )
    except ReferenceError as e:
        return CompileResult(errors=[e])
    except CompilationError as e:
        return CompileResult(errors=[e])
    except Exception as e:
        return CompileResult(errors=[CompilationError(f"Normalization error: {e}")])

    # ════════════════════════════════════════════════════════════════════
    # STEP 7: Calculate Layout Dimensions
    # ════════════════════════════════════════════════════════════════════
    # Calculate pixel dimensions for all layout items.
    try:
        compiled = calculate_layout(compiled)
    except Exception as e:
        # Layout errors are warnings, not failures
        warnings.append(f"Layout calculation warning: {e}")

    return CompileResult(
        board=compiled,
        errors=errors,
        warnings=warnings,
        query_registry=query_registry,
    )

compile_file¶

Compile from a file path:

compile_file ¶

compile_file(file_path: Path, options: Optional[Dict[str, Any]] = None) -> CompileResult

Compile a YAML file to a CompiledBoard.

Convenience function that reads a file and compiles it.

PARAMETER	DESCRIPTION
`file_path`	Path to YAML file TYPE: `Path`
`options`	Optional compilation options TYPE: `Optional[Dict[str, Any]]` DEFAULT: `None`

RETURNS	DESCRIPTION
`CompileResult`	CompileResult with compiled board or errors

RAISES	DESCRIPTION
`FileNotFoundError`	If file doesn't exist

Example

result = compile_file(Path("dashboard.yml")) if result.success: ... print(result.board.title)

Source code in dataface/compile/compiler.py

def compile_file(
    file_path: Path,
    options: Optional[Dict[str, Any]] = None,
) -> CompileResult:
    """Compile a YAML file to a CompiledBoard.

    Convenience function that reads a file and compiles it.

    Args:
        file_path: Path to YAML file
        options: Optional compilation options

    Returns:
        CompileResult with compiled board or errors

    Raises:
        FileNotFoundError: If file doesn't exist

    Example:
        >>> result = compile_file(Path("dashboard.yml"))
        >>> if result.success:
        ...     print(result.board.title)
    """
    if isinstance(file_path, str):
        file_path = Path(file_path)

    if not file_path.exists():
        return CompileResult(errors=[CompilationError(f"File not found: {file_path}")])

    try:
        yaml_content = file_path.read_text()
    except Exception as e:
        return CompileResult(errors=[CompilationError(f"Failed to read file: {e}")])

    return compile(yaml_content, options=options, base_dir=file_path.parent)

CompileResult¶

The result object returned by compilation:

CompileResult `dataclass` ¶

CompileResult(board: Optional[CompiledBoard] = None, errors: List[CompilationError] = list(), warnings: List[str] = list(), query_registry: Dict[str, CompiledQuery] = dict())

Result of compilation.

Contains the compiled board (if successful), any errors encountered, and warnings that don't prevent compilation.

ATTRIBUTE	DESCRIPTION
`board`	Compiled board (None if errors occurred) TYPE: `Optional[CompiledBoard]`
`errors`	List of compilation errors TYPE: `List[CompilationError]`
`warnings`	List of non-fatal warnings TYPE: `List[str]`
`query_registry`	All normalized queries (for executor) TYPE: `Dict[str, CompiledQuery]`

Example

result = compile(yaml_content) if result.success: ... print(f"Compiled: {result.board.title}") ... else: ... for error in result.errors: ... print(f"Error: {error}")

success `property` ¶

success: bool

Check if compilation was successful.

is_success `property` ¶

is_success: bool

Check if compilation was successful (alias).

Compiled Types¶

These types represent the output of compilation, with all fields guaranteed to be present.

CompiledBoard¶

CompiledBoard ¶

Bases: BaseModel

Compiled board ready for execution and rendering.

Stage: COMPILE output / EXECUTE and RENDER input

This is the main output of compilation. It contains: - Resolved definitions (variables, queries, charts) - Unified layout structure - Calculated dimensions - Applied defaults and metadata

Guarantees

id: Always set
title: Always set (empty string if not provided)
layout: Always present (unified structure)
All charts resolved to CompiledChart objects
All queries resolved to query objects

ATTRIBUTE	DESCRIPTION
`id`	Unique identifier for this board TYPE: `str`
`title`	Display title TYPE: `str`
`description`	Description text TYPE: `str`
`variables`	Variable definitions (Dict[str, Variable]) TYPE: `Dict[str, Variable]`
`queries`	Compiled queries (Dict[str, CompiledQuery]) TYPE: `Dict[str, CompiledQuery]`
`charts`	Compiled charts (Dict[str, CompiledChart]) TYPE: `Dict[str, CompiledChart]`
`layout`	Unified layout structure TYPE: `Layout`
`variable_defaults`	Resolved default values for variables TYPE: `VariableValues`
`style`	Board styling TYPE: `Dict[str, Any]`
`depth`	Nesting depth (0 = root) TYPE: `int`

Example

result = compile(yaml_content) board = result.board print(board.id) # "sales-dashboard" print(board.title) # "Sales Dashboard" print(board.layout.type) # "rows" for item in board.layout.items: ... chart = item.chart ... print(f"Chart: {chart.id}, Query: {chart.query_name}")

CompiledChart¶

CompiledChart ¶

Bases: BaseModel

Compiled chart ready for execution and rendering.

Stage: COMPILE output / EXECUTE and RENDER input

This type represents a fully normalized chart where: - All optional fields from Chart have been resolved - The query reference has been resolved to a query object - An ID has been assigned - Title and description have defaults applied

Guarantees vs Chart (input type): | Field | Chart (input) | CompiledChart (compiled) | |-------------|-----------------------|----------------------------| | id | Optional[str] | str (always present) | | title | Optional[str] | str (empty string default) | | description | Optional[str] | str (empty string default) | | query | str (reference) | Query object (resolved) |

Downstream code can access fields directly without checking for None.

ATTRIBUTE	DESCRIPTION
`id`	Unique identifier within the dashboard. TYPE: `str`
`query`	Resolved query object (SqlQuery, CsvQuery, etc.) TYPE: `CompiledQuery`
`query_name`	String name for executor lookup. TYPE: `str`
`type`	Chart type (line, bar, table, etc.) TYPE: `ChartTypeLiteral`
`title`	Display title. Empty string if not provided. TYPE: `str`
`description`	Description. Empty string if not provided. TYPE: `str`

validate_kpi ¶

validate_kpi() -> CompiledChart

Validate KPI charts have metric field.

Source code in dataface/compile/compiled_types.py

@model_validator(mode="after")
def validate_kpi(self) -> "CompiledChart":
    """Validate KPI charts have metric field."""
    if self.type == "kpi" and not self.metric:
        raise ValueError("KPI charts must specify a 'metric' field")
    return self

LayoutItem¶

LayoutItem ¶

Bases: BaseModel

A single item in a layout.

Represents either a chart or a nested board in the layout. The unified structure makes iteration simple:

for item in layout.items:
    if item.type == "chart":
        render_chart(item.chart)
    else:
        render_board(item.board)

ATTRIBUTE	DESCRIPTION
`type`	Either "chart" or "board" TYPE: `Literal['chart', 'board']`
`chart`	CompiledChart if type == "chart" TYPE: `Optional[CompiledChart]`
`board`	CompiledBoard if type == "board" TYPE: `Optional[CompiledBoard]`

Grid-specific (optional): row, col: Grid position (0-based) row_span, col_span: Grid span (in grid units)

Calculated dimensions (set by sizing module): width_fraction: What fraction of parent width (0.0-1.0) width: Pixel width height: Pixel height x: X position in pixels y: Y position in pixels

Query Types¶

Dataface supports multiple query types through a unified interface.

SqlQuery¶

SqlQuery ¶

Bases: Query

SQL query against a database.

Executes raw SQL against the configured database connection. Supports Jinja templating for variable substitution.

Required Fields

sql: SQL query string (may contain Jinja templates)

Connection Fields (one required): source: Source name or inline config (new preferred way) profile: dbt profile name (legacy, alias for source)

Optional Fields

target: dbt target name (defaults to 'dev' if not specified)

Example

query = SqlQuery( ... sql="SELECT * FROM users WHERE id = {{ user_id }}", ... source="my_postgres" ... ) query.query_type # "sql" query.source_description # "SQL: SELECT * FROM users WHERE id = {{ user_..."

query_type `property` ¶

query_type: str

Return 'sql' as query type.

source_description `property` ¶

source_description: str

Return truncated SQL for description.

effective_source `property` ¶

effective_source: Optional[Union[str, Dict[str, Any]]]

Get effective source (source takes precedence over profile).

CsvQuery¶

CsvQuery ¶

Bases: Query

Query data from a CSV file.

Loads and queries data from CSV files. Supports column selection, filtering, and limiting results.

Fields (one required): file: Path to CSV file (relative to project root or absolute) source: Source reference or inline CSV source config

Optional Fields

columns: List of columns to select (default: all) filter: Dict of column->value for exact match filtering

Example

query = CsvQuery(file="data/sales.csv", columns=["date", "amount"]) query.query_type # "csv" query.source_description # "CSV: data/sales.csv"

query_type `property` ¶

query_type: str

Return 'csv' as query type.

source_description `property` ¶

source_description: str

Return file path for description.

effective_file `property` ¶

effective_file: Optional[str]

Get effective file path from file or source config.

HttpQuery¶

HttpQuery ¶

Bases: Query

Query data from an HTTP API.

Fetches data from REST API endpoints. Supports various HTTP methods, headers, query parameters, and request bodies.

Fields

url: Full URL of the API endpoint (legacy) source: Source reference or inline HTTP source config (new) path: API path (appended to base URL from source)

Optional Fields

method: HTTP method (default: GET) headers: Request headers (merged with source headers) params: Query parameters body: Request body (for POST/PUT/PATCH) json_path: JSONPath expression to extract data from response

Example

query = HttpQuery( ... source="model_api", ... path="/predict", ... method="POST", ... body={"input": "data"} ... ) query.query_type # "http" query.source_description # "HTTP: POST model_api/predict"

query_type `property` ¶

query_type: str

Return 'http' as query type.

source_description `property` ¶

source_description: str

Return method and URL for description.

effective_url `property` ¶

effective_url: Optional[str]

Get effective URL from url or source config.

MetricFlowQuery¶

MetricFlowQuery ¶

Bases: Query

Query the dbt Semantic Layer.

Executes queries against dbt's Semantic Layer using MetricFlow. Supports metrics, dimensions, filters, and time grains.

Required Fields

metrics: List of metric names to query

Optional Fields

dimensions: List of dimension names to group by time_grain: Time grain for time-based dimensions (day, week, month, etc.)

Example

query = MetricFlowQuery( ... metrics=["revenue", "orders"], ... dimensions=["date_day", "region"] ... ) query.query_type # "metricflow" query.source_description # "MetricFlow: revenue, orders"

query_type `property` ¶

query_type: str

Return 'metricflow' as query type.

source_description `property` ¶

source_description: str

Return metrics list for description.

Input Types¶

These types represent the raw YAML structure before normalization.

Board¶

Board ¶

Bases: BaseModel

Board (dashboard) definition from YAML.

This is the top-level input type representing a complete dashboard. A board contains definitions (variables, queries, charts) and a layout.

Example YAML

title: Sales Dashboard description: Overview of sales metrics

sources: default: my_postgres # Default source for all queries

variables: date_range: input: daterange default: ["2024-01-01", "2024-12-31"]

queries: sales: SELECT * FROM sales WHERE date BETWEEN ...

charts: revenue: query: sales type: line x: date y: amount

rows: - revenue

Layout

Exactly one layout type should be present (rows, cols, grid, or tabs). Layout items can be: - Chart name (string reference) - Inline chart definition (dict with query and type) - Nested board (dict with layout keys)

Sources

Optional sources section with: - default: Default source name for all queries in this board - Inline source definitions (source_name: {...config...})

reject_dashboard_key `classmethod` ¶

reject_dashboard_key(data: Any) -> Any

Reject 'dashboard' as a top-level key - it's not valid syntax.

Source code in dataface/compile/types.py

@model_validator(mode="before")
@classmethod
def reject_dashboard_key(cls, data: Any) -> Any:
    """Reject 'dashboard' as a top-level key - it's not valid syntax."""
    if isinstance(data, dict) and "dashboard" in data:
        raise ValueError(
            "'dashboard:' is not valid. Board properties (title, rows, queries, etc.) "
            "should be at the top level of the YAML file, not nested under 'dashboard:'."
        )
    return data

validate_layout ¶

validate_layout() -> Board

Ensure at least one layout type or content is defined.

Source code in dataface/compile/types.py

@model_validator(mode="after")
def validate_layout(self) -> "Board":
    """Ensure at least one layout type or content is defined."""
    layouts = [self.rows, self.cols, self.grid, self.tabs]
    defined = [l for l in layouts if l is not None]
    if len(defined) > 1:
        raise ValueError(
            "Board can only have one layout type (rows, cols, grid, or tabs)"
        )
    if len(defined) == 0 and not self.content:
        raise ValueError("Board must have at least one layout type or content")
    return self

get_default_source ¶

get_default_source() -> Optional[str]

Get the default source for this board.

RETURNS	DESCRIPTION
`Optional[str]`	Default source name, or None if not set

Source code in dataface/compile/types.py

def get_default_source(self) -> Optional[str]:
    """Get the default source for this board.

    Returns:
        Default source name, or None if not set
    """
    # Single source shorthand takes precedence
    if self.source:
        return self.source
    # Then check sources.default
    if self.sources:
        if isinstance(self.sources, SourcesSection):
            return self.sources.default
        elif isinstance(self.sources, dict):
            return self.sources.get("default")
    return None

Chart¶

Chart ¶

Bases: BaseModel

Chart definition from YAML.

Charts visualize query results. The type determines the visualization.

Example YAML

charts: revenue_trend: query: sales_by_date type: line title: "Revenue Trend" x: date y: amount color: category

ATTRIBUTE	DESCRIPTION
`query`	Reference to a query name (required) TYPE: `Union[str, Dict[str, Any]]`
`type`	Chart type (line, bar, etc.) TYPE: `ChartTypeLiteral`
`title`	Display title (optional, derived from chart name if not set) TYPE: `Optional[str]`
`x,`	Data field mappings for axes TYPE: `y`
`color,`	Additional encodings TYPE: `(size, shape)`

validate_kpi ¶

validate_kpi() -> Chart

Validate KPI charts have metric field.

Source code in dataface/compile/types.py

@model_validator(mode="after")
def validate_kpi(self) -> "Chart":
    """Validate KPI charts have metric field."""
    if self.type == "kpi" and not self.metric:
        raise ValueError("KPI charts must specify a 'metric' field")
    return self

Query¶

Query ¶

Bases: BaseModel

Query definition from YAML.

Queries fetch data from various sources. The type field determines which adapter handles execution.

Supported types: - sql: Raw SQL query (default) - metricflow: dbt Semantic Layer query - dbt_model: Query against a dbt model - http: REST API query - csv: CSV file query

Example YAML

queries: sales_by_date: sql: SELECT date, SUM(amount) FROM sales GROUP BY date source: my_postgres # Source reference

metrics_query: type: metricflow metrics: [revenue, orders] dimensions: [date_day]

api_data: type: http url: https://api.example.com/data

Errors¶

errors ¶

Compilation error types.

Stage: COMPILE Purpose: Define error types for all compilation failures.

These errors are raised during: - YAML parsing (ParseError, YAMLError) - Schema validation (ValidationError) - Reference resolution (ReferenceError) - Jinja template rendering (JinjaError)

All errors inherit from CompilationError for easy catching.

CompilationError ¶

CompilationError(message: str, location: Optional[str] = None)

Bases: DatafaceError

Base error for all compilation failures.

This is the parent class for all compilation-related errors. Catch this to handle any compilation error.

ATTRIBUTE	DESCRIPTION
`message`	Human-readable error description
`location`	Optional location in source (line number, path, etc.)

Source code in dataface/compile/errors.py

def __init__(self, message: str, location: Optional[str] = None):
    self.message = message
    self.location = location
    super().__init__(self._format_message())

parse_error `classmethod` ¶

parse_error(message: str) -> CompilationError

Create a parse error from a message.

Source code in dataface/compile/errors.py

@classmethod
def parse_error(cls, message: str) -> "CompilationError":
    """Create a parse error from a message."""
    return ParseError(message)

validation_error `classmethod` ¶

validation_error(message: str, location: Optional[str] = None) -> CompilationError

Create a validation error from a message.

Source code in dataface/compile/errors.py

@classmethod
def validation_error(
    cls, message: str, location: Optional[str] = None
) -> "CompilationError":
    """Create a validation error from a message."""
    return ValidationError(message, location)

reference_error `classmethod` ¶

reference_error(ref: str, context: Optional[str] = None) -> CompilationError

Create a reference error for an unresolved reference.

Source code in dataface/compile/errors.py

@classmethod
def reference_error(
    cls, ref: str, context: Optional[str] = None
) -> "CompilationError":
    """Create a reference error for an unresolved reference."""
    return ReferenceError(ref, context)

jinja_error `classmethod` ¶

jinja_error(message: str, template: Optional[str] = None) -> CompilationError

Create a Jinja template error.

Source code in dataface/compile/errors.py

@classmethod
def jinja_error(
    cls, message: str, template: Optional[str] = None
) -> "CompilationError":
    """Create a Jinja template error."""
    return JinjaError(message, template)

normalization_error `classmethod` ¶

normalization_error(message: str) -> CompilationError

Create a normalization error from a message.

Source code in dataface/compile/errors.py

@classmethod
def normalization_error(cls, message: str) -> "CompilationError":
    """Create a normalization error from a message."""
    return CompilationError(f"Normalization error: {message}")

ParseError ¶

ParseError(message: str, line: Optional[int] = None)

Bases: CompilationError

Error during YAML parsing.

Raised when: - YAML syntax is invalid - YAML structure cannot be parsed

Example

try: ... compile("invalid: yaml: content") ... except ParseError as e: ... print(f"Parse failed: {e}")

Source code in dataface/compile/errors.py

def __init__(self, message: str, line: Optional[int] = None):
    location = f"line {line}" if line else None
    super().__init__(f"YAML parse error: {message}", location)

ValidationError ¶

ValidationError(message: str, location: Optional[str] = None)

Bases: CompilationError

Error during schema validation.

Raised when: - Required fields are missing - Field values are invalid type - Enum values are not recognized - Schema constraints are violated

Example

try: ... compile("charts:\n my_chart:\n type: invalid_type") ... except ValidationError as e: ... print(f"Validation failed: {e}")

Source code in dataface/compile/errors.py

def __init__(self, message: str, location: Optional[str] = None):
    super().__init__(f"Validation error: {message}", location)

ReferenceError ¶

ReferenceError(ref: str, context: Optional[str] = None)

Bases: CompilationError

Error resolving a reference.

Raised when: - Chart references unknown query - Layout references unknown chart - Remote reference cannot be resolved - Partial file not found

ATTRIBUTE	DESCRIPTION
`ref`	The reference that could not be resolved
`context`	Where the reference was used

Example

try: ... compile("charts:\n my_chart:\n query: unknown_query") ... except ReferenceError as e: ... print(f"Reference '{e.ref}' not found")

Source code in dataface/compile/errors.py

def __init__(self, ref: str, context: Optional[str] = None):
    self.ref = ref
    message = f"Reference '{ref}' not found"
    location = f"in {context}" if context else None
    super().__init__(message, location)

JinjaError ¶

JinjaError(message: str, template: Optional[str] = None)

Bases: CompilationError

Error during Jinja template resolution.

Raised when: - Jinja syntax is invalid - Variable not found in template context - Filter not found

ATTRIBUTE	DESCRIPTION
`template`	The template that caused the error (if available)

Example

try: ... compile("queries:\n q: SELECT * FROM {{ undefined_var }}") ... except JinjaError as e: ... print(f"Template error: {e}")

Source code in dataface/compile/errors.py

def __init__(self, message: str, template: Optional[str] = None):
    self.template = template
    location = (
        f"template: {template[:50]}..."
        if template and len(template) > 50
        else template
    )
    super().__init__(f"Jinja error: {message}", location)

Internal Modules¶

These are used internally by the compiler. Most users won't need to use them directly.

Parser¶

parse_yaml ¶

parse_yaml(content: str) -> Board

Parse YAML content into a Board object.

Stage: COMPILE (Step 1 of 4: Parsing)

This is the first step of compilation. It converts a raw YAML string into a structured Board object. Only basic syntax validation happens here - schema validation is the next step.

PARAMETER	DESCRIPTION
`content`	Raw YAML string to parse TYPE: `str`

RETURNS	DESCRIPTION
`Board`	Board object with parsed structure

RAISES	DESCRIPTION
`ParseError`	If YAML syntax is invalid or parsing fails

Example

yaml_content = ''' ... title: My Dashboard ... queries: ... sales: SELECT * FROM sales ... charts: ... revenue: ... query: sales ... type: line ... rows: ... - revenue ... ''' board = parse_yaml(yaml_content) board.title 'My Dashboard'

Source code in dataface/compile/parser.py

def parse_yaml(content: str) -> Board:
    """Parse YAML content into a Board object.

    Stage: COMPILE (Step 1 of 4: Parsing)

    This is the first step of compilation. It converts a raw YAML string
    into a structured Board object. Only basic syntax validation happens
    here - schema validation is the next step.

    Args:
        content: Raw YAML string to parse

    Returns:
        Board object with parsed structure

    Raises:
        ParseError: If YAML syntax is invalid or parsing fails

    Example:
        >>> yaml_content = '''
        ... title: My Dashboard
        ... queries:
        ...   sales: SELECT * FROM sales
        ... charts:
        ...   revenue:
        ...     query: sales
        ...     type: line
        ... rows:
        ...   - revenue
        ... '''
        >>> board = parse_yaml(yaml_content)
        >>> board.title
        'My Dashboard'
    """
    # Step 1a: Parse YAML to dict
    try:
        parsed_data = yaml.safe_load(content)
    except yaml.YAMLError as e:
        raise ParseError(f"Invalid YAML syntax: {e}")

    # Step 1b: Handle empty content
    if parsed_data is None:
        raise ParseError("Empty YAML document")

    if not isinstance(parsed_data, dict):
        raise ParseError(f"YAML must be a mapping, got {type(parsed_data).__name__}")

    # Step 1c: Normalize queries (infer types)
    if "queries" in parsed_data and isinstance(parsed_data["queries"], dict):
        parsed_data["queries"] = _normalize_query_definitions(parsed_data["queries"])

    # Step 1d: Convert to Board
    try:
        return Board(**parsed_data)
    except Exception as e:
        # Try to enhance Pydantic validation errors with valid fields
        try:
            from pydantic import ValidationError as PydanticValidationError

            if isinstance(e, PydanticValidationError):
                from dataface.playground.error_formatter import (
                    format_pydantic_error_with_valid_fields,
                )

                enhanced_msg = format_pydantic_error_with_valid_fields(e, Board)
                # Use enhanced message directly (it already includes field path info)
                raise ParseError(enhanced_msg)
        except ImportError:
            # Fallback if error_formatter not available
            pass
        except ParseError:
            # Re-raise ParseError (from enhanced message)
            raise
        raise ParseError(f"Failed to parse board structure: {e}")

Validator¶

validate_board ¶

validate_board(board: Board) -> List[ValidationError]

Validate a Board structure and cross-references.

Stage: COMPILE (Step 2 of 4: Validation)

Performs semantic validation beyond what Pydantic provides: - Chart → Query references exist - Layout items reference existing charts

Note: Pydantic already validates types, required fields, and nested structures during parsing. This function adds cross-reference validation.

PARAMETER	DESCRIPTION
`board`	Parsed Board to validate TYPE: `Board`

RETURNS	DESCRIPTION
`List[ValidationError]`	List of ValidationError objects (empty if valid)

Example

board = parse_yaml(yaml_content) errors = validate_board(board) if errors: ... for e in errors: ... print(f"Error: {e}")

Source code in dataface/compile/validator.py

def validate_board(board: Board) -> List[ValidationError]:
    """Validate a Board structure and cross-references.

    Stage: COMPILE (Step 2 of 4: Validation)

    Performs semantic validation beyond what Pydantic provides:
    - Chart → Query references exist
    - Layout items reference existing charts

    Note: Pydantic already validates types, required fields, and nested
    structures during parsing. This function adds cross-reference validation.

    Args:
        board: Parsed Board to validate

    Returns:
        List of ValidationError objects (empty if valid)

    Example:
        >>> board = parse_yaml(yaml_content)
        >>> errors = validate_board(board)
        >>> if errors:
        ...     for e in errors:
        ...         print(f"Error: {e}")
    """
    errors: List[ValidationError] = []

    # Collect available names
    query_names = set((board.queries or {}).keys())
    chart_names = set((board.charts or {}).keys())

    # Validate chart → query references
    errors.extend(_validate_chart_query_references(board.charts or {}, query_names))

    # Validate SQL queries have required fields
    errors.extend(_validate_sql_queries(board.queries or {}))

    # Validate layout → chart references
    errors.extend(_validate_layout_references(board, chart_names, query_names))

    return errors

Normalizer¶

normalize_board ¶

normalize_board(board: Board, board_id: Optional[str] = None, parent_context: Optional[Dict[str, Any]] = None, query_registry: Optional[Dict[str, CompiledQuery]] = None, chart_registry: Optional[Dict[str, Any]] = None, depth: int = 0, base_path: Optional[Path] = None) -> CompiledBoard

Normalize a Board into a CompiledBoard with guaranteed structure.

Stage: COMPILE (Step 3 of 4: Normalization)

This is the core transformation from user-provided YAML structure to the internal compiled representation. After this step, all references are resolved, IDs are assigned, and downstream code can rely on guaranteed field presence.

Normalization performs: 1. Resolve all chart references (queries, extends, partials) 2. Generate unique IDs for all entities 3. Apply defaults (title from ID, empty description) 4. Transform input types to compiled types 5. Create unified Layout structure 6. Propagate default source to queries without explicit source

PARAMETER	DESCRIPTION
`board`	Input Board from parsing/validation step. TYPE: `Board`
`board_id`	Optional explicit ID for this board TYPE: `Optional[str]` DEFAULT: `None`
`parent_context`	Context from parent board (for nested boards) TYPE: `Optional[Dict[str, Any]]` DEFAULT: `None`
`query_registry`	Complete query registry for reference resolution TYPE: `Optional[Dict[str, CompiledQuery]]` DEFAULT: `None`
`chart_registry`	Complete chart registry for reference resolution TYPE: `Optional[Dict[str, Any]]` DEFAULT: `None`
`depth`	Current nesting depth (for cycle detection) TYPE: `int` DEFAULT: `0`
`base_path`	Base path for resolving external file references TYPE: `Optional[Path]` DEFAULT: `None`

RETURNS	DESCRIPTION
`CompiledBoard`	CompiledBoard with guarantees: - All fields required by compiled types are present - All references resolved to actual objects - All IDs are unique and deterministic - Layout is unified (rows/cols/grid/tabs → Layout type)

RAISES	DESCRIPTION
`ReferenceError`	When a chart references an undefined query
`CompilationError`	When normalization fails

Example

board = parse_yaml(yaml_content) errors = validate_board(board) if not errors: ... compiled = normalize_board(board) ... print(compiled.charts['revenue'].id) # Guaranteed to exist 'revenue'

Source code in dataface/compile/normalizer.py

def normalize_board(
    board: Board,
    board_id: Optional[str] = None,
    parent_context: Optional[Dict[str, Any]] = None,
    query_registry: Optional[Dict[str, CompiledQuery]] = None,
    chart_registry: Optional[Dict[str, Any]] = None,
    depth: int = 0,
    base_path: Optional[Path] = None,
) -> CompiledBoard:
    """Normalize a Board into a CompiledBoard with guaranteed structure.

    Stage: COMPILE (Step 3 of 4: Normalization)

    This is the core transformation from user-provided YAML structure to the
    internal compiled representation. After this step, all references are resolved,
    IDs are assigned, and downstream code can rely on guaranteed field presence.

    Normalization performs:
    1. Resolve all chart references (queries, extends, partials)
    2. Generate unique IDs for all entities
    3. Apply defaults (title from ID, empty description)
    4. Transform input types to compiled types
    5. Create unified Layout structure
    6. Propagate default source to queries without explicit source

    Args:
        board: Input Board from parsing/validation step.
        board_id: Optional explicit ID for this board
        parent_context: Context from parent board (for nested boards)
        query_registry: Complete query registry for reference resolution
        chart_registry: Complete chart registry for reference resolution
        depth: Current nesting depth (for cycle detection)
        base_path: Base path for resolving external file references

    Returns:
        CompiledBoard with guarantees:
            - All fields required by compiled types are present
            - All references resolved to actual objects
            - All IDs are unique and deterministic
            - Layout is unified (rows/cols/grid/tabs → Layout type)

    Raises:
        ReferenceError: When a chart references an undefined query
        CompilationError: When normalization fails

    Example:
        >>> board = parse_yaml(yaml_content)
        >>> errors = validate_board(board)
        >>> if not errors:
        ...     compiled = normalize_board(board)
        ...     print(compiled.charts['revenue'].id)  # Guaranteed to exist
        'revenue'
    """
    # Prevent infinite recursion
    MAX_DEPTH = 50
    if depth > MAX_DEPTH:
        raise CompilationError(f"Maximum nesting depth ({MAX_DEPTH}) exceeded")

    # Initialize context
    parent_context = parent_context or {}

    # Get base_path from parent context if not provided
    if base_path is None:
        base_path = parent_context.get("base_path")

    # Build registries if not provided (for standalone normalize_board calls)
    # This allows normalize_board to be called directly without going through compile()
    if query_registry is None:
        from dataface.compile.compiler import _build_query_registry

        default_source = (
            board.get_default_source() if hasattr(board, "get_default_source") else None
        )
        query_registry = _build_query_registry(
            board, base_dir=base_path, default_source=default_source
        )

    if chart_registry is None:
        from dataface.compile.compiler import _build_chart_registry

        chart_registry = _build_chart_registry(board, base_dir=base_path)

    # Generate board ID
    if not board_id:
        board_id = _generate_board_id(board, depth, parent_context.get("parent_id"))

    # ════════════════════════════════════════════════════════════════════
    # STEP 3a-0: Resolve default source
    # ════════════════════════════════════════════════════════════════════
    # Get default source from board, falling back to parent context
    default_source = (
        board.get_default_source() if hasattr(board, "get_default_source") else None
    )
    if not default_source:
        default_source = parent_context.get("default_source")

    # ════════════════════════════════════════════════════════════════════
    # STEP 3a: Normalize variables
    # ════════════════════════════════════════════════════════════════════
    # Variables are GLOBAL - all names must be unique across the entire dashboard.
    #
    # board.variables: Local variable definitions (for UI controls in this section)
    # The renderer will build the global variable registry by traversing the board tree
    # at render time (for template resolution, validation, etc.)

    # Local variables (for UI controls in this board section)
    local_variables: Dict[str, Variable] = {}
    if board.variables:
        for var_name, var_def in board.variables.items():
            if isinstance(var_def, Variable):
                local_variables[var_name] = var_def
            elif isinstance(var_def, dict):
                local_variables[var_name] = Variable(**var_def)
            elif isinstance(var_def, str) and ".variables." in var_def:
                # Handle cross-file variable reference (e.g., "file.variables.var_name")
                from dataface.compile.compiler import _load_variable_from_reference

                local_variables[var_name] = _load_variable_from_reference(
                    var_def, base_path
                )

    # Variable defaults (from local variables only - renderer builds global defaults)
    variable_defaults: VariableValues = {}
    for var_name, var in local_variables.items():
        if var.default is not None:
            variable_defaults[var_name] = var.default

    # ════════════════════════════════════════════════════════════════════
    # STEP 3b: Extract local queries from global registry
    # ════════════════════════════════════════════════════════════════════
    # Queries are normalized at compile-time and stored in global registry.
    # For this board, extract only the queries defined locally (for CompiledBoard.queries).
    # All queries are available in query_registry for chart resolution.
    local_queries: Dict[str, CompiledQuery] = {}
    if board.queries:
        for query_name in board.queries.keys():
            if query_name in query_registry:
                local_queries[query_name] = query_registry[query_name]
            else:
                # This shouldn't happen if registry was built correctly, but handle gracefully
                raise CompilationError(
                    f"Query '{query_name}' not found in registry. "
                    "This may indicate a compilation error."
                )

    # ════════════════════════════════════════════════════════════════════
    # STEP 3c: Normalize charts with query resolution
    # ════════════════════════════════════════════════════════════════════
    # Charts are GLOBAL - all charts are available from the global registry.
    # Normalize all charts from registry (for layout resolution).
    # Then extract only the charts defined locally (for CompiledBoard.charts).
    if chart_registry is None:
        chart_registry = {}

    # Normalize all charts from registry (charts are global - any layout can reference any chart)
    normalized_charts: Dict[str, CompiledChart] = {}
    for chart_name, chart_def in chart_registry.items():
        if chart_name not in normalized_charts:
            normalized_charts[chart_name] = _normalize_chart(
                chart_name, chart_def, query_registry, base_path, default_source
            )

    # Extract only locally defined charts (for CompiledBoard.charts)
    local_charts: Dict[str, CompiledChart] = {}
    if board.charts:
        for chart_name in board.charts.keys():
            if chart_name in normalized_charts:
                local_charts[chart_name] = normalized_charts[chart_name]
            else:
                # This shouldn't happen if registry was built correctly, but handle gracefully
                raise CompilationError(
                    f"Chart '{chart_name}' not found in registry. "
                    "Charts must be defined in the board tree or imported."
                )

    charts = local_charts

    # ════════════════════════════════════════════════════════════════════
    # STEP 3d: Build unified layout
    # ════════════════════════════════════════════════════════════════════
    layout = _build_unified_layout(
        board,
        normalized_charts,
        query_registry,
        board_id,
        depth,
        base_path,
        default_source,
        chart_registry,
    )

    # ════════════════════════════════════════════════════════════════════
    # STEP 3e: Create CompiledBoard
    # ════════════════════════════════════════════════════════════════════
    compiled_board = CompiledBoard(
        id=board_id,
        title=board.title or "",
        description=board.description or "",
        tags=board.tags or [],
        content=board.content or "",
        variables=local_variables,  # Local variables for UI controls
        queries=local_queries,
        charts=charts,
        layout=layout,
        variable_defaults=variable_defaults,  # Local variable defaults
        style=board.style or {},
        inherited_style=parent_context.get("style", {}),
        depth=depth,
        meta={
            "compiled_at": datetime.now(timezone.utc).isoformat(),
            "version": "0.1.0",
        },
    )

    # ════════════════════════════════════════════════════════════════════
    # STEP 3f: Build global variable registry (root board only)
    # ════════════════════════════════════════════════════════════════════
    if depth == 0:
        compiled_board.variable_registry = _build_variable_registry(compiled_board)

        # Validate that all referenced variables are defined
        warnings = _validate_variable_references(compiled_board)
        if warnings:
            compiled_board.meta["variable_warnings"] = warnings

    return compiled_board

Sizing¶

calculate_layout ¶

calculate_layout(board: CompiledBoard) -> CompiledBoard

Calculate dimensions for all layout items.

Stage: COMPILE (Step 4 of 4: Layout Calculation)

This is the final compilation step. It calculates pixel dimensions for all charts and nested boards based on layout type and nesting.

The sizing is content-aware: - KPIs get smaller heights (~100px) - Charts get standard heights (~300px) - Titles are measured for text wrapping - Nested boards accumulate their content heights

PARAMETER	DESCRIPTION
`board`	CompiledBoard with layout structure TYPE: `CompiledBoard`

RETURNS	DESCRIPTION
`CompiledBoard`	CompiledBoard with calculated dimensions on layout items

Example

board = normalize_board(parsed_board) board = calculate_layout(board) print(board.layout.width, board.layout.height) 1200.0 800.0

Source code in dataface/compile/sizing.py

def calculate_layout(board: CompiledBoard) -> CompiledBoard:
    """Calculate dimensions for all layout items.

    Stage: COMPILE (Step 4 of 4: Layout Calculation)

    This is the final compilation step. It calculates pixel dimensions
    for all charts and nested boards based on layout type and nesting.

    The sizing is content-aware:
    - KPIs get smaller heights (~100px)
    - Charts get standard heights (~300px)
    - Titles are measured for text wrapping
    - Nested boards accumulate their content heights

    Args:
        board: CompiledBoard with layout structure

    Returns:
        CompiledBoard with calculated dimensions on layout items

    Example:
        >>> board = normalize_board(parsed_board)
        >>> board = calculate_layout(board)
        >>> print(board.layout.width, board.layout.height)
        1200.0 800.0
    """
    config = get_default_config()

    container_width = float(config.board.width)
    gap = float(config.board.row_gap)
    min_height = float(config.board.min_height)

    # Calculate required height based on content (content-aware)
    container_height = _calculate_content_aware_height(
        board.layout,
        gap,
        min_height,
        container_width,
    )

    # Add height for root board title if present
    if board.title:
        title_height = estimate_title_height(board.title, container_width, board.depth)
        container_height += title_height + gap

    # Set root layout dimensions
    board.layout.width = container_width
    board.layout.height = container_height

    # Calculate all item dimensions in a single pass
    _calculate_layout_items(
        board.layout,
        container_width,
        container_height,
        gap,
    )

    return board

Compile Stage¶

Entry Points¶

compile¶

compile ¶

compile_file¶

compile_file ¶

CompileResult¶

CompileResult dataclass ¶

success property ¶

is_success property ¶

Compiled Types¶

CompiledBoard¶

CompiledBoard ¶

CompiledChart¶

CompiledChart ¶

validate_kpi ¶

LayoutItem¶

LayoutItem ¶

Query Types¶

SqlQuery¶

SqlQuery ¶

query_type property ¶

source_description property ¶

effective_source property ¶

CsvQuery¶

CsvQuery ¶

query_type property ¶

source_description property ¶

effective_file property ¶

HttpQuery¶

HttpQuery ¶

query_type property ¶

source_description property ¶

effective_url property ¶

MetricFlowQuery¶

MetricFlowQuery ¶

query_type property ¶

source_description property ¶

Input Types¶

Board¶

Board ¶

reject_dashboard_key classmethod ¶

validate_layout ¶

get_default_source ¶

Chart¶

Chart ¶

validate_kpi ¶

Query¶

Query ¶

Errors¶

errors ¶

CompilationError ¶

parse_error classmethod ¶

validation_error classmethod ¶

reference_error classmethod ¶

jinja_error classmethod ¶

normalization_error classmethod ¶

ParseError ¶

ValidationError ¶

ReferenceError ¶

JinjaError ¶

Internal Modules¶

Parser¶

parse_yaml ¶

Validator¶

validate_board ¶

Normalizer¶

normalize_board ¶

Sizing¶

calculate_layout ¶

CompileResult `dataclass` ¶

success `property` ¶

is_success `property` ¶

query_type `property` ¶

source_description `property` ¶

effective_source `property` ¶

query_type `property` ¶

source_description `property` ¶

effective_file `property` ¶

query_type `property` ¶

source_description `property` ¶

effective_url `property` ¶

query_type `property` ¶

source_description `property` ¶

reject_dashboard_key `classmethod` ¶

parse_error `classmethod` ¶

validation_error `classmethod` ¶

reference_error `classmethod` ¶

jinja_error `classmethod` ¶

normalization_error `classmethod` ¶