Parsing and Semantic Validation

Parsing Validation ensures the config files you've authored conform to the schema MetricFlow expects. Semantic validations are a set of arbitrary rules that run against your model to make sure your model is correct. This page outlines the validation rules that run as part of Semantic Validation and the schemas MetricFlow expects during Parsing Validation for various core objects.

This page is organized by object type, so you can easily reference which rules apply to which object.

Data Source Validation

We run the following validation checks to ensure that your data source configs are correct.

1. Schema validation: We ensure that your data source YAML conforms to the following schema.

data_source_schema = {
    "$id": "data_source",
    "type": "object",
    "properties": {
        "name": {
            "type": "string",
            "pattern": TRANSFORM_OBJECT_NAME_PATTERN,
        },
        "sql_table": {"type": "string"},
        "sql_query": {"type": "string"},
        "dbt_model": {"type": "string"},
        "identifiers": {"type": "array", "items": {"$ref": "identifier_schema"}},
        "measures": {"type": "array", "items": {"$ref": "measure_schema"}},
        "dimensions": {"type": "array", "items": {"$ref": "dimension_schema"}},
        "mutability": {"$ref": "mutability_schema"},
        "constraint": {"$ref": "constraint_schema"},
    },
    "additionalProperties": False,
    "required": ["name", "mutability"],
}

2. Check that Data Sources with measures have a valid primary time dimension: If you define measures in your data sources, you must have a primary time dimension defined in that data source.

3. Check that there is only one primary identifier defined in each data source: Each data source can only have one primary identifier

Identifier Validation

We run the following validation checks to ensure that your Identifiers are specified correctly.

1. Schema validation: We ensure that your Identifiers conform to the following schema.

identifier_schema = {
    "$id": "identifier_schema",
    "type": "object",
    "properties": {
        "name": {
            "type": "string",
            "pattern": TRANSFORM_OBJECT_NAME_PATTERN,
        },
        "type": {"enum": identifier_type_enum_values},
        "role": {"type": "string"},
        "expr": {"type": ["string", "boolean"]},
        "entity": {"type": "string"},
        "identifiers": {
            "type": "array",
            "items": {"$ref": "composite_sub_identifier_schema"},
        },
    },
    "additionalProperties": False,
    "required": ["name", "type"],
}

2. Check validity of composite identifiers: Composite identifiers must be defined with a ref tag or name/expr but, not both. If using ref you must reference an existing identifier.

3. Check that identifiers with the same name are defined with the same set of sub-identifiers across data sources: If identifiers have the same name and are defined with sub-identifiers, then the sub-identifiers should be the same.

Dimension Validation

1. Schema validation: We ensure that your Dimensions conform to the following schema.

dimension_schema = {
    "$id": "dimension_schema",
    "type": "object",
    "properties": {
        "name": {
            "type": "string",
            "pattern": TRANSFORM_OBJECT_NAME_PATTERN,
        },
        "type": {"enum": dimension_type_values},
        "is_partition": {"type": "boolean"},
        "expr": {"type": ["string", "boolean"]},
        "type_params": {"$ref": "dimension_type_params_schema"},
        
    },
    "additionalProperties": False,
    "required": ["name", "type"],

dimension_type_params_schema = {
    "$id": "dimension_type_params_schema",
    "type": "object",
    "properties": {
        "is_primary": {"type": "boolean"},
        "time_format": {"type": "string"},
        "time_granularity": {"enum": time_granularity_values},
    },
    "additionalProperties": False,
    "required": ["time_granularity"],
}

2. Dimension Consistency: Checks that dimensions with the same name are of the same type. Checks that time dimensions have consistent is_primary designations.

Measure Validation

1. Schema Validation: We ensure that your Measures conform to the following schema.

measure_schema = {
    "$id": "measure_schema",
    "type": "object",
    "properties": {
        "name": {
            "type": "string",
            "pattern": TRANSFORM_OBJECT_NAME_PATTERN,
        },
        "agg": {"enum": aggregation_type_values},
        "expr": {"type": ["string", "integer", "boolean"]},
        "create_metric": {"type": "boolean"},
        "create_metric_display_name": {"type": "string"},
    },
    "additionalProperties": False,
    "required": ["name", "agg"],
}

2. Unique Measures in Data Sources: Measures need to have unique names across the data sources in your model.

Metric Validation

1. Schema Validation: We ensure that your Metrics conform to the following schema.

metric_schema = {
    "$id": "metric",
    "type": "object",
    "properties": {
        "name": {
            "type": "string",
            "pattern": TRANSFORM_OBJECT_NAME_PATTERN,
        },
        "type": {"enum": metric_types_enum_values},
        "type_params": {"$ref": "metric_type_params"},
        "constraint": {"type": "string"},
        "where_constraint": {"type": "string"},
    },
    "additionalProperties": False,
    "required": ["name", "type", "type_params"],

metric_type_params_schema = {
    "$id": "metric_type_params",
    "type": "object",
    "properties": {
        "numerator": {"type": "string"},
        "denominator": {"type": "string"},
        "measure": {"type": "string"},
        "measures": {
            "type": "array",
            "items": {"type": "string"},
        },
        "expr": {"type": ["string", "boolean"]},
        "window": {"type": "string"},
        "grain_to_date": {"type": "string"},
    },
    "additionalProperties": False,
}

2. Measures in metrics are valid: Checks that measures referenced in metrics exist
3. Cumulative metrics are configured properly: Checks that the metric parameters are valid for cumulative metrics