TimeRangePhenotype

Bases: Phenotype

As the name implies, TimeRangePhenotype is designed for working with time ranges. If the input data has a start and an end date, use TimeRangePhenotype to identify other events (or patients) that occur within this time range. The most common use case of this is working with 'health insurance coverage' data i.e. on 'OBSERVATION_PERIOD' table. These tables have one or many rows per patient with the start of coverage and end of coverage i.e. domains compatible with TimeRangePhenotype require a START_DATE and an END_DATE column. At it's simplest, TimeRangePhenotype identifies patients who have their INDEX_DATE (or other anchor date of interest) within this time range. Additionally, a minimum or maximum number of days from the anchor date to the beginning/end of the time range can be defined. The returned Phenotype has the following interpretation:

DATE: If relative_time_range.when='before', then DATE is the beginning of the coverage period containing the anchor phenotype. If relative_time_range.when='after', then DATE is the end of the coverage period containing the anchor date. VALUE: Coverage (in days) relative to the anchor date. By convention, always non-negative.

There are two primary use cases for TimeRangePhenotype

Identify patients with some minimum duration of coverage prior to anchor_phenotype date e.g. "identify patients with 1 year of continuous coverage prior to index date"
Determine the date of loss to followup (right censoring) i.e. the duration of coverage after the anchor_phenotype event

Data for TimeRangePhenotype

This phenotype requires a table with PersonID and a coverage start date and end date. Depending on the datasource used, this information is a separate ObservationPeriod table or found in the PersonTable. Use an PhenexObservationPeriodTable to map required coverage start and end date columns. For tables with overlapping time ranges, use the CombineOverlappingPeriods derived table to combine time ranges into a single time range.

PersonID	startDate	endDate
1	2009-01-01	2010-01-01
2	2008-01-01	2010-01-02

One assumption that is made by TimeRangePhenotype is that there are NO overlapping coverage periods.

Parameters:

Name	Type	Description	Default
`name`	`Optional[str]`	The name of the phenotype.	`'TIME_RANGE'`
`domain`	`Optional[str]`	The domain of the phenotype. Default is 'observation_period'.	`'OBSERVATION_PERIOD'`
`date_range`	`Optional[DateFilter]`	A DateFilter to apply. min_date clips START_DATE (periods starting before min_date are trimmed to min_date); max_date clips END_DATE (periods ending after max_date are trimmed to max_date). Periods entirely outside the range are excluded. VALUE is then computed on the clipped period.	`None`
`relative_time_range`	`Optional[RelativeTimeRangeFilter]`	Filter returned persons based on the duration of coverage in days. The relative_time_range.anchor_phenotype defines the reference date with respect to calculate coverage. In typical applications, the anchor phenotype will be the entry criterion. The relative_time_range.when 'before', 'after'. If before, the return date is the start of the coverage period containing the anchor_phenotype. If after, the return date is the end of the coverage period containing the anchor_phenotype.	`None`
`allow_null_end_date`	`bool`	TimeRangePhenotype checks that anchor date is within the time range of interest. This requires that the start date is not null, and the end date is either null or after the anchor date. If you want to require that the end date is not null, set allow_null_end_date to False.	`True`

Example:

# make sure to create an entry phenotype, for example 'atrial fibrillation diagnosis'
entry_phenotype = CodelistPhenotype(...)
# one year continuous coverage prior to index
one_year_coverage = TimeRangePhenotype(
    relative_time_range = RelativeTimeRangeFilter(
        min_days=GreaterThanOrEqualTo(365),
        anchor_phenotype = entry_phenotype,
        when = 'before',
    ),
)
# determine the date of loss to followup
loss_to_followup = TimeRangePhenotype(
    relative_time_range = RelativeTimeRangeFilter(
        anchor_phenotype = entry_phenotype
        when = 'after',
    )
)

# determine the date when a drug was discontinued
drug_discontinuation = TimeRangePhenotype(
    relative_time_range = RelativeTimeRangeFilter(
        anchor_phenotype = entry_phenotype
        when = 'after',
    )
)

Source code in phenex/phenotypes/time_range_phenotype.py

class TimeRangePhenotype(Phenotype):
    """
    As the name implies, TimeRangePhenotype is designed for working with time ranges. If the input data has a start and an end date, use TimeRangePhenotype to identify other events (or patients) that occur within this time range. The most common use case of this is working with 'health insurance coverage' data i.e. on 'OBSERVATION_PERIOD' table. These tables have one or many rows per patient with the start of coverage and end of coverage i.e. domains compatible with TimeRangePhenotype require a START_DATE and an END_DATE column. At it's simplest, TimeRangePhenotype identifies patients who have their INDEX_DATE (or other anchor date of interest) within this time range. Additionally, a minimum or maximum number of days from the anchor date to the beginning/end of the time range can be defined. The returned Phenotype has the following interpretation:

    DATE: If relative_time_range.when='before', then DATE is the beginning of the coverage period containing the anchor phenotype. If relative_time_range.when='after', then DATE is the end of the coverage period containing the anchor date.
    VALUE: Coverage (in days) relative to the anchor date. By convention, always non-negative.

    There are two primary use cases for TimeRangePhenotype:
        1. Identify patients with some minimum duration of coverage prior to anchor_phenotype date e.g. "identify patients with 1 year of continuous coverage prior to index date"
        2. Determine the date of loss to followup (right censoring) i.e. the duration of coverage after the anchor_phenotype event

    ## Data for TimeRangePhenotype
    This phenotype requires a table with PersonID and a coverage start date and end date. Depending on the datasource used, this information is a separate ObservationPeriod table or found in the PersonTable. Use an PhenexObservationPeriodTable to map required coverage start and end date columns. For tables with overlapping time ranges, use the CombineOverlappingPeriods derived table to combine time ranges into a single time range.

    | PersonID    |   startDate          |   endDate          |
    |-------------|----------------------|--------------------|
    | 1           |   2009-01-01         |   2010-01-01       |
    | 2           |   2008-01-01         |   2010-01-02       |

    One assumption that is made by TimeRangePhenotype is that there are **NO overlapping coverage periods**.

    Parameters:
        name: The name of the phenotype.
        domain: The domain of the phenotype. Default is 'observation_period'.
        date_range: A DateFilter to apply. min_date clips START_DATE (periods starting before min_date are trimmed to min_date); max_date clips END_DATE (periods ending after max_date are trimmed to max_date). Periods entirely outside the range are excluded. VALUE is then computed on the clipped period.
        relative_time_range: Filter returned persons based on the duration of coverage in days. The relative_time_range.anchor_phenotype defines the reference date with respect to calculate coverage. In typical applications, the anchor phenotype will be the entry criterion. The relative_time_range.when 'before', 'after'. If before, the return date is the start of the coverage period containing the anchor_phenotype. If after, the return date is the end of the coverage period containing the anchor_phenotype.
        allow_null_end_date: TimeRangePhenotype checks that anchor date is within the time range of interest. This requires that the start date is not null, and the end date is either null or after the anchor date. If you want to require that the end date is not null, set allow_null_end_date to False.

    Example:
    ```python
    # make sure to create an entry phenotype, for example 'atrial fibrillation diagnosis'
    entry_phenotype = CodelistPhenotype(...)
    # one year continuous coverage prior to index
    one_year_coverage = TimeRangePhenotype(
        relative_time_range = RelativeTimeRangeFilter(
            min_days=GreaterThanOrEqualTo(365),
            anchor_phenotype = entry_phenotype,
            when = 'before',
        ),
    )
    # determine the date of loss to followup
    loss_to_followup = TimeRangePhenotype(
        relative_time_range = RelativeTimeRangeFilter(
            anchor_phenotype = entry_phenotype
            when = 'after',
        )
    )

    # determine the date when a drug was discontinued
    drug_discontinuation = TimeRangePhenotype(
        relative_time_range = RelativeTimeRangeFilter(
            anchor_phenotype = entry_phenotype
            when = 'after',
        )
    )
    ```
    """

    output_display_type = "value"

    def __init__(
        self,
        name: Optional[str] = "TIME_RANGE",
        domain: Optional[str] = "OBSERVATION_PERIOD",
        date_range: Optional[DateFilter] = None,
        relative_time_range: Optional["RelativeTimeRangeFilter"] = None,
        allow_null_end_date: bool = True,
        **kwargs,
    ):
        super(TimeRangePhenotype, self).__init__(name=name, **kwargs)
        self.domain = domain
        self.date_range = date_range
        self.relative_time_range = relative_time_range
        self.allow_null_end_date = allow_null_end_date
        if self.relative_time_range is not None:
            if self.relative_time_range.anchor_phenotype is not None:
                self.add_children(self.relative_time_range.anchor_phenotype)

    def _execute(self, tables: Dict[str, Table]) -> PhenotypeTable:
        table = tables[self.domain]
        table = self._perform_anchor_join(table)
        table = self._perform_date_range_clipping(table)
        table = self._perform_anchor_containment_filtering(table)
        table = self._perform_value_date_assignment(table)
        table = self._perform_coverage_duration_filtering(table)
        return self._perform_final_processing(table)

    def _perform_anchor_join(self, table):
        """Join the domain table with the anchor phenotype to get the reference date column."""
        table, reference_column = attach_anchor_and_get_reference_date(
            table, self.relative_time_range.anchor_phenotype
        )
        self._reference_column = reference_column
        return table

    def _perform_date_range_clipping(self, table):
        """Clip START_DATE to min_date and END_DATE to max_date, then exclude periods that fall entirely outside the range."""
        if self.date_range is None:
            return table
        if self.date_range.min_value is not None:
            min_date = self.date_range.min_value.value
            table = table.mutate(
                START_DATE=ibis.greatest(table.START_DATE, ibis.literal(min_date))
            )
        if self.date_range.max_value is not None:
            max_date = self.date_range.max_value.value
            table = table.mutate(
                END_DATE=ibis.least(table.END_DATE, ibis.literal(max_date))
            )
        return table.filter(table.START_DATE <= table.END_DATE)

    def _perform_anchor_containment_filtering(self, table):
        """Keep only periods that contain the anchor date. Null END_DATE is treated as ongoing if allow_null_end_date is True."""
        ref = self._reference_column
        if self.allow_null_end_date:
            return table.filter(
                (table.START_DATE <= ref)
                & ((ref <= table.END_DATE) | table.END_DATE.isnull())
            )
        return table.filter((table.START_DATE <= ref) & (ref <= table.END_DATE))

    def _perform_value_date_assignment(self, table):
        """Assign VALUE (coverage days) and EVENT_DATE based on the direction of the relative time range."""
        ref = self._reference_column
        if (
            self.relative_time_range is None
            or self.relative_time_range.when == "before"
        ):
            value = ref.cast("date").delta(table.START_DATE.cast("date"), "day")
            event_date = table.START_DATE
        else:
            value = table.END_DATE.cast("date").delta(ref.cast("date"), "day")
            event_date = table.END_DATE
        return table.mutate(VALUE=value, EVENT_DATE=event_date)

    def _perform_coverage_duration_filtering(self, table):
        """Filter by min_days / max_days from the relative time range."""
        if self.relative_time_range is None:
            return table
        value_filter = ValueFilter(
            min_value=self.relative_time_range.min_days,
            max_value=self.relative_time_range.max_days,
            column_name="VALUE",
        )
        return value_filter.filter(table)

`dependencies` `property`

Recursively collect all dependencies of a node (including dependencies of dependencies).

Returns:

Type	Description
`Set[Node]`	List[Node]: A list of Node objects on which this Node depends.

`dependency_graph` `property`

Build a dependency graph where each node maps to its direct dependencies (children).

Returns:

Type	Description
`Dict[Node, Set[Node]]`	Dict[Node, Set[Node]: A mapping of Node's to their children Node's.

`execution_metadata` `property`

Retrieve the execution metadata rows for this node from phenex.db.

Returns:

Type	Description
	pandas.DataFrame: One row per database the node ran against, or None if it has never been executed. Columns: NODE_NAME (its table name), NODE_HASH (fingerprint of the node's settings), EXECUTION_PARAMS (where it ran: connector type, source and destination database), EXECUTION_ID, EXECUTION_START_TIME, EXECUTION_END_TIME, EXECUTION_DURATION, NODE_SQL (the query that built its table).

`namespaced_table` `property`

A PhenotypeTable has generic column names 'person_id', 'boolean', 'event_date', and 'value'. The namespaced_table prepends the phenotype name to all of these columns. This is useful when joining multiple phenotype tables together.

Returns:

Name	Type	Description
`table`	`Table`	The namespaced table for the current phenotype.

`reverse_dependency_graph` `property`

Build a reverse dependency graph where each node maps to nodes that depend on it (parents).

Returns:

Type	Description
`Dict[Node, Set[Node]]`	Dict[Node, List[Node]: A mapping of Node's to their parent Node's.

`table = None` `instance-attribute` `property` `writable`

This step's result table. After load(), the first read fetches it from the database and keeps it.

`unique_name` `property`

The name of this node's table. A Cohort sets it to the phenotype's place, e.g. OUTCOMES__BLEEDING__CHILD_1. Otherwise it is name.

`attach_lazy_table(con, db_name)`

Note where this step's saved table lives, without reading it. The first read of table fetches it. Used by load().

Source code in phenex/node.py

def attach_lazy_table(self, con: "DatabaseConnector", db_name: str) -> None:
    """Note where this step's saved table lives, without reading it.
    The first read of `table` fetches it. Used by load()."""
    self._table = None

    def _load() -> Optional[Table]:
        try:
            return con.get_dest_table(db_name)
        except Exception as e:
            logger.warning(
                f"Node '{self.name}': could not attach table '{db_name}': {e}"
            )
            return None

    self._table_loader = _load

`clear_cache(con=None, recursive=False)`

Clear the cached state for this node, forcing re-execution on the next call to execute().

This method removes the node's hash from the node states table and optionally drops the materialized table from the database. After calling this method, the node will be treated as if it has never been executed before.

Parameters:

Name	Type	Description	Default
`con`	`Optional[object]`	Database connector. If provided, clears only runs with matching execution context and drops the materialized table. If None, clears all runs for the node.	`None`
`recursive`	`bool`	If True, also clear the cache for all child nodes recursively. Defaults to False.	`False`

Example

# Clear all cached runs for a single node
my_node.clear_cache()

# Clear runs with specific execution context and drop materialized table
my_node.clear_cache(con=my_connector)

# Clear cache for node and all its dependencies
my_node.clear_cache(recursive=True)

Source code in phenex/node.py

def clear_cache(self, con: Optional[object] = None, recursive: bool = False):
    """
    Clear the cached state for this node, forcing re-execution on the next call to execute().

    This method removes the node's hash from the node states table and optionally drops the materialized table from the database. After calling this method, the node will be treated as if it has never been executed before.

    Parameters:
        con: Database connector. If provided, clears only runs with matching execution context and drops the materialized table. If None, clears all runs for the node.
        recursive: If True, also clear the cache for all child nodes recursively. Defaults to False.

    Example:
        ```python
        # Clear all cached runs for a single node
        my_node.clear_cache()

        # Clear runs with specific execution context and drop materialized table
        my_node.clear_cache(con=my_connector)

        # Clear cache for node and all its dependencies
        my_node.clear_cache(recursive=True)
        ```
    """
    # Delegate all logic to NodeManager
    return Node._node_manager.clear_cache(self, con=con, recursive=recursive)

`copy()`

Return a deep copy of this node with _table_name_prefix cleared on all nodes.

Source code in phenex/node.py

def copy(self):
    """Return a deep copy of this node with _table_name_prefix cleared on all nodes."""
    clone = copy.deepcopy(self)
    clone._table_name_prefix = None
    for dep in clone.dependencies:
        dep._table_name_prefix = None
    return clone

`delete_table(con, table_name_prefix=None)`

Delete this node's materialized table from the destination database.

Parameters:

Name	Type	Description	Default
`con`		Database connector.	required
`table_name_prefix`	`Optional[str]`	Optional prefix. If None, uses the prefix from the last execution.	`None`

Returns:

Name	Type	Description
`bool`	`bool`	True if the table was dropped, False if it didn't exist.

Source code in phenex/node.py

def delete_table(self, con, table_name_prefix: Optional[str] = None) -> bool:
    """
    Delete this node's materialized table from the destination database.

    Parameters:
        con: Database connector.
        table_name_prefix: Optional prefix. If None, uses the prefix from the last execution.

    Returns:
        bool: True if the table was dropped, False if it didn't exist.
    """
    db_name = self.get_table_name(table_name_prefix)
    try:
        con.drop_table(db_name)
        logger.info(f"Node '{self.name}': dropped table '{db_name}'")
        return True
    except Exception as e:
        logger.warning(f"Node '{self.name}': could not drop table '{db_name}': {e}")
        return False

`execute(tables=None, con=None, overwrite=False, lazy_execution=False, n_threads=1, table_name_prefix=None)`

Executes the Node computation for the current node and its dependencies.

Lazy Execution

When lazy_execution=True, nodes are only recomputed if changes are detected. The system tracks: 1. Node definition changes: Detected by hashing the node's parameters (from to_dict()) and class name 2. Execution environment changes: Detected by tracking source/destination database configurations

A node will be rerun if either: - The node's defining parameters have changed (different hash than last execution) - The database connector's source or destination databases have changed - The node has never been executed before

If no changes are detected, the node uses its cached result from the database instead of recomputing.

Requirements for lazy execution: - A database connector (con) must be provided to store and retrieve cached results - overwrite=True must be set to allow updating existing cached tables

State tracking is maintained in a local DuckDB database (__PHENEX_META__NODE_STATES table) that stores: - Node hashes, parameters, and execution metadata - Database connector configuration used during execution - Execution timing information

Parameters:

Name	Type	Description	Default
`tables`	`Dict[str, Table]`	A dictionary mapping domains to Table objects.	`None`
`con`	`Optional[object]`	Connection to database for materializing outputs. If provided, outputs from the node and all children nodes will be materialized (written) to the database using the connector. Required for lazy_execution.	`None`
`overwrite`	`bool`	If True, will overwrite any existing tables found in the database while writing. If False, will throw an error when an existing table is found. Has no effect if con is not passed. Must be True when using lazy_execution.	`False`
`lazy_execution`	`bool`	If True, only re-executes nodes when changes are detected in either the node definition or execution environment. Defaults to False. Requires con to be provided.	`False`
`n_threads`	`int`	Max number of Node's to execute simultaneously when this node has multiple children.	`1`

Returns:

Name	Type	Description
`Table`	`Table`	The resulting table for this node. Also accessible through self.table after calling self.execute().

Raises:

Type	Description
`ValueError`	If lazy_execution=True but overwrite=False or con=None.

Source code in phenex/node.py

def execute(
    self,
    tables: Dict[str, Table] = None,
    con: Optional[object] = None,
    overwrite: bool = False,
    lazy_execution: bool = False,
    n_threads: int = 1,
    table_name_prefix: Optional[str] = None,
) -> Table:
    """
    Executes the Node computation for the current node and its dependencies.

    Lazy Execution:
        When lazy_execution=True, nodes are only recomputed if changes are detected. The system tracks:
        1. Node definition changes: Detected by hashing the node's parameters (from to_dict()) and class name
        2. Execution environment changes: Detected by tracking source/destination database configurations

        A node will be rerun if either:
        - The node's defining parameters have changed (different hash than last execution)
        - The database connector's source or destination databases have changed
        - The node has never been executed before

        If no changes are detected, the node uses its cached result from the database instead of recomputing.

        Requirements for lazy execution:
        - A database connector (con) must be provided to store and retrieve cached results
        - overwrite=True must be set to allow updating existing cached tables

        State tracking is maintained in a local DuckDB database (__PHENEX_META__NODE_STATES table) that stores:
        - Node hashes, parameters, and execution metadata
        - Database connector configuration used during execution
        - Execution timing information

    Parameters:
        tables: A dictionary mapping domains to Table objects.
        con: Connection to database for materializing outputs. If provided, outputs from the node and all children nodes will be materialized (written) to the database using the connector. Required for lazy_execution.
        overwrite: If True, will overwrite any existing tables found in the database while writing. If False, will throw an error when an existing table is found. Has no effect if con is not passed. Must be True when using lazy_execution.
        lazy_execution: If True, only re-executes nodes when changes are detected in either the node definition or execution environment. Defaults to False. Requires con to be provided.
        n_threads: Max number of Node's to execute simultaneously when this node has multiple children.

    Returns:
        Table: The resulting table for this node. Also accessible through self.table after calling self.execute().

    Raises:
        ValueError: If lazy_execution=True but overwrite=False or con=None.
    """
    if table_name_prefix:
        table_name_prefix = re.sub(r"[^A-Za-z0-9_]", "_", table_name_prefix).upper()

    # Store prefix on all nodes so get_table_name() works after execution
    all_deps = self.dependencies
    for dep in all_deps:
        dep._table_name_prefix = table_name_prefix
    self._table_name_prefix = table_name_prefix

    # Handle None tables
    if tables is None:
        tables = {}
    else:
        # Work on a shallow copy so publishing completed nodes' tables (below)
        # for intra-execution sibling lookups never mutates the caller's dict,
        # which is reused as subset_tables_entry/index (domain -> PhenexTable).
        tables = dict(tables)

    # Build dependency graph for all dependencies
    nodes = self._graph_nodes()

    # Build dependency and reverse graphs
    dependency_graph = self._build_dependency_graph(nodes)
    reverse_graph = self._build_reverse_graph(dependency_graph)

    # Track completion status and results
    completed = set()
    completion_lock = threading.Lock()
    # Guards reads (snapshotting) and writes of the shared `tables` dict so a node's
    # _execute() never iterates it while another thread concurrently inserts into it.
    tables_lock = threading.Lock()
    worker_exceptions = []  # Track exceptions from worker threads
    stop_all_workers = (
        threading.Event()
    )  # Signal to stop all workers on first error

    # Track in-degree for scheduling
    in_degree = {}
    for node_name, dependencies in dependency_graph.items():
        in_degree[node_name] = len(dependencies)
    for node_name in nodes:
        if node_name not in in_degree:
            in_degree[node_name] = 0

    # Queue for nodes ready to execute
    ready_queue = queue.Queue()

    # Add nodes with no dependencies to ready queue
    for node_name, degree in in_degree.items():
        if degree == 0:
            ready_queue.put(node_name)

    def _restore_phenex_wrapper(materialized, original):
        # Re-wrap: get_dest_table() strips PhenexTable metadata, restore the original subclass.
        if isinstance(original, PhenexTable) and not isinstance(
            materialized, PhenexTable
        ):
            return type(original)(
                materialized, name=original.NAME_TABLE, column_mapping={}
            )
        return materialized

    def _run_and_materialise(node, node_name, node_tables):
        """Execute *node*, materialise the result, record timing, and update the run hash."""
        db_name = node.get_table_name(table_name_prefix)
        node.lastexecution_start_time = datetime.now()
        table = node._execute(node_tables)
        # Save the generating query before create_table() materializes it (to_sql layer 1).
        node._expression = table
        if table is not None:
            original = table
            con.create_table(table, db_name, overwrite=overwrite)
            table = _restore_phenex_wrapper(con.get_dest_table(db_name), original)
        node.lastexecution_end_time = datetime.now()
        node.lastexecution_duration = (
            node.lastexecution_end_time - node.lastexecution_start_time
        ).total_seconds()
        Node._node_manager.update_run_params(node, con)
        return table

    def worker():
        """Worker function for thread pool"""
        while not stop_all_workers.is_set():
            try:
                node_name = ready_queue.get(timeout=1)
                # timeout forces to wait 1 second to avoid busy waiting
                if node_name is None:  # Sentinel value to stop worker
                    break
            except queue.Empty:
                continue

            try:
                logger.info(
                    f"Thread {threading.current_thread().name}: executing node '{node_name}'"
                )
                node = nodes[node_name]
                active_display().node_started(node_name)
                served_from_cache = False

                # Snapshot `tables` once per node so this node's _execute() (which may
                # iterate the dict, e.g. LoadTable) never races with another thread
                # inserting a newly-completed sibling's table into the shared dict.
                with tables_lock:
                    node_tables = dict(tables)

                # Execute the node (without recursive child execution since we handle dependencies here)
                if lazy_execution:
                    if not overwrite:
                        raise ValueError(
                            "lazy_execution only works with overwrite=True."
                        )
                    if con is None:
                        raise ValueError(
                            "A DatabaseConnector is required for lazy execution."
                        )

                    # Sampler nodes write no dest table (the SAMPLER_STAGE group returns
                    # None; a fraction=1.0 node is a no-op via _skip_cache). Skip the cache
                    # for them — with no table to look up they would perpetually MISS.
                    if isinstance(node, NodeGroup) or getattr(
                        node, "_skip_cache", False
                    ):
                        table = node._execute(node_tables)
                    elif Node._node_manager.should_rerun(node, con):
                        table = _run_and_materialise(node, node_name, node_tables)
                    else:
                        db_name = node.get_table_name(table_name_prefix)
                        try:
                            table = con.get_dest_table(db_name)
                            served_from_cache = True
                        except Exception:
                            # Cached table was dropped or is inaccessible; recompute.
                            logger.warning(
                                f"Cached table for '{node_name}' not found at {db_name}; recomputing."
                            )
                            table = _run_and_materialise(
                                node, node_name, node_tables
                            )
                else:
                    # Time the execution
                    node.lastexecution_start_time = datetime.now()
                    table = node._execute(node_tables)
                    # Save the generating query before create_table() materializes it (to_sql layer 1).
                    node._expression = table

                    if (
                        con and table is not None
                    ):  # Only create table if _execute returns something
                        original = table
                        db_name = node.get_table_name(table_name_prefix)
                        logger.info(
                            f"Thread {threading.current_thread().name}: materializing '{node_name}' to database ..."
                        )
                        _t_mat = datetime.now()
                        con.create_table(table, db_name, overwrite=overwrite)
                        logger.info(
                            f"Thread {threading.current_thread().name}: materialized '{node_name}' "
                            f"in {(datetime.now() - _t_mat).total_seconds():.3f}s"
                        )
                        table = _restore_phenex_wrapper(
                            con.get_dest_table(db_name), original
                        )

                    node.lastexecution_end_time = datetime.now()
                    node.lastexecution_duration = (
                        node.lastexecution_end_time - node.lastexecution_start_time
                    ).total_seconds()

                node.table = table
                # Publish immediately (not only after the whole graph finishes) so
                # sibling nodes depending on this domain (declared via add_children)
                # can look it up in `tables` during the same execute() call.
                # Only DerivedTable nodes register a domain: ordinary nodes
                # (phenotypes, reporters, subset tables) must never be added to the
                # domains dict, e.g. a phenotype named 'DEATH' must not overwrite the
                # 'DEATH' source domain that later subset nodes still read from.
                if table is not None and isinstance(node, DerivedTable):
                    with tables_lock:
                        tables[node.name] = table

                with completion_lock:
                    completed.add(node_name)

                    # Update in-degree for dependent nodes and add ready ones to queue
                    for dependent in reverse_graph.get(node_name, set()):
                        in_degree[dependent] -= 1
                        if in_degree[dependent] == 0:
                            # Check if all dependencies are completed
                            deps_completed = all(
                                dep in completed
                                for dep in dependency_graph.get(dependent, set())
                            )
                            if deps_completed:
                                ready_queue.put(dependent)

                # Log completion with timing info
                if (
                    not served_from_cache
                    and node.lastexecution_duration is not None
                ):
                    logger.info(
                        f"Thread {threading.current_thread().name}: completed node '{node_name}' "
                        f"in {node.lastexecution_duration:.3f} seconds"
                    )
                else:
                    logger.info(
                        f"Thread {threading.current_thread().name}: completed node '{node_name}' (cached)"
                    )

                active_display().node_finished(
                    node_name,
                    served_from_cache,
                    duration=node.lastexecution_duration,
                )

            except Exception as e:
                logger.error(f"Error executing node '{node_name}': {str(e)}")
                with completion_lock:
                    # Store exception for main thread
                    worker_exceptions.append(e)
                    # Signal all workers to stop immediately and exit worker loop
                    stop_all_workers.set()
                    break
            finally:
                ready_queue.task_done()

    # Start worker threads
    threads = []
    for i in range(min(n_threads, len(nodes))):
        thread = threading.Thread(target=worker, name=f"PhenexWorker-{i}")
        thread.daemon = True
        thread.start()
        threads.append(thread)

    # Wait for all nodes to complete or for an error to occur
    _last_heartbeat = datetime.now()
    while (
        len(completed) < len(nodes)
        and not worker_exceptions
        and not stop_all_workers.is_set()
    ):
        threading.Event().wait(0.1)  # Small delay to prevent busy waiting
        _now = datetime.now()
        if (_now - _last_heartbeat).total_seconds() >= 30:
            _pending = sorted(set(nodes.keys()) - completed)
            logger.info(
                f"Node '{self.name}': still waiting for {len(_pending)} nodes: {_pending}"
            )
            _last_heartbeat = _now

    if not stop_all_workers.is_set():
        # Time to stop workers and cleanup
        stop_all_workers.set()

    # Check if any worker thread had an exception
    if worker_exceptions:
        # Signal workers to stop
        for _ in threads:
            ready_queue.put(None)
        # Wait for threads to finish
        for thread in threads:
            thread.join(timeout=1)
        # Re-raise the first exception
        raise worker_exceptions[0]

    # Signal workers to stop and wait for them
    for _ in threads:
        ready_queue.put(None)  # Sentinel value to stop workers

    for thread in threads:
        thread.join(timeout=1)

    logger.info(
        f"Node '{self.name}': completed multithreaded execution of {len(nodes)} nodes"
    )
    return self.table

`get_sql_filename()`

Filesystem-safe .sql filename for this node's table.

<>:"/\|?* become _, plus a short hash when that changed the name so two names that clean up alike never collide. Filenames longer than 250 characters (including .sql) are truncated with the same hash suffix, staying below the common 255-character per-filename filesystem limit. Only the filename changes, not the SQL.

Source code in phenex/node.py

def get_sql_filename(self) -> str:
    """Filesystem-safe `.sql` filename for this node's table.

    `<>:"/\\|?*` become `_`, plus a short hash when that changed the name so two
    names that clean up alike never collide. Filenames longer than 250 characters
    (including `.sql`) are truncated with the same hash suffix, staying below the
    common 255-character per-filename filesystem limit. Only the filename changes,
    not the SQL.
    """
    MAX_FILENAME_LEN = 250  # includes ".sql"
    name = self.get_table_name()
    safe = re.sub(r'[<>:"/\\|?*]', "_", name)
    if safe != name:
        safe = f"{safe}__{hashlib.md5(name.encode()).hexdigest()[:8]}"
    if len(safe) + len(".sql") > MAX_FILENAME_LEN:
        digest = hashlib.md5(name.encode()).hexdigest()[:8]
        keep = MAX_FILENAME_LEN - len(".sql") - len("__") - len(digest)
        safe = f"{safe[:keep]}__{digest}"
    return f"{safe}.sql"

`get_table_name(table_name_prefix=None)`

Get the table name used in the destination database for this node.

Parameters:

Name	Type	Description	Default
`table_name_prefix`	`Optional[str]`	Optional prefix. If None, uses the prefix from the last execution.	`None`

Returns:

Name	Type	Description
`str`	`str`	The table name in the destination database.

Source code in phenex/node.py

def get_table_name(self, table_name_prefix: Optional[str] = None) -> str:
    """
    Get the table name used in the destination database for this node.

    Parameters:
        table_name_prefix: Optional prefix. If None, uses the prefix from the last execution.

    Returns:
        str: The table name in the destination database.
    """
    prefix = table_name_prefix or self._table_name_prefix
    if prefix:
        prefix = re.sub(r"[^A-Za-z0-9_]", "_", prefix).upper()
        # a phenotype named by its place always gets the prefix
        if self._unique_name is not None or not self.name.startswith(prefix):
            return f"{prefix}__{self.unique_name}"
    return self.unique_name

`to_sql(sql_dir='./sql', connector=None)`

Return the SQL that produces this node's table, read-only (execute() writes it).

Returns the first source available, in order: the in-memory expression (this session), the saved {sql_dir}/{name}.sql file, the phenex.db cache, or a recompile from dependency tables. Pass connector to pin the dialect.

Raises:

Type	Description
`RuntimeError`	If required upstream tables are missing, call `execute()` first.

Source code in phenex/node.py

def to_sql(self, sql_dir: str = "./sql", connector=None) -> str:
    """Return the SQL that produces this node's table, read-only (execute() writes it).

    Returns the first source available, in order: the in-memory expression (this
    session), the saved `{sql_dir}/{name}.sql` file, the `phenex.db` cache, or a
    recompile from dependency tables. Pass `connector` to pin the dialect.

    Raises:
        RuntimeError: If required upstream tables are missing, call `execute()` first.
    """
    # Explicit compile target when a connector is given; else each layer self-identifies.
    target_dialect = (
        ibis_dialect_of_connector(connector) if connector is not None else None
    )
    name = self.get_table_name()  # cohort-prefixed; matches the written filename

    # 1. in-memory expression: compile + stamp in the target (or its own backend) dialect.
    if self._expression is not None:
        logger.info(f"to_sql('{name}') ← memory (this session)")
        return compile_sql(self._expression, dialect=target_dialect)

    # 2. saved file, keyed by name (cohort-prefixed) to match the written filename.
    # sql_dir=None means "no directory to read from", skip straight to the cache.
    if sql_dir is not None:
        filepath = os.path.join(sql_dir, self.get_sql_filename())
        if os.path.exists(filepath):
            file_sql = read_sql_file(filepath)
            # Use the file unless it's stamped for a different backend (wrong dialect).
            stamp = read_dialect_stamp(file_sql)
            if target_dialect is None or stamp is None or stamp == target_dialect:
                logger.info(f"to_sql('{name}') ← file: {filepath}")
                return file_sql

    # 3. phenex.db SQL dialect-aware with a connector (only its SQL), else hash-only lookup.
    try:
        sql = Node._node_manager.get_sql(self, con=connector)
    except Exception:
        sql = None
    if sql is not None:
        logger.info(f"to_sql('{name}') ← phenex.db cache")
        return sql

    # 4. auto-collect dependency tables and recompile
    collected = {dep.name: dep.table for dep in self.dependencies}
    missing = [dep_name for dep_name, tbl in collected.items() if tbl is None]
    if missing:
        raise RuntimeError(
            f"Node '{self.name}' cannot compile SQL. Missing upstream tables: "
            f"{', '.join(missing)}. Call `load()` or `execute()` first."
        )
    # KeyError = a missing source table -> clear error; a real node bug -> re-raise.
    try:
        # Recompiled from dependency tables: compile + stamp in the target (or backend) dialect.
        recompiled = compile_sql(self._execute(collected), dialect=target_dialect)
        logger.info(
            f"to_sql('{self.get_table_name()}') ← recompiled from dependencies"
        )
        return recompiled
    except KeyError as e:
        key = e.args[0]
        if key not in collected:
            raise RuntimeError(
                f"Node '{self.name}' cannot compile SQL. "
                f"Missing source-domain table: '{key}'. "
                "Call `load()` or `execute()` first."
            ) from e
        raise

`visualize_dependencies()`

Create a text visualization of the dependency graph for this node and its dependencies.

Returns:

Name	Type	Description
`str`	`str`	A text representation of the dependency graph

Source code in phenex/node.py

def visualize_dependencies(self) -> str:
    """
    Create a text visualization of the dependency graph for this node and its dependencies.

    Returns:
        str: A text representation of the dependency graph
    """
    lines = [f"Dependencies for Node '{self.name}':"]

    nodes = self._graph_nodes()

    # Build dependency graph
    dependency_graph = self._build_dependency_graph(nodes)

    for node_name in sorted(nodes.keys()):
        dependencies = dependency_graph.get(node_name, set())
        if dependencies:
            deps_str = ", ".join(sorted(dependencies))
            lines.append(f"  {node_name} depends on: {deps_str}")
        else:
            lines.append(f"  {node_name} (no dependencies)")

    return "\n".join(lines)

TimeRangePhenotype

Data for TimeRangePhenotype

dependencies property

dependency_graph property

execution_metadata property

namespaced_table property

reverse_dependency_graph property

table = None instance-attribute property writable

unique_name property

attach_lazy_table(con, db_name)

clear_cache(con=None, recursive=False)

copy()

delete_table(con, table_name_prefix=None)

execute(tables=None, con=None, overwrite=False, lazy_execution=False, n_threads=1, table_name_prefix=None)

get_sql_filename()

get_table_name(table_name_prefix=None)

to_sql(sql_dir='./sql', connector=None)

visualize_dependencies()

`dependencies` `property`

`dependency_graph` `property`

`execution_metadata` `property`

`namespaced_table` `property`

`reverse_dependency_graph` `property`

`table = None` `instance-attribute` `property` `writable`

`unique_name` `property`

`attach_lazy_table(con, db_name)`

`clear_cache(con=None, recursive=False)`

`copy()`

`delete_table(con, table_name_prefix=None)`

`execute(tables=None, con=None, overwrite=False, lazy_execution=False, n_threads=1, table_name_prefix=None)`

`get_sql_filename()`

`get_table_name(table_name_prefix=None)`

`to_sql(sql_dir='./sql', connector=None)`

`visualize_dependencies()`