Skip to content

io

Utilities for reading and writing data to disk.

load_aman(obs_id, preprocess_cfg, dets, job, min_dets, logger, fp_flag=False, save=False)

Load and preprocess an observation.

Parameters:

Name Type Description Default
obs_id str

The obs_id to load.

 required
preprocess_cfg dict

The loaded preprocess configuration.

 required
dets dict

Detector selections dictionairy. Check your preprocess config to see what the minimum set of selections needed here are.

 required
job Job

The Job that we are loading this observation for. If we fail to load it then the job is marked as failed and the reason we couldn't load will be saved in the message tag.

 required
min_dets int

The minimum number of detectors allowed after preprocessing that we want. If fewer than min_dets detectors remain then the job is marked as failed. and None is returned.

 required
logger LoggerLike

Logger to log to when preprocessing. Note that the log level will be set to ERROR for preprocess.

 required
fp_flag bool

If True then keep only detectors with valid pointing.

 False
save bool

If True then try to save the preprocess result.

 False

Returns:

Name Type Description
aman Optional[AxisManager]

If we loaded and preprocessed successfully this is the loaded observation. If something failed this is None.
Source code in lat_beams/utils/io.py 
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
def load_aman(
    obs_id: str,
    preprocess_cfg: dict,
    dets: dict,
    job: jobdb.Job,
    min_dets: int,
    logger: LoggerLike,
    fp_flag: bool = False,
    save: bool = False,
) -> Optional[AxisManager]:
    """
    Load and preprocess an observation.

    Parameters
    ----------
    obs_id : str
        The `obs_id` to load.
    preprocess_cfg : dict
        The loaded preprocess configuration.
    dets : dict
        Detector selections dictionairy.
        Check your preprocess config to see what the minimum set of selections needed here are.
    job : jobdb.Job
        The `Job` that we are loading this observation for.
        If we fail to load it then the job is marked as failed and the reason we couldn't load
        will be saved in the `message` tag.
    min_dets : int
        The minimum number of detectors allowed after preprocessing that we want.
        If fewer than `min_dets` detectors remain then the job is marked as failed.
        and `None` is returned.
    logger : LoggerLike
        Logger to log to when preprocessing.
        Note that the log level will be set to `ERROR` for preprocess.
    fp_flag : bool, default: False
        If `True` then keep only detectors with valid pointing.
    save : bool, default: False
        If `True` then try to save the preprocess result.

    Returns
    -------
    aman : Optional[AxisManager]
        If we loaded and preprocessed successfully this is the loaded observation.
        If something failed this is `None`.
    """
    try:
        with log_lvl(logger, logging.ERROR):
            aman, _, _, err = preproc_or_load_group(
                obs_id,
                preprocess_cfg,
                dets=dets,
                save_archive=save,
                save_proc_aman=save,
                overwrite=True,
                logger=logger,
            )
    except Exception as e:
        msg = f"Failed to load or preprocess with error {e}"
        fail(job, ErrCode.PREPROC, msg, logger)
        return None
    if aman is None:
        msg = f"Preprocess failed with error {err}"
        fail(job, ErrCode.PREPROC, msg, logger)
        return None

    if fp_flag:
        aman.restrict(
            "dets",
            np.isfinite(aman.focal_plane.xi)
            * np.isfinite(aman.focal_plane.eta)
            * np.isfinite(aman.focal_plane.gamma),
        )

    if aman.dets.count < min_dets:
        msg = f"Only {aman.dets.count} dets!"
        fail(job, ErrCode.MIN_DETS, msg, logger)
        return None
    return aman