Open In Colab

astrohack

VLA Holography Tutorial

Important External Information

  • xarray Official Documentation (docs).

  • Dask Official Documentation (docs).

  • zarr Official Documentation (docs)

[1]:
import os

try:
    import astrohack

    print("AstroHACK version", astrohack.__version__, "already installed.")
except ImportError as e:
    print(e)
    print("Installing AstroHACK")

    os.system("pip install astrohack")

    import astrohack

    print("astrohack version", astrohack.__version__, " installed.")
AstroHACK version 1.0.1 already installed.

Download Tutorial Data

[2]:
import toolviper

toolviper.utils.data.download(file="ea25_cal_small_after_fixed.split.ms", folder="data")
[2026-03-19 15:38:25,643]     INFO   astrohack:  Module path: /export/home/arya/miniforge3/envs/casadev/lib/python3.12/site-packages/toolviper
[2026-03-19 15:38:25,646]     INFO   astrohack:  Downloading from [cloudflare] ....
  Download List                       
 ─────────────────────────────────────
  ea25_cal_small_after_fixed.split.ms

[2026-03-19 15:38:25,650]     INFO   astrohack:  Creating path:/export/home/arya/work/Holography-1022/astrohack/docs/tutorials/data

Holography Data File API

As part of the astroHACK API a set of functions to allow users to easily open on disk holography files has been provided. Each function takes an astroHACK holography file name as an argument and returns an object related to the given file type, i.e. holog, image, panel, point. Each object allows the user to access data via dictionary keys with values consisting of the relevant holography dataset. Each object also provides a summary() helper function to list available keys for each file. An example call for each file type is show below and the API documentation for all data-io functions can be found here.

from astrohack import open_holog
from astrohack import open_image
from astrohack import open_panel
from astrohack import open_pointing

holog_data = open_holog('./data/ea25_cal_small_spw1_4_60_ea04_after.holog.zarr')
image_data = open_image('./data/ea25_cal_small_spw1_4_60_ea04_after.image.zarr')
panel_data = open_panel('./data/ea25_cal_small_spw1_4_60_ea04_after.panel.zarr')
pointing_data = open_pointing('./data/ea25_cal_small_spw1_4_60_ea04_after.point.zarr')

Setup Dask Local Cluster

The local Dask client handles scheduling and worker managment for the parallelization. The user has the option of choosing the number of cores and memory allocations for each worker howerver, we recommend a minimum of 8Gb per core with standard settings.

A significant amount of information related to the client and scheduling can be found using the Dask Dashboard. This is a built-in dashboard native to Dask and allows the user to monitor the workers during processing. This is especially useful for profilling. For those that are interested in working soley within Jupyterlab a dashboard extension is availabe for Jupyterlab.

dashboard

[3]:
from toolviper.dask.client import local_client

parallel = True

if parallel:
    client = local_client(cores=4, memory_limit="4GB")
    print(client)
else:
    client = None
[2026-03-19 15:38:29,201]     INFO   astrohack:  Module path: /export/home/arya/miniforge3/envs/casadev/lib/python3.12/site-packages/toolviper
[2026-03-19 15:38:29,204]  WARNING   astrohack:  It is recommended that the local cache directory be set using the dask_local_dir parameter.
[2026-03-19 15:38:30,139]     INFO   astrohack:  Client <MenrvaClient: 'tcp://127.0.0.1:37175' processes=4 threads=4, memory=14.90 GiB>
<MenrvaClient: 'tcp://127.0.0.1:37175' processes=4 threads=4, memory=14.90 GiB>

Holography processing

Extract Holog

The extraction and restructuring of the holography data is done using the extract_holog function. This function is similar in function to the UVHOL task in AIPS. The holography data that is extracted can be set using the compound dictionary holog_obs_description: mapping, scan, and antenna id. A detailed description of the structure of the holog_obs_description dictionary can be found in the documentation here. The extract_holog can automatically generate the holog_obs_description by inspecting the pointing table.

Inline information on the input parameters can also be gotten using help(extract_holog) in the cell.

[4]:
from astrohack import extract_pointing, extract_holog

extract_pointing(
    ms_name="data/ea25_cal_small_after_fixed.split.ms",
    point_name="data/ea25_cal_small_after_fixed.split.point.zarr",
    parallel=parallel,
    overwrite=True,
)

holog_mds = extract_holog(
    ms_name="data/ea25_cal_small_after_fixed.split.ms",
    point_name="data/ea25_cal_small_after_fixed.split.point.zarr",
    data_column="CORRECTED_DATA",
    parallel=parallel,
    overwrite=True,
)
[2026-03-19 15:38:30,144]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
[2026-03-19 15:38:30,146]     INFO   astrohack:  Creating output file name: data/ea25_cal_small_after_fixed.split.point.zarr
[2026-03-19 15:38:33,252]     INFO   astrohack:  Consolidating data/ea25_cal_small_after_fixed.split.point.zarr...
[2026-03-19 15:38:33,351]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
[2026-03-19 15:38:33,356]     INFO   astrohack:  Creating output file name: data/ea25_cal_small_after_fixed.split.holog.zarr
[2026-03-19 15:38:33,376]     INFO   astrohack:  Pre-processing ddi: 0, scans: [8 ... 57]
[2026-03-19 15:38:33,377]     INFO   astrohack:  Pre-processing ddi: 1, scans: [8 ... 57]
[2026-03-19 15:38:38,080]     INFO   astrohack:  EA06: DDI 0: Suggested cell size 2.47 amin, FOV: (1.11 deg, 1.11 deg)
[2026-03-19 15:38:38,082]     INFO   astrohack:  EA25: DDI 0: Suggested cell size 2.47 amin, FOV: (1.11 deg, 1.11 deg)
[2026-03-19 15:38:39,423]     INFO   astrohack:  Finished extracting holography chunk for DDI 0, map_0.
[2026-03-19 15:38:40,589]     INFO   astrohack:  EA06: DDI 1: Suggested cell size 2.20 amin, FOV: (1.11 deg, 1.11 deg)
[2026-03-19 15:38:40,591]     INFO   astrohack:  EA25: DDI 1: Suggested cell size 2.20 amin, FOV: (1.11 deg, 1.11 deg)
[2026-03-19 15:38:40,788]     INFO   astrohack:  Finished extracting holography chunk for DDI 1, map_0.
[2026-03-19 15:38:40,790]     INFO   astrohack:  Consolidating data/ea25_cal_small_after_fixed.split.holog.zarr...

extract_pointing creates a file for holding the extracted pointing information in the form of <point_name>.point.zarr.

point_name.point.zarr: The pointing zarr file contains position and pointing information extracted from the pointing table of the input measurement set. In addition, the antenna and mapping scan information is listed for each antenna. The pointing object is structured in a single depth data tree with the key being the antenna id and the value being the pointing dataset.

point_mds =
{
   ant_0: point_ds,
            ⋮
   ant_n: point_ds
}

extract_holog creates a file for holding the extracted holography data as <holog_name>.holog.zarr. In addition, a holography data object is returned. This is the same holography data object returned by the hologrphy data API above. The holog_mds object is a python dict containing the extracted holography data found in .holog.zarr but with extended functionality such as providing a summary of the run infomation in table form. Below for each DDI we can see the available scan and antenna information.

holog_name.holog.zarr: The holog zarr file contains ungridded data extracted from the pointing and main tables in the measurement set. The holog file includes the directional, visibility and weight information recorded on a shared time axis; a time resampling is done because the sampling rates of the pointing and main tables are different. In addition, the metadata such as sampled parallactic data (beginning, middle and end of scan) and l and m extents is recorded in the file attributes. The holog file structure is a data tree, with ant -> ddi -> map as the keys for each depth.

holog_mds =
{
   ant_0:{
          ddi_0:{
                 map_0: holog_ds,
                          ⋮
                 map_n: holog_ds
                },
              ⋮
          ddi_p: …
         },
       ⋮
   ant_m: …
}

An example of the holog dataset object is show below.

[5]:
holog_mds["ant_ea25"]["ddi_0"]["map_0"]
[5]:
<xarray.DataTree 'map_0'>
Group: /ant_ea25/ddi_0/map_0
    Dimensions:                    (time: 8914, lm: 2, chan: 64, pol: 4)
    Coordinates:
      * time                       (time) float64 71kB 5.17e+09 ... 5.17e+09
      * chan                       (chan) float64 512B 1.41e+10 ... 1.423e+10
      * pol                        (pol) <U2 32B 'RR' 'RL' 'LR' 'LL'
    Dimensions without coordinates: lm
    Data variables:
        DIRECTIONAL_COSINES        (time, lm) float64 143kB dask.array<chunksize=(8914, 2), meta=np.ndarray>
        IDEAL_DIRECTIONAL_COSINES  (time, lm) float64 143kB dask.array<chunksize=(8914, 2), meta=np.ndarray>
        VIS                        (time, chan, pol) complex128 37MB dask.array<chunksize=(8914, 64, 4), meta=np.ndarray>
        WEIGHT                     (time, chan, pol) float64 18MB dask.array<chunksize=(8914, 64, 4), meta=np.ndarray>
    Attributes:
        parallactic_samples:      [5.30815743314105, 5.357028871429483, 5.5029779...
        scan_list:                [8, 9, 10, 12, 13, 14, 16, 17, 18, 23, 24, 25, ...
        scan_time_ranges:         [[5170354117.500001, 5170354438.499999], [51703...
        summary:                  {'aperture': None, 'beam': {'cell size': 0.0007...
        time_smoothing_interval:  1.0

In this case, there is only one selection in the holography file as seen in the summary. Using the available keys we can see an overview of the Dask dataset structure. In addition, the numpy arrays for the data are accessed by calling values on a given dataset variable. For instance accessing the data for the DIRECTIONAL_COSINES below would be simply

>> holog_mds['ddi_0']['map_0']['ant_ea25'].DIRECTIONAL_COSINES.values
>> array([[-0.00433549, -0.0027946 ],
       [-0.00870191, -0.00682571],
       [-0.00965634, -0.00908509],
       ...,
       [ 0.00966373,  0.00957556],
       [ 0.00966267,  0.00957601],
       [ 0.00965895,  0.00956941]])

>> holog_mds['ddi_0']['map_0']['ant_ea25'].DIRECTIONAL_COSINES.values.shape
>> (9145, 2)

where the dimension are given in the mds output for each data variable (in this case (time, lm)). A more in-depth overview of how to interact with Dask dataset can be found here.

A summary of the available key values can be obtained using the summary convenience function and a summary of the observation characteristics can be accessed with the observation_summary function.

[6]:
holog_mds.summary()
print(f"\n\n{100*'*'}\n\n")
holog_mds.observation_summary(
    "data/ea25_cal_small_after_fixed.split.holog.zarr.summary.txt"
)
################################################################################
###                               Summary for:                               ###
###             data/ea25_cal_small_after_fixed.split.holog.zarr             ###
################################################################################

Data origin:
creation_time:    2026-03-19 15:38:33 MDT
creator_function: extract_holog
origin:           astrohack
version:          1.0.1

Input Parameters:
+-------------------------------+--------------------------------------------------+
| Parameter                     | Value                                            |
+-------------------------------+--------------------------------------------------+
| ant                           | all                                              |
| append                        | False                                            |
| baseline_average_distance     | all                                              |
| baseline_average_nearest      | 1                                                |
| data_column                   | CORRECTED_DATA                                   |
| ddi                           | all                                              |
| exclude_antennas              | None                                             |
| holog_name                    | data/ea25_cal_small_after_fixed.split.holog.zarr |
| holog_obs_dict                | None                                             |
| map                           | all                                              |
| ms_name                       | data/ea25_cal_small_after_fixed.split.ms         |
| overwrite                     | True                                             |
| parallel                      | True                                             |
| point_name                    | data/ea25_cal_small_after_fixed.split.point.zarr |
| pointing_interpolation_method | linear                                           |
| time_smoothing_interval       | None                                             |
+-------------------------------+--------------------------------------------------+

Available methods:
+------------------------------+-----------------------------------------------+
| Methods                      | Description                                   |
+------------------------------+-----------------------------------------------+
| add_node                     | Add a node to the data tree file structure,   |
|                              | however this node is not yet consolidated     |
|                              | into the data tree         structure,         |
|                              | consolidate must be called to integrate all   |
|                              | nodes writen by add_node onto the tree        |
|                              | structure.                                    |
| consolidate                  | Traverse own file structure on disk           |
|                              | consolidating metadata to create a unified    |
|                              | data tree entity.                             |
| create_from_input_parameters | Create an AstrohackBaseFile object from a     |
|                              | filename and initializes xdtree root          |
|                              | attributes.                                   |
| export_to_aips               | Export data compatible to AIPS's HOLOG task   |
| is_close_to                  | Tests if self and other_mds are close to each |
|                              | other.                                        |
| items                        | Get children items                            |
| keys                         | Get children keys                             |
| observation_summary          | Create a Summary of observation information   |
| open                         | Open Base file.                               |
| plot_diagnostics             | Plot diagnostic calibration plots from the    |
|                              | holography data file.                         |
| plot_lm_sky_coverage         | Plot directional cosine coverage.             |
| summary                      | Prints summary of this Astrohack File object, |
|                              | with available data, attributes and methods   |
| values                       | Get children values                           |
| write                        | Write mds to disk by saving the data tree to  |
|                              | a file                                        |
+------------------------------+-----------------------------------------------+

Data Contents:
+----------+-------+-----------+
| Antenna  | DDI   | Mapping   |
+----------+-------+-----------+
| ant_ea06 | ddi_0 | ['map_0'] |
| ant_ea06 | ddi_1 | ['map_0'] |
| ant_ea25 | ddi_0 | ['map_0'] |
| ant_ea25 | ddi_1 | ['map_0'] |
+----------+-------+-----------+


****************************************************************************************************


[2026-03-19 15:38:40,863]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
############################################################
###                ant_ea06, ddi_0, map_0                ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea06
      Station            => N04
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (101.8, 45.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 2.000 MHz
      Frequency range    => 14.103 GHz to 14.231 GHz
      Number of channels => 64
      Rep. frequency     => 14.168 GHz
      Rep. wavelength    => 2.116 cm

   Beam:
      Cell size          => 2.47 amin
      Grid size          => 27 by 27 pixels
      L extent           => From -33.28 amin to 36.85 amin
      M extent           => From -33.94 amin to 34.81 amin

############################################################
###                ant_ea06, ddi_1, map_0                ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea06
      Station            => N04
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (101.8, 45.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 2.000 MHz
      Frequency range    => 15.903 GHz to 16.017 GHz
      Number of channels => 57
      Rep. frequency     => 15.960 GHz
      Rep. wavelength    => 1.878 cm

   Beam:
      Cell size          => 2.20 amin
      Grid size          => 31 by 31 pixels
      L extent           => From -33.28 amin to 36.85 amin
      M extent           => From -33.94 amin to 34.81 amin

############################################################
###                ant_ea25, ddi_0, map_0                ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea25
      Station            => E02
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (104.4, 48.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 2.000 MHz
      Frequency range    => 14.103 GHz to 14.231 GHz
      Number of channels => 64
      Rep. frequency     => 14.168 GHz
      Rep. wavelength    => 2.116 cm

   Beam:
      Cell size          => 2.47 amin
      Grid size          => 27 by 27 pixels
      L extent           => From -33.26 amin to 33.26 amin
      M extent           => From -33.53 amin to 33.34 amin

############################################################
###                ant_ea25, ddi_1, map_0                ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea25
      Station            => E02
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (104.4, 48.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 2.000 MHz
      Frequency range    => 15.903 GHz to 16.017 GHz
      Number of channels => 57
      Rep. frequency     => 15.960 GHz
      Rep. wavelength    => 1.878 cm

   Beam:
      Cell size          => 2.20 amin
      Grid size          => 31 by 31 pixels
      L extent           => From -33.26 amin to 33.26 amin
      M extent           => From -33.53 amin to 33.34 amin


Holog

The holog function processes the holography data and produces a holog image file on disk with the suffix, .image.zarr. This function is a direct replacement for the task HOLOG in AIPS. It is required that the user provide the grid_size and cell_size when processing holography data. The grid_size defines the number of l x m points used to when doing the gridding. The cell_size defines the value in arseconds of each grid spacing. More in-depth parameter information can be found in readthedocs here.

Inline information on the input paramters can also be gotten using help(holog) in the cell.

[7]:
import numpy as np

from astrohack import holog

cell_size = np.array([-0.0006442, 0.0006442])  # arcseconds
grid_size = np.array([31, 31])  # pixels

image_mds = holog(
    holog_name="data/ea25_cal_small_after_fixed.split.holog.zarr",
    overwrite=True,
    phase_fit_engine="perturbations",
    to_stokes=True,
    parallel=parallel,
)
[2026-03-19 15:38:40,995]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
[2026-03-19 15:38:40,999]     INFO   astrohack:  Creating output file name: data/ea25_cal_small_after_fixed.split.image.zarr
[2026-03-19 15:38:41,047]     INFO   astrohack:  Processing EA06, DDI 0
[2026-03-19 15:38:41,047]     INFO   astrohack:  EA06, DDI 0: Using a grid of 27 by 27 pixels for the beam
[2026-03-19 15:38:41,048]     INFO   astrohack:  EA06, DDI 0: Using a cell size of -2.47 amin by 2.47 amin for the beam
[2026-03-19 15:38:41,855]     INFO   astrohack:  Finished processing EA06, DDI 0
[2026-03-19 15:38:41,856]     INFO   astrohack:  Processing EA06, DDI 1
[2026-03-19 15:38:41,857]     INFO   astrohack:  EA06, DDI 1: Using a grid of 31 by 31 pixels for the beam
[2026-03-19 15:38:41,857]     INFO   astrohack:  EA06, DDI 1: Using a cell size of -2.20 amin by 2.20 amin for the beam
[2026-03-19 15:38:42,745]     INFO   astrohack:  Finished processing EA06, DDI 1
[2026-03-19 15:38:42,747]     INFO   astrohack:  Processing EA25, DDI 0
[2026-03-19 15:38:42,747]     INFO   astrohack:  EA25, DDI 0: Using a grid of 27 by 27 pixels for the beam
[2026-03-19 15:38:42,747]     INFO   astrohack:  EA25, DDI 0: Using a cell size of -2.47 amin by 2.47 amin for the beam
[2026-03-19 15:38:43,611]     INFO   astrohack:  Finished processing EA25, DDI 0
[2026-03-19 15:38:43,612]     INFO   astrohack:  Processing EA25, DDI 1
[2026-03-19 15:38:43,612]     INFO   astrohack:  EA25, DDI 1: Using a grid of 31 by 31 pixels for the beam
[2026-03-19 15:38:43,613]     INFO   astrohack:  EA25, DDI 1: Using a cell size of -2.20 amin by 2.20 amin for the beam
[2026-03-19 15:38:44,487]     INFO   astrohack:  Finished processing EA25, DDI 1
[2026-03-19 15:38:44,488]     INFO   astrohack:  Consolidating data/ea25_cal_small_after_fixed.split.image.zarr...

image_name.image.zarr: The image zarr file contains gridded image data the beam, extracted aperture and the amplitude and phase components. It also contains all the relevant coordinate information. The image file structure is a 2 level data tree with keys in the ant -> ddi order with the leaves consisting of the image dataset.

image_mds =
{
   ant_0:{
          ddi_0: image_ds,
                 ⋮
          ddi_m: image_ds
         },
       ⋮
   ant_n: …
}

An example of the image dataset object is show below.

[8]:
image_mds["ant_ea25"]["ddi_0"]
[8]:
<xarray.DataTree 'ddi_0'>
Group: /ant_ea25/ddi_0
    Dimensions:               (time: 1, chan: 1, pol: 4, u_prime: 510,
                               v_prime: 510, u: 512, v: 512, l: 27, m: 27,
                               orig_pol: 4, osa: 15)
    Coordinates:
      * chan                  (chan) float64 8B 1.417e+10
      * pol                   (pol) <U1 16B 'I' 'Q' 'U' 'V'
      * u_prime               (u_prime) float64 4kB 14.62 14.56 ... -14.56 -14.62
      * v_prime               (v_prime) float64 4kB -14.62 -14.56 ... 14.56 14.62
      * u                     (u) float64 4kB 14.68 14.62 14.56 ... -14.62 -14.68
      * v                     (v) float64 4kB -14.68 -14.62 -14.56 ... 14.62 14.68
      * l                     (l) float64 216B 0.008994 0.008274 ... -0.009713
      * m                     (m) float64 216B -0.008994 -0.008274 ... 0.009713
      * orig_pol              (orig_pol) <U2 32B 'RR' 'RL' 'LR' 'LL'
      * osa                   (osa) <U9 540B 'N=0, M=0' 'N=1, M=-1' ... 'N=4, M=4'
    Dimensions without coordinates: time
    Data variables:
        AMPLITUDE             (time, chan, pol, u_prime, v_prime) float64 8MB dask.array<chunksize=(1, 1, 4, 510, 510), meta=np.ndarray>
        APERTURE              (time, chan, pol, u, v) complex128 17MB dask.array<chunksize=(1, 1, 4, 512, 512), meta=np.ndarray>
        BEAM                  (time, chan, pol, l, m) complex128 47kB dask.array<chunksize=(1, 1, 4, 27, 27), meta=np.ndarray>
        CORRECTED_PHASE       (time, chan, pol, u_prime, v_prime) float64 8MB dask.array<chunksize=(1, 1, 4, 510, 510), meta=np.ndarray>
        ZERNIKE_COEFFICIENTS  (time, chan, orig_pol, osa) complex128 960B dask.array<chunksize=(1, 1, 4, 15), meta=np.ndarray>
        ZERNIKE_FIT_RMS       (time, chan, orig_pol) complex128 64B dask.array<chunksize=(1, 1, 4), meta=np.ndarray>
        ZERNIKE_MODEL         (time, chan, orig_pol, u, v) complex128 17MB dask.array<chunksize=(1, 1, 4, 512, 512), meta=np.ndarray>
    Attributes:
        ddi:              ddi_0
        phase_fitting:    {'map_0': {'14167000000.0': {'I': {'phase_offset': {'er...
        summary:          {'aperture': {'cell size': [-0.05744485294117574, 0.057...
        time_centroid:    5170359434.000001
        zernike_N_order:  4

A summary of the available key values can be obtained using the summary convenience function.

[9]:
image_mds.summary()
################################################################################
###                               Summary for:                               ###
###             data/ea25_cal_small_after_fixed.split.image.zarr             ###
################################################################################

Data origin:
creation_time:    2026-03-19 15:38:41 MDT
creator_function: holog
origin:           astrohack
version:          1.0.1

Input Parameters:
+-------------------------+--------------------------------------------------+
| Parameter               | Value                                            |
+-------------------------+--------------------------------------------------+
| alma_osf_pad            | None                                             |
| ant                     | all                                              |
| cell_size               | None                                             |
| chan_average            | True                                             |
| chan_tolerance_factor   | 0.005                                            |
| ddi                     | all                                              |
| grid_interpolation_mode | gaussian                                         |
| grid_size               | None                                             |
| holog_name              | data/ea25_cal_small_after_fixed.split.holog.zarr |
| image_name              | data/ea25_cal_small_after_fixed.split.image.zarr |
| overwrite               | True                                             |
| padding_factor          | 10                                               |
| parallel                | True                                             |
| phase_fit_control       | [True, True, True, True, True]                   |
| phase_fit_engine        | perturbations                                    |
| scan_average            | True                                             |
| to_stokes               | True                                             |
| zernike_n_order         | 4                                                |
+-------------------------+--------------------------------------------------+

Available methods:
+------------------------------+-----------------------------------------------+
| Methods                      | Description                                   |
+------------------------------+-----------------------------------------------+
| add_node                     | Add a node to the data tree file structure,   |
|                              | however this node is not yet consolidated     |
|                              | into the data tree         structure,         |
|                              | consolidate must be called to integrate all   |
|                              | nodes writen by add_node onto the tree        |
|                              | structure.                                    |
| consolidate                  | Traverse own file structure on disk           |
|                              | consolidating metadata to create a unified    |
|                              | data tree entity.                             |
| create_from_input_parameters | Create an AstrohackBaseFile object from a     |
|                              | filename and initializes xdtree root          |
|                              | attributes.                                   |
| export_phase_fit_results     | Export perturbations phase fit results from   |
|                              | the data in an AstrohackImageFIle object to   |
|                              | ASCII files.                                  |
| export_to_fits               | Export contents of an AstrohackImageFile      |
|                              | object to several FITS files in the           |
|                              | destination folder                            |
| export_zernike_fit_results   | Export Zernike coefficients from the data in  |
|                              | an AstrohackImageFIle object to ASCII files.  |
| is_close_to                  | Tests if self and other_mds are close to each |
|                              | other.                                        |
| items                        | Get children items                            |
| keys                         | Get children keys                             |
| observation_summary          | Create a Summary of observation information   |
| open                         | Open Base file.                               |
| plot_apertures               | Aperture amplitude and phase plots from the   |
|                              | data in an AstrohackImageFIle object.         |
| plot_beams                   | Beam plots from the data in an                |
|                              | AstrohackImageFIle object.                    |
| plot_zernike_model           | Plot Zernike models from the data in an       |
|                              | AstrohackImageFile object.                    |
| summary                      | Prints summary of this Astrohack File object, |
|                              | with available data, attributes and methods   |
| values                       | Get children values                           |
| write                        | Write mds to disk by saving the data tree to  |
|                              | a file                                        |
+------------------------------+-----------------------------------------------+

Data Contents:
+----------+--------------------+
| Antenna  | DDI                |
+----------+--------------------+
| ant_ea06 | ['ddi_0', 'ddi_1'] |
| ant_ea25 | ['ddi_0', 'ddi_1'] |
+----------+--------------------+

Each of the holography output files is a compound dictionary with respect to the run parameters and contains a xarray Dataset, this means that the holography files have access to all native xarray functionality. The user can use their favorite plotting package to visualize the data or use xarray’s internal functions to do simple filtering and plotting.

[10]:
image_mds["ant_ea25"]["ddi_0"].CORRECTED_PHASE.isel(chan=0, pol=0).plot()
[10]:
<matplotlib.collections.QuadMesh at 0x7f2cb8a9c380>
../_images/tutorials_vla_holography_tutorial_24_1.png

Panel

The panel function takes the place of and expands the PANEL AIPS function to processes the image information and derives adjustements to the dish panels. This produces a file on disk of format .panel.zarr containing information on corrections, residuals and screw adjustments. As an added bonus the panel function has a helper function to convert aips data to astrohack format and process it using the aips_holog_to_astrohack function. For a full description of the operation and arguments of the panel function see docs.

[11]:
from astrohack import panel

panel_model = "rigid"

panel_mds = panel(
    image_name="data/ea25_cal_small_after_fixed.split.image.zarr",
    panel_model=panel_model,
    panel_margins=0.2,
    clip_type="relative",
    clip_level=0.2,
    parallel=parallel,
    overwrite=True,
)
[2026-03-19 15:38:44,955]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
[2026-03-19 15:38:44,958]     INFO   astrohack:  Creating output file name: data/ea25_cal_small_after_fixed.split.panel.zarr
[2026-03-19 15:38:45,020]     INFO   astrohack:  processing EA06: DDI 0
[2026-03-19 15:38:45,869]     INFO   astrohack:  processing EA06: DDI 1
[2026-03-19 15:38:46,437]  WARNING   astrohack:  EA06: DDI 1: Fit failed with the rigid model and a simple mean has been used instead for the following panels:
[2026-03-19 15:38:46,438]  WARNING   astrohack:  ['6-10']
[2026-03-19 15:38:46,713]     INFO   astrohack:  processing EA25: DDI 0
[2026-03-19 15:38:47,556]     INFO   astrohack:  processing EA25: DDI 1
[2026-03-19 15:38:48,399]     INFO   astrohack:  Consolidating data/ea25_cal_small_after_fixed.split.panel.zarr...

panel_name.panel.zarr: The panel zarr file contains process information regarding the per panel screw corrections as well as residuals, masks and phase corrections used to produce them. The panel file structure is a 2 level data tree with keys in the ant -> ddi order with the leaves consisting of the panel dataset.

panel_mds =
{
   ant_0:{
          ddi_0: panel_ds,
                 ⋮
          ddi_m: panel_ds
         },
       ⋮
   ant_n: …

An example of the panel dataset object is show below.

[12]:
panel_mds["ant_ea25"]["ddi_0"]
[12]:
<xarray.DataTree 'ddi_0'>
Group: /ant_ea25/ddi_0
    Dimensions:             (u: 510, v: 510, labels: 172, pars: 3, screws: 4)
    Coordinates:
      * u                   (u) float64 4kB 14.62 14.56 14.5 ... -14.5 -14.56 -14.62
      * v                   (v) float64 4kB -14.62 -14.56 -14.5 ... 14.5 14.56 14.62
      * labels              (labels) <U22 15kB '1-1' '1-2' '1-3' ... '6-39' '6-40'
      * pars                (pars) int64 24B 0 1 2
      * screws              (screws) <U2 32B 'il' 'ir' 'ol' 'or'
    Data variables: (12/15)
        AMPLITUDE           (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        AMP_NOISE           (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        CORRECTIONS         (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        DEVIATION           (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        MASK                (u, v) bool 260kB dask.array<chunksize=(510, 510), meta=np.ndarray>
        PANEL_DISTRIBUTION  (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        ...                  ...
        PANEL_SCREWS        (labels, screws) float64 6kB dask.array<chunksize=(172, 4), meta=np.ndarray>
        PHASE               (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        PHASE_CORRECTIONS   (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        PHASE_RESIDUALS     (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        RADIUS              (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
        RESIDUALS           (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>
    Attributes: (12/16)
        amp_unit:             V
        aperture_resolution:  [1.4366515837104072, 1.4366515837104072]
        clip:                 0.5412720936531105
        ddi:                  ddi_0
        fitted:               True
        input_gain:           41.57934084050393
        ...                   ...
        panel_model:          rigid
        pol_state:            I
        solved:               True
        summary:              {'aperture': {'cell size': [-0.05744485294117574, 0...
        theoretical_gain:     41.71607063670329
        wavelength:           0.021161322651231735

A summary of the available key values can be obtained using the summary convenience function. Additionally, a final summary of the observation characteristics can be accessed through the observation_summary method.

[13]:
panel_mds.summary()
print(f"\n\n{100*'*'}\n\n")
panel_mds.observation_summary(
    "data/ea25_cal_small_after_fixed.split.panel.zarr.summary.txt"
)
################################################################################
###                               Summary for:                               ###
###             data/ea25_cal_small_after_fixed.split.panel.zarr             ###
################################################################################

Data origin:
creation_time:    2026-03-19 15:38:45 MDT
creator_function: panel
origin:           astrohack
version:          1.0.1

Input Parameters:
+--------------------+--------------------------------------------------+
| Parameter          | Value                                            |
+--------------------+--------------------------------------------------+
| ant                | all                                              |
| clip_level         | 0.2                                              |
| clip_type          | relative                                         |
| ddi                | all                                              |
| image_name         | data/ea25_cal_small_after_fixed.split.image.zarr |
| overwrite          | True                                             |
| panel_margins      | 0.2                                              |
| panel_model        | rigid                                            |
| panel_name         | data/ea25_cal_small_after_fixed.split.panel.zarr |
| parallel           | True                                             |
| polarization_state | I                                                |
| use_detailed_mask  | True                                             |
+--------------------+--------------------------------------------------+

Available methods:
+------------------------------+-----------------------------------------------+
| Methods                      | Description                                   |
+------------------------------+-----------------------------------------------+
| add_node                     | Add a node to the data tree file structure,   |
|                              | however this node is not yet consolidated     |
|                              | into the data tree         structure,         |
|                              | consolidate must be called to integrate all   |
|                              | nodes writen by add_node onto the tree        |
|                              | structure.                                    |
| consolidate                  | Traverse own file structure on disk           |
|                              | consolidating metadata to create a unified    |
|                              | data tree entity.                             |
| create_from_input_parameters | Create an AstrohackBaseFile object from a     |
|                              | filename and initializes xdtree root          |
|                              | attributes.                                   |
| export_gain_tables           | Compute estimated antenna gains in dB and     |
|                              | saves them to ASCII files.                    |
| export_screws                | Export screw adjustments to text files and    |
|                              | optionally plots.                             |
| export_to_fits               | Export contents of an Astrohack MDS file to   |
|                              | several FITS files in the destination folder  |
| get_antenna                  | Retrieve an AntennaSurface object for         |
|                              | interaction                                   |
| is_close_to                  | Tests if self and other_mds are close to each |
|                              | other.                                        |
| items                        | Get children items                            |
| keys                         | Get children keys                             |
| observation_summary          | Create a Summary of observation information   |
| open                         | Open Base file.                               |
| plot_antennas                | Create diagnostic plots of antenna surfaces   |
|                              | from panel data file.                         |
| summary                      | Prints summary of this Astrohack File object, |
|                              | with available data, attributes and methods   |
| values                       | Get children values                           |
| write                        | Write mds to disk by saving the data tree to  |
|                              | a file                                        |
+------------------------------+-----------------------------------------------+

Data Contents:
+----------+--------------------+
| Antenna  | DDI                |
+----------+--------------------+
| ant_ea06 | ['ddi_0', 'ddi_1'] |
| ant_ea25 | ['ddi_0', 'ddi_1'] |
+----------+--------------------+


****************************************************************************************************


[2026-03-19 15:38:49,190]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
############################################################
###                   ant_ea06, ddi_0                    ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea06
      Station            => N04
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (101.8, 45.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 128.000 MHz
      Frequency range    => 14.103 GHz to 14.231 GHz
      Number of channels => 1
      Rep. frequency     => 14.168 GHz
      Rep. wavelength    => 2.116 cm

   Beam:
      Cell size          => -2.47 amin by 2.47 amin
      Grid size          => 27 by 27 pixels
      L extent           => From -33.28 amin to 36.85 amin
      M extent           => From -33.94 amin to 34.81 amin

   Aperture:
      Cell size          => -5.74 cm by 5.74 cm
      Grid size          => 512 by 512 pixels
      Resolution         => 1.44 m by 1.44 m

############################################################
###                   ant_ea06, ddi_1                    ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea06
      Station            => N04
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (101.8, 45.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 114.000 MHz
      Frequency range    => 15.903 GHz to 16.017 GHz
      Number of channels => 1
      Rep. frequency     => 15.960 GHz
      Rep. wavelength    => 1.878 cm

   Beam:
      Cell size          => -2.20 amin by 2.20 amin
      Grid size          => 31 by 31 pixels
      L extent           => From -33.28 amin to 36.85 amin
      M extent           => From -33.94 amin to 34.81 amin

   Aperture:
      Cell size          => -5.74 cm by 5.74 cm
      Grid size          => 512 by 512 pixels
      Resolution         => 1.25 m by 1.25 m

############################################################
###                   ant_ea25, ddi_0                    ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea25
      Station            => E02
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (104.4, 48.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 128.000 MHz
      Frequency range    => 14.103 GHz to 14.231 GHz
      Number of channels => 1
      Rep. frequency     => 14.168 GHz
      Rep. wavelength    => 2.116 cm

   Beam:
      Cell size          => -2.47 amin by 2.47 amin
      Grid size          => 27 by 27 pixels
      L extent           => From -33.26 amin to 33.26 amin
      M extent           => From -33.53 amin to 33.34 amin

   Aperture:
      Cell size          => -5.74 cm by 5.74 cm
      Grid size          => 512 by 512 pixels
      Resolution         => 1.44 m by 1.44 m

############################################################
###                   ant_ea25, ddi_1                    ###
############################################################

   General:
      Telescope name     => EVLA
      Antenna name       => ea25
      Station            => E02
      Reference antennas => ['ea04 @ E03']
      Source             => HOLORASTER
      Phase center       => 22h53m57.748s +16°08m53.561s [FK5]
      Az el info         => @ l,m = (0,0), Az, El = (104.4, 48.6) [deg]
      Start time         => 20 Sep 2022, 01:19:00 (UTC)
      Stop time          => 20 Sep 2022, 04:29:47 (UTC)
      Duration           => 3 hour, 10 min, 47.00 sec

   Spectral:
      Channel width      => 114.000 MHz
      Frequency range    => 15.903 GHz to 16.017 GHz
      Number of channels => 1
      Rep. frequency     => 15.960 GHz
      Rep. wavelength    => 1.878 cm

   Beam:
      Cell size          => -2.20 amin by 2.20 amin
      Grid size          => 31 by 31 pixels
      L extent           => From -33.26 amin to 33.26 amin
      M extent           => From -33.53 amin to 33.34 amin

   Aperture:
      Cell size          => -5.74 cm by 5.74 cm
      Grid size          => 512 by 512 pixels
      Resolution         => 1.25 m by 1.25 m


Additional Functions

The panel_mds object provides two helper functions for the user to export or investigate the results of the panel function. - export_screws(): This method exports the screw and panel adjustements from the panel output file. - plot_antennas(): This method plots one of three diagnostics plots from the panel output file data. The plots types are: deviation, phase and ancillary.

Examples usage for each helper functions are given below and more detailed documentation can be found in the visualization tutorial.

[14]:
export_folder = "exports"

panel_mds.export_screws(
    destination=export_folder,
    ant="ea25",
    ddi=0,
    unit="mm",
    threshold=0.5,  # Threshold in mm for significant adjustments
    display=True,
)
[2026-03-19 15:38:49,255]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
../_images/tutorials_vla_holography_tutorial_32_1.png
[15]:
with open(export_folder + "/panel_screws_ant_ea25_ddi_0.txt", "r") as file:
    for _ in range(30):
        print(file.readline()[:-1])
# Screw adjustments for VLA's EA25: DDI 0, pol. state I
# Frequency = 14.1670 GHz
# Antenna surface RMS before adjustment: 0.50 mm
# Antenna surface RMS after adjustment: 0.22 mm
# Lower means away from subreflector
# Raise means toward the subreflector
# LOWER the panel if the number is POSITIVE
# RAISE the panel if the number is NEGATIVE
# Adjustments are in mm

# Panel      il        ir        ol        or    Fallback    Model
  1-1     -0.26      0.13      0.16      0.89      no       rigid
  1-2     -0.25     -0.44      0.42      0.06      no       rigid
  1-3      0.43      0.52      0.03      0.20      no       rigid
  1-4     -0.53     -0.74     -0.33     -0.71      no       rigid
  1-5     -0.41     -0.28     -0.39     -0.16      no       rigid
  1-6     -0.52     -0.35     -0.06      0.26      no       rigid
  1-7     -1.19     -1.21      0.32      0.28      no       rigid
  1-8     -1.28     -1.01      0.16      0.67      no       rigid
  1-9      0.54      0.09      0.03     -0.82      no       rigid
 1-10      0.55      0.87     -0.69     -0.09      no       rigid
 1-11      0.98      1.29     -0.53      0.03      no       rigid
 1-12      0.78      0.10      0.75     -0.51      no       rigid
  2-1      0.59      1.07      0.03      0.75      no       rigid
  2-2     -0.32     -0.38      0.05     -0.04      no       rigid
  2-3      0.21      0.16     -0.21     -0.29      no       rigid
  2-4      0.08     -0.04     -0.23     -0.40      no       rigid
  2-5     -0.66      0.19     -0.66      0.64      no       rigid
  2-6      0.23     -0.30      0.81      0.01      no       rigid
  2-7     -0.84     -0.45     -0.49      0.10      no       rigid
[16]:
panel_mds.plot_antennas(
    destination=export_folder,
    ant="ea25",
    ddi=0,
    plot_type="deviation",
    plot_screws=False,
    dpi=300,
    parallel=False,
    display=True,
)
[2026-03-19 15:38:51,218]     INFO   astrohack:  Module path: /export/home/arya/work/Holography-1022/astrohack/src/astrohack
../_images/tutorials_vla_holography_tutorial_34_1.png
../_images/tutorials_vla_holography_tutorial_34_2.png
../_images/tutorials_vla_holography_tutorial_34_3.png
[17]:
if parallel:
    client.close()