{ "cells": [ { "cell_type": "markdown", "id": "756761ad-fe5f-4de7-b02e-26ab85d7e2fe", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "[](https://colab.research.google.com/github/nrao/astrohack/blob/v1.0.1/docs/tutorial_vla.ipynb)" ] }, { "cell_type": "markdown", "id": "80ca8c71-4c92-41f6-8aee-a8aa4ee5fb2f", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "e46c2166", "metadata": {}, "source": [ "# VLA Holography Tutorial\n", "\n", "### Important External Information\n", "\n", "- #### xarray Official Documentation ([docs](https://docs.xarray.dev/en/stable/)).\n", "- #### Dask Official Documentation ([docs](https://www.dask.org/)).\n", "- #### zarr Official Documentation ([docs](https://zarr.readthedocs.io/en/stable/))" ] }, { "cell_type": "code", "execution_count": 1, "id": "7db6f868-030c-41ee-8188-c236aa675c27", "metadata": { "ExecuteTime": { "end_time": "2026-02-10T16:57:04.144047874Z", "start_time": "2026-02-10T16:57:01.623314850Z" }, "execution": { "iopub.execute_input": "2026-03-19T21:38:23.129416Z", "iopub.status.busy": "2026-03-19T21:38:23.129263Z", "iopub.status.idle": "2026-03-19T21:38:25.640720Z", "shell.execute_reply": "2026-03-19T21:38:25.640280Z" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "AstroHACK version 1.0.1 already installed.\n" ] } ], "source": [ "import os\n", "\n", "try:\n", " import astrohack\n", "\n", " print(\"AstroHACK version\", astrohack.__version__, \"already installed.\")\n", "except ImportError as e:\n", " print(e)\n", " print(\"Installing AstroHACK\")\n", "\n", " os.system(\"pip install astrohack\")\n", "\n", " import astrohack\n", "\n", " print(\"astrohack version\", astrohack.__version__, \" installed.\")" ] }, { "cell_type": "markdown", "id": "8f192eb8-7e2f-48a4-a04b-543629f3af79", "metadata": {}, "source": [ "## Download Tutorial Data" ] }, { "cell_type": "code", "execution_count": 2, "id": "ffb79bcd", "metadata": { "ExecuteTime": { "end_time": "2026-02-10T16:57:04.323388989Z", "start_time": "2026-02-10T16:57:04.146740565Z" }, "execution": { "iopub.execute_input": "2026-03-19T21:38:25.642235Z", "iopub.status.busy": "2026-03-19T21:38:25.641973Z", "iopub.status.idle": "2026-03-19T21:38:29.154860Z", "shell.execute_reply": "2026-03-19T21:38:29.154232Z" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\u001b[38;2;128;05;128m2026-03-19 15:38:25,643\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m astrohack: \u001b[0m Module path: \u001b[38;2;50;50;205m/export/home/arya/miniforge3/envs/casadev/lib/python3.12/site-packages/toolviper\u001b[0m \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[\u001b[38;2;128;05;128m2026-03-19 15:38:25,646\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m astrohack: \u001b[0m Downloading from [cloudflare] .... \n" ] }, { "data": { "text/html": [ "
\n",
" Download List \n",
" ───────────────────────────────────── \n",
" ea25_cal_small_after_fixed.split.ms \n",
" \n",
"\n"
],
"text/plain": [
" \n",
" \u001b[1m \u001b[0m\u001b[1mDownload List \u001b[0m\u001b[1m \u001b[0m \n",
" ───────────────────────────────────── \n",
" \u001b[35mea25_cal_small_after_fixed.split.ms\u001b[0m \n",
" \n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\u001b[38;2;128;05;128m2026-03-19 15:38:25,650\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m astrohack: \u001b[0m Creating path:\u001b[38;2;50;50;205m/export/home/arya/work/Holography-1022/astrohack/docs/tutorials/data\u001b[0m \n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "04a4dcb344174dd0a76e13c07601f7fb",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Output()"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n"
],
"text/plain": []
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import toolviper\n",
"\n",
"toolviper.utils.data.download(file=\"ea25_cal_small_after_fixed.split.ms\", folder=\"data\")"
]
},
{
"cell_type": "markdown",
"id": "81fa2ab3-b718-46e3-9dd0-b29eba815a57",
"metadata": {},
"source": [
"## Holography Data File API\n",
"\n",
"As part of the `astroHACK` API a set of functions to allow users to easily open on disk holography files has been provided. Each function takes an `astroHACK` holography file name as an argument and returns an object related to the given file type, i.e. holog, image, panel, point. Each object allows the user to access data via dictionary keys with values consisting of the relevant holography dataset. Each object also provides a `summary()` helper function to list available keys for each file. An example call for each file type is show below and the API documentation for all data-io functions can be found [here](https://astrohack.readthedocs.io/en/latest/_api/autoapi/astrohack/dio/index.html)."
]
},
{
"cell_type": "markdown",
"id": "85088acd-1247-4469-aa88-a012c9ffc089",
"metadata": {
"tags": []
},
"source": [
"```python\n",
"from astrohack import open_holog\n",
"from astrohack import open_image\n",
"from astrohack import open_panel\n",
"from astrohack import open_pointing\n",
"\n",
"holog_data = open_holog('./data/ea25_cal_small_spw1_4_60_ea04_after.holog.zarr')\n",
"image_data = open_image('./data/ea25_cal_small_spw1_4_60_ea04_after.image.zarr')\n",
"panel_data = open_panel('./data/ea25_cal_small_spw1_4_60_ea04_after.panel.zarr')\n",
"pointing_data = open_pointing('./data/ea25_cal_small_spw1_4_60_ea04_after.point.zarr')\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "42331121-cc19-4f28-914b-fb5e495565bb",
"metadata": {
"tags": []
},
"source": [
"## Setup Dask Local Cluster\n",
"\n",
"The local Dask client handles scheduling and worker managment for the parallelization. The user has the option of choosing the number of cores and memory allocations for each worker howerver, we recommend a minimum of 8Gb per core with standard settings.\n",
"\n",
"\n",
"A significant amount of information related to the client and scheduling can be found using the [Dask Dashboard](https://docs.dask.org/en/stable/dashboard.html). This is a built-in dashboard native to Dask and allows the user to monitor the workers during processing. This is especially useful for profilling. For those that are interested in working soley within Jupyterlab a dashboard extension is availabe for [Jupyterlab](https://github.com/dask/dask-labextension#dask-jupyterlab-extension).\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "10dffc63-1907-497f-b025-392b5813eac9",
"metadata": {
"ExecuteTime": {
"end_time": "2026-02-10T16:57:04.375119964Z",
"start_time": "2026-02-10T16:57:04.342121836Z"
},
"execution": {
"iopub.execute_input": "2026-03-19T21:38:29.184867Z",
"iopub.status.busy": "2026-03-19T21:38:29.184754Z",
"iopub.status.idle": "2026-03-19T21:38:30.141950Z",
"shell.execute_reply": "2026-03-19T21:38:30.141423Z"
},
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\u001b[38;2;128;05;128m2026-03-19 15:38:29,201\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m astrohack: \u001b[0m Module path: \u001b[38;2;50;50;205m/export/home/arya/miniforge3/envs/casadev/lib/python3.12/site-packages/toolviper\u001b[0m \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\u001b[38;2;128;05;128m2026-03-19 15:38:29,204\u001b[0m] \u001b[38;2;255;160;0m WARNING\u001b[0m\u001b[38;2;112;128;144m astrohack: \u001b[0m It is recommended that the local cache directory be set using the \u001b[38;2;50;50;205mdask_local_dir\u001b[0m parameter. \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\u001b[38;2;128;05;128m2026-03-19 15:38:30,139\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m astrohack: \u001b[0m Client <xarray.DataTree 'map_0'>\n",
"Group: /ant_ea25/ddi_0/map_0\n",
" Dimensions: (time: 8914, lm: 2, chan: 64, pol: 4)\n",
" Coordinates:\n",
" * time (time) float64 71kB 5.17e+09 ... 5.17e+09\n",
" * chan (chan) float64 512B 1.41e+10 ... 1.423e+10\n",
" * pol (pol) <U2 32B 'RR' 'RL' 'LR' 'LL'\n",
" Dimensions without coordinates: lm\n",
" Data variables:\n",
" DIRECTIONAL_COSINES (time, lm) float64 143kB dask.array<chunksize=(8914, 2), meta=np.ndarray>\n",
" IDEAL_DIRECTIONAL_COSINES (time, lm) float64 143kB dask.array<chunksize=(8914, 2), meta=np.ndarray>\n",
" VIS (time, chan, pol) complex128 37MB dask.array<chunksize=(8914, 64, 4), meta=np.ndarray>\n",
" WEIGHT (time, chan, pol) float64 18MB dask.array<chunksize=(8914, 64, 4), meta=np.ndarray>\n",
" Attributes:\n",
" parallactic_samples: [5.30815743314105, 5.357028871429483, 5.5029779...\n",
" scan_list: [8, 9, 10, 12, 13, 14, 16, 17, 18, 23, 24, 25, ...\n",
" scan_time_ranges: [[5170354117.500001, 5170354438.499999], [51703...\n",
" summary: {'aperture': None, 'beam': {'cell size': 0.0007...\n",
" time_smoothing_interval: 1.0<xarray.DataTree 'ddi_0'>\n",
"Group: /ant_ea25/ddi_0\n",
" Dimensions: (time: 1, chan: 1, pol: 4, u_prime: 510,\n",
" v_prime: 510, u: 512, v: 512, l: 27, m: 27,\n",
" orig_pol: 4, osa: 15)\n",
" Coordinates:\n",
" * chan (chan) float64 8B 1.417e+10\n",
" * pol (pol) <U1 16B 'I' 'Q' 'U' 'V'\n",
" * u_prime (u_prime) float64 4kB 14.62 14.56 ... -14.56 -14.62\n",
" * v_prime (v_prime) float64 4kB -14.62 -14.56 ... 14.56 14.62\n",
" * u (u) float64 4kB 14.68 14.62 14.56 ... -14.62 -14.68\n",
" * v (v) float64 4kB -14.68 -14.62 -14.56 ... 14.62 14.68\n",
" * l (l) float64 216B 0.008994 0.008274 ... -0.009713\n",
" * m (m) float64 216B -0.008994 -0.008274 ... 0.009713\n",
" * orig_pol (orig_pol) <U2 32B 'RR' 'RL' 'LR' 'LL'\n",
" * osa (osa) <U9 540B 'N=0, M=0' 'N=1, M=-1' ... 'N=4, M=4'\n",
" Dimensions without coordinates: time\n",
" Data variables:\n",
" AMPLITUDE (time, chan, pol, u_prime, v_prime) float64 8MB dask.array<chunksize=(1, 1, 4, 510, 510), meta=np.ndarray>\n",
" APERTURE (time, chan, pol, u, v) complex128 17MB dask.array<chunksize=(1, 1, 4, 512, 512), meta=np.ndarray>\n",
" BEAM (time, chan, pol, l, m) complex128 47kB dask.array<chunksize=(1, 1, 4, 27, 27), meta=np.ndarray>\n",
" CORRECTED_PHASE (time, chan, pol, u_prime, v_prime) float64 8MB dask.array<chunksize=(1, 1, 4, 510, 510), meta=np.ndarray>\n",
" ZERNIKE_COEFFICIENTS (time, chan, orig_pol, osa) complex128 960B dask.array<chunksize=(1, 1, 4, 15), meta=np.ndarray>\n",
" ZERNIKE_FIT_RMS (time, chan, orig_pol) complex128 64B dask.array<chunksize=(1, 1, 4), meta=np.ndarray>\n",
" ZERNIKE_MODEL (time, chan, orig_pol, u, v) complex128 17MB dask.array<chunksize=(1, 1, 4, 512, 512), meta=np.ndarray>\n",
" Attributes:\n",
" ddi: ddi_0\n",
" phase_fitting: {'map_0': {'14167000000.0': {'I': {'phase_offset': {'er...\n",
" summary: {'aperture': {'cell size': [-0.05744485294117574, 0.057...\n",
" time_centroid: 5170359434.000001\n",
" zernike_N_order: 4<xarray.DataTree 'ddi_0'>\n",
"Group: /ant_ea25/ddi_0\n",
" Dimensions: (u: 510, v: 510, labels: 172, pars: 3, screws: 4)\n",
" Coordinates:\n",
" * u (u) float64 4kB 14.62 14.56 14.5 ... -14.5 -14.56 -14.62\n",
" * v (v) float64 4kB -14.62 -14.56 -14.5 ... 14.5 14.56 14.62\n",
" * labels (labels) <U22 15kB '1-1' '1-2' '1-3' ... '6-39' '6-40'\n",
" * pars (pars) int64 24B 0 1 2\n",
" * screws (screws) <U2 32B 'il' 'ir' 'ol' 'or'\n",
" Data variables: (12/15)\n",
" AMPLITUDE (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" AMP_NOISE (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" CORRECTIONS (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" DEVIATION (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" MASK (u, v) bool 260kB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" PANEL_DISTRIBUTION (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" ... ...\n",
" PANEL_SCREWS (labels, screws) float64 6kB dask.array<chunksize=(172, 4), meta=np.ndarray>\n",
" PHASE (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" PHASE_CORRECTIONS (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" PHASE_RESIDUALS (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" RADIUS (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" RESIDUALS (u, v) float64 2MB dask.array<chunksize=(510, 510), meta=np.ndarray>\n",
" Attributes: (12/16)\n",
" amp_unit: V\n",
" aperture_resolution: [1.4366515837104072, 1.4366515837104072]\n",
" clip: 0.5412720936531105\n",
" ddi: ddi_0\n",
" fitted: True\n",
" input_gain: 41.57934084050393\n",
" ... ...\n",
" panel_model: rigid\n",
" pol_state: I\n",
" solved: True\n",
" summary: {'aperture': {'cell size': [-0.05744485294117574, 0...\n",
" theoretical_gain: 41.71607063670329\n",
" wavelength: 0.021161322651231735ea25_cal_small_after_fix ... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00\n\n", "text/plain": "ea25_cal_small_after_fix ... \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[36m0:00:00\u001b[0m\n" }, "metadata": {}, "output_type": "display_data" } ], "tabbable": null, "tooltip": null } }, "6b046745e01d49c8985f3a9993efcf5b": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } } }, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 5 }