AMON

Files for this exercise

• All files are available on the git repository stored in the faculty’s Gitlab server.
• The address of the project is the following:
  https://git.kpi.fei.tuke.sk/svd/lab-amon
• The repository contains the following files:
  • Visualizing AMON SN02 data.ipynb
    • 7 tasks in total
  • Observing variations in the data.ipynb
    • depends on the result of Visualizing AMON SN02 data.ipynb
    • 3 tasks in total
  • space-lab:
    • plot_aac_data.ipynb – 2 mandatory tasks
    • plot_amon_data.ipynb – only bonus tasks, just run the notebook and observe the results
    • plot_gnss_data.ipynb – only bonus tasks, just run the notebook and observe the results
    • plot_weather_data.ipynb – only bonus tasks, just run the notebook and observe the results
• Data processed in the notebooks are available here:
  • http://michalvrabel.sk/tsf/svd/amon/data/

Setup of the environment

• Setup of the python environment
  • The python and necessary libraries can be installed locally, or the examined jupyter notebooks can be run in cloud-based solutions such as Google Colaboratory.
  • Python 3.X installation
    • Follow the installation steps for your specific platform. For instance, nice installation guide is available at https://realpython.com/installing-python/.
  • Environment setup:
    • Virtual environments (venv, conda) are recommended for the local installation.
      • For pip, the virtual environment can be set-up by the following command:
        python3 -m virtualenv -p python3 venv
# activation
. venv/bin/activate
    • Required non-standard libraries are listed in requirements.txt file
    • If you have problems with some dependency versions don’t bother, try installing any available versions:
      • pip install numpy pandas matplotlib seaborn astropy timezonefinder
    • With pip package installer, all required packages can be installed via:
      pip install -r requirements.txt
  • Google Colab
    • Although the environment already provides many popular packages not all packages are installed.
    • Google Colab supports pip package manager. Thus packages can be installed using pip from the notebook itself (use built-in magic command %%bash).
    • To use files in Google Drive (such as requirements.txt), the Google Drive directories can be mounted in a notebook using google.colab.drive library (see the example notebook). Shell commands can be executed from a jupyter notebook by putting an exclamation point as the first letter of a line (example of shell commands in IPython).

Visualization, filtering and resampling of AMON SN02 data

TASK #1: Visualize the subset referenced by variable day_around_the_peak_df

• Take an inspiration from the code above.
• No resampling is required in this case.

TASK #2: Review the data in AMON-new viewer web application (no code required)

• The viewer is presently avaliable at the following address: http://147.213.198.111/
• The main section of the website related to this activity are “Detail view” and “Single night analysis”

TASK 3#: Calculate sun positions from the detector’s location

• Base your implementation on the code which does this task for the Moon
• Plot zenth angles for both the Sun and the Moon
• See:
  • https://docs.astropy.org/en/stable/api/astropy.coordinates.get_sun.html
  • https://docs.astropy.org/en/stable/coordinates/index.html
  • https://docs.astropy.org/en/stable/coordinates/solarsystem.html

TASK #4: Use appropriate function to convert time zone of subset data (referenced by day_around_the_peak_df)

• Save the result in DataFrame day_around_the_peak_local_df
• See: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#time-series-date-functionality

TASK 5#: Apply mask to the localized data and visualize altitude (elevation) in the local time

• First, apply masks to dataframe with local time (day_around_the_peak_local_df).
• Second, create an updated visualization
• You can base your implementation on the code above

TASK #6: Load file with the precalculated zenith-angles

• Preserve column names date, moon_zenith, sun_zenith,
• Make column date index, and do not forget to parse is as datetime
• Either read the file
  • form a local directory: data/sn02_zen_angles_2017-09-27_2019-02-01.csv.gz
  • or download it from a remote server: http://michalvrabel.sk/tsf/svd/amon/data/sn02_zen_angles_2017-09-27_2018-06-01.csv.gz

TASK #7: Apply the same filter criteria to all data (mexico_w_angles_df) as for the subset analysed above

• Store the result in DataFrame referenced by variable night_mexico_w_angles_df
• See: pd.DataFrame.query()

Observing variations in the data

TASK #1: Resample the data into weekly bins

• Store the result in DataFrame referenced by variable night_mexico_w_angles_df in DataFrame night_mexico_1w_sample
• Make a selection to only use column counts
• See:
  • https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.resample.html
  • Note: when pd.Timedelta() is used more, string formats are supported
  • pd.Timedelta('1day') works, resample('1day') does not.
  • https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#offset-aliases
  • https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Timedelta.html

TASK 2#: overlay observations for each day on top of each other

• Resample date into 1 minute bins and select an average value for each bin.
• Base your code on the code above.
• Find an appropriate workaround to plot different dates on top of each other.
• See:
  • https://matplotlib.org/stable/api/_as_gen/matplotlib.axis.Axis.set_major_formatter.html
  • https://matplotlib.org/stable/api/dates_api.html#date-formatters
  • https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.plot_date.html

TASK 3#: overlay histograms of the observations for each day on top of each other

• Find a suitable visualization apporach so that all data are sufficiently visible.
• Base your code on the code above.
• See:
  • https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.hist.html
  • https://matplotlib.org/stable/gallery/statistics/histogram_histtypes.html

Exercise – AAC data

TASK 1#: Calculate 50% and 90% percentiles for values in a picture captured with the green filter

• Store results in: acc_G_percentile_50, acc_G_percentile_90
• Base your code on the code above.

TASK #2: Subtract N image from G image

• Consider current datatypes of the arrays, allow negative results.
• This task will be expanded at later labs.