Reading the detector data

The aim of the workshop is to familiarize students with the CERN ROOT framework. The focus of the activities will be on working with CERN ROOT files, which is a necessary basis for other activities of this summer school. The workshop also includes a small demonstration of visualization facilities in the framework. Then students will use the framework to finish the implementation of a reader required to read the EUSO-SPB1 data. Another task is to finish the implementation of a reader for ESAF ROOT files. Finally, data read from the files will be visualized.

The Air Shower Pattern Recognition summer school consists of the following five workshops:

  1. Introduction to the JEM-EUSO and running air shower event simulations
  2. Reading the detector data
  3. Implementation of a feature extraction method
  4. Classification of the data through the extracted features
  5. Classification of the data using a convolutional neural network

Slides

Practical activity

  1. Writing and reading and ROOT files
  2. Partial Implementation of a Acquisition format reader
  3. Partial implementation of ESAF ROOT file reader (simu2npy) 
  4. Visualization of data from the files

#1 Writing and reading and ROOT files

  1. Login to the remote machine (school.spaceforum.sk) using the personal credentials provided to each student. 
    1. If you also want to be able to visualize plots in GUI connect to the ssh server with X11 forwarding (Optional).
    2. You can use Visual Studio Code’s extension “Visual Studio Code Remote – SSH” (Developing on Remote Machines using SSH and Visual Studio Code)
    3. If you have CERN ROOT on your computer, you can do this task locally.
  2. Change your current working directory to writing_reading_data.
  3. Activate the ROOT environment using thisroot.sh script. Paths apply for the machine school.spaceforum.sk.
    source /opt/ROOT6/bin/thisroot.sh
  4. Create a CERN ROOT file filled by random histograms
    1. Based on official CERN ROOT tutorial https://root.cern/manual/storing_root_objects/ 
    2. Review the file named example_writing.C
    3. Open ROOT by running command: root
      Now you are in the ROOT shell environment (Cling).
    4. Execute macro:
      .x example_writing.C
    5. List contents of your current working directory using:
      .!ls
    6. Check your current directory in ROOT environment:
      gDirectory->pwd()
    7. In the ROOT shell environment, open the created file for reading:
      TFile fdemo("demo.root", “READ”)
    8. Check your current directory in ROOT environment:
      gDirectory->pwd()
    9. List contents of the file
      fdemo.ls()
    10. Create a pointer variable:
      TH1F *h3;
    11. Retrieve the entry “h3” from the file:
      fdemo.GetObject("h3",h3);
    12. Check if it was retrieved:
      h3
    13. Save the histogram as a macro:
      h3->SaveAs("h3.C")
    14. Create a canvas (if you have X11 forwarding, a new window should open):
      TCanvas c1;
    15. Draw the histogram:
      h3->Draw()
    16. Save the canvas as an image:
      c.SaveAs("c1.png")
    17. Exit the ROOT shell:
      .q
  5. Creating and writing a ROOT tree.
    1. Based on the official CERN ROOT tutorial
      https://root.cern/manual/trees/#writing-a-tree 
    2. Copy a file from ROOT tutorials directory to your directory:
      cp -v $ROOTSYS/tutorials/tree/cernbuild.C .
    3. Edit the file, by changing the output file name to the “demo.root” and change the call of the method TFile::Open() to UPDATE mode.
    4. Run the macro
      root -b -q cernbuild.C
    5. Open the demo file in ROOT.
      root demo.root
    6. List the contents of the file:
      _file0->ls()
    7. Print tree information (GetObject happens automatically)
      T->Print()
    8. See more possible activities with the tree here: https://root.cern/manual/trees/#writing-a-tree
    9. If you have X11, open TBrowser,  and browse the file through GUI.
      new TBrowser

#2 Partial Implementation of the acquisition file format reader

The aim of the task is to enable running of the feature extraction procedure on an input file. The feature extraction procedure will be developed at the next lesson. In the framework used here, the feature extraction for a single file is executed by feature_extraction.py script.

  1. TTree-related operations in acquisition data reader (AcqL1EventReader)
    1. event_reading.ttree_reader.TTreeReader:
      • Add calls of appropriate ROOT function to retrieve branch and leaf of a TTree object.
    2. event_reading.acq_l1_event_reader.AcqL1EventReader.__init__:
      • Provide allocated memory block of sufficient size for the branch photon_count_data
    3. event_reading.gtu_pdm_data.GtuPdmDataIterator.__next__ :
      • Retrieve an entry from the tree. The index of the entry is stored in the member variable aer._current_tevent_entry
  2. Run the feature_extraction.py to visualize the data.
    • The script can be executed in the following way:
      python feature_extraction.py ~/data/workshop2/acq.root
    • The script is preconfigured to run the skeleton feature extraction code designated for this workshop.
    • Basic visualization is implemented as a part of the method for processing of a single frame sequence (event_processing.workshop.EventProcessingWorkshop.process_event). The visualization draws three projections of the frame sequence.
  3. Visualize data in ETOS software and compare the results.
  4. Use NumPy to apply an appropriate transformation to match visualization in the ETOS software.
    Make the changes in event_reading.GtuPdmData.__init__

#3 Partial implementation of ESAF ROOT file reader (simu2npy)

  • simu2npy: A converter from ESAF output files to numpy files
  • Activities:
  1. Review the code, observe steps required for working with ESAF under python
    1. Locate method load_etree(self, pathnames, use_tchain=True), observe the usage of ROOT’s TChain.
    2. Locate method create_hierarchy_dicts(self, ev=None)
  2. Review the following ESAF’s files:
    • packages/common/root/include/EEvent.hh
    • packages/common/root/src/EEvent.cc 
    • packages/common/root/include/EDetector.hh
    • packages/common/root/src/EDetector.cc
    • packages/common/root/include/EFee.hh
  3. Finish the implementation of the method create_hierarchy_dicts(), call appropriate EFee getter methods
  4. Run the simu2npy program:
    python simu2npy.py

#4 Visualization of data from the files

  1. Connect to the JupyterHub at http://school.spaceforum.sk
  2. Launch notebook “Event visualization.ipynb

Further reading

The task of ROOT file reading could also be likely handled by using root_numpy package. It is developed within the Scikit-HEP project.

A potentially useful package is Uproot, which enables I/O with ROOT files without ROOT dependency.

Another related package is root_pandas. This allows reading a ROOT file into Pandas DataFrame.

CERN ROOT package provides wide range of functions. We encourage you to review the Getting Started page at the root.cern.