Getting started

Scientific motivations of YSPEX

The purpose of “YSPEX” is to quickly and efficiently determinate alleged photometric and astrometric members of a fixed star-forming region through a set of default selection functions, that can also be modified by the user to comply with the pecualiarities of the region.

Its main goal is to automate the standard procedures for the selection of candidates, such as the generation of cross-matching catalogues, spatial and flux completeness, and the construction of colour-magnitude, colour-colour, and kinematic diagrams.

YSPEX will also produce preliminary determinations of Luminosity Function, Initial Mass Function, spatial distribution and the detection of objects with infrared excess. The output catalogue of candidates and their associated selection function will allow further analysis with different tools. The outcome catalogue will also help with the selection of samples for their spectroscopic observation.

(This is a dummy text v2)

Installation

Automatic installation through Docker

Since YSPEX depends on multiple softwares, we provide a simple installation method: a Docker container.

To install Docker, please refer to their official docs.

After you’ve installed Docker on your machine, you can install the container by running the following commands:

docker login docker.pkg.github.com --username <your-github-username>
docker pull docker.pkg.github.com/tcaram/YSPEX/yspex:1.0.0

To run the container, execute:

docker run -d -p 9999:9999 YSPEX/yspex:1.0.0

Manual installation

You will need the following applications:

First, clone the repository from GitHub:

git clone https://github.com/tcaram/YSPEX.git

Install virtualenv from pip, and make an environment so you can have an isolated workspace for the project that doesn’t interfere with other Python versions/packages. This process might take a few minutes.

pip install virtualenv
cd yspex
virtualenv env

Activate the environment:

  • On Unix-like OS:

    cd /env/bin
source activate

  • On Windows:

    cd env/Scripts
activate.bat

Install the required Python packages with pip:

cd ../.. # Go back to the root directory
pip install -r requirements.txt

Install plotly/orca. Please refer to their official installation docs.

Download the STILTS Jarfile and place it in bin/.

The default tab layout of YSPEX

Home

The “Home” tab includes a set of basic user account mannagement tools. The sub-tabs allow to register new users, login into an existing account and logout from it, reset the password from an existing user account and manage the complete set of working sessions (WS) by defining the active WS, creating new ones and delete those that are no longer required.

The “Register”, “Login” and “Reset Password” tools become active only before the user login, while “Sessions” and “Logout” become active only for logged users.

User registration

In order to secure your work, you’re required to register an account before starting to work with YSPEX. To register an account you only need an active mail account and a secure password. You’ll be asked to answer two security questions, whose answers must be kept private, as they can be used to reset your password.

You can create an account in Home -> Register.

Creating working sessions

Users can use YSPEX for working in several projects. In order to keep the work efficiently organized each data set is saved in the user account as a independent Working Sesion. A session is an isolated workspace with its own catalogues. Sessions are linked to accounts, though they can be shared with other users (link to add other users to my session). An account can have multiple sessions, and they’re meant to help to have your catalogues nicely separated by project.

You can create a session in Home -> Sessions -> New session.

After creating the session, make sure to set it as the current working session.

1. Login

2. Register

3. Reset password

4. Logout

5. Sessions

Data Handling

This tab consists on applications related to the management of data and catalogues. You can fetch Photometric and Extinction data from multiple sources (GAIA, 2MASS, WISE, DUST, Gontcharov, etc.) or upload your own. You can also generate random samples, perform cross-matches between catalogues, print a summary of the catalogues or create your own synthetic columns.

1. Own Data

Upload your own catalog

One of the available input options is uploading your own catalog. Your catalog must include at least a column for Declination and a column for Right Ascension. YSPEX will attempt to identify the columns by using their name or their UCDs, if these cannot be found, you’ll be prompted to manually identify them.

The following table formats are supported:

  1. FITS (first table will be read)

  2. ASCII

  3. CSV

Note

Decimal and sexagesimal formats are both supported.

The following compression formats are supported:

  1. gzip

  2. bz2

Attention

‘tar’ files are not supported.

After you’ve uploaded the catalog, a circular region of the sky matching your catalog will be calculated. A mean of the right ascension and declination columns is calculated, then the farthest object is sought and the distance from that object to the previously calculated means will be the radius. A threshold of 1’’ is added so the cone covers the farthest object’s region.

Select a region of the sky

The other available input option is selecting a circular sky region. You’ll be asked to input the corresponding central right ascension and declination, and radius. After you select it, that region will be associated to your working session, and future cone-searches will be performed using that selected region.

2. Photometry

This tab’s function is to look for all point sources in the online catalogues from GAIA, 2MASS and WISE surveys, within the boundaries of the user’s catalog or within the region defined by the user in the ‘Own Data’ tab.

3. Extinction

This tab’s function is to look for the total integrated interstellar extinction towards the direction of each star from the extinction data included in GAIA, Gontcharov and DUST catalogues.

4. Cross Matches

The matches are performed with the tool STILTS from the Starlink project. A new master catalog will be created unifying the data from every catalog, but without repeating the objects (a match will be performed to achieve such thing).

In order to keep the resulting catalog as complete as possible, the general cross-match process was made over a reference catalog which was created as follow:

  1. A temporary catalog is created with the concatenation of the coordinates (right ascension and declination) of the photometric catalogues GAIA, WISE and 2MASS in that order.

  2. A reference catalog is created by performing an internal match in the temporary catalog with the tolerance inputted by the user, in such way that only one row of each object meeting that separation tolerance is kept.

  3. A new master catalog is created by cross-matching the reference catalog to the other catalogues using a ‘left outer join’ (keeping only the point sources that are in the reference catalog, since the reference catalog was internally matched, there should be no repeated sources). Meaning that if the same object was in multiple catalogues, the information from those catalogues is now combined into the same row. When a repetition is found, STILTS keeps the first occurrence, meaning that it will prioritize the order in which the concatenation was made (GAIA, WISE and then 2MASS).

Please note that after performing the cross-match, columns are renamed following YSPEX’s column naming convention. (ref. required).

5. Make Synthetic Column

There are two different methods to make synthetic columns: basic arithmetic operations between columns and for more complex calculations that aren’t offered by default: running Python code.

Additionally, there’s the functionality to randomly select a subset containing a given percentage of the master catalog. Such subset could be useful for a quick look of the data through plotting only the randomly selected subsample which will improve the performance of plot generation.

Arithmetic Operations between columns

Using Python operators you can perform basic arithmetic operations such as addition, subtraction, multiplication and division. Simple math functions (trigonometric, logarithmic and exponential) are also supported.

You’ll also be asked to input a name for the new synthetic column.

Examples:

(gaia.ra + gaia.dec)/2

log(gaia.rpmag)

Run Python code

In this environment you’re allowed to run Python functions from different packages (astropy, numpy, pandas, scipy, math), given that you’ll be coding at a low-level, basic knowledge of the YSPEX internals (ref to dev) is required.

Due to the complexity of this function, a separate section (ref.) with examples has been created.

Random selected portion of the catalog

This tab’s function is to add a new column to the master catalog with two possible values: 1 or 0. The probability of getting whether a 0 or a 1 is set by the user.

Biases

1. Photometry

In this application, we show first approach to the flux uncompletenesses and flux uncertainities through plots showing the magnitude distribution and the photometric error as a function of magnitude for the complete set of ten photometric bands including photometry from 2MASS (J, H, K), GAIA (G, BP, RP) & WISE (W1, W2, W3, W4).

2. Astrometry and RVs

This tab includes plots of the proper motion errors and radial velocity as a function of the brightness in different magnitudes.

3. Spatial Distribution

This tab includes plots of the spatial distribution for every catalog (Master, GAIA, WISE and 2MASS).

Selection

1. Colour-Magnitude Diagram