Quickstart for Carsus SQL

This notebook provides a quick introduction in using Carsus with the SQL interface.

Creating a Database

To start working with a database you need to initialize it. This requires an url, for example sqlite:///path/to/example.db.

In this guide we’ll use a in-memory sqlite database, so we leave the url empty:

[1]:
from carsus import init_db
session = init_db()
[                carsus.base][   INFO] - Initializing the database at sqlite:// (base.py:32)
[                carsus.base][   INFO] - Ingesting basic atomic data. (base.py:45)

Now, we have an SQLAlchemy Session object named session. We can use session to make simple queries. As you can see from the output, some basic atomic data has been ingested into our database.

Let’s examine it:

[2]:
from carsus.model import Atom

print('Number of Atoms in the database: {}'.format(session.query(Atom).count()))
si = session.query(Atom).get(14)

print('Symbol: {}, atomic_number: {}, name: {}'.format(si.symbol, si.atomic_number, si.name))
Number of Atoms in the database: 118
Symbol: Si, atomic_number: 14, name: Silicon

So, our database already contains basic information about atoms. But this information has not been commited to the database yet. It is always your responsibility to commit the data!.

[3]:
session.commit()

Atomic Weights and Ionization Energies

To store more interesting data we are going to use ingesters. For each data source supported by Carsus there is a corresponding ingester class.

For example, to ingest atomic weights from the NIST Atomic Weights and Isotopic Compositions database:

[4]:
from carsus.io.nist import NISTWeightsCompIngester

weightscomp_ingester = NISTWeightsCompIngester(session)
weightscomp_ingester.ingest()

session.commit()
 ChiantiPy version 0.8.4
 found PyQt5 widgets
 using PyQt5 widgets
[ carsus.io.nist.weightscomp][   INFO] - Downloading data from the carsus-dat-nist repository (weightscomp.py:72)
[                py.warnings][WARNING] - /home/runner/micromamba/envs/carsus/lib/python3.7/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'raw.githubusercontent.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  InsecureRequestWarning,
 (warnings.py:110)
[ carsus.io.nist.weightscomp][   INFO] - Ingesting atomic weights from `nist`. (weightscomp.py:225)
[5]:
print(si.weights)
[<Quantity: 28.085 u>]

And the ionization energies from the NIST Atomic Spectra Database:

[6]:
from carsus.io.nist import NISTIonizationEnergiesIngester

ioniz_energies_ingester = NISTIonizationEnergiesIngester(session, spectra='H-Mg')
ioniz_energies_ingester.ingest(ionization_energies=True, ground_levels=True)

session.commit()
[  carsus.io.nist.ionization][   INFO] - Downloading ionization energies from the carsus-data-nist repo. (ionization.py:89)
[                py.warnings][WARNING] - /home/runner/micromamba/envs/carsus/lib/python3.7/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'raw.githubusercontent.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  InsecureRequestWarning,
 (warnings.py:110)
[  carsus.io.nist.ionization][   INFO] - Ingesting ionization energies from `nist-asd`. (ionization.py:331)
[  carsus.io.nist.ionization][   INFO] - Ingesting ground levels from `nist-asd`. (ionization.py:356)

Levels, Lines and Collisions

The Carsus SQL interface supports only two sources of energy levels and spectral lines: GFALL and CHIANTI.

GFALL

The Robert Kurucz’s Atomic Linelist (GFALL) is the main source of levels and lines in Carsus.

[7]:
from carsus.io.kurucz import GFALLIngester

gfall_url = 'http://kurucz.harvard.edu/linelists/gfall/gfall.dat'
gfall_ingester = GFALLIngester(session, fname=gfall_url, ions='H-Mg')
gfall_ingester.ingest(levels=True, lines=True)

session.commit()
[     carsus.io.kurucz.gfall][WARNING] - A specific combination to identify unique levels from GFALL data has not been given. Defaulting to ["energy", "j"]. (gfall.py:129)
[     carsus.io.kurucz.gfall][   INFO] - Parsing GFALL from: http://kurucz.harvard.edu/linelists/gfall/gfall.dat (gfall.py:182)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels from `ku_latest`. (gfall.py:574)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for He 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for He 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Li 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Li 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Be 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Be 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Be 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for B 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for B 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for B 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for B 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for C 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for C 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for C 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for C 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for N 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for N 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for N 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for N 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for N 4. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for N 5. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for O 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for O 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for O 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for O 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for O 4. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for O 5. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for F 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for F 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for F 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for F 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for F 4. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for F 5. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Ne 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Ne 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Ne 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Ne 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Ne 4. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Ne 5. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Na 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Na 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Na 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Na 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Na 4. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Na 5. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Mg 0. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Mg 1. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Mg 2. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Mg 3. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Mg 4. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting levels for Mg 5. (gfall.py:586)
[     carsus.io.kurucz.gfall][   INFO] - Extracting line data: atomic_number, ion_charge, energy_lower, j_lower, energy_upper, j_upper, wavelength, loggf. (gfall.py:425)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines from `ku_latest`. (gfall.py:628)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for He 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for He 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Li 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Li 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Be 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Be 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Be 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for B 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for B 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for B 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for B 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for C 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for C 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for C 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for C 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for N 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for N 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for N 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for N 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for N 4. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for N 5. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for O 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for O 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for O 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for O 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for O 4. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for O 5. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for F 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for F 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for F 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for F 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for F 4. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for F 5. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Ne 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Ne 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Ne 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Ne 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Ne 4. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Ne 5. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Na 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Na 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Na 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Na 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Na 4. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Na 5. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Mg 0. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Mg 1. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Mg 2. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Mg 3. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Mg 4. (gfall.py:640)
[     carsus.io.kurucz.gfall][   INFO] - Ingesting lines for Mg 5. (gfall.py:640)

CHIANTI

The Chianti Atomic Database is also a source of collision strengths.

[8]:
from carsus.io.chianti_ import ChiantiIngester

chianti_ingester = ChiantiIngester(session, ions='O 0-2')
chianti_ingester.ingest(levels=True, lines=True, collisions=True)

session.commit()
[carsus.io.chianti_.chianti_][   INFO] - Ingesting levels from `chianti_v9.0.1`. (chianti_.py:349)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting levels for O 0. (chianti_.py:367)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting levels for O 1. (chianti_.py:367)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting levels for O 2. (chianti_.py:367)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting lines from `chianti_v9.0.1`. (chianti_.py:391)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting lines for O 0. (chianti_.py:409)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting lines for O 1. (chianti_.py:409)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting lines for O 2. (chianti_.py:409)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting collisions from `chianti_v9.0.1`. (chianti_.py:451)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting collisions for O 0. (chianti_.py:469)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting collisions for O 1. (chianti_.py:469)
[carsus.io.chianti_.chianti_][   INFO] - Ingesting collisions for O 2. (chianti_.py:469)

Zeta Data

The Knox S. Long’s ground state recombinations fractions (\(\zeta\)).

[9]:
from carsus.io.zeta import KnoxLongZetaIngester

zeta_url = 'https://raw.githubusercontent.com/tardis-sn/carsus/master/carsus/data/knox_long_recombination_zeta.dat'
zeta_ingester = KnoxLongZetaIngester(session, zeta_url)
zeta_ingester.ingest()

session.commit()

Dump to HDF5

Once you have a database, it’s very easy to create atomic data files for TARDIS using a special class called AtomData. This class takes session as its first argument; other important parameters are selected_atoms and chianti_ions (if any).

Only data for selected_atoms will be stored in the output DataFrames and levels and lines for chianti_ions will be taken from the data source with the same name.

[10]:
from carsus.io.output import AtomData
atom_data = AtomData(session,
                     selected_atoms='H-Mg',
                     chianti_short_name='chianti_v9.0.1',
                     chianti_ions='O 0-2')

Warning:

If chianti_short_name does NOT match with your current version of Chianti DB the above command will fail.

The stored DataFrames can be easily accessed as attributes of atom_data:

[11]:
atom_data.atom_masses_prepared
[11]:
symbol name mass
atomic_number
1 b'H' b'Hydrogen' 1.007975
2 b'He' b'Helium' 4.002602
3 b'Li' b'Lithium' 6.967500
4 b'Be' b'Beryllium' 9.012183
5 b'B' b'Boron' 10.813500
6 b'C' b'Carbon' 12.010600
7 b'N' b'Nitrogen' 14.006855
8 b'O' b'Oxygen' 15.999400
9 b'F' b'Fluorine' 18.998403
10 b'Ne' b'Neon' 20.179700
11 b'Na' b'Sodium' 22.989769
12 b'Mg' b'Magnesium' 24.305500
[12]:
atom_data.levels_prepared.head(20)
[                py.warnings][WARNING] - /home/runner/micromamba/envs/carsus/lib/python3.7/site-packages/pandas/core/series.py:679: RuntimeWarning: divide by zero encountered in log10
  result = getattr(ufunc, method)(*inputs, **kwargs)
 (warnings.py:110)
[12]:
energy g metastable
atomic_number ion_number level_number
1 0 0 0.000000 2 True
1 0 0.000000 1 True
2 0 0 0.000000 1 True
1 19.819626 3 True
2 20.615788 1 True
3 20.964099 5 False
4 20.964109 3 False
5 20.964117 5 True
6 20.964117 9 True
7 20.964231 1 False
8 21.218036 3 False
9 22.718479 3 False
10 22.920331 1 False
11 23.007085 5 False
12 23.007087 3 False
13 23.007090 5 True
14 23.007121 1 False
15 23.073663 7 False
16 23.073663 5 False
17 23.073664 7 True
[13]:
atom_data.macro_atom_prepared.head(20)
[13]:
atomic_number ion_number source_level_number destination_level_number transition_type transition_probability transition_line_id
0 2 0 0 142 1 0.000000e+00 69
1 2 0 0 132 1 0.000000e+00 70
2 2 0 0 122 1 0.000000e+00 71
3 2 0 0 110 1 0.000000e+00 72
4 2 0 0 96 1 0.000000e+00 73
5 2 0 0 81 1 0.000000e+00 74
6 2 0 0 64 1 0.000000e+00 75
7 2 0 0 50 1 0.000000e+00 76
8 2 0 0 35 1 0.000000e+00 77
9 2 0 0 20 1 0.000000e+00 78
10 2 0 0 8 1 0.000000e+00 79
11 2 0 1 164 1 1.012837e+09 145
12 2 0 1 155 1 1.248469e+09 146
13 2 0 1 146 1 1.561061e+09 147
14 2 0 1 136 1 1.994064e+09 148
15 2 0 1 126 1 2.596821e+09 149
16 2 0 1 114 1 1.149997e+09 151
17 2 0 1 115 1 1.908520e+09 152
18 2 0 1 100 1 1.602380e+09 154
19 2 0 1 99 1 2.659288e+09 155

Finally, use the to_hdf() method to create an HDF5 file.

[14]:
atom_data.to_hdf('kurucz_H-Mg_chianti_O_I-III.h5',
                 store_atom_masses=True,
                 store_ionization_energies=True,
                 store_levels=True,
                 store_lines=True,
                 store_collisions=True,
                 store_macro_atom=True,
                 store_zeta_data=True)
[                py.warnings][WARNING] - /home/runner/micromamba/envs/carsus/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3552: PerformanceWarning:
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->bytes,key->block0_values] [items->Index(['symbol', 'name'], dtype='object')]

  exec(code_obj, self.user_global_ns, self.user_ns)
 (warnings.py:110)
[                py.warnings][WARNING] - /home/runner/micromamba/envs/carsus/lib/python3.7/site-packages/pyarrow/serialization.py:165: FutureWarning: The SparseDataFrame class is removed from pandas. Accessing it from the top-level namespace will also be removed in the next version
  if isinstance(obj, pd.SparseDataFrame):
 (warnings.py:110)
[                py.warnings][WARNING] - /home/runner/micromamba/envs/carsus/lib/python3.7/site-packages/pyarrow/serialization.py:176: FutureWarning: The SparseSeries class is removed from pandas. Accessing it from the top-level namespace will also be removed in the next version
  if isinstance(obj, pd.SparseSeries):
 (warnings.py:110)
[   carsus.io.output.tardis_][   INFO] - Signing AtomData:
MD5: 4204f1d3261dd980fe309e34b5d74592
UUID1: d4583d06fb3a11ee91f86045bd4cbafe (tardis_.py:1180)

You are done! Now you can use your file to run TARDIS simulations.