Outputs

<output?> Block in Input File

Data output in Athena++ is controlled by the <output[n]> blocks in the input file, where [n] is an integer. There is no limit on the total number of outputs. Here is an example:

    <output1>
    file_type   = vtk       # VTK data dump
    variable    = prim      # variables to be output
    id          = primitive # file identifier
    dt          = 0.01      # time interval between outputs
    x3_slice    = 0.0       # slicing at z=0.0
    ghost_zones = 1         # include ghost zones

• file_type (mandatory) : output format (see below)
• variable (mandatory, except for .rst and .hst): output variable (see below)
• id (optional) : output ID used in file names (default = out[n])
• dt (mandatory) : time interval between outputs
• x[1,2,3]_slice (optional) : sliced output at specified position
• ghost_zones (optional) : include ghost zones in output (default = false)
• cartesian_vector (optional) : add vector variables converted into Cartesian coordinates (default = false)

Output Variables

Either a single variable or a set of variables can be specified in the variable field.

• prim : primitive variables (density, velocities, pressure, and magnetic fields)
• cons : conserved variables (density, momenta, total energy, and magnetic fields)
• d : primitive gas density
• D : conservative gas density (the same as d in non-relativistic simulations)
• p : gas pressure
• E : total energy
• v : gas velocities
• m : gas momenta
• bcc : (cell-centered) magnetic fields
• uov (user_out_var) : user output variables (see below)

Output Formats

Currently the following output formats are supported.

• History File (file_type = hst)*
• Formatted Table (tab)
• VTK (vtk)
• HDF5 (hdf5)*
• Restart (rst)*

* = parallel output

History File

A history file (.hst) is a special file that contains global sums of basic physical variables such as the mass, momenta, and energy. The format is a simple text table. This file is useful for roughly monitoring the simulation as well as a sanity check of the code. By default, it includes the time, time step, total mass, momenta, kinetic energies in three directions, total energy, and magnetic energies in three directions. Users can add user-defined variables (see below).

Formatted Table

This is a simple text output, and is useful for relatively small 1D and 2D simulations. The data can be visualized using gnuplot or similar plotting software. Because the data size tends to become large, this format is not recommended for large 3D simulations. The file extension is .tab.

VTK

VTK (Visualization Tool Kit) is a standard data format commonly used in numerical simulations. It can be easily visualized using various visualization software, such as VisIt and ParaView. The file extension is .vtk.

HDF5

Athena++ can output files formatted according to the HDF5 (Hierarchical Data Format) standard. This format is the most suitable one for simulations with mesh refinement. It also has the advantage of working with parallel IO, meaning more than one process can write into a single file, scaling well when used with a parallel file system like GPFS.

This output generates two files per output timestep: .athdf and .athdf.xdmf. The .athdf file is HDF5 and contains the data. The .athdf.xdmf (eXtensible Data Model and Format) file is an auxiliary file containing a description of the data structure stored in the .athdf file. This file is used when visualizing the results using VisIt or ParaView. Also, yt supports the Athena++ HDF5 output.

In order to enable this option, the HDF5 library is needed and the code must be configured with the -hdf5 option. Also note that a parallel version of the HDF5 library is required for parallel simulations with MPI.

HDF5 files are self-describing, rather than adhering to a single, exact layout. One can view the structure using a tool like HDFVIEW provided by the HDF Group. Athena++ has its own particular HDF5 format.

Restart File

A restarting file (.rst) is used literally for restarting simulations. For example, on many supercomputing systems, the time allowed for one run is limited, and you may want to continue your simulations over multiple runs. This is a parallel format using MPI-IO, so only one file per output timestep is generated but no external library other than MPI is required. Because this file necessarily contains all the information in the simulation, the variable field is ignored. When the code is launched with -t hh:mm:ss option, a restarting file named *.final.rst that contains the data at the end of the simulation is created. Allow some time for the final output as the code makes the final output after the specified time limit is reached.

To restart a simulation, simply specify a restarting file using the -r option instead of -i

    > athena -r example.out1.00010.rst

Note that you can change the number of MPI processes when you restart, as long as the number of MPI processes is fewer than that of MeshBlocks. This is useful for AMR simulations, when many refinement levels are created as simulations proceed.

File Names

For serial output formats (Formatted table and VTK), one file is created for each MeshBlock per every output timestep. Output filenames follow the convention

    [problem_id].block[block_id].[output_id].?????.[type]

where problem_id is the Problem ID specified in the input file, [block_id] is the global ID of MeshBlock, [output_id] is the output ID, ????? is the output step, and [type] is the file extension for the specified output format.

For parallel output formats (HDF5), only one file is created per every output timestep, no matter how many MeshBlocks exist. This is convenient for massively parallel simulations. Output filenames follow the convention

    [problem_id].[output_id].?????.[type]

And the restarting output uses the following file names:

    [problem_id].?????.rst

Please note that these file names are different from those in the old Athena code. In particular, now MeshBlock ID is used instead of process ID in Athena.

User Output Variables

User Output Variables (user_out_var in the code) are user-defined variables associated with each cell. To enable these variables, specify the number of fields in the MeshBlock::InitUserMeshBlockData function in your problem generator file:

    void MeshBlock::InitUserMeshBlockData(ParameterInput *pin)
    {
      AllocateUserOutputVariables(2);
      return;
    }

This function sets MeshBlock::nuser_out_var to be 2, and allocates a 4-dimensional AthenaArray<Real> user_out_var. Then specify an output block in your input file:

    <output2>
    file_type   = hdf5
    dt          = 0.1
    variable    = uov  # (or user_out_var)

Store the data you want to output in the MeshBlock::UserWorkBeforeOutput function in your problem generator file (note: formerly UserWorkInLoop was used for this purpose, but now this new dedicated interface is provided because calculation of uov can be expensive). For example, in order to calculate and save the gas temperature and plasma beta:

    void MeshBlock::UserWorkBeforeOutput(ParameterInput *pin)
    {
      for(int k=ks; k<=ke; k++) {
        for(int j=js; j<=je; j++) {
          for(int i=is; i<=ie; i++) {
            Real pmag = 0.5*(SQR(pfield->bcc(IB1,k,j,i))
                            +SQR(pfield->bcc(IB2,k,j,i))
                            +SQR(pfield->bcc(IB3,k,j,i)));
            user_out_var(0,k,j,i) = phydro->w(IPR,k,j,i)/phydro->w(IDN,k,j,i);
            user_out_var(1,k,j,i) = phydro->w(IPR,k,j,i)/pmag;
          }
        }
      }
    }

Note that if you want to output the data in the boundary ghost cells, the loop limits should be from ie-NGHOST to ie+NGHOST, etc. The user_out_var array will be automatically destroyed at the end of the simulation, so you do not have to delete it.

Also, this function is called before output, except the history output because the history output is intended for more frequent and quick analysis. To add user-defined variables in the history file, please see the next section.

User-Defined History Output

Users can add any variable in the history output. First, define a function to calculate a sum of user-defined variables within a MeshBlock. The history output automatically calculates a global sum using this function. The example below calculates divergence of magnetic fields (note: Athena++'s constrained transport scheme should conserve ∇⋅B = 0 to machine precision as long as the code and the initial condition are correct).

Real DivergenceB(MeshBlock *pmb, int iout)
{
  Real divb=0;
  int is=pmb->is, ie=pmb->ie, js=pmb->js, je=pmb->je, ks=pmb->ks, ke=pmb->ke;
  AthenaArray<Real> face1, face2p, face2m, face3p, face3m;
  FaceField &b = pmb->pfield->b;

  face1.NewAthenaArray((ie-is)+2*NGHOST+2);
  face2p.NewAthenaArray((ie-is)+2*NGHOST+1);
  face2m.NewAthenaArray((ie-is)+2*NGHOST+1);
  face3p.NewAthenaArray((ie-is)+2*NGHOST+1);
  face3m.NewAthenaArray((ie-is)+2*NGHOST+1);

  for(int k=ks; k<=ke; k++) {
    for(int j=js; j<=je; j++) {
      pmb->pcoord->Face1Area(k,   j,   is, ie+1, face1);
      pmb->pcoord->Face2Area(k,   j+1, is, ie,   face2p);
      pmb->pcoord->Face2Area(k,   j,   is, ie,   face2m);
      pmb->pcoord->Face3Area(k+1, j,   is, ie,   face3p);
      pmb->pcoord->Face3Area(k,   j,   is, ie,   face3m);
      for(int i=is; i<=ie; i++) {
        divb+=(face1(i+1)*b.x1f(k,j,i+1)-face1(i)*b.x1f(k,j,i)
              +face2p(i)*b.x2f(k,j+1,i)-face2m(i)*b.x2f(k,j,i)
              +face3p(i)*b.x3f(k+1,j,i)-face3m(i)*b.x3f(k,j,i));
      }
    }
  }

  face1.DeleteAthenaArray();
  face2p.DeleteAthenaArray();
  face2m.DeleteAthenaArray();
  face3p.DeleteAthenaArray();
  face3m.DeleteAthenaArray();

  return divb;
}

Then, in the InitUserMeshData function, set the number of the additional variables you want using the AllocateUserHistoryOutput function, and enroll the function defined above.

    void Mesh::InitUserMeshData(ParameterInput *pin)
    {
      AllocateUserHistoryOutput(1);
      EnrollUserHistoryOutput(0, DivergenceB, "divB");
      return;
    }

The first parameter of EnrollUserHistoryOutput is the index of the user-defined variable, and the third is its name. If you need more history outputs, simply put a larger number in AllocateUserHistoryOutput(n) and enroll the output function with a larger index. The index should be from 0 to n-1 where n is the number of the allocated output variables.

A user-defined history output function receives the index of the output in the second parameter (e.g., iout = 0 means it is the first user-defined output, and iout = 1 means it is the second). Users can reuse the same output function using this parameter to control its behavior. For example, one can calculate the enclosed masses within different radii using the same function (e.g., r < 1 for iout = 0, r < 2 for iout = 1, ...).

Visualization Software

See Analysis Tools.