3 Exporting tree with data

3.1 Introduction

The treeio package (Wang et al., 2020) supports parsing various phylogenetic tree file formats including software outputs that contain evolutionary evidence. Some of the formats are just log files (e.g., PAML and r8s output), while some of the others are non-standard formats (e.g., BEAST and MrBayes output that introduce a square bracket, which was reserved to store comments in standard Nexus format, to store inferences). With treeio, we are now able to parse these files to extract phylogenetic trees and map associated data on the tree structure. Exporting tree structure is easy, users can use the as.phyo() method defined in treeio to convert a treedata object to a phylo object and then use write.tree() or write.nexus() implemented in ape package (Paradis et al., 2004) to export the tree structure as Newick text or Nexus file. This is quite useful for converting non-standard formats to a standard format and for extracting trees from software outputs, such as log files.

However, exporting a tree with associated data is still challenging. These associated data can be parsed from analysis programs or obtained from external sources (e.g., phenotypic data, experimental data, and clinical data). The major obstacle here is that there is no standard format designed for storing a tree with data. NeXML (Vos et al., 2012) may be the most flexible format. However, it is currently not widely supported. Most of the analysis programs in this field rely extensively on Newick string and Nexus format. In my opinion, although BEAST Nexus format may not be the best solution, it is currently a good approach for storing heterogeneous associated data. The beauty of the format is that all the annotated elements are stored within square brackets, which are reserved for comments. In this way, existing programs that can read standard Nexus format are able to parse it by ignoring the annotated elements.

3.2 Exporting Tree Data to BEAST Nexus Format

3.2.1 Exporting/converting software output

The treeio package (Wang et al., 2020) provides the write.beast() function to export treedata object as BEAST Nexus file (Bouckaert et al., 2014). With treeio, it is easy to convert software output to BEAST format if the output can be parsed by treeio (see Chapter 1).

Here is an example of converting NHX file to BEAST format:

nhxfile <- system.file("extdata/NHX", "phyldog.nhx", package="treeio")
nhx <- read.nhx(nhxfile)
# write.beast(nhx, file = "phyldog.tree")
write.beast(nhx)

#NEXUS
[R-package treeio, Thu Oct 14 11:24:19 2021]

BEGIN TAXA;
    DIMENSIONS NTAX = 16;
    TAXLABELS
        Prayidae_D27SS7@2825365
        Kephyes_ovata@2606431
        Chuniphyes_multidentata@1277217
        Apolemia_sp_@1353964
        Bargmannia_amoena@263997
        Bargmannia_elongata@946788
        Physonect_sp_@2066767
        Stephalia_dilata@2960089
        Frillagalma_vityazi@1155031
        Resomia_ornicephala@3111757
        Lychnagalma_utricularia@2253871
        Nanomia_bijuga@717864
        Cordagalma_sp_@1525873
        Rhizophysa_filiformis@3073669
        Hydra_magnipapillata@52244
        Ectopleura_larynx@3556167
    ;
END;
BEGIN TREES;
    TRANSLATE
        1   Prayidae_D27SS7@2825365,
        2   Kephyes_ovata@2606431,
        3   Chuniphyes_multidentata@1277217,
        4   Apolemia_sp_@1353964,
        5   Bargmannia_amoena@263997,
        6   Bargmannia_elongata@946788,
        7   Physonect_sp_@2066767,
        8   Stephalia_dilata@2960089,
        9   Frillagalma_vityazi@1155031,
        10  Resomia_ornicephala@3111757,
        11  Lychnagalma_utricularia@2253871,
        12  Nanomia_bijuga@717864,
        13  Cordagalma_sp_@1525873,
        14  Rhizophysa_filiformis@3073669,
        15  Hydra_magnipapillata@52244,
        16  Ectopleura_larynx@3556167
    ;
    TREE * UNTITLED = [&R] (((1[&Ev=S,ND=0,S=58]:0.0682841,(2[&Ev=S,ND=1,
S=69]:0.0193941,3[&Ev=S,ND=2,S=70]:0.0121378)[&Ev=S,ND=3,S=60]:0.0217782)
[&Ev=S,ND=4,S=36]:0.0607598,((4[&Ev=S,ND=9,S=31]:0.11832,(((5[&Ev=S,ND=10,
S=37]:0.0144549,6[&Ev=S,ND=11,S=38]:0.0149723)[&Ev=S,ND=12,S=33]:0.0925388,
7[&Ev=S,ND=13,S=61]:0.077429)[&Ev=S,ND=14,S=24]:0.0274637,(8[&Ev=S,ND=15,
S=52]:0.0761163,((9[&Ev=S,ND=16,S=53]:0.0906068,10[&Ev=S,ND=17,S=54]:1e-06)
[&Ev=S,ND=18,S=45]:1e-06,((11[&Ev=S,ND=19,S=65]:0.120851,12[&Ev=S,ND=20,
S=71]:0.133939)[&Ev=S,ND=21,S=56]:1e-06,13[&Ev=S,ND=22,S=64]:0.0693814)
[&Ev=S,ND=23,S=46]:1e-06)[&Ev=S,ND=24,S=40]:0.0333823)[&Ev=S,ND=25,S=35]:
1e-06)[&Ev=D,ND=26,S=24]:0.0431861)[&Ev=S,ND=27,S=19]:1e-06,14[&Ev=S,ND=28,
S=26]:0.22283)[&Ev=S,ND=29,S=17]:0.0292362)[&Ev=D,ND=8,S=17]:0.185603,
(15[&Ev=S,ND=5,S=16]:0.0621782,16[&Ev=S,ND=6,S=15]:0.332505)[&Ev=S,ND=7,
S=12]:0.185603)[&Ev=S,ND=30,S=9];
END;

Another example of converting CODEML output to BEAST format:

mlcfile <- system.file("extdata/PAML_Codeml", "mlc", package="treeio")
ml <- read.codeml_mlc(mlcfile)
# write.beast(ml, file = "codeml.tree")
write.beast(ml) # output not shown

Some software tools that do not support these outputs can be supported through data conversion. For example, we can convert the NHX file to BEAST file and use NHX tags to color the tree using FigTree (Figure 3.1A) or convert CODEML output and use d_N/d_S, d_N, or d_S to color the tree in FigTree (Figure 3.1B). Before conversion, these files could not be opened in Figtree. Treeio’s conversion function makes data available to other software tools and expands the application range of these tools.

Visualizing BEAST file in FigTree. Directly visualizing NHX file (A) and CodeML output (B) in FigTree is not supported. treeio can convert these files to BEAST compatible NEXUS format which can be directly opened in FigTree and visualized together with annotated data.

FIGURE 3.1: Visualizing BEAST file in FigTree. Directly visualizing NHX file (A) and CodeML output (B) in FigTree is not supported. treeio can convert these files to BEAST compatible NEXUS format which can be directly opened in FigTree and visualized together with annotated data.

3.2.2 Combining tree with external data

Using the utilities provided by tidytree and treeio, it is easy to link external data onto the corresponding phylogeny. The write.beast() function enables users to export the tree with external data to a single tree file.

phylo <- as.phylo(nhx)
## save space for printing the tree text
phylo$edge.length <- round(phylo$edge.length, 2)

## print the newick text
write.tree(phylo)

(((Prayidae_D27SS7@2825365:0.07,(Kephyes_ovata@2606431:0.02,
Chuniphyes_multidentata@1277217:0.01):0.02):0.06,((Apolemia_sp_@1353964:0.12,
(((Bargmannia_amoena@263997:0.01,Bargmannia_elongata@946788:0.01):0.09,
Physonect_sp_@2066767:0.08):0.03,(Stephalia_dilata@2960089:0.08,
((Frillagalma_vityazi@1155031:0.09,Resomia_ornicephala@3111757:0):0,
((Lychnagalma_utricularia@2253871:0.12,Nanomia_bijuga@717864:0.13):0,
Cordagalma_sp_@1525873:0.07):0):0.03):0):0.04):0,Rhizophysa_filiformis@3073669:
0.22):0.03):0.19,(Hydra_magnipapillata@52244:0.06,
Ectopleura_larynx@3556167:0.33):0.19);

N <- Nnode2(phylo)
fake_data <- tibble(node = 1:N, fake_trait = round(rnorm(N), 2), 
                    another_trait = round(runif(N), 2))
fake_tree <- full_join(phylo, fake_data, by = "node")
# write.beast(fake_tree)

## to save space, use a subtree
fake_tree2 = tree_subset(fake_tree, node=27, levels_back=0)
write.beast(fake_tree2)

#NEXUS
[R-package treeio, Tue Nov 16 10:13:32 2021]

BEGIN TAXA;
    DIMENSIONS NTAX = 5;
    TAXLABELS
        Frillagalma_vityazi@1155031
        Resomia_ornicephala@3111757
        Lychnagalma_utricularia@2253871
        Nanomia_bijuga@717864
        Cordagalma_sp_@1525873
    ;
END;
BEGIN TREES;
    TRANSLATE
        1   Frillagalma_vityazi@1155031,
        2   Resomia_ornicephala@3111757,
        3   Lychnagalma_utricularia@2253871,
        4   Nanomia_bijuga@717864,
        5   Cordagalma_sp_@1525873
    ;
    TREE * UNTITLED = [&R] ((1[&fake_trait=-1.42,another_trait=0.17]:0.09,
2[&fake_trait=0.19,another_trait=0.04]:0)[&fake_trait=0.85,
another_trait=0.56]:0,(5[&fake_trait=0.22,another_trait=0.73]:0.07,
(3[&fake_trait=0.02,another_trait=0.29]:0.12,4[&fake_trait=-1.29,
another_trait=0.35]:0.13)[&fake_trait=-0.33,another_trait=0.88]:0)
[&fake_trait=0.27,another_trait=0.94]:0):0.29;
END;

After merging, the fake_trait and another_trait stored in fake_data will be linked to the tree, phylo, and stored in the treedata object, the fake_tree. The write.beast() function exports the tree with associated data to a single BEAST format file. The associated data can be used to visualize the tree using ggtree (Figure 5.8) or FigTree (Figure 3.1).

3.2.3 Merging tree data from different sources

Not only Newick tree text can be combined with associated data, but also tree data obtained from software output can be combined with external data, as well as different tree objects can be merged (for details, see Chapter 2).

## combine tree object with data
tree_with_data <- full_join(nhx, fake_data, by = "node")
tree_with_data

## 'treedata' S4 object that stored information
## of
##  '/home/ygc/R/library/treeio/extdata/NHX/phyldog.nhx'.
## 
## ...@ phylo:
## 
## Phylogenetic tree with 16 tips and 15 internal nodes.
## 
## Tip labels:
##   Prayidae_D27SS7@2825365, Kephyes_ovata@2606431,
## Chuniphyes_multidentata@1277217,
## Apolemia_sp_@1353964, Bargmannia_amoena@263997,
## Bargmannia_elongata@946788, ...
## 
## Rooted; includes branch lengths.
## 
## with the following features available:
##   'Ev', 'ND', 'S', 'fake_trait', 'another_trait'.

## merge two tree object
tree2 <- merge_tree(nhx, fake_tree)
identical(tree_with_data, tree2)

## [1] TRUE

After merging data from different sources, the tree with the associated data can be exported into a single file.

outfile <- tempfile(fileext = ".tree")
write.beast(tree2, file = outfile)

The output BEAST Nexus file can be imported into R using the read.beast function and all the associated data can be used to annotate the tree using ggtree (Yu et al., 2017).

read.beast(outfile)

## 'treedata' S4 object that stored information
## of
##  '/tmp/RtmpfSfWKq/file1aaef634ae0b9.tree'.
## 
## ...@ phylo:
## 
## Phylogenetic tree with 16 tips and 15 internal nodes.
## 
## Tip labels:
##   Prayidae_D27SS7@2825365, Kephyes_ovata@2606431,
## Chuniphyes_multidentata@1277217,
## Apolemia_sp_@1353964, Bargmannia_amoena@263997,
## Bargmannia_elongata@946788, ...
## 
## Rooted; includes branch lengths.
## 
## with the following features available:
##   'another_trait', 'Ev', 'fake_trait', 'ND', 'S'.

3.3 Exporting Tree Data to the jtree Format

The treeio package (Wang et al., 2020) provides the write.beast() function to export treedata to BEAST Nexus file. This is quite useful to convert file format, combine tree with data and merge tree data from different sources as we demonstrated in the previous session. The treeio package also supplies the read.beast() function to parse the output file of the write.beast() function. Although with treeio, the R community has the ability to manipulate BEAST Nexus format and process tree data, there is still a lacking library/package for parsing BEAST files in other programming languages.

JSON (JavaScript Object Notation) is a lightweight data-interchange format and is widely supported in almost all modern programming languages. To make it easy to import a tree with data in other programming languages, treeio supports exporting a tree with data in the jtree format, which is JSON-based and can be easy to parse using any language that supports JSON.

# write.jtree(tree2)

# to save space, use a subtree
tree3 <- tree_subset(tree2, node=24, levels_back=0) 
write.jtree(tree3)

{
    "tree": "(Physonect_sp_@2066767:0.077429{3},(Bargmannia_amoena@263997:0.0144549
{1},Bargmannia_elongata@946788:0.0149723{2}):0.0925388{5}):0.28549{4};",
    "data":[
  {
    "edge_num": 1,
    "Ev": "S",
    "ND": 10,
    "S": 37,
    "fake_trait": -0.69,
    "another_trait": 0.42
  },
  {
    "edge_num": 2,
    "Ev": "S",
    "ND": 11,
    "S": 38,
    "fake_trait": -0.95,
    "another_trait": 0.38
  },
  {
    "edge_num": 3,
    "Ev": "S",
    "ND": 13,
    "S": 61,
    "fake_trait": 0.59,
    "another_trait": 0.65
  },
  {
    "edge_num": 4,
    "Ev": "S",
    "ND": 14,
    "S": 24,
    "fake_trait": -0.69,
    "another_trait": 0.06
  },
  {
    "edge_num": 5,
    "Ev": "S",
    "ND": 12,
    "S": 33,
    "fake_trait": -0.58,
    "another_trait": 0.4
  }
],
    "metadata": {"info": "R-package treeio", "data": "Tue Nov 16 10:21:20 2021"}
}

The jtree format is based on JSON and can be parsed using JSON parser.

jtree_file <- tempfile(fileext = '.jtree')
write.jtree(tree3, file = jtree_file)
jsonlite::fromJSON(jtree_file)

$tree
[1] "(Physonect_sp_@2066767:0.077429{3},(Bargmannia_amoena@263997:0.0144549{1},
Bargmannia_elongata@946788:0.0149723{2}):0.0925388{5}):0.28549{4};"

$data
  edge_num Ev ND  S fake_trait another_trait
1        1  S 10 37      -0.69          0.42
2        2  S 11 38      -0.95          0.38
3        3  S 13 61       0.59          0.65
4        4  S 14 24      -0.69          0.06
5        5  S 12 33      -0.58          0.40

$metadata
$metadata$info
[1] "R-package treeio"

$metadata$data
[1] "Tue Nov 16 10:24:34 2021"

The jtree file can be directly imported as a treedata object using the read.jtree() function provided also in treeio package (see also session 1.3).

read.jtree(jtree_file)

## 'treedata' S4 object that stored information
## of
##  '/tmp/RtmpfSfWKq/file1aaef5c82b4cf.jtree'.
## 
## ...@ phylo:
## 
## Phylogenetic tree with 3 tips and 2 internal nodes.
## 
## Tip labels:
##   Physonect_sp_@2066767, Bargmannia_amoena@263997,
## Bargmannia_elongata@946788
## 
## Rooted; includes branch lengths.
## 
## with the following features available:
##   'Ev', 'ND', 'S', 'fake_trait', 'another_trait'.

3.4 Summary

Phylogenetic tree-associated data is often stored in a separate file and needs the expertise to map the data to the tree structure. Lacking standardization to store and represent phylogeny and associated data makes it difficult for researchers to access and integrate the phylogenetic data into their studies. The treeio package provides functions to import phylogeny with associated data from several sources, including analysis findings from commonly used software and external data such as experimental data, clinical data, or meta-data. These trees and their associated data can be exported into a single file as BEAST or jtree formats, and the output file can be parsed back to R by treeio and the data is easy to access. The input and output utilities supplied by treeio package lay the foundation for phylogenetic data integration for downstream comparative study and visualization. It creates the possibility of integrating a tree with associated data from different sources and extends the applications of phylogenetic analysis in different disciplines.

2 Manipulating Tree with Data

4 Phylogenetic Tree Visualization