How to use MetaEgress to generate EML document from LTER-core-metabase

Last updated: 15 November, 2019

This is an example workflow from metabase to EML document. This guide assumes you have an installed and populated instance of LTER-core-metabase, plus log-in credentials for an user with at least read or SELECT access to this database. See the LTER-core-metabase Github repository for information on installation and population. If you would like to test drive MetaEgress, use the database dump specifically made for this purpose; see this document.

Outline

• Preparation
• Set up in R
• Connect and query from LTER-core-metabase
• Create entities
• Create EML
• Validate and write to file
• Troubleshoot invalid EML

Preparation

Necessary components

(aside from well populated metabase)

• An abstract document in pandoc-compatible format. The file name should be exactly as specified in metabase DataSet.Abstract.
• A method document in pandoc-compatible format. The file name should be exactly as specified in metabase DataSetMethods.methodDocument.
• Data files for all listed entities. File names should be exactly as listed in metabase DataSetEntities.FileName.

Note on document formats

MetaEgress uses the function set_TextType from the EML R package to wrap documents in XML tags. See set_TextType documentation to learn more about compatible formats: ?EML::set_TextType.

Set up a local project folder

Put data files and other documents (e.g. abstract, methods, boilerplate, license) in a folder.

Recommended approach for local files: Create a R script in this folder and run MetaEgress from this script.

For remotely hosted data: TODO

Remotely hosted data

TODO

Set up in R

In the project R script:

# Load MetaEgress into R environment
library(MetaEgress)

# Set working directory to current script's directory
setwd(dirname(rstudioapi::getActiveDocumentContext()$path))

Connect and query from LTER-core-metabase

# View documentation
?get_meta

# Use function
metadata <-
  get_meta(
    dbname = "metabase",       # change to your DB name
    schema = "mb2eml_r",       # change to schema containing views
    dataset_ids = 1,           # change to ID or numeric vector of IDs wanted
    host = "localhost",        # change to IP address if remote host
    port = 5432,
    user = NULL,               # change to username and password to save time or if not using RStudio
    password = NULL            # if NULL, RStudio will create pop-up windows asking for username and password
  )

Create entities

# View documentation
?create_entity_all

# Use the function
entities <- create_entity_all(
  meta_list = metadata,   # list returned by `get_meta`
  dataset_id = 1,         # a singular dataset ID
  file_dir = getwd()      # directory containing data files
)

Note that even if the function runs successfully, many warnings will be generated. Use warnings() to see them. Most of the time the warnings will concern custom units and can safely be ignored.

Create EML

# View documentation
?create_EML

# Use the function
EML <-
  create_EML(
    meta_list = metadata,                 # list returned by `get_meta`
    entity_list = entities,               # list returned by `create_entity_all`
    dataset_id = 1                       # a singular dataset ID
)

Validate and write to file

First validate EML document using the function eml_validate from EML R package. The input could be an EML list object or output from create_entity, or a XML file.

EML::eml_validate(EML)

Desired outcome is TRUE:

[1] TRUE attr(,“errors”) character(0)

Should eml_validate return FALSE, examine the list object returned by create_EML. Error messages generated by eml_validate could be quite cryptic and might not point to the real problem.

Then serialize or write to XML file using the function write_eml from the EML R package.

EML::write_eml(EML, file = "EML.xml")

Troubleshoot invalid EML

TODO