BOLDconnectR is a package designed for retrieval, transformation and analysis of the data available in the Barcode Of Life Data Systems (BOLD) database. This package provides the functionality to obtain public and private user data available in the database in the Barcode Core Data Model (BCDM) format. Data include information on the taxonomy,geography,collection,identification and DNA barcode sequence of every submission.
BOLDconnectR requires R version 4.0 or above to function properly. The versions of dependent packages have also been set such that they would work with R >= 4.0. In addition, there are a few suggested packages that are not mandatory for the package to download and install properly, but, are essential for a couple of functions to work. The names and exact versions of the dependencies/suggestions are given here (https://github.com/boldsystems-central/BOLDconnectR/blob/main/DESCRIPTION). More details on Suggested packages provided below.
The package can be installed either by install.packages() for the CRAN
version or using the devtools::install_github or pak::pak functions from the
devtools/ pak packages in R (which need to be installed before installing
BOLDConnectR).
## CRAN install
# install.packages('BOLDconnectR')
## GitHub install
# devtools::install_github("https://github.com/boldsystems-central/BOLDconnectR")
# OR
# pak::pak("boldsystems-central/BOLDconnectR")
library(BOLDconnectR)
Note on Suggested packages Function 5: bold.data.summarize
requires the packages Biostrings to be installed and imported in the R
session beforehand for generating the barcode_summary. msa and
Biostrings can be installed using using BiocManager package.
Function 6: bold.analyze.align requires the packages msa, muscle
and Biostrings to be installed and imported in the R session
beforehand. Function 7 also uses the the output generated from function
6. msa, muscle and Biostrings can be installed using using
BiocManager package.
if (!requireNamespace("BiocManager", quietly=TRUE))
install.packages("BiocManager")
BiocManager::install("msa")
BiocManager::install("Biostrings")
BiocManager::install("muscle")
library(msa)
library(Biostrings)
library(muscle)
The function bold.fetch requires an api key internally in order to
access and download all public + private user data. The API key needed
to retrieve BOLD records is found in the BOLD ‘Workbench’
https://bench.boldsystems.org/index.php/Login/page?destination=MAS_Management_UserConsole.
After logging in, navigate to ‘Your Name’ (located at the top left-hand
side of the window) and click ‘Edit User Preferences’. You can find the
API key in the ‘User Data’ section. Please note that to have an API key
available in the workbench, a user must have uploaded at least 10,000
records to BOLD. API key can be saved in the R session using
bold.apikey() function.
# Substitute ‘00000000-0000-0000-0000-000000000000’ with your key
# bold.apikey(‘00000000-0000-0000-0000-000000000000’)
API key function must be run prior to using the fetch function (Please see above).
BCDM_data<-bold.fetch(get_by = "processid",
identifiers = test.data$processid)
knitr::kable(head(BCDM_data,4))
| processid | record_id | insdc_acs | sampleid | specimenid | taxid | short_note | identification_method | museumid | fieldid | collection_code | processid_minted_date | inst | funding_src | sex | life_stage | reproduction | habitat | collectors | site_code | specimen_linkout | collection_event_id | sampling_protocol | tissue_type | collection_date_start | collection_time | associated_taxa | associated_specimens | voucher_type | notes | taxonomy_notes | collection_notes | geoid | marker_code | kingdom | phylum | class | order | family | subfamily | tribe | genus | species | subspecies | identification | identification_rank | species_reference | identified_by | sequence_run_site | nuc | nuc_basecount | sequence_upload_date | bin_uri | bin_created_date | elev | depth | coord | coord_source | coord_accuracy | elev_accuracy | depth_accuracy | realm | biome | ecoregion | region | sector | site | country_iso | country.ocean | province.state | bold_recordset_code_arr | collection_date_end |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BBCNP2615-14 | BBCNP2615-14.COI-5P | KP654931 | BIOUG12571-E02 | 4468216 | 9199 | Prince Albert NP | NA | BIOUG12571-E02 | L#12BIOBUS-1007 | BIOUG | 2014-04-21 | Centre for Biodiversity Genomics | iBOL:WG1.9 | NA | I | S | NA | BIOBus 2012 | NA | NA | NA | Free Hand Collection | NA | 2012-07-13 | NA | NA | NA | museum voucher | hand collecting|sunny with haze|25C | CollectionsID | NA | 511 | COI-5P | Animalia | Arthropoda | Arachnida | Araneae | Salticidae | NA | NA | Eris | Eris militaris | NA | Eris militaris | species | (Hentz, 1845) | Gergin A. Blagoev | Centre for Biodiversity Genomics | ACGTTATATTTAATTTTTGGAGCTTGATCAGCTATAGTTGGTACTGCTATAAGAGTATTAATTCGAATAGAATTAGGACAAACTGGATCATTTTTAGGTAATGATCATATATATAATGTAATTGTAACTGCTCATGCTTTTGTAATGATTTTTTTTATAGTAATACCAATTATAATTGGGGGATTTGGTAATTGGTTAGTTCCTTTAATGTTAGGGGCTCCGGATATAGCTTTTCCTCGAATAAATAATTTAAGTTTTTGATTATTACCTCCTTCTTTATTTTTATTGTTTATTTCTTCTATAGCTGAAATAGGGGTTGGAGCTGGATGAACAGTATATCCTCCTTTGGCATCTATTGTTGGACATAATGGTAGATCAGTAGATTTTGCTATTTTTTCTTTACATTTAGCTGGTGCTTCATCAATTATAGGAGCTATTAATTTTATTTCTACTATTATTAATATACGATCAGTAGGAATATCTTTAGATAAAATTCCTTTATTTGTTTGATCTGTAATAATTACTGCTGTATTATTATTGTTATCATTACCTGTTTTAGCAGGA——————————————————————— | 564 | 2014-06-26 | BOLD:AAA5654 | 2010-07-15 | 549 | NA | 53.59,-106.278 | GPSmap 60Cx | NA | NA | NA | Nearctic | NA | Mid-Canada_Boreal_Plains_forests | Prince Albert NP | Hunters Lake Trail | predominately aspen forest | CA | Canada | Saskatchewan | BBCNP,DS-SOC2014,DS-ARANCCYH,DATASET-BBPANP1,DS-SPCANADA,DS-JUMPGLOB | NA |
| BBCAN226-09 | BBCAN226-09.COI-5P | GU683124 | CCDB-04550-C12 | 1274762 | 9199 | Kouchibouguac NP | BOLD ID Engine | CCDB-04550-C12 | L#09KC-053 | BIOUG | 2009-11-23 | Centre for Biodiversity Genomics | iBOL:WG1.9 | F | I | S | Wetland | BIObus 2009 | NA | NA | NA | Free Hand | 2009-08-17 | NA | NA | NA | Vouchered:Registered Collection | NA | NA | Free Hand|Mixed sun and cloud after rain|Boardwalk through a bog | 521 | COI-5P | Animalia | Arthropoda | Arachnida | Araneae | Salticidae | NA | NA | Eris | Eris militaris | NA | Eris militaris | species | (Hentz, 1845) | Gergin A. Blagoev | Centre for Biodiversity Genomics | GACGTTATATTTAATTTTTGGAGCTTGATCAGCTATAGTTGGTACTGCTATAAGAGTATTAATTCGAATAGAATTAGGACAAACTGGATCATTTTTAGGTAATGATCATATATATAATGTAATCGTAACTGCTCATGCTTTTGTAATGATTTTTTTTATAGTAATACCAATTATAATTGGGGGATTTGGTAATTGGTTAGTTCCTTTAATGTTAGGGGCTCCGGATATAGCTTTTCCTCGAATAAATAATTTAAGTTTTTGATTATTACCTCCTTCTTTATTTTTATTATTTATTTCTTCTATAGCTGAAATAGGGGTTGGAGCTGGATGAACAGTATATCCTCCTTTGGCATCTATTGTTGGACATAATGGCAGATCAGTAGATTTTGCTATTTTTTCTTTACATTTAGCTGGTGCTTCATCAATTATAGGAGCTATTAATTTTATTTCTACTATTATTAATATACGATCAGTAGGAATATCTTTAGATAAAATTCCTTTATTTGTTTGATCTGTAATAATTACTGCTGTATTATTATTGTTATCATTACCTGTTTTAGCAGGAGCTATTACTATATTATTAACTGATCGAAATTTTAATACTTCTTTTTTTGATCCTGCAGGAGGAGGAGATCCAATTTTGTTTCAACATTTATTT | 658 | 2011-03-25 | BOLD:AAA5654 | 2010-07-15 | 7 | NA | 46.816,-64.953 | NA | NA | NA | NA | Nearctic | NA | Gulf_of_St._Lawrence_lowland_forests | Kouchibouguac NP | The Bog Trail | NA | CA | Canada | New Brunswick | SPIBB,DATASET-BBKCNP1,DATASET-BBCNP1,DS-MOB113,DS-BICNP02,DS-SOC2014,DS-ARANCCYH,DS-DDSG,DS-SPCANADA,DS-JUMPGLOB,DS-MOB112 | NA | |
| ARSO594-09 | ARSO594-09.COI-5P | KM826219 | 08BBARAC-0466 | 990633 | 9199 | Riding Mountain NP | 08BBARAC-0466 | L#08RIDMO-097 | BIOUG | NA | Centre for Biodiversity Genomics | NA | F | I | S | NA | BIObus 2008 | NA | NA | NA | Sweep Net | NA | 2009-08-19 | NA | NA | NA | Vouchered:Registered Collection | NA | NA | Sweep Net||Hiking trail | 531 | COI-5P | Animalia | Arthropoda | Arachnida | Araneae | Salticidae | NA | NA | Eris | Eris militaris | NA | Eris militaris | species | (Hentz, 1845) | Gergin A. Blagoev | Centre for Biodiversity Genomics | AACGTTATATTTAATTTTTGGAGCTTGATCAGCTATAGTTGGTACTGCTATAAGAGTATTAATTCGAATAGAATTAGGACAAACTGGATCATTTTTAGGTAATGATCATATATATAATGTAATTGTAACTGCTCATGCTTTTGTAATGATTTTTTTTATAGTAATACCAATTATAATTGGGGGATTTGGTAATTGGTTAGTTCCTTTAATGTTAGGGGCTCCGGATATAGCTTTTCCTCGAATAAATAATTTAAGTTTTTGATTATTACCTCCTTCTTTATTTTTATTATTTATTTCTTCTATAGCTGAAATAGGGGTTGGAGCTGGATGAACAGTATATCCTCCTTTGGCATCTATTGTTGGACATAATGGTAGATCAGTAGATTTTGCTATTTTTTCTTTACATTTAGCTGGTGCTTCATCAATTATAGGAGCTATTAATTTTATTTCTACTATTATTAATATACGATCAGTAGGAATATCTTTAGATAAAATTCCTTTATTTGTTTGATCTGTAATAATTACTGCTGTATTATTATTGTTATCATTACCTGTTTTAGCAGGAGCTATTACTATATTATTAACTGATCGAAATTTTAATACTTCTTTTTTTGATCCTGCAGGAGGAGGAGATCCAATTTTGTTTCAACATTtaTTt | 658 | 2009-05-30 | BOLD:AAA5654 | 2010-07-15 | 662 | NA | 50.881,-100.056 | NA | NA | NA | NA | Nearctic | NA | Mid-Canada_Boreal_Plains_forests | Riding Mountain NP | Moon Lake | Hiking Trail | CA | Canada | Manitoba | SPIBB,DATASET-BBRMNP1,DS-MOB113,DS-BICNP02,DS-SOC2014,DS-ARANCCYH,DS-SPCANADA,DS-JUMPGLOB,DS-MOB112 | NA | |
| BBCNP1308-14 | BBCNP1308-14.COI-5P | KP648859 | BIOUG09535-G01 | 4141099 | 9199 | Elk Island NP | NA | BIOUG09535-G01 | L#12BIOBUS-0772 | BIOUG | 2014-01-17 | Centre for Biodiversity Genomics | iBOL:WG1.9 | M | A | S | NA | BIOBus 2012 | NA | NA | NA | Free Hand Collection | NA | 2012-07-02 | NA | NA | NA | museum voucher | hand collecting|mostly sunny|24C | CollectionsID | NA | 533 | COI-5P | Animalia | Arthropoda | Arachnida | Araneae | Salticidae | NA | NA | Eris | Eris militaris | NA | Eris militaris | species | (Hentz, 1845) | Gergin A. Blagoev | Centre for Biodiversity Genomics | GAACGTTATATTTAATTTTTGGAGCTTGATCAGCTATAGTTGGTACTGCTATAAGAGTATTAATTCGAATAGAATTAGGACAAACTGGATCATTTTTAGGTAATGATCATATATATAATGTGATTGTAACTGCTCATGCTTTTGTAATGATTTTTTTTATAGTAATACCAATTATAATTGGGGGATTTGGTAATTGGTTAGTTCCTTTAATGTTAGGGGCTCCGGATATAGCTTTTCCTCGAATAAATAATTTAAGTTTTTGATTATTACCTCCTTCTTTATTTTTATTATTTATTTCTTCTATAGCTGAAATAGGGGTTGGAGCTGGATGAACAGTATATCCTCCTTTGGCATCTATTGTTGGACATAATGGTAGATCAGTAGATTTTGCTATTTTTTCTTTACATTTAGCTGGTGCTTCATCAATTATAGGAGCTATTAATTTTATTTCTACTATTATTAATATACGATCAGTAGGAATATCTTTAGATAAAATTCCTTTATTTGTTTGATCTGTAATAATTACTGCTGTATTATTATTGTTATCATTACCTGTTTTAGCAGGAGCTATTACTATATTATTAACTGATCGA | 593 | 2014-03-20 | BOLD:AAA5654 | 2010-07-15 | 721 | NA | 53.618,-112.875 | GPSmap 60Cx | NA | NA | NA | Nearctic | NA | Canadian_Aspen_forests_and_parklands | Elk Island NP | Tawayik Lake Trail | aspen, birch, rose bushes, adjacent to wetland/lake | CA | Canada | Alberta | BBCNP,DS-SOC2014,DS-ARANCCYH,DATASET-BBEINP1,DS-SPCANADA,DS-JUMPGLOB,DS-MOB112 | NA |
Similarly, sampleids or dataset_codes or project_codes can also be used
to fetch data. The data can also be filtered on different parameters
such as Geography, Attributions and DNA Sequence information using the
_filt arguments available in the function
Downloaded data can then be summarized in different ways. Options currently include a concise summary of all the data, detailed taxonomic counts, data completeness and a barcode based summary
BCDM_data_summary<-bold.data.summarize(bold_df = BCDM_data,
summary_type = "concise_summary")
BCDM_data_summary$concise_summary
#> Category Value
#> 1 Total_records 1336
#> 2 Total_records_w_sequences 1336
#> 3 Unique_species 79
#> 4 Unique_BINs 117
#> 5 Unique_countries 1
#> 6 Unique_institutes 6
#> 7 Unique_identified_by 7
#> 8 Unique_specimen_depositories 3
#> 9 Unique_markers COI-5P
#> 10 Amplicon_length_range 508-658
A concise summary providing a high level overview of the data
Downloaded data can also be exported to the local machine either as a flat file or as a FASTA file for any third party sequence analysis tools.The flat file contents can be modified as per user requirements (entire data or specific presets or individual fields).
# Preset dataframe
# bold.export(bold_df = BCDM_data,
# export_type = "preset_df",
# presets = 'taxonomy',
# cols_for_fas_names = NULL,
# export = "file_path_with_intended_name.csv")
# Unaligned fasta file
# bold.export(bold_df = BCDM_data,
# export_type = "fas",
# cols_for_fas_names = c("bin_uri","genus","species"),
# export = "file_path_with_intended_name.fas")
Downloaded data be aligned by either ClustalOmega or muscle
algorithms with custom headers for the aligned sequences. The function
uses msa, ape, muscle and Biostrings internally for the sequence
alignment. The packages muscle, msa and Biostrings must be
imported prior to using the align function.
# bold.apikey('')
# fetch.test.data.4.align<-bold.fetch(identifiers = "DS-IBOLR24",
# get_by = "dataset_codes",
# filt_taxonomy = "Manduca",
# filt_basecount = c(600,670))
# Sequence is aligned using ClustalOmega with default settings
# align.test.data<-bold.analyze.align(bold_df = fetch.test.data.4.align,
# marker = "COI-5P",
# cols_for_seq_names = c("species","bin_uri"),
# align_method = "ClustalOmega")
# Confirm the result
# head(subset(align.test.data,select = c(aligned_seq,msa.seq.name)),5)
The aligned sequences cane also be visualized as a Neighbour joining
tree. The function uses the ape nj function to help create the
visualization. All additional arguments available for the nj function
can be passed on to this function as well.
# bold.apikey('')
# fetch.test.data.4.align<-bold.fetch(identifiers = "DS-IBOLR24",
# get_by = "dataset_codes",
# filt_taxonomy = "Manduca",
# filt_basecount = c(600,670))
# Sequence is aligned using ClustalOmega with default settings
# align.test.data<-bold.analyze.align(bold_df = fetch.test.data.4.align,
# marker = "COI-5P",
# cols_for_seq_names = c("species","bin_uri"),
# align_method = "ClustalOmega")
# NJ tree with basic settings
# test.tree<-bold.analyze.tree(bold_df=align.test.data,
# dist_model = "K80",
# clus_method = "njs",
# tree_plot = TRUE,
# tree_plot_type = 'p')
#
# The phylo object which can be used for further customization of the plot
# test.tree$data_for_plot
#
# base frequencies
# test.tree$base_freq
Basic biodiversity analyses (preston plots, taxa richness estimation,
shannon diversity and beta diversity) of the downloaded data can be
carried out at any level of taxonomic hierarchy and geographic category.
The function uses vegan, BAT and betapart internally. The result
also provides an occurrence matrix that can be used for other
ecological analyses.
# bold.apikey('')
# fetch.test.data<-bold.fetch(identifiers = "DS-IBOLR24",
# get_by = "dataset_codes")
# 1. Generic richness
# test.richness.diversity<-bold.analyze.diversity(bold_df=fetch.test.data,
# taxon_rank = "genus",
# site_type = "locations",
# location_type = "country.ocean",
# diversity_profile = "richness")
# View the richness estimator results
# View(test.richness.diversity$richness)
# Occurrence matrix (site X species)
# test.richness.diversity$comm.matrix
# 2. Preston plots
# test.richness.diversity2<-bold.analyze.diversity(bold_df=fetch.test.data,
# taxon_rank = "genus",
# site_type = "locations",
# location_type = "country.ocean",
# diversity_profile = "preston")
# View the preston plot
# test.richness.diversity2$preston.plot
The package also offers functionality to generate occurrence maps of the
downloaded data at different scales (Complete geographic extent of the
downloaded data, country specific occurrence and a bounding box based
occurrenc map). The result also provides an sf object that can be used
for other spatial analyses.
# bold.apikey('')
# fetch.test.data<-bold.fetch(identifiers = "DS-IBOLR24",
# get_by = "dataset_codes")
# All occurrences
# test.map<-bold.analyze.map(fetch.test.data)
# Specific country
# test.map.brazil=bold.analyze.map(fetch.test.data,country = "Australia")
BOLDconnectR is able to retrieve data very fast (~100k records in a minute on a fast wired connection).
This work was funded by the New Frontiers in Research Fund (NFRF) - Transformation 2020
Padhye, S. M., Ballesteros-Mejia, L., Agda, J., Agda, J., Hebert, P. D., & Ratnasingham, S. (2026). BOLDconnectR: An R package for streamlined retrieval, transformation, and analysis of DNA barcode data on BOLD. PloS one, 21(8), e0355496 (https://doi.org/10.1371/journal.pone.0355496)