Getting MGnify data#
The MGnify API provides access to MGnify analyses datasets and important metadata such as biome, sample, study, run, analysis details. On this page we demonstrate how to:
Get the metadata using
MGnifierGet the datasets as a
MGazine
# uncomment below if colab
#!pip install mgnipy
Recall the typical workflow (from What is MGni.Py? ):
Start up a
mgnipy.MGnipyclient with your desired configurationSearch in MGnify resources using a MGnifier glass
Receive a MGazine of MGnify datasets
which we will follow in this notebook
1. mgnipy.MGnipy to init session#
from mgnipy import MGnipy
MG = MGnipy(cache_dir=None)
2. MGnifier to query MGnify#
for this example we will search MGnify .studies for a list of pea studies.
After we .get() the list we will populate each of the study’s details / metadata using .enrich_details()
# access studies MGnifier and pass search params (build query set)
pea_studies = MG.studies(biome_lineage="root:Host-associated:Plants", search="pea")
# check out the request url
pea_studies.explain()
https://www.ebi.ac.uk/metagenomics/api/v2/studies?biome_lineage=root%3AHost-associated%3APlants&search=pea&page=1
now executing the query url(s). If there were multiple urls in the query set then we could also use .get_all() rather than iteratively .getting page by page.
# MG as client context manager
with MG:
# get a page of pea study list
pea_studies.get()
# now filling with metadata
pea_studies.enrich_details()
We can access the detailed metadata via .metadata attribute which will return a MGnifyMetadata instance that allows you to view as a list, polars or pandas dataframe.
# accessing metadata
meta = pea_studies.metadata
# as pandas dataframe
meta.to_pandas(expand_nested_dicts=True)
3. MGazine of MGnify datasets#
To access the study’s mgazine use .datasets
Notice how in study details printed above there is a “downloads” field with information about the data.
this “downloads” information is used by
mgnipy.MGazineto allow us to download or read them into our notebook.To access the study’s mgazine use
.datasetsthe str representaiton of mgazine gives us a peak into the pipeline versions within, number of downloads and the short description categories
# access study mgazine
MZ = pea_studies.datasets
# print for more info
print(MZ)
# also can view more as df
MZ.downloads_df()
You can read in whole or stream in chunks a dataset by passing its alias or url to MGazine.stream()
alias = "ERP014435_GO-slim_abundances_v3.0.tsv"
# reading in above file
df_go = MZ.stream(
alias=alias,
chunksize=None, # default to read in all, set int for chunked reading
df_engine="pandas", # or polars
)
df_go.head()
# run accessions as a list
run_ids = df_go.columns[3:].to_list()
# check it out
print(run_ids)
['ERR1299314', 'ERR1299315', 'ERR1299316', 'ERR1299317', 'ERR1299319', 'ERR1299320', 'ERR1299321', 'ERR1299323', 'ERR1299324', 'ERR1299325', 'ERR1299326', 'ERR1299330', 'ERR1299331', 'ERR1299333']
you can also .download() or .download_all() of the files to a directory of your choosing
MZ.download(alias=alias, to_dir="downloads")
Wrap Up:#
This page was a quick start demonstration of:
✅ Start up a
mgnipy.MGnipyclient with your desired configuration✅ Querying MGnify using a
MGnifierglass✅ Accessing the resulting
MGazineof MGnify datasets
Next we will see how we can collect even more metadata for the above list of run_ids using mgnipy’s MGnetizer and BioSampler helpers.