Getting MGnify datasets#
The MGnify API provides access to MGnify analyses datasets and important metadata such as biome, sample, study, run, analysis details.
# uncomment below if colab
#!pip install mgnipy
🎯 The Goal: Retrieve taxonomic datasets of tomato rhizosphere studies#
Let’s request tomato rhizosphere datasets and metadata from MGnify API.
Recall the typical workflow (from What is MGni.Py? ):
Start up a
mgnipy.MGnipyclient with your desired configurationSearch in MGnify resources using a MGnifier glass
Receive a MGazine of MGnify datasets
which we will follow in this notebook
from mgnipy import MGnipy
# 1. init with default config
MG = MGnipy(
cache_dir="downloads"
)
# 2.a) setup studies mgnifier (build queries)
tomato_studies = MG.studies(
biome_lineage="root:Host-associated:Plants:Rhizosphere", search="tomato"
)
with MG:
# 2.b) execute the list query (get the study list)
tomato_studies.get_all()
# 2.c) get the study list (execute all detail queries)
tomato_studies.enrich_details()
# take a look at the studies details results as a pandas df
tomato_studies.metadata.to_pandas(expand_nested_dicts=True)
3. Accessing the MGazine of datasets#
study details have a
mgnipy.MGazinewhich allow us to download and interact with study-level datasets outputed from MGnify.We can use
mgnipy.MGazineto download the datasets onto disk or read them into our notebook.To access the study’s mgazine use
.datasetsthe str representaiton of mgazine gives us a peak into the pipeline versions within, number of downloads and the short_description categories
# access study mgazine
MZ = tomato_studies.datasets
# print for more info
print(MZ)
# also can view more as df
MZ.downloads_df()
You can read in whole or stream in chunks a dataset by passing its alias or url to MGazine.stream()
MZ.stream(
alias = MZ.aliases[5],
chunksize=None, # default to read in all, set int for chunked reading
).head()
| GO | description | category | ERZ12343720 | ERZ12343730 | ERZ12343740 | ERZ12343750 | ERZ12343760 | ERZ12343770 | ERZ12343780 | ... | ERZ12590947 | ERZ12593067 | ERZ12590878 | ERZ12590908 | ERZ12590968 | ERZ12590988 | ERZ12590849 | ERZ12590879 | ERZ12590889 | ERZ12590909 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | GO:0000015 | phosphopyruvate hydratase complex | cellular component | 106 | 77 | 69 | 106 | 96 | 88 | 102 | ... | 107 | 103 | 57 | 84 | 70 | 76 | 97 | 79 | 148 | 94 |
| 1 | GO:0000150 | recombinase activity | molecular function | 152 | 111 | 112 | 126 | 137 | 154 | 178 | ... | 146 | 165 | 97 | 117 | 123 | 136 | 167 | 123 | 283 | 135 |
| 2 | GO:0000160 | phosphorelay signal transduction system | biological process | 5337 | 4117 | 4340 | 5774 | 5273 | 5160 | 5794 | ... | 5912 | 5731 | 2975 | 4728 | 4834 | 4776 | 5820 | 4669 | 8623 | 5539 |
| 3 | GO:0000166 | nucleotide binding | molecular function | 27506 | 23975 | 22572 | 28502 | 28606 | 25597 | 30817 | ... | 28526 | 31371 | 16457 | 25486 | 26749 | 23192 | 28760 | 24531 | 43168 | 29333 |
| 4 | GO:0003674 | molecular function | molecular function | 3919 | 3307 | 3217 | 4040 | 3915 | 3621 | 4337 | ... | 4138 | 4391 | 2305 | 3594 | 3695 | 3329 | 4124 | 3479 | 6271 | 4144 |
5 rows × 117 columns
You can filter by short descriptioins by passing them as you would an index into square brackets i..e, getitem
ssu = MZ['Taxonomic assignments SSU']
print(ssu)
# now with additional taxonomic helpers
tax = ssu.taxonomic
MGazine containing:
- MGnify pipeline versions: ['v5']
- Number of downloads: 5
- Short descriptions: ['Taxonomic assignments SSU']
- Nonempty metadata sets: .mgnify_studies
TaxaMGazine containing:
- MGnify pipeline versions: ['v5']
- Number of downloads: 5
- Short descriptions: ['Taxonomic assignments SSU']
- Nonempty metadata sets: .mgnify_studies
-----------------------
Next steps: Use `.load()` to initialize.
The MGazine informtion page also delves into how to download as well as other options for reading in the files
We will carry on with our filtered TaxaMGazine given our goal for now.
for example, we can also combine the taxonomic assignment results into one dataframe e.g. .to_pandas(), .to_polars, .X()
# first loading
tax.load()
# accessing the 5 datasets in one df
tax.to_polars().head()
# also as an annotated dataframe (AnnData)
tax.to_anndata()
AnnData object with n_obs × n_vars = 149 × 558
var: 'Superkingdom', 'Kingdom', 'Phylum', 'Class', 'Order', 'Family', 'Genus', 'Species'
We can see that only the var or features (taxonomy) are annotated. The obs or observations (runs/samples) are not yet annotated because we did not collect their metadata.
From here we could use a MGnetizer to collect all the detailed metadata for the .runs_accession in our MGazine. See the following notebooks for more information
# tidying up cache
MG.clear_subcaches()
Wrap Up:#
This page was a quick start demonstration of:
✅ Start up a
mgnipy.MGnipyclient with your desired configuration✅ Search in MGnify resources using a MGnifier glass
✅ Receive a MGazine of MGnify datasets