mgnipy.V2.collect.mgnetizer module

mgnipy.V2.collect.mgnetizer module#

class MGnetizer(resource, all_ids, config=None, client=None, mgnify_metadata=None, detail_proxy=None)[source]#

Bases: CheckpointMixin, ClientManagerMixin

Fetch detailed metadata for a given list of MGnify accessions.

MGnetizer is designed to retrieve the rich metadata from `MGnify`_ for a list of accessions/ids.

Unlike MGnifier or the mgnipy.V2.proxies module, which are designed to search for MGnify lists OR fetch detailed metadata for a single accession at a time, MGnetizer allows for batch processing of multiple accessions. It uses the MGnifyDetail proxy to fetch detailed metadata for each accession and stores the results in a MGnifyMetadata instance.

Parameters:

resource (DetailResourceStr) – The type of resource to fetch metadata for. Must be one of the supported MGnifyDetail resource types (e.g., “study”, “sample”, “run”, etc.).
all_ids (list of str ) – A list of MGnify accessions/ids for which to fetch detailed metadata.
config (MGnipyConfig, optional) – An optional configuration object for MGnipy. If not provided, a default configuration will be used.
client (Client or AuthenticatedClient, optional) – An optional HTTP client for making requests. If not provided, a default client will be initialized using the provided or default configuration.
mgnify_metadata (MGnifyMetadata, optional) – An optional MGnifyMetadata instance to store the enriched metadata.
detail_proxy (MGnifyDetail, optional) – An optional MGnifyDetail proxy instance to use for fetching detailed metadata. If not provided, the appropriate proxy will be selected based on the specified resource.

resource#

The type of resource being processed.

Type:: DetailResourceStr

all_ids#

The complete list of MGnify accessions/ids provided during initialization.

Type:: list of str

mgnify_metadata#

The enriched metadata as a MGnifyMetadata instance.

Type:: MGnifyMetadata

metadata#

Alias for mgnify_metadata.

Type:: MGnifyMetadata

params#

A dictionary of parameters used for checkpointing, including the resource type and a sorted list of accessions/ids. Used for caching and resuming enrichment processes.

Type:: dict

async aclose()#

async aenrich(limit=200, hide_progress=False)[source]#

Async version of enrich().

See enrich() for details on parameters and behavior.

Parameters:

limit (int | None)
hide_progress (bool )

Return type:

None

property all_ids: list [str ]#: The list of MGnify accessions/ids set during initialization.

async aload_cache()#

Async wrapper for load_cache.

Return type:: list [int ]

property async_httpx_client: AsyncClient#

Get the asynchronous httpx client instance from the AuthenticatedClient.

Returns:: The asynchronous httpx client instance.
Return type:: httpx.AsyncClient

async awrite_results(request_num, items)#

Async wrapper for write_results.

Parameters:

request_num (int )
items (Any )

Return type:

None

property cache_key: str #

Generate deterministic hash from resource + params.

Returns:: A unique cache key for the current query parameters and resource. For a query to the ‘samples’ resource with parameters {‘biome_lineage’: ‘root:Environmental:Terrestrial’}, the cache key will be a SHA256 hash of the string representation of the resource and parameters, ensuring that identical queries will have the same cache key and thus access the same cached results.
Return type:: str

Example

>>> from mgnipy.V2.mixins import CheckpointMixin
>>> from mgnipy import MGnipyConfig
>>> # Prepare parameters and config
>>> params = {'lineage': 'root:Environmental:Terrestrial'}
>>> resource = 'biome'
>>> config = MGnipyConfig(cache_dir="/path/to/cache")
>>> # Create CheckpointMixin instance and compute cache key
>>> cache_handler = CheckpointMixin()
>>> cache_handler.params = params
>>> cache_handler.resource = resource
>>> cache_handler.config = config
>>> cache_handler.cache_key
'1eb56ddf5a2e7d60d8155c8bbe01f032f959a2519d43e99f31f533abffa3166f'

property cache_path: Path | None #: Directory for this query’s cached pages.

clear_cache()#

Remove all cached pages for this set of queries.

Return type:: None

close()#

property datasets: MGazine#: Returns a MGazine instance containing the enriched datasets.

property downloads: list [str ]#: Returns the list of downloads from the enriched metadata.

enrich(limit=200, hide_progress=False)[source]#

Fetches MGnify metadata for the given accessions.

This method iterates through the list of MGnify or ENA run accessions provided during initialization and retrieves their corresponding MGnify detail metadata. The results are stored in the MGnifyMetadata instance associated with this class. This does not return anything.

Parameters:

limit (Optional[int ], default=200) – An optional integer to limit the number of detaile metadata to retrieve. If set to None, there will be no limit on the number of accessions enriched.
hide_progress (bool , default=False) – Whether to hide the progress bar during enrichment.

Return type:

None

explain()[source]#

property httpx_client: Client#

Get the synchronous httpx client instance from the AuthenticatedClient.

Returns:: The synchronous httpx client instance.
Return type:: httpx.Client

load_cache()#

Pick up where you left off. Loads cached results and manifest into memory.

Returns:: A list of request numbers (page numbers) that were successfully loaded from the cache.
Return type:: list of int

load_cache_manifest()#

Load the cache manifest file if present, and update total records and total requests.

Returns:: The contents of the manifest file, or an empty dictionary if the manifest is not found or fails to load.
Return type:: dict

load_cache_results()#

Load cached pages/request nums into results.

Loads cached pages from disk into the in-memory results dictionary (self._results), if available.

Returns:: A list of request numbers (page numbers) that were successfully loaded from the cache.
Return type:: list of int

property manifest_path: Path | None #: Path to mgnipy_manifest.json storing metadata.

property metadata: MGnifyMetadata#: Returns the enriched metadata as an MGnifyMetadata instance.

property mgnify_metadata: MGnifyMetadata#: The enriched metadata as an MGnifyMetadata instance.

property params: dict [str , Any ]#: For CheckpointMixin

renew_client()#: Init a new client instance and replace the existing one. This is useful if the current client has been closed or is no longer valid, allowing for a fresh start with a new HTTP client session.

property resource: str #: For CheckpointMixin

status()#

Print the status of the MGnipy client, including the type of client and whether the synchronous and asynchronous httpx client sessions are open.

Return type:: None

try_load_cache()#

Attempt to load cached results and manifest into memory if not already loaded. This method checks if the cache has already been loaded to avoid redundant operations. If the cache has not been loaded, it will attempt to load it and set the _cache_loaded attribute accordingly.

Notes

This method is intended to be called internally before accessing cached results.
If cache_dir is None then _cache_loaded will be True after initial attempt.
If an error occurs during cache loading, it will be logged, and _cache_loaded will be set to False.
Dependent on .mixins.CheckpointMixin

Return type:: None

write_results(request_num, items, include_manifest=True)#

Auto atomic write to disk.

Parameters:

request_num (int )
items (Any )
include_manifest (bool )

Return type:

None

mgnipy.V2.collect.mgnetizer module

Contents

mgnipy.V2.collect.mgnetizer module#