Core Modules

Core pipeline orchestration, data acquisition, and configuration management.

Configuration and data models for SPXQuery package.

class spxquery.core.config.Source(ra: float, dec: float, name: str | None = None)

Bases: object

Astronomical source coordinates.

ra: float

dec: float

name: str | None = None

to_skycoord() → SkyCoord

Convert source coordinates to astropy SkyCoord object.

Returns:

Astropy SkyCoord object with ICRS frame coordinates.

Return type:

SkyCoord

Examples

>>> source = Source(ra=304.69, dec=42.44, name="Deneb")
>>> coord = source.to_skycoord()
>>> print(coord.ra, coord.dec)
304d41m24s 42d26m24s

__init__(ra: float, dec: float, name: str | None = None) → None

class spxquery.core.config.PhotometryConfig(aperture_method: str = 'fwhm', aperture_diameter: float = 3.0, fwhm_multiplier: float = 2.5, max_processing_workers: int = 10, bad_flags: ~typing.List[int] = <factory>, annulus_inner_offset: float = 1.414, min_annulus_area: int = 10, max_outer_radius: float = 5.0, min_usable_pixels: int = 10, bg_sigma_clip_sigma: float = 3.0, bg_sigma_clip_maxiters: int = 3, subtract_zodi: bool = True, zodi_scale_min: float = 0.0, zodi_scale_max: float = 10.0, pixel_scale_fallback: float = 6.2, max_annulus_attempts: int = 5, annulus_expansion_step: float = 0.5, background_method: str = 'annulus', window_size: int | ~typing.Tuple[int, int] = 50)

Bases: object

Advanced photometry configuration.

aperture_method

Method for determining aperture size. Options: ‘fixed’ (use aperture_diameter), ‘fwhm’ (use PSF FWHM). Default: ‘fwhm’.

Type:

str

aperture_diameter

Aperture diameter in pixels. Default: 3.0. Used when aperture_method=’fixed’, or as fallback when FWHM estimation fails.

Type:

float

fwhm_multiplier

Multiplier for FWHM to determine aperture diameter. Default: 2.5. Aperture diameter = FWHM × fwhm_multiplier. Only used when aperture_method=’fwhm’.

Type:

float

max_processing_workers

Number of parallel workers for photometry processing. Default: 10. Adjust based on CPU cores.

Type:

int

bad_flags

List of flag bit positions to reject in quality control. Default: [0, 1, 2, 6, 7, 9, 10, 11, 14, 15, 17, 19]. These flags indicate problematic pixels.

Type:

List[int]

annulus_inner_offset

Gap between aperture edge and inner annulus radius (pixels). Default: 1.414 (√2). Reduce for crowded fields, increase for extended sources.

Type:

float

min_annulus_area

Minimum area for background annulus (pixels). Default: 10. Increase for better statistics.

Type:

int

max_outer_radius

Maximum outer radius for background annulus (pixels). Default: 5.0. Increase for faint sources.

Type:

float

min_usable_pixels

Minimum number of unflagged pixels required in annulus. Default: 10. Increase for higher quality.

Type:

int

bg_sigma_clip_sigma

Sigma threshold for sigma-clipped background statistics. Default: 3.0. Common values: 2.5 (strict), 3.0 (standard), 3.5 (lenient).

Type:

float

bg_sigma_clip_maxiters

Maximum iterations for sigma clipping of background. Default: 3. Usually 1-5 is sufficient.

Type:

int

subtract_zodi

Whether to subtract zodiacal light background from images before photometry. Default: True. Set to False to work with raw images.

Type:

bool

zodi_scale_min

Minimum allowed zodiacal scaling factor. Default: 0.0. Negative values may indicate model failure.

Type:

float

zodi_scale_max

Maximum allowed zodiacal scaling factor. Default: 10.0. Increase if studying high-zodiacal periods.

Type:

float

pixel_scale_fallback

Fallback pixel scale (arcsec/pixel) when WCS fails. Default: 6.2 (SPHEREx). Change for other missions.

Type:

float

max_annulus_attempts

Maximum attempts to expand annulus when insufficient pixels. Default: 5. Rarely needs adjustment.

Type:

int

annulus_expansion_step

Step size in pixels when expanding annulus. Default: 0.5. Usually 0.3-1.0 is reasonable.

Type:

float

background_method

Background estimation method. Options: ‘annulus’ (ring around source), ‘window’ (rectangular region). Default: ‘annulus’. Use ‘window’ for crowded fields.

Type:

str

window_size

Size of background window in pixels (for window method). Default: 50. If int: square window. If tuple: (height, width). Pixels intersecting the aperture are automatically excluded.

Type:

Union[int, Tuple[int, int]]

aperture_method: str = 'fwhm'

aperture_diameter: float = 3.0

fwhm_multiplier: float = 2.5

max_processing_workers: int = 10

bad_flags: List[int]

annulus_inner_offset: float = 1.414

min_annulus_area: int = 10

max_outer_radius: float = 5.0

min_usable_pixels: int = 10

bg_sigma_clip_sigma: float = 3.0

bg_sigma_clip_maxiters: int = 3

subtract_zodi: bool = True

zodi_scale_min: float = 0.0

zodi_scale_max: float = 10.0

pixel_scale_fallback: float = 6.2

max_annulus_attempts: int = 5

annulus_expansion_step: float = 0.5

background_method: str = 'annulus'

window_size: int | Tuple[int, int] = 50

__post_init__()

Validate parameters.

to_dict() → Dict[str, Any]

Convert to dictionary for serialization.

classmethod from_dict(data: Dict[str, Any]) → PhotometryConfig

Create from dictionary.

__init__(aperture_method: str = 'fwhm', aperture_diameter: float = 3.0, fwhm_multiplier: float = 2.5, max_processing_workers: int = 10, bad_flags: ~typing.List[int] = <factory>, annulus_inner_offset: float = 1.414, min_annulus_area: int = 10, max_outer_radius: float = 5.0, min_usable_pixels: int = 10, bg_sigma_clip_sigma: float = 3.0, bg_sigma_clip_maxiters: int = 3, subtract_zodi: bool = True, zodi_scale_min: float = 0.0, zodi_scale_max: float = 10.0, pixel_scale_fallback: float = 6.2, max_annulus_attempts: int = 5, annulus_expansion_step: float = 0.5, background_method: str = 'annulus', window_size: int | ~typing.Tuple[int, int] = 50) → None

class spxquery.core.config.VisualizationConfig(sigma_threshold: float = 5.0, use_magnitude: bool = False, show_errorbars: bool = True, wavelength_cmap: str = 'rainbow', date_cmap: str = 'viridis', sigma_clip_sigma: float = 3.0, sigma_clip_maxiters: int = 10, ylim_percentile_min: float = 1.0, ylim_percentile_max: float = 99.0, ylim_padding_fraction: float = 0.1, marker_size_good: float = 1.5, marker_size_rejected: float = 2.0, marker_size_upper_limit: float = 3.0, errorbar_alpha: float = 0.2, marker_alpha: float = 0.9, errorbar_linewidth: float = 0.5, figsize: Tuple[float, float] = (10, 8), dpi: int = 150)

Bases: object

Advanced visualization configuration.

sigma_threshold

Minimum SNR (flux/flux_err) for quality control filtering. Default: 5.0. Measurements below this are marked as rejected.

Type:

float

use_magnitude

If True, plot AB magnitude instead of flux. Default: False (plot flux in μJy).

Type:

bool

show_errorbars

If True, show error bars on plots. Default: True.

Type:

bool

wavelength_cmap

Matplotlib colormap name for wavelength coding in light curves. Default: “rainbow”. Alternatives: “viridis”, “plasma”, “cividis”.

Type:

str

date_cmap

Matplotlib colormap name for date coding in spectra. Default: “viridis”. Should differ from wavelength_cmap.

Type:

str

sigma_clip_sigma

Sigma threshold for outlier removal in plots. Default: 3.0. Set to 100+ to disable outlier removal.

Type:

float

sigma_clip_maxiters

Maximum iterations for sigma clipping. Default: 10. Usually sufficient.

Type:

int

ylim_percentile_min

Lower percentile for smart y-axis limits (0-100). Default: 1.0. Use 0.0 to show all data.

Type:

float

ylim_percentile_max

Upper percentile for smart y-axis limits (0-100). Default: 99.0. Use 100.0 to show all data.

Type:

float

ylim_padding_fraction

Padding fraction added to y-axis range. Default: 0.1 (10%). Usually 0.05-0.2.

Type:

float

marker_size_good

Marker size for good measurements. Default: 1.5. Increase for print publications.

Type:

float

marker_size_rejected

Marker size for rejected measurements. Default: 2.0. Should be visible but not dominant.

Type:

float

marker_size_upper_limit

Marker size for upper limit arrows. Default: 3.0. Should be clearly visible.

Type:

float

errorbar_alpha

Transparency for error bars (0-1). Default: 0.2. Increase for print, decrease for screen.

Type:

float

marker_alpha

Transparency for markers (0-1). Default: 0.9. Usually keep near 1.0.

Type:

float

errorbar_linewidth

Line width for error bars in points. Default: 0.5. Increase for print publications.

Type:

float

figsize

Figure size in inches (width, height). Default: (10, 8). Common journal sizes: (7.5, 6), (3.5, 3).

Type:

Tuple[float, float]

dpi

Resolution in dots per inch for saved figures. Default: 150. Use 300 for print publications.

Type:

int

sigma_threshold: float = 5.0

use_magnitude: bool = False

show_errorbars: bool = True

wavelength_cmap: str = 'rainbow'

date_cmap: str = 'viridis'

sigma_clip_sigma: float = 3.0

sigma_clip_maxiters: int = 10

ylim_percentile_min: float = 1.0

ylim_percentile_max: float = 99.0

ylim_padding_fraction: float = 0.1

marker_size_good: float = 1.5

marker_size_rejected: float = 2.0

marker_size_upper_limit: float = 3.0

errorbar_alpha: float = 0.2

marker_alpha: float = 0.9

errorbar_linewidth: float = 0.5

figsize: Tuple[float, float] = (10, 8)

dpi: int = 150

__post_init__()

Validate parameters.

to_dict() → Dict[str, Any]

Convert to dictionary for serialization.

classmethod from_dict(data: Dict[str, Any]) → VisualizationConfig

Create from dictionary.

__init__(sigma_threshold: float = 5.0, use_magnitude: bool = False, show_errorbars: bool = True, wavelength_cmap: str = 'rainbow', date_cmap: str = 'viridis', sigma_clip_sigma: float = 3.0, sigma_clip_maxiters: int = 10, ylim_percentile_min: float = 1.0, ylim_percentile_max: float = 99.0, ylim_padding_fraction: float = 0.1, marker_size_good: float = 1.5, marker_size_rejected: float = 2.0, marker_size_upper_limit: float = 3.0, errorbar_alpha: float = 0.2, marker_alpha: float = 0.9, errorbar_linewidth: float = 0.5, figsize: Tuple[float, float] = (10, 8), dpi: int = 150) → None

class spxquery.core.config.DownloadConfig(max_download_workers: int = 4, cutout_size: str | None = None, cutout_center: str | None = None, chunk_size: int = 8192, timeout: int = 300, max_retries: int = 3, retry_delay: int = 5, user_agent: str = <factory>)

Bases: object

Advanced download configuration.

max_download_workers

Number of parallel workers for file downloads. Default: 4. Adjust based on network bandwidth and connection limits.

Type:

int

cutout_size

Image cutout size for downloads (e.g., “200px”, “3arcmin”). Default: None (download full images).

Type:

Optional[str]

cutout_center

Cutout center coordinates (e.g., “70,20”, “300.5,120px”). Default: None (use source position).

Type:

Optional[str]

chunk_size

Download chunk size in bytes. Default: 8192 (8 KB). Increase for fast connections.

Type:

int

timeout

HTTP request timeout in seconds. Default: 300 (5 minutes). Increase for slow connections.

Type:

int

max_retries

Number of retry attempts for failed downloads. Default: 3. Increase for unreliable connections.

Type:

int

retry_delay

Delay between retry attempts in seconds. Default: 5. Consider exponential backoff for many retries.

Type:

int

user_agent

User agent string for HTTP requests. Default: “SPXQuery/<version>”. Usually no need to change.

Type:

str

max_download_workers: int = 4

cutout_size: str | None = None

cutout_center: str | None = None

chunk_size: int = 8192

timeout: int = 300

max_retries: int = 3

retry_delay: int = 5

user_agent: str

__post_init__()

Validate parameters.

to_dict() → Dict[str, Any]

Convert to dictionary for serialization.

classmethod from_dict(data: Dict[str, Any]) → DownloadConfig

Create from dictionary.

__init__(max_download_workers: int = 4, cutout_size: str | None = None, cutout_center: str | None = None, chunk_size: int = 8192, timeout: int = 300, max_retries: int = 3, retry_delay: int = 5, user_agent: str = <factory>) → None

class spxquery.core.config.QueryConfig(source: ~spxquery.core.config.Source, output_dir: ~pathlib._local.Path = <factory>, bands: ~typing.List[str] | None = None)

Bases: object

Query-specific configuration (sub-config of AdvancedConfig).

This class handles only query-specific parameters. It is contained within AdvancedConfig, not the other way around.

Parameters:

• source (Source) – Source coordinates and name

• output_dir (Path) – Output directory for results and data

• bands (Optional[List[str]]) – List of bands to query (e.g., [‘D1’, ‘D2’]). None = all bands.

source: Source

output_dir: Path

bands: List[str] | None = None

to_dict() → Dict[str, Any]

Convert to dictionary for serialization.

classmethod from_dict(data: Dict[str, Any]) → QueryConfig

Create from dictionary.

__init__(source: ~spxquery.core.config.Source, output_dir: ~pathlib._local.Path = <factory>, bands: ~typing.List[str] | None = None) → None

class spxquery.core.config.AdvancedConfig(query: ~spxquery.core.config.QueryConfig, photometry: ~spxquery.core.config.PhotometryConfig = <factory>, visualization: ~spxquery.core.config.VisualizationConfig = <factory>, download: ~spxquery.core.config.DownloadConfig = <factory>, pipeline_stages: ~typing.List[str] = <factory>)

Bases: object

Central configuration “bus” for all advanced parameters.

This class manages ALL sub-configurations including query parameters. It provides intelligent parameter routing so users can update any parameter without needing to know which sub-config it belongs to.

query

Query-specific parameters (source, output_dir, bands)

Type:

QueryConfig

photometry

Photometry-related parameters

Type:

PhotometryConfig

visualization

Visualization and plotting parameters

Type:

VisualizationConfig

download

Download and cutout parameters

Type:

DownloadConfig

pipeline_stages

List of pipeline stages to execute. Default: [‘query’, ‘download’, ‘processing’, ‘visualization’]

Type:

List[str]

query: QueryConfig

photometry: PhotometryConfig

visualization: VisualizationConfig

download: DownloadConfig

pipeline_stages: List[str]

to_dict() → Dict[str, Any]

Convert to dictionary for serialization.

classmethod from_dict(data: Dict[str, Any]) → AdvancedConfig

Create from dictionary.

update(**kwargs) → None

Intelligently update parameters across all sub-configs.

This method automatically determines which sub-config each parameter belongs to and updates it accordingly. Users don’t need to know the internal structure.

Parameters:

**kwargs – Parameter names and values to update. Can include: - Query parameters: source, output_dir, bands - Photometry parameters: aperture_diameter, etc. - Visualization parameters: sigma_threshold, use_magnitude, etc. - Download parameters: cutout_size, max_download_workers, etc. - Pipeline control: pipeline_stages

Raises:

ValueError – If a parameter name doesn’t exist in any sub-config

Examples

>>> config = AdvancedConfig(query=QueryConfig(source=Source(ra=10, dec=20)))
>>> config.update(aperture_diameter=5.0, use_magnitude=True, bands=['D1', 'D2'])
>>> # Automatically routes to correct sub-configs:
>>> # - aperture_diameter -> photometry
>>> # - use_magnitude -> visualization
>>> # - bands -> query

to_yaml_file(filepath: Path) → None

Save to YAML file with comments.

classmethod from_yaml_file(filepath: Path) → AdvancedConfig

Load from YAML file.

classmethod from_saved_state(source_name: str, output_dir: Path, **user_overrides) → AdvancedConfig

Create AdvancedConfig by loading from saved state with optional overrides.

Parameters are loaded with priority: user_overrides > saved state > defaults.

Parameters:

• source_name (str) – Name of the source (used to find {source_name}.yaml)

• output_dir (Path) – Output directory where state file is located

• **user_overrides – Any parameters to override from saved state (can include parameters from any sub-config: query, photometry, visualization, download)

Returns:

Complete configuration loaded from state with overrides applied

Return type:

AdvancedConfig

Examples

>>> # Load everything from saved state
>>> config = AdvancedConfig.from_saved_state("cloverleaf", Path("output"))
>>>
>>> # Load from state but override some parameters
>>> config = AdvancedConfig.from_saved_state(
...     "cloverleaf", Path("output"),
...     aperture_diameter=5.0,
...     use_magnitude=True,
...     bands=['D1', 'D2']
... )

classmethod create(source: Source, output_dir: Path, bands: List[str] | None = None, **advanced_params) → AdvancedConfig

Convenient factory method to create AdvancedConfig from basic parameters.

This is the recommended way to create a new AdvancedConfig for users who want to specify some advanced parameters without creating all sub-configs.

Parameters:

• source (Source) – Source coordinates and name

• output_dir (Path) – Output directory for results

• bands (Optional[List[str]]) – List of bands to query (e.g., [‘D1’, ‘D2’]). None = all bands.

• **advanced_params – Any additional parameters to override from defaults. Can include parameters from photometry, visualization, or download configs.

Returns:

Complete configuration with specified parameters

Return type:

AdvancedConfig

Examples

>>> # Create with defaults
>>> source = Source(ra=304.69, dec=42.44, name="My_Star")
>>> config = AdvancedConfig.create(source, Path("output"))
>>>
>>> # Create with custom parameters
>>> config = AdvancedConfig.create(
...     source=Source(ra=304.69, dec=42.44, name="My_Star"),
...     output_dir=Path("output"),
...     bands=['D1', 'D2'],
...     aperture_diameter=5.0,
...     use_magnitude=True,
...     cutout_size='200px'
... )

__init__(query: ~spxquery.core.config.QueryConfig, photometry: ~spxquery.core.config.PhotometryConfig = <factory>, visualization: ~spxquery.core.config.VisualizationConfig = <factory>, download: ~spxquery.core.config.DownloadConfig = <factory>, pipeline_stages: ~typing.List[str] = <factory>) → None

class spxquery.core.config.ObservationInfo(obs_id: str, band: str, mjd: float, wavelength_min: float, wavelength_max: float, download_url: str, t_min: float, t_max: float)

Bases: object

Information about a single SPHEREx observation.

obs_id: str

band: str

mjd: float

wavelength_min: float

wavelength_max: float

download_url: str

t_min: float

t_max: float

property wavelength_center: float

Central wavelength in microns.

property bandwidth: float

Bandwidth in microns.

__init__(obs_id: str, band: str, mjd: float, wavelength_min: float, wavelength_max: float, download_url: str, t_min: float, t_max: float) → None

class spxquery.core.config.QueryResults(observations: List[ObservationInfo], query_time: datetime, source: Source, total_size_gb: float, time_span_days: float, band_counts: Dict[str, int])

Bases: object

Results from SPHEREx archive query.

observations: List[ObservationInfo]

query_time: datetime

source: Source

total_size_gb: float

time_span_days: float

band_counts: Dict[str, int]

filter_by_band(bands: List[str]) → QueryResults

Return new QueryResults filtered by bands.

__init__(observations: List[ObservationInfo], query_time: datetime, source: Source, total_size_gb: float, time_span_days: float, band_counts: Dict[str, int]) → None

class spxquery.core.config.PhotometryResult(obs_id: str, mjd: float, flux: float, flux_error: float, wavelength: float, bandwidth: float, flag: int, pix_x: float, pix_y: float, band: str, mag_ab: float | None = None, mag_ab_error: float | None = None)

Bases: object

Result from aperture photometry on a single observation.

obs_id: str

mjd: float

flux: float

flux_error: float

wavelength: float

bandwidth: float

flag: int

pix_x: float

pix_y: float

band: str

mag_ab: float | None = None

mag_ab_error: float | None = None

property is_upper_limit: bool

Check if measurement should be treated as upper limit.

__init__(obs_id: str, mjd: float, flux: float, flux_error: float, wavelength: float, bandwidth: float, flag: int, pix_x: float, pix_y: float, band: str, mag_ab: float | None = None, mag_ab_error: float | None = None) → None

class spxquery.core.config.DownloadResult(url: str, local_path: Path, success: bool, error: str | None = None, size_mb: float | None = None)

Bases: object

Result from file download attempt.

url: str

local_path: Path

success: bool

error: str | None = None

size_mb: float | None = None

__init__(url: str, local_path: Path, success: bool, error: str | None = None, size_mb: float | None = None) → None

class spxquery.core.config.PipelineState(stage: str, config: ~spxquery.core.config.AdvancedConfig, query_results: ~spxquery.core.config.QueryResults | None = None, downloaded_files: ~typing.List[~pathlib._local.Path] = <factory>, photometry_results: ~typing.List[~spxquery.core.config.PhotometryResult] = <factory>, csv_path: ~pathlib._local.Path | None = None, plot_path: ~pathlib._local.Path | None = None, completed_stages: ~typing.List[str] = <factory>)

Bases: object

State for resumable pipeline execution.

This class uses AdvancedConfig as the complete configuration container, which includes query, photometry, visualization, download, and pipeline stages.

stage: str

config: AdvancedConfig

query_results: QueryResults | None = None

downloaded_files: List[Path]

photometry_results: List[PhotometryResult]

csv_path: Path | None = None

plot_path: Path | None = None

completed_stages: List[str]

to_dict() → Dict[str, Any]

Convert to dictionary for serialization.

classmethod from_dict(data: Dict[str, Any]) → PipelineState

Create from dictionary.

__init__(stage: str, config: ~spxquery.core.config.AdvancedConfig, query_results: ~spxquery.core.config.QueryResults | None = None, downloaded_files: ~typing.List[~pathlib._local.Path] = <factory>, photometry_results: ~typing.List[~spxquery.core.config.PhotometryResult] = <factory>, csv_path: ~pathlib._local.Path | None = None, plot_path: ~pathlib._local.Path | None = None, completed_stages: ~typing.List[str] = <factory>) → None

TAP query functionality for SPHEREx data from IRSA.

spxquery.core.query.query_spherex_observations(source: Source, bands: List[str] | None = None, cutout_size: str | None = None, mjd_range: tuple | None = None, max_images: int = 0) → QueryResults

Query SPHEREx observations for a given source position.

Uses CONTAINS(POINT, polygon) to find images that cover the source.

Parameters:

• source (Source) – Target source with RA/Dec coordinates.

• bands (list of str, optional) – Bands to query (e.g., ['D1', 'D2']). None = all.

• cutout_size (str, optional) – Ignored (kept for backward compatibility). Cutout parameters are appended during the download phase.

• mjd_range (tuple, optional) – (mjd_min, mjd_max) to restrict by observation time. Applied server-side in ADQL.

• max_images (int, optional) – Safety cap on result count. 0 = no limit (default).

Return type:

QueryResults

spxquery.core.query.query_spherex_region(center_ra: float, center_dec: float, radius: float, coverage_mode: str = 'any', bands: List[str] | None = None, mjd_range: tuple | None = None, max_images: int = 500) → QueryResults

Query SPHEREx observations covering a circular sky region.

Parameters:

• center_ra (float) – Region center in degrees (ICRS).

• center_dec (float) – Region center in degrees (ICRS).

• radius (float) – Search radius in degrees.

• coverage_mode (str) – "any" — image overlaps with search circle (INTERSECTS). "full" — image fully contains the search circle (CONTAINS).

• bands (list of str, optional) – Bands to query (e.g., ['D1', 'D3']). None = all.

• mjd_range (tuple, optional) – (mjd_min, mjd_max) to restrict by observation time. Applied server-side in ADQL.

• max_images (int) – Safety cap. Raise if exceeded (default 500).

Return type:

QueryResults

spxquery.core.query.print_query_summary(query_results: QueryResults) → None

Print a summary of query results.

Parallel download manager for SPHEREx FITS files.

spxquery.core.download.download_file(url: str, output_path: Path, timeout: int = 300, retries: int = 3, retry_delay: int = 5, chunk_size: int = 8192, user_agent: str = 'SPXQuery/0.4.1') → DownloadResult

Download a single file with retry logic.

Parameters:

• url (str) – URL to download

• output_path (Path) – Local path to save file

• timeout (int) – Download timeout in seconds (default: 300)

• retries (int) – Number of retry attempts (default: 3)

• retry_delay (int) – Delay between retry attempts in seconds (default: 5)

• chunk_size (int) – Download chunk size in bytes (default: 8192)

• user_agent (str) – User agent string for HTTP requests (default: “SPXQuery/0.4.1”)

Returns:

Result of download attempt

Return type:

DownloadResult

spxquery.core.download.parallel_download(download_info: List[Tuple[ObservationInfo, str]], output_dir: Path, max_workers: int = 4, show_progress: bool = True, skip_existing: bool = True, download_config: DownloadConfig | None = None) → List[DownloadResult]

Download multiple files in parallel with progress tracking.

Parameters:

• download_info (List[Tuple[ObservationInfo, str]]) – List of (observation, download_url) tuples

• output_dir (Path) – Directory to save downloaded files

• max_workers (int) – Maximum number of parallel downloads (default: 4)

• show_progress (bool) – Whether to show progress bar (default: True)

• skip_existing (bool) – If True, skip files that already exist. If False, re-download all files (default: True)

• download_config (DownloadConfig, optional) – Download configuration with timeout, retries, etc. If None, uses defaults.

Returns:

Results for all download attempts

Return type:

List[DownloadResult]

Notes

Priority: explicit max_workers parameter > download_config.max_download_workers > default

spxquery.core.download.print_download_summary(results: List[DownloadResult]) → None

Print a summary of download results.

Parameters:

results (List[DownloadResult]) – Download results to summarize

Main pipeline orchestrator for SPXQuery package with flexible, resumable execution.

class spxquery.core.pipeline.SPXQueryPipeline(config: AdvancedConfig, pipeline_stages: List[str] | None = None)

Bases: object

Main pipeline for SPHEREx data query, download, and analysis.

Supports: - Flexible stage configuration (add/remove pipeline stages) - Full automatic execution or step-by-step mode - Resumable execution with state persistence - Dependency checking for manual execution

STAGE_DEPENDENCIES = {'download': ['query'], 'processing': ['query', 'download'], 'query': [], 'visualization': ['query', 'download', 'processing']}

__init__(config: AdvancedConfig, pipeline_stages: List[str] | None = None)

Initialize pipeline with configuration.

Parameters:

• config (AdvancedConfig) – Complete pipeline configuration including query, photometry, visualization, and download settings

• pipeline_stages (List[str], optional) – List of stages to execute. If None, uses config.pipeline_stages. Default: [‘query’, ‘download’, ‘processing’, ‘visualization’] Allows customization of pipeline flow (e.g., skip visualization, add custom stages)

save_state() → None

Save current pipeline state to disk.

load_state() → bool

Load pipeline state from disk.

Returns:

True if state was loaded successfully

Return type:

bool

check_dependencies(stage: str) → None

Check if all dependencies for a stage are satisfied.

Parameters:

stage (str) – Stage name to check

Raises:

RuntimeError – If dependencies are not satisfied

mark_stage_complete(stage: str) → None

Mark a stage as completed.

Parameters:

stage (str) – Stage name

get_status_message() → str

Get a human-readable status message.

Returns:

Status message describing completed and pending stages

Return type:

str

print_status() → None

Print current pipeline status.

run_full_pipeline(skip_existing_downloads: bool = True) → None

Run the complete pipeline through all configured stages.

Parameters:

skip_existing_downloads (bool) – If True, skip already downloaded files. If False, re-download everything.

run_query() → None

Execute query stage.

run_download(skip_existing: bool = True) → None

Execute download stage.

Parameters:

skip_existing (bool) – If True, skip files that already exist. If False, re-download all files.

run_processing() → None

Execute processing stage.

run_visualization() → None

Execute visualization stage.

resume(skip_existing_downloads: bool = True) → None

Resume pipeline from saved state.

Parameters:

skip_existing_downloads (bool) – If True, skip files that already exist during download. If False, re-download.

spxquery.core.pipeline.run_pipeline(ra: float, dec: float, output_dir: Path | None = None, bands: List[str] | None = None, aperture_diameter: float = 3.0, source_name: str | None = None, resume: bool = False, log_level: str = 'INFO', max_download_workers: int = 4, max_processing_workers: int = 10, cutout_size: str | None = None, cutout_center: str | None = None, sigma_threshold: float = 5.0, bad_flags: List[int] | None = None, use_magnitude: bool = False, show_errorbars: bool = True, skip_existing_downloads: bool = True, pipeline_stages: List[str] | None = None, advanced_params_file: str | Path | None = None) → None

Convenience function to run the pipeline with sensible defaults.

Parameters:

• ra (float) – Right ascension in degrees

• dec (float) – Declination in degrees

• output_dir (Path, optional) – Output directory (default: current directory)

• bands (List[str], optional) – Bands to query (e.g., [‘D1’, ‘D2’])

• aperture_diameter (float) – Aperture diameter in pixels (default: 3)

• source_name (str, optional) – Name of the source

• resume (bool) – Whether to resume from saved state

• log_level (str) – Logging level

• max_download_workers (int) – Number of worker threads for downloading (default: 4)

• max_processing_workers (int) – Number of worker processes for photometry (default: 10)

• cutout_size (str, optional) – Cutout size parameter (e.g., “200px”, “3arcmin”)

• cutout_center (str, optional) – Cutout center parameter (e.g., “70,20”) or None to use source position

• sigma_threshold (float) – Minimum SNR (flux/flux_err) for quality control (default: 5.0)

• bad_flags (List[int], optional) – List of bad flag bit positions to reject (default: [0, 1, 2, 6, 7, 9, 10, 11, 14, 15, 17, 19])

• use_magnitude (bool) – If True, plot AB magnitude instead of flux (default: False)

• show_errorbars (bool) – If True, show errorbars on plots (default: True)

• skip_existing_downloads (bool) – If True, skip already downloaded files. If False, re-download all (default: True)

• pipeline_stages (List[str], optional) – List of stages to execute (default: [‘query’, ‘download’, ‘processing’, ‘visualization’])

• advanced_params_file (str or Path, optional) – Path to JSON file with advanced parameters (photometry, visualization, download settings). If provided, these parameters are loaded with priority: user input > JSON file > defaults

Examples

>>> # Basic usage
>>> run_pipeline(ra=304.69, dec=42.44, output_dir="output")

>>> # With advanced parameters
>>> from spxquery.utils.params import export_default_parameters
>>> params_file = export_default_parameters("output")
>>> # Edit output/spxquery_default_params.json
>>> run_pipeline(ra=304.69, dec=42.44, advanced_params_file=params_file)