Utility Modules
Shared utilities for logging, file I/O, parameter management, and FITS handling.
Helper utility functions for SPXQuery package.
- spxquery.utils.helpers.setup_logging(level: str = 'INFO') None[source]
Set up logging configuration.
- Parameters:
level (str) – Logging level (DEBUG, INFO, WARNING, ERROR)
- spxquery.utils.helpers.save_yaml(data: Dict[str, Any], filepath: Path) None[source]
Save dictionary to YAML file.
- Parameters:
data (Dict[str, Any]) – Data to save
filepath (Path) – Output file path
- spxquery.utils.helpers.load_yaml(filepath: Path) Dict[str, Any][source]
Load dictionary from YAML file.
- Parameters:
filepath (Path) – Input file path
- Returns:
Loaded data
- Return type:
Dict[str, Any]
- spxquery.utils.helpers.format_file_size(size_bytes: float) str[source]
Format file size in human-readable format.
- spxquery.utils.helpers.validate_directory(path: Path, create: bool = True) bool[source]
Validate and optionally create directory.
- spxquery.utils.helpers.get_file_list(directory: Path, pattern: str = '*.fits') list[Path][source]
Get list of files matching pattern in directory.
- spxquery.utils.helpers.validate_cutout_size(size_str: str) bool[source]
Validate cutout size parameter format.
Valid formats: - Single value: “200”, “0.1”, “3.5” - Two values: “100,200”, “0.5,1.0” - With units: “200px”, “100,200pixels”, “3arcmin”, “0.1deg”
- spxquery.utils.helpers.validate_cutout_center(center_str: str) bool[source]
Validate cutout center parameter format.
Valid formats: - Degrees (default): “70,20”, “304.5,42.3” - Pixels: “1020,1020px”, “500,600pixels” - Other angular: “1.5,0.8rad”, “304.5,42.3deg”
- spxquery.utils.helpers.format_cutout_url_params(cutout_size: str | None, cutout_center: str | None, source_ra: float, source_dec: float) str[source]
Format cutout parameters as URL query string.
If cutout_size is None, returns empty string (no cutout). If cutout_size is specified but cutout_center is None, uses source position. If both specified, uses provided center.
- Parameters:
cutout_size (str or None) – Size parameter (e.g., “200px”, “3arcmin”)
cutout_center (str or None) – Center parameter (e.g., “70,20”) or None to use source position
source_ra (float) – Source RA in degrees (used if cutout_center is None)
source_dec (float) – Source Dec in degrees (used if cutout_center is None)
- Returns:
URL query string (e.g., “?size=200px” or “?center=70,20&size=200px”) Empty string if cutout_size is None
- Return type:
Examples
>>> format_cutout_url_params("200px", None, 304.69, 42.44) '?center=304.69,42.44&size=200px'
>>> format_cutout_url_params("3arcmin", "70,20", 304.69, 42.44) '?center=70,20&size=3arcmin'
>>> format_cutout_url_params(None, None, 304.69, 42.44) ''
- spxquery.utils.helpers.estimate_cutout_size_mb(cutout_size: str | None, full_size_mb: float = 71.6, min_size_mb: float = 5.0) float[source]
Estimate cutout file size based on dimensions.
Assumes full SPHEREx image is 2040x2040 pixels (~71.6 MB). Estimates cutout size proportional to pixel area, with minimum 5 MB due to auxiliary data (PSF, WCS, etc.) that’s always included.
SPHEREx pixel scale: ~6.2 arcsec/pixel
- Parameters:
- Returns:
Estimated cutout size in MB
- Return type:
Examples
>>> estimate_cutout_size_mb("200px") 5.0 # minimum size due to auxiliary data
>>> estimate_cutout_size_mb("500px") 6.1 # (500*500)/(2040*2040) * 71.6 ≈ 4.3, but min 5.0
>>> estimate_cutout_size_mb("1000px") 17.2 # (1000*1000)/(2040*2040) * 71.6
>>> estimate_cutout_size_mb("3arcmin") 9.7 # 3 arcmin ≈ 29 pixels, but minimum 5 MB applies
>>> estimate_cutout_size_mb(None) 71.6 # full image
- spxquery.utils.helpers.create_flag_mask(bad_flags: list[int]) int[source]
Convert list of bad flag bit positions to a single integer mask.
- Parameters:
- Returns:
Integer mask with all bad flag bits set
- Return type:
Examples
>>> create_flag_mask([0, 1, 2]) 7 # 0b0111
>>> create_flag_mask([0, 2, 4]) 21 # 0b10101
- spxquery.utils.helpers.check_flag_bits(flag: int, bad_flags_mask: int) bool[source]
Check if any bad flag bits are set in the given flag value.
- Parameters:
- Returns:
True if any bad flags are set, False otherwise
- Return type:
Examples
>>> mask = create_flag_mask([0, 1, 2]) >>> check_flag_bits(0b0000, mask) # No flags set False
>>> check_flag_bits(0b0001, mask) # Bit 0 is set True
>>> check_flag_bits(0b1000, mask) # Only bit 3 is set (not in bad mask) False
- spxquery.utils.helpers.apply_quality_filters(photometry_results: list, sigma_threshold: float = 5.0, bad_flags: list[int] | None = None) tuple[list, dict][source]
Apply quality control filters to photometry results.
Filters based on: 1. SNR threshold: flux/flux_err >= sigma_threshold 2. Flag rejection: reject points with any bad flags set
- Parameters:
photometry_results (list[PhotometryResult]) – Input photometry measurements
sigma_threshold (float) – Minimum SNR (flux/flux_err) to accept (default: 5.0)
bad_flags (list[int], optional) – List of bad flag bit positions to reject Default: [0, 1, 2, 6, 7, 9, 10, 11, 14, 15, 17, 19]
- Returns:
filtered_results (list[PhotometryResult]) – Filtered photometry results
filter_stats (dict) – Statistics about filtering (rejected counts by reason)
Examples
>>> from spxquery.core.config import PhotometryResult >>> results = [ ... PhotometryResult(..., flux=10, flux_error=1, flag=0), # Good ... PhotometryResult(..., flux=10, flux_error=5, flag=0), # Low SNR ... PhotometryResult(..., flux=10, flux_error=1, flag=0b0001), # Bad flag ... ] >>> filtered, stats = apply_quality_filters(results, sigma_threshold=5.0) >>> len(filtered) 1
- class spxquery.utils.helpers.ClassifiedPhotometry(good_regular: List, rejected_regular: List, good_upper_limits: List, rejected_upper_limits: List)[source]
Bases:
objectQuality-classified photometry points.
- spxquery.utils.helpers.classify_photometry_by_quality(photometry_results: List, sigma_threshold: float, bad_flags_mask: int | None = None, separate_upper_limits: bool = True) ClassifiedPhotometry[source]
Classify photometry points as good/rejected based on SNR and flag filters.
This function separates photometry measurements into four categories: - Good regular measurements (SNR >= threshold, no bad flags, not upper limit) - Rejected regular measurements (SNR < threshold or has bad flags, not upper limit) - Good upper limits (SNR >= threshold, no bad flags, is upper limit) - Rejected upper limits (SNR < threshold or has bad flags, is upper limit)
- Parameters:
photometry_results (List[PhotometryResult]) – Photometry measurements to classify
sigma_threshold (float) – Minimum SNR (flux/flux_err) threshold
bad_flags_mask (int, optional) – Integer mask with bad flag bits set (from create_flag_mask) If None, no flag filtering is applied
separate_upper_limits (bool) – If True, separate upper limits from regular measurements
- Returns:
Classified photometry points in four categories
- Return type:
Examples
>>> from spxquery.core.config import PhotometryResult >>> from spxquery.utils.helpers import create_flag_mask, classify_photometry_by_quality >>> >>> # Create sample data >>> results = [...] # List of PhotometryResult objects >>> >>> # Create flag mask for bad flags >>> bad_flags_mask = create_flag_mask([0, 1, 2, 6, 7, 9, 10, 11, 15]) >>> >>> # Classify photometry >>> classified = classify_photometry_by_quality( ... results, ... sigma_threshold=5.0, ... bad_flags_mask=bad_flags_mask, ... separate_upper_limits=True ... ) >>> >>> print(f"Good regular: {len(classified.good_regular)}") >>> print(f"Rejected regular: {len(classified.rejected_regular)}")
Utilities for exporting and importing advanced parameter templates.
- spxquery.utils.params.export_default_parameters(output_path: str | Path, filename: str = 'spxquery_default_params.yaml') Path[source]
Export default advanced parameters to YAML file with comments.
This creates a template file that users can modify to customize photometry, visualization, and download parameters. The template does NOT include source-specific information (ra/dec/name).
- Parameters:
output_path (str or Path) – Directory to save the parameter file, or full path to YAML file. If directory, saves as {output_path}/{filename}. If file path ending in .yaml/.yml, uses that path directly.
filename (str, optional) – Filename to use if output_path is a directory. Default: “spxquery_default_params.yaml”
- Returns:
Full path to the created parameter file
- Return type:
Path
Examples
>>> # Save to directory with default filename >>> params_file = export_default_parameters("my_folder") >>> # Returns: Path("my_folder/spxquery_default_params.yaml")
>>> # Save to specific file >>> params_file = export_default_parameters("my_folder/custom_params.yaml") >>> # Returns: Path("my_folder/custom_params.yaml")
>>> # Save to current directory >>> params_file = export_default_parameters(".") >>> # Returns: Path("./spxquery_default_params.yaml")
- spxquery.utils.params.load_advanced_config(filepath: Path) AdvancedConfig[source]
Load advanced configuration from YAML file.
- Parameters:
filepath (Path) – Path to YAML parameter file
- Returns:
Loaded configuration
- Return type:
- Raises:
FileNotFoundError – If file doesn’t exist
ValueError – If file contains invalid parameters
Enhanced SPHEREx Multi-Extension FITS (MEF) file handling with utility methods.
This module provides the SPHERExMEF class and related utilities for working with SPHEREx spectral imaging data, including: - Unit conversion (MJy/sr to μJy/arcsec² or other units) - PSF extraction from 121-zone (11×11) oversampled grid - WCS coordinate transformation wrappers - Image cutout extraction - Zodiacal background subtraction
- spxquery.utils.spherex_mef.suppress_astropy_info()[source]
Context manager to temporarily suppress astropy INFO messages and FITS warnings.
This is needed because SPHEREx provides non-standard WCS headers that trigger harmless but annoying INFO messages about SIP distortion coefficients and warnings about redundant SCAMP distortion parameters.
- class spxquery.utils.spherex_mef.SPHERExMEF(filepath: Path, image: ndarray, flags: ndarray, variance: ndarray, zodi: ndarray, psf: ndarray, spatial_wcs: WCS, spectral_wcs: WCS, header: Header, psf_header: Header, obs_id: str, detector: int, mjd: float, psf_fwhm: float = 6.0, image_unit: str = 'MJy/sr', native_unit: str = 'MJy/sr', _psf_zone_centers: Dict[int, Tuple[float, float]] | None = None, _psf_oversamp: int | None = None)[source]
Bases:
objectContainer for SPHEREx Multi-Extension FITS data with enhanced methods.
- filepath
Path to original FITS file
- Type:
Path
- image
Calibrated flux (units specified by image_unit)
- Type:
np.ndarray
- flags
Bitmap flags for pixel quality
- Type:
np.ndarray
- variance
Variance array (units: image_unit²)
- Type:
np.ndarray
- zodi
Zodiacal light model (same units as image)
- Type:
np.ndarray
- psf
PSF cube (101×101×121) - 11×11 zones, each oversampled 10× (101×101 pixels)
- Type:
np.ndarray
- spatial_wcs
Primary astrometric WCS (RA/Dec)
- Type:
WCS
- spectral_wcs
Alternative spectral WCS (wavelength/bandwidth)
- Type:
WCS
- header
Primary IMAGE extension header
- Type:
fits.Header
- psf_header
PSF extension header (contains XCTR_*, YCTR_*, OVERSAMP)
- Type:
fits.Header
- _psf_zone_centers
Cached PSF zone center coordinates (zone_id -> (x, y))
- filepath: Path
- property image_zodi_subtracted: ndarray
Return zodiacal light subtracted image with amplitude scaling.
- property psf_oversamp: int
Get PSF oversampling factor from header.
- Returns:
Oversampling factor (typically 10 for SPHEREx)
- Return type:
- property psf_pixel_scale: float
Get PSF pixel scale in arcsec/pixel using oversampling factor from header.
The PSF pixel scale is the detector pixel scale divided by the oversampling factor, which is read from the OVERSAMP keyword in the PSF header.
- Returns:
PSF pixel scale in arcsec/pixel
- Return type:
- world_to_pixel(ra: float, dec: float) Tuple[float, float][source]
Convert world coordinates (RA/Dec) to pixel coordinates.
- pixel_to_world(x: float, y: float) SkyCoord[source]
Convert pixel coordinates to world coordinates (RA/Dec).
- pixel_to_wavelength(x: float, y: float) Tuple[float, float][source]
Get wavelength and bandwidth at pixel position.
Uses the spectral WCS (alternative ‘W’) which provides wavelength information via lookup table for SPHEREx’s 2D spectral mapping.
- get_pixel_scale(x: float, y: float, fallback: float = 6.2) float[source]
Calculate the pixel scale in arcsec/pixel at a given position.
This accounts for any WCS distortions and provides the actual pixel scale at the specified position.
- extract_psf_at_position(x: float, y: float) ndarray[source]
Extract PSF at a specific pixel position by finding the closest PSF zone.
SPHEREx PSF cube structure: - 121 spatial zones distributed across the detector - Zone centers defined by XCTR_i and YCTR_i keywords in PSF header - Each PSF is 101×101 pixels (oversampled by OVERSAMP factor, typically 10)
This method: 1. Converts pixel position to parent image coordinates (if cutout) 2. Loads PSF zone centers from PSF header (XCTR_*, YCTR_*) 3. Finds the zone with minimum distance to the position 4. Returns the corresponding PSF from the cube
- get_psf_fwhm_estimate(x: float, y: float) float[source]
Estimate PSF FWHM at a given position using radial profile analysis.
Uses photutils.profiles.RadialProfile to convert the 2D PSF to a 1D radial profile, then interpolates to find the half-maximum radius.
- Parameters:
- Returns:
Estimated FWHM in arcseconds
- Return type:
- Raises:
Exception – If FWHM estimation fails. Caller should handle the error and decide on fallback behavior.
Notes
PSF is 101×101 pixels with 10× oversampling. The PSF center is determined from the peak pixel position (not geometric center), and radial sampling is adapted to the PSF size. Uses constant extrapolation at radial profile boundaries for robust interpolation.
- get_cutout(position: Tuple[float, float] | SkyCoord, size: int | Tuple[int, int], include_extensions: list | None = None, mode: str = 'trim') Dict[str, ndarray][source]
Extract a cutout from the MEF data using astropy.nddata.Cutout2D.
Returns a dictionary with cutout arrays and updated WCS. Uses Cutout2D for proper WCS handling, which automatically updates WCS keywords for the cutout region.
- Parameters:
position (tuple of float or SkyCoord) – Position of cutout center. Can be either: - Tuple (x, y) of pixel coordinates (0-based) - SkyCoord object with RA/Dec
size (int or tuple of int) – Size of cutout in pixels. If int, creates square cutout. If tuple (height, width), creates rectangular cutout. Note: Order is (height, width) to match numpy array shape convention.
include_extensions (list of str, optional) – List of extensions to include in cutout. Default: [‘image’, ‘flags’, ‘variance’, ‘zodi’] Available: [‘image’, ‘flags’, ‘variance’, ‘zodi’, ‘psf’]
mode (str, optional) – Mode for handling cutouts that extend beyond image boundaries. Options: ‘trim’ (default), ‘partial’, ‘strict’ See astropy.nddata.Cutout2D documentation for details.
- Returns:
Dictionary containing: - Requested extension cutouts (np.ndarray) - ‘wcs’: Updated WCS for the cutout region - ‘position_original’: Original position in parent image (x, y) - ‘position_cutout’: Position in cutout coordinates (x, y) - ‘bbox_original’: Bounding box in original image (BoundingBox object) - ‘shape’: Shape of cutout (height, width)
- Return type:
Examples
>>> # Square cutout using pixel coordinates >>> cutout = mef.get_cutout(position=(1020, 1020), size=50) >>> cutout_image = cutout['image'] >>> cutout_wcs = cutout['wcs'] >>> >>> # Rectangular cutout using SkyCoord >>> from astropy.coordinates import SkyCoord >>> import astropy.units as u >>> coord = SkyCoord(ra=304.5*u.deg, dec=42.3*u.deg) >>> cutout = mef.get_cutout(position=coord, size=(100, 50), ... include_extensions=['image', 'flags']) >>> >>> # Access cutout center in cutout pixel coordinates >>> center_x_cutout, center_y_cutout = cutout['position_cutout']
- __init__(filepath: Path, image: ndarray, flags: ndarray, variance: ndarray, zodi: ndarray, psf: ndarray, spatial_wcs: WCS, spectral_wcs: WCS, header: Header, psf_header: Header, obs_id: str, detector: int, mjd: float, psf_fwhm: float = 6.0, image_unit: str = 'MJy/sr', native_unit: str = 'MJy/sr', _psf_zone_centers: Dict[int, Tuple[float, float]] | None = None, _psf_oversamp: int | None = None) None
- spxquery.utils.spherex_mef.read_spherex_mef(filepath: Path, target_unit: str | None = None) SPHERExMEF[source]
Read SPHEREx Multi-Extension FITS file with optional unit conversion.
- Parameters:
filepath (Path) – Path to SPHEREx MEF file
target_unit (str, optional) – Target unit for image data. Options: - None: Keep native MJy/sr (default) - ‘uJy/arcsec2’ or ‘microJy/arcsec2’: Convert to μJy/arcsec² - ‘Jy/arcsec2’: Convert to Jy/arcsec² - ‘MJy/sr’: No conversion (native)
- Returns:
Container with all MEF data. If unit conversion was applied, image, variance, and zodi arrays are in target units.
- Return type:
Notes
Conversion from MJy/sr to μJy/arcsec²: 1 MJy/sr = 1e6 Jy/sr = 1e6 Jy / (206265 arcsec)² = 0.02350443 μJy/arcsec² Conversion factor: 1 MJy/sr × 0.02350443 = μJy/arcsec²
- spxquery.utils.spherex_mef.get_pixel_coordinates(mef: SPHERExMEF, ra: float, dec: float) Tuple[float, float][source]
Convert RA/Dec to pixel coordinates (legacy function).
Use mef.world_to_pixel() instead for new code.
- spxquery.utils.spherex_mef.get_wavelength_at_position(mef: SPHERExMEF, x: float, y: float) Tuple[float, float][source]
Get wavelength and bandwidth at pixel position (legacy function).
Use mef.pixel_to_wavelength() instead for new code.
- spxquery.utils.spherex_mef.get_pixel_scale_at_position(wcs: WCS, x: float, y: float, pixel_scale_fallback: float = 6.2) float[source]
Calculate pixel scale at position (legacy function).
Use mef.get_pixel_scale() instead for new code.
- spxquery.utils.spherex_mef.get_flag_info(flag_value: int) Dict[str, bool][source]
Decode flag bitmap into individual flags.
- spxquery.utils.spherex_mef.format_flag_binary(flag_value: int, num_bits: int = 22) str[source]
Format flag value as binary string.
- spxquery.utils.spherex_mef.create_background_mask(flags: ndarray, exclude_source: bool = True) ndarray[source]
Create mask for background pixels (good for zodiacal matching).
Masks out pixels with problematic flags including non-functional pixels, outliers, etc. By default, SOURCE-flagged pixels are also excluded.
- Parameters:
flags (np.ndarray) – Flag bitmap array
exclude_source (bool, optional) – If True (default), also exclude SOURCE-flagged pixels (bit 21). If False, SOURCE pixels are kept for local background estimation.
- Returns:
Boolean mask (True = good background pixel)
- Return type:
np.ndarray
Notes
Bad flag bits: 0=TRANSIENT, 1=OVERFLOW, 2=SUR_ERROR, 6=NONFUNC, 7=DICHROIC, 9=MISSING_DATA, 10=HOT, 11=COLD, 14=PHANMISS, 15=NONLINEAR, 17=PERSIST, 19=OUTLIER, optionally 21=SOURCE.
- spxquery.utils.spherex_mef.estimate_zodiacal_scaling(image: ndarray, zodi: ndarray, mask: ndarray, variance: ndarray | None = None) float[source]
Estimate scaling factor to match zodiacal model to observed background.
Uses least-squares fitting on uncontaminated pixels to find the multiplicative factor that best matches the zodi model to the data.
- Parameters:
image (np.ndarray) – Observed image data
zodi (np.ndarray) – Zodiacal model
mask (np.ndarray) – Boolean mask (True = good background pixels)
variance (np.ndarray, optional) – Variance array for weighted fitting
- Returns:
Scaling factor for zodiacal model
- Return type:
- spxquery.utils.spherex_mef.subtract_zodiacal_background(image: ndarray, zodi: ndarray, flags: ndarray, variance: ndarray | None = None, zodi_scale_min: float = 0.0, zodi_scale_max: float = 10.0, bg_fraction_reject_level: float = 0.5, static_zodi: bool = False) Tuple[ndarray, float][source]
Subtract zodiacal light background from image with amplitude scaling.
Uses uncontaminated background pixels to determine the optimal scaling factor for the zodiacal model before subtraction.
- Parameters:
image (np.ndarray) – Original image
zodi (np.ndarray) – Zodiacal light model
flags (np.ndarray) – Flag bitmap array
variance (np.ndarray, optional) – Variance array for weighted fitting
zodi_scale_min (float) – Minimum allowed zodiacal scaling factor
zodi_scale_max (float) – Maximum allowed zodiacal scaling factor
bg_fraction_reject_level (float, optional) – Minimum fraction of background pixels required for zodiacal estimation. If the fraction of available background pixels is below this threshold, the function will use a fallback scale factor of 1.0. Default is 0.5.
static_zodi (bool, optional) – If True, use the zodiacal model as-is (scale=1.0) without fitting. Skips background masking and scaling factor estimation entirely.
- Returns:
corrected_image (np.ndarray) – Background-subtracted image
scale_factor (float) – Applied scaling factor for the zodiacal model