Files
openswarm/.venv/lib/python3.12/site-packages/PIL/ImageFile.py
T
3008f083ca [aidan] feat: scheduled tasks (#105)
* [eric] ci: gitleaks-ignore the known historical secrets so our branch stops failing on leaks it didnt add

* [eric] workflows: restore scheduled-tasks on the workflow line (revert removal, keep windows fixes + 1.1.69)

* [eric] workflows: re-apply uncommitted scheduling wip (schedule pill, calendar view, slice)

* [eric] ops: gitignore dev-team local state files

* [eric] ops: backlog item for download-tracking visibility

* [eric] ci: allowlist the cdp-routes redaction-test token in gitleaks

* [aidan] feat/scheduled-tasks: keep step labels in sync on edit and show chevron on every step

* [aidan] fix: schedule time in chat

* [aidan] ux/workflows: add workflow step removal (#91)

* [aidan] feat/scheduled-tasks: remember workflow tool permissions across runs

* [aidan] feat/task-scheduling: add hourly and minute (15-min minimum) schedule intervals (#93)

* [aidan] feat/scheduled-tasks: calendar, rename, and edit workflows (#94)

* [aidan] bug: fix schedule button

* [aidan] fix/agent-errors: surface provider rate limits

* [aidan] ux/cards: click-to-rename for chat and workflow titles

Single-click a card's title to enter edit mode inline. Commit on Enter/blur,
cancel on Escape. Rename persists via PATCH for workflows and sessions.

* [aidan] feat/workflows: seed build prompt for zero-step workflows

When a new workflow has no steps, seed the agent with a prompt asking
the user to describe what the workflow should do, rather than starting blank.

* [aidan] feat/workflows: add-to-schedule popover for unscheduled workflows

Clicking the "+" on an unscheduled workflow row opens a popover with two options:
- Keep this schedule: enables the workflow's existing cadence and moves it to Scheduled
- Change schedule: opens the scheduling editor to pick a different time

* [aidan] ux/workflows: wire add-to-schedule popover and simplify New button

- Made the "+" icon on unscheduled workflow rows clickable, opening a popover
  to keep or change the schedule
- Removed AddIcon from toolbar "New" button (now reads "New" instead of "+ New")

* [aidan] fix/scheduled-tasks: open schedule calendar when Schedule pill clicked

Fixed the Schedule pill click being swallowed by the toolbar's dismiss handler.
Exempted the toolbar pills via data-toolbar-pills so their click handlers fire.

* [aidan] ux/workflows: open New workflow in agent build chat instead of empty card

When creating a new workflow from the hub, open it in edit_agent view (with the
agent builder chat) instead of a preview card. The workflow is created on the
backend first so the embedded session has a real ID.

* [aidan] feat/workflow-edit: add draft testing save flow

* [aidan] ux/chat: remove continue chat button

* [aidan] ux/workflows: polish workflow card interactions

* [aidan] fix/workflow-scheduling: save unscheduled workflows as drafts

* aidan ui: schedule naming changes

* [aidan] ui: tool calling desc/naming

* [aidan] ui: calendar sidebar naming

* [aidan] ui: fix stop viewing closing chat

* [aidan] feat/workflows: auto-name workflows and polish the build flow (#95)

* [aidan] feat/workflow-auto-naming: auto-generate workflow titles from steps

Generate a title + description from a workflow's steps (one aux call,
reused for step labels) whenever it is still auto_named, so a workflow
built in the Edit Agent names itself on commit instead of staying
"New workflow". A manual rename sets auto_named=False and is never
overwritten. Stream the aux call (non-streaming drops content on some
9router lanes) and fall back to a step-derived title when the model is
unavailable.

* [aidan] feat/workflows: hide unsaved new workflows until first save

A brand-new "+ New" workflow is created with unsaved=true and kept out
of the hub's scheduled/unscheduled lists while the user is still building
it in the Edit Agent. The first commit (Save) clears the flag and the
workflow appears. Every other create path stays visible immediately.

* [aidan] ux/workflows: remove redundant save workflow button

The Edit Agent already has Discard/Save controls in its strip, so the
header "Save Workflow" button was a duplicate save path. Remove it and
its pulse/edit-session-id wiring; the model/time subtitle stays.

* [aidan] ux/workflows: animate title on auto-rename

Wrap the workflow card title in the same Typewriter the chat card uses,
so when the auto-generated name replaces the placeholder after Save it
retypes letter-by-letter. Gated on a real (non-placeholder) title so it
never animates on mount or for already-named workflows.

* [aidan] ux/workflows: animate sidebar title on auto-rename

Wrap the calendar hub's sidebar row title in the same Typewriter the
workflow card uses, so a title that auto-renames retypes letter-by-letter
in the sidebar too. Extract the placeholder/isRealTitle guard into the
shared workflowVisuals so the card and sidebar stay in sync.

* [aidan] fix/workflows: connect watch tether, keep watched chat open, wire draft run/history

* [aidan] ui: grey out chat pill when not selected

* [aidan] ui: fix running agent display

* [aidan] feat/history-popover: add chat history and scheduled tasks run log tabs (#96)

* [aidan] ux/schedule: toast when calendar view already open on expand

* [aidan] ui: fix popover descs

* [aidan] feat/workflow-runs: add pause, resume, and stop controls for live runs

* [aidan] feat/schedule-calendar: add calendar occurrences endpoint and concrete timezones

* [aidan] feat/workflows: require at least one step to save a workflow

* [aidan] ux/edit-agent: hide Discard for an unsaved new workflow

* [aidan] ux/edit-agent: move fix-prefix card below the step list

* [aidan] feat/workflows: toast when an unattended scheduled run starts

* [aidan] fix/dashboard-tethers: anchor workflow-sidecar tethers to measured card rects

* [aidan] feat/workflows: validate steps before scheduling and keep chat tool memory

* [aidan] feat/mcp-suggestions: dismissable integration banner with per-session cooldown

* [aidan] feat/workflows: add scheduled-run "running now" toast with click-to-view (#97)

* [aidan] ux/workflows: surface paused state on card, sidebar, and calendar; tidy run history

* [aidan] feat/mcp-suggestions: suggest both Google and Microsoft when provider is ambiguous

* [aidan] fix/agent-tokens: friendly out-of-tokens card across all agent surfaces

* [aidan] feat/workflow-model: persist edit-agent model on save with switch notice and fresh drafts

* [aidan] fix/workflow-chat: force stop on watched run mirrors workflow card stop

* [aidan] fix/workflow-cards: keep watched run tethered on finish to avoid duplicate chat

* [aidan] feat/schedule-calendar: mark current time with a now line in week view

* [aidan] refactor/private-names: rename error and schedule classifiers from _ to p_

* [aidan] feat/scheduled-tasks: agent workflow scheduling and in-chat convert (#98)

* [aidan] feat/workflow-suggest: nudge user to convert repeatable chat to workflow

Add SuggestConvertToWorkflow MCP tool that agents call at the end of a task
when they've completed something worth repeating (daily report, weekly check,
recurring data pull). Frontend detects the tool call and glows the "Convert to
workflow" button 3 times to draw the eye. When user clicks it, the suggested
cadence (e.g. "every weekday at 9am") is stored in the draft and seeded into
the scheduling agent's first prompt, so the agent can act on the suggestion
rather than asking the user again.

Tool is never auto-called — agents decide when a task is genuinely repeatable
(not debugging, creative work, one-off lookup). Tool description emphasizes
sparse, high-confidence use only (once per session max).

Files changed:
- backend/apps/agents/schedule_mcp_server.py: add SuggestConvertToWorkflow tool
- frontend/src/shared/mcpToolMeta.ts: add label for new tool
- frontend/src/app/pages/Dashboard/cards/AgentCard.tsx: detect suggestion in
  session messages, show+glow "Convert to workflow" button, pass cadence to draft
- frontend/src/shared/state/workflowsSlice.ts: add suggested_cadence field to
  Workflow interface
- frontend/src/app/pages/Workflows/SchedulingView.tsx: seed scheduling agent
  prompt with suggested cadence hint

* [aidan] feat/agent-scheduling: route recurring asks through native workflows, deny claude cron skill

* [aidan] feat/workflow-convert: in-chat convert popup and auto-open scheduled workflow card

* [aidan] ux/calendar-page: schedule calendar restyle + popover fixes (#99)

* [aidan] fix/dashboard-delete: remove workflows calendar panel on delete key

* [aidan] ux/workflows-calendar: restyle hub, fix today highlight, add toolbar toggle

* [aidan] ux/schedule-popover: compact density, fix sticky header bleed, add header spacing

* [aidan] ux/schedule-calendar: hollow ring dot for past fires in month view

* [aidan] ux/schedule-calendar: clickable +N more opens day's full run list

* [aidan] feat/run-log-filters: add success and skipped pills to scheduled task history

* [aidan] fix/convert-button: stop drag capture so convert-to-workflow click fires

* [aidan] ux/calendar-card: match border color and radius to chat and workflow cards

* [aidan] fix/minimap: render missed-runs card on the minimap

* [aidan] ux/run-sparkline: simplify tooltip to plain run tally

* [aidan] ux/run-history: collapse expanded run view to one clickable line

* [aidan] ux/calendar-card: match corner radius to browser cards

* [aidan] feat/workflows: launch-time scheduling UX and workflow-card polish (#101)

* [aidan] feat/schedule-list: lazy-load list view via scroll sentinel

* [aidan] feat/missed-runs: launch toast with per-workflow counts and pan-to-card

* [aidan] fix/dashboard-tethers: keep watching line anchored on canvas zoom

* [aidan] feat/scheduled-tasks: review missed runs at launch instead of auto-firing on_missed

* [aidan] refactor/workflow-cards: use radius and status design tokens, polish card chrome

* [aidan] ux/agent-card: keep convert-to-workflow visible during runs with mid-turn toast

* [aidan] ux/mcp-bubble: drop redundant verb label when a workflow label is shown

* [aidan] chore/backend: remove stale explanatory comments

* [aidan] fix/workflows-hub: load workflows on hub mount so calendar fills at launch

* [aidan] feat/workflows: generate title, description, step labels at convert time

* [aidan] fix/tidy-layout: include workflows hub in tidy and fit-to-view

* [aidan] feat/schedule-list: window long list via measured-height virtualizer

* [aidan] ux/workflows-hub: remove time-saved badge from calendar header

* [aidan] fix/types: add missing semantic-type labels and drop stray fade arg

* [aidan] feat/schedule: pin monthly day-of-month and honor repeat-every intervals

* [aidan] feat/schedule: inherit source-session tool surface for scheduled runs

* [aidan] ux/calendar: restack hour-cell events as bars with overflow affordance

* [aidan] feat/calendar: open the run card when clicking a scheduled occurrence

* [aidan] ux/missed-runs: add per-group select-all toggle and rename skip action

* [aidan] feat: new scheduled task design ported

* [aidan] ui: sidebar reorder, repeat controls on schedule card

* [aidan] ui: sidebar, scheduling time

* [aidan] feat/schedule: pin monthly last-day-of-month

* [aidan] feat/steps: per-step enable toggle

* [aidan] feat/workflows: per-workflow color swatch

* [aidan] feat/trash: soft-delete workflows with restore and purge

* [aidan] feat/run-monitor: live run monitor card on the canvas

* [aidan] feat/run-context: attach a run as removable chat context

* [aidan] feat/compose: new-workflow landing page and auto-commit build flow

* [aidan] ui/workflows: dark mode and design-system cohesion

* [aidan] ui/calendar: overflow popover, condensed week view, scroll fix

* [aidan] feat/home: ongoing runs, missed review, and accurate Coming-up counts

* [aidan] fix/run-status: sync ongoing runs and heal stuck/interrupted runs

* [aidan] ux/schedule: last-day-of-month UI, Run-at time typing, interval input

* [aidan] ux/workflows: default window size and toolbar icon

* [aidan] fix/schedule: measure ran_late from start and anchor recurrences to created_at

* [aidan] feat/calendar: render fire times from backend, drop JS recurrence reimpl

* [aidan] chore/dashboard: drop dead configure/missed-run cards, refetch on reconnect

* [aidan] chore/agent-card: remove unreachable convert-to-workflow action

* [aidan] fix/workflows: don't bump updated_at on a no-op draft commit so viewing a workflow doesn't reorder the sidebar

* [aidan] feat/schedule: warn when scheduling a workflow that has no steps

* [aidan] fix/selection-tool: never select the workflows app, and exit the tool on Escape without dropping selections

* [aidan] ux/compose: diversify new-workflow starter prompts across personas

* [aidan] ux/run-monitor: spawn the run card a bit farther right of the workflows app

* [aidan] fix/schedule: harden run recovery and storage writes against crashes

* [aidan] ux/compose: restyle new-workflow starters as a clean pill cluster with rich prompts

* [aidan] fix/workflows: optimistically apply edits so the schedule banner updates instantly

* [aidan] ui/workflows: three-tone surface depth so the window lifts off the canvas in both themes

* [aidan] ui/workflows: close buttons turn red on hover, matching the chat card

* [aidan] test/schedule: cover executor pipeline, storage durability, and recurrence gaps

* [aidan] fix: remove package-lock json

* [aidan] fix/workflows-compose: keep compose view until edit agent replies

* [aidan] feat/workflows: auto-generate workflow + step titles with typewriter animation

* [eric] deps: restore frontend/package-lock.json (PR #105 deletion broke npm ci)

---------

Co-authored-by: Eric <ciregenz@berkeley.edu>
Co-authored-by: cire <134991075+ciregenz@users.noreply.github.com>
2026-06-24 18:58:43 -07:00

936 lines
29 KiB
Python

#
# The Python Imaging Library.
# $Id$
#
# base class for image file handlers
#
# history:
# 1995-09-09 fl Created
# 1996-03-11 fl Fixed load mechanism.
# 1996-04-15 fl Added pcx/xbm decoders.
# 1996-04-30 fl Added encoders.
# 1996-12-14 fl Added load helpers
# 1997-01-11 fl Use encode_to_file where possible
# 1997-08-27 fl Flush output in _save
# 1998-03-05 fl Use memory mapping for some modes
# 1999-02-04 fl Use memory mapping also for "I;16" and "I;16B"
# 1999-05-31 fl Added image parser
# 2000-10-12 fl Set readonly flag on memory-mapped images
# 2002-03-20 fl Use better messages for common decoder errors
# 2003-04-21 fl Fall back on mmap/map_buffer if map is not available
# 2003-10-30 fl Added StubImageFile class
# 2004-02-25 fl Made incremental parser more robust
#
# Copyright (c) 1997-2004 by Secret Labs AB
# Copyright (c) 1995-2004 by Fredrik Lundh
#
# See the README file for information on usage and redistribution.
#
from __future__ import annotations
import abc
import io
import itertools
import logging
import os
import struct
from typing import IO, Any, NamedTuple, cast
from . import ExifTags, Image
from ._util import DeferredError, is_path
TYPE_CHECKING = False
if TYPE_CHECKING:
from ._typing import StrOrBytesPath
logger = logging.getLogger(__name__)
MAXBLOCK = 65536
"""
By default, Pillow processes image data in blocks. This helps to prevent excessive use
of resources. Codecs may disable this behaviour with ``_pulls_fd`` or ``_pushes_fd``.
When reading an image, this is the number of bytes to read at once.
When writing an image, this is the number of bytes to write at once.
If the image width times 4 is greater, then that will be used instead.
Plugins may also set a greater number.
User code may set this to another number.
"""
SAFEBLOCK = 1024 * 1024
LOAD_TRUNCATED_IMAGES = False
"""Whether or not to load truncated image files. User code may change this."""
ERRORS = {
-1: "image buffer overrun error",
-2: "decoding error",
-3: "unknown error",
-8: "bad configuration",
-9: "out of memory error",
}
"""
Dict of known error codes returned from :meth:`.PyDecoder.decode`,
:meth:`.PyEncoder.encode` :meth:`.PyEncoder.encode_to_pyfd` and
:meth:`.PyEncoder.encode_to_file`.
"""
#
# --------------------------------------------------------------------
# Helpers
def _get_oserror(error: int, *, encoder: bool) -> OSError:
try:
msg = Image.core.getcodecstatus(error)
except AttributeError:
msg = ERRORS.get(error)
if not msg:
msg = f"{'encoder' if encoder else 'decoder'} error {error}"
msg += f" when {'writing' if encoder else 'reading'} image file"
return OSError(msg)
def _tilesort(t: _Tile) -> int:
# sort on offset
return t[2]
class _Tile(NamedTuple):
codec_name: str
extents: tuple[int, int, int, int] | None
offset: int = 0
args: tuple[Any, ...] | str | None = None
#
# --------------------------------------------------------------------
# ImageFile base class
class ImageFile(Image.Image):
"""Base class for image file format handlers."""
def __init__(
self, fp: StrOrBytesPath | IO[bytes], filename: str | bytes | None = None
) -> None:
super().__init__()
self._min_frame = 0
self.custom_mimetype: str | None = None
self.tile: list[_Tile] = []
""" A list of tile descriptors """
self.readonly = 1 # until we know better
self.decoderconfig: tuple[Any, ...] = ()
self.decodermaxblock = MAXBLOCK
self.fp: IO[bytes] | None
self._fp: IO[bytes] | DeferredError
if is_path(fp):
# filename
self.fp = open(fp, "rb")
self.filename = os.fspath(fp)
self._exclusive_fp = True
else:
# stream
self.fp = cast(IO[bytes], fp)
self.filename = filename if filename is not None else ""
# can be overridden
self._exclusive_fp = False
try:
try:
self._open()
if isinstance(self, StubImageFile):
if loader := self._load():
loader.open(self)
except (
IndexError, # end of data
TypeError, # end of data (ord)
KeyError, # unsupported mode
EOFError, # got header but not the first frame
struct.error,
) as v:
raise SyntaxError(v) from v
if not self.mode or self.size[0] <= 0 or self.size[1] <= 0:
msg = "not identified by this driver"
raise SyntaxError(msg)
except BaseException:
# close the file only if we have opened it this constructor
if self._exclusive_fp:
self.fp.close()
raise
def _open(self) -> None:
pass
# Context manager support
def __enter__(self) -> ImageFile:
return self
def _close_fp(self) -> None:
if getattr(self, "_fp", False) and not isinstance(self._fp, DeferredError):
if self._fp != self.fp:
self._fp.close()
self._fp = DeferredError(ValueError("Operation on closed image"))
if self.fp:
self.fp.close()
def __exit__(self, *args: object) -> None:
if getattr(self, "_exclusive_fp", False):
self._close_fp()
self.fp = None
def close(self) -> None:
"""
Closes the file pointer, if possible.
This operation will destroy the image core and release its memory.
The image data will be unusable afterward.
This function is required to close images that have multiple frames or
have not had their file read and closed by the
:py:meth:`~PIL.Image.Image.load` method. See :ref:`file-handling` for
more information.
"""
try:
self._close_fp()
self.fp = None
except Exception as msg:
logger.debug("Error closing: %s", msg)
super().close()
def get_child_images(self) -> list[ImageFile]:
child_images = []
exif = self.getexif()
ifds = []
if ExifTags.Base.SubIFDs in exif:
subifd_offsets = exif[ExifTags.Base.SubIFDs]
if subifd_offsets:
if not isinstance(subifd_offsets, tuple):
subifd_offsets = (subifd_offsets,)
ifds = [
(exif._get_ifd_dict(subifd_offset), subifd_offset)
for subifd_offset in subifd_offsets
]
ifd1 = exif.get_ifd(ExifTags.IFD.IFD1)
if ifd1 and ifd1.get(ExifTags.Base.JpegIFOffset):
assert exif._info is not None
ifds.append((ifd1, exif._info.next))
offset = None
for ifd, ifd_offset in ifds:
assert self.fp is not None
current_offset = self.fp.tell()
if offset is None:
offset = current_offset
fp = self.fp
if ifd is not None:
thumbnail_offset = ifd.get(ExifTags.Base.JpegIFOffset)
if thumbnail_offset is not None:
thumbnail_offset += getattr(self, "_exif_offset", 0)
self.fp.seek(thumbnail_offset)
length = ifd.get(ExifTags.Base.JpegIFByteCount)
assert isinstance(length, int)
data = self.fp.read(length)
fp = io.BytesIO(data)
with Image.open(fp) as im:
from . import TiffImagePlugin
if thumbnail_offset is None and isinstance(
im, TiffImagePlugin.TiffImageFile
):
im._frame_pos = [ifd_offset]
im._seek(0)
im.load()
child_images.append(im)
if offset is not None:
assert self.fp is not None
self.fp.seek(offset)
return child_images
def get_format_mimetype(self) -> str | None:
if self.custom_mimetype:
return self.custom_mimetype
if self.format is not None:
return Image.MIME.get(self.format.upper())
return None
def __getstate__(self) -> list[Any]:
return super().__getstate__() + [self.filename]
def __setstate__(self, state: list[Any]) -> None:
self.tile = []
if len(state) > 5:
self.filename = state[5]
super().__setstate__(state)
def verify(self) -> None:
"""Check file integrity"""
# raise exception if something's wrong. must be called
# directly after open, and closes file when finished.
if self._exclusive_fp and self.fp:
self.fp.close()
self.fp = None
def load(self) -> Image.core.PixelAccess | None:
"""Load image data based on tile list"""
if not self.tile and self._im is None:
msg = "cannot load this image"
raise OSError(msg)
pixel = Image.Image.load(self)
if not self.tile:
return pixel
self.map: mmap.mmap | None = None
use_mmap = self.filename and len(self.tile) == 1
assert self.fp is not None
readonly = 0
# look for read/seek overrides
if hasattr(self, "load_read"):
read = self.load_read
# don't use mmap if there are custom read/seek functions
use_mmap = False
else:
read = self.fp.read
if hasattr(self, "load_seek"):
seek = self.load_seek
use_mmap = False
else:
seek = self.fp.seek
if use_mmap:
# try memory mapping
decoder_name, extents, offset, args = self.tile[0]
if isinstance(args, str):
args = (args, 0, 1)
if (
decoder_name == "raw"
and isinstance(args, tuple)
and len(args) >= 3
and args[0] == self.mode
and args[0] in Image._MAPMODES
):
if offset < 0:
msg = "Tile offset cannot be negative"
raise ValueError(msg)
try:
# use mmap, if possible
import mmap
with open(self.filename) as fp:
self.map = mmap.mmap(fp.fileno(), 0, access=mmap.ACCESS_READ)
if offset + self.size[1] * args[1] > self.map.size():
msg = "buffer is not large enough"
raise OSError(msg)
self.im = Image.core.map_buffer(
self.map, self.size, decoder_name, offset, args
)
readonly = 1
# After trashing self.im,
# we might need to reload the palette data.
if self.palette:
self.palette.dirty = 1
except (AttributeError, OSError, ImportError):
self.map = None
self.load_prepare()
err_code = -3 # initialize to unknown error
if not self.map:
# sort tiles in file order
self.tile.sort(key=_tilesort)
# FIXME: This is a hack to handle TIFF's JpegTables tag.
prefix = getattr(self, "tile_prefix", b"")
# Remove consecutive duplicates that only differ by their offset
self.tile = [
list(tiles)[-1]
for _, tiles in itertools.groupby(
self.tile, lambda tile: (tile[0], tile[1], tile[3])
)
]
for i, (decoder_name, extents, offset, args) in enumerate(self.tile):
seek(offset)
decoder = Image._getdecoder(
self.mode, decoder_name, args, self.decoderconfig
)
try:
decoder.setimage(self.im, extents)
if decoder.pulls_fd:
decoder.setfd(self.fp)
err_code = decoder.decode(b"")[1]
else:
b = prefix
while True:
read_bytes = self.decodermaxblock
if i + 1 < len(self.tile):
next_offset = self.tile[i + 1].offset
if next_offset > offset:
read_bytes = next_offset - offset
try:
s = read(read_bytes)
except (IndexError, struct.error) as e:
# truncated png/gif
if LOAD_TRUNCATED_IMAGES:
break
else:
msg = "image file is truncated"
raise OSError(msg) from e
if not s: # truncated jpeg
if LOAD_TRUNCATED_IMAGES:
break
else:
msg = (
"image file is truncated "
f"({len(b)} bytes not processed)"
)
raise OSError(msg)
b = b + s
n, err_code = decoder.decode(b)
if n < 0:
break
b = b[n:]
finally:
# Need to cleanup here to prevent leaks
decoder.cleanup()
self.tile = []
self.readonly = readonly
self.load_end()
if self._exclusive_fp and self._close_exclusive_fp_after_loading:
self.fp.close()
self.fp = None
if not self.map and not LOAD_TRUNCATED_IMAGES and err_code < 0:
# still raised if decoder fails to return anything
raise _get_oserror(err_code, encoder=False)
return Image.Image.load(self)
def load_prepare(self) -> None:
# create image memory if necessary
if self._im is None:
self.im = Image.core.new(self.mode, self.size)
# create palette (optional)
if self.mode == "P":
Image.Image.load(self)
def load_end(self) -> None:
# may be overridden
pass
# may be defined for contained formats
# def load_seek(self, pos: int) -> None:
# pass
# may be defined for blocked formats (e.g. PNG)
# def load_read(self, read_bytes: int) -> bytes:
# pass
def _seek_check(self, frame: int) -> bool:
if (
frame < self._min_frame
# Only check upper limit on frames if additional seek operations
# are not required to do so
or (
not (hasattr(self, "_n_frames") and self._n_frames is None)
and frame >= getattr(self, "n_frames") + self._min_frame
)
):
msg = "attempt to seek outside sequence"
raise EOFError(msg)
return self.tell() != frame
class StubHandler(abc.ABC):
def open(self, im: StubImageFile) -> None:
pass
@abc.abstractmethod
def load(self, im: StubImageFile) -> Image.Image:
pass
class StubImageFile(ImageFile, metaclass=abc.ABCMeta):
"""
Base class for stub image loaders.
A stub loader is an image loader that can identify files of a
certain format, but relies on external code to load the file.
"""
@abc.abstractmethod
def _open(self) -> None:
pass
def load(self) -> Image.core.PixelAccess | None:
loader = self._load()
if loader is None:
msg = f"cannot find loader for this {self.format} file"
raise OSError(msg)
image = loader.load(self)
assert image is not None
# become the other object (!)
self.__class__ = image.__class__ # type: ignore[assignment]
self.__dict__ = image.__dict__
return image.load()
@abc.abstractmethod
def _load(self) -> StubHandler | None:
"""(Hook) Find actual image loader."""
pass
class Parser:
"""
Incremental image parser. This class implements the standard
feed/close consumer interface.
"""
incremental = None
image: Image.Image | None = None
data: bytes | None = None
decoder: Image.core.ImagingDecoder | PyDecoder | None = None
offset = 0
finished = 0
def reset(self) -> None:
"""
(Consumer) Reset the parser. Note that you can only call this
method immediately after you've created a parser; parser
instances cannot be reused.
"""
assert self.data is None, "cannot reuse parsers"
def feed(self, data: bytes) -> None:
"""
(Consumer) Feed data to the parser.
:param data: A string buffer.
:exception OSError: If the parser failed to parse the image file.
"""
# collect data
if self.finished:
return
if self.data is None:
self.data = data
else:
self.data = self.data + data
# parse what we have
if self.decoder:
if self.offset > 0:
# skip header
skip = min(len(self.data), self.offset)
self.data = self.data[skip:]
self.offset = self.offset - skip
if self.offset > 0 or not self.data:
return
n, e = self.decoder.decode(self.data)
if n < 0:
# end of stream
self.data = None
self.finished = 1
if e < 0:
# decoding error
self.image = None
raise _get_oserror(e, encoder=False)
else:
# end of image
return
self.data = self.data[n:]
elif self.image:
# if we end up here with no decoder, this file cannot
# be incrementally parsed. wait until we've gotten all
# available data
pass
else:
# attempt to open this file
try:
with io.BytesIO(self.data) as fp:
im = Image.open(fp)
except OSError:
pass # not enough data
else:
flag = hasattr(im, "load_seek") or hasattr(im, "load_read")
if not flag and len(im.tile) == 1:
# initialize decoder
im.load_prepare()
d, e, o, a = im.tile[0]
im.tile = []
self.decoder = Image._getdecoder(im.mode, d, a, im.decoderconfig)
self.decoder.setimage(im.im, e)
# calculate decoder offset
self.offset = o
if self.offset <= len(self.data):
self.data = self.data[self.offset :]
self.offset = 0
self.image = im
def __enter__(self) -> Parser:
return self
def __exit__(self, *args: object) -> None:
self.close()
def close(self) -> Image.Image:
"""
(Consumer) Close the stream.
:returns: An image object.
:exception OSError: If the parser failed to parse the image file either
because it cannot be identified or cannot be
decoded.
"""
# finish decoding
if self.decoder:
# get rid of what's left in the buffers
self.feed(b"")
self.data = self.decoder = None
if not self.finished:
msg = "image was incomplete"
raise OSError(msg)
if not self.image:
msg = "cannot parse this image"
raise OSError(msg)
if self.data:
# incremental parsing not possible; reopen the file
# not that we have all data
with io.BytesIO(self.data) as fp:
try:
self.image = Image.open(fp)
finally:
self.image.load()
return self.image
# --------------------------------------------------------------------
def _save(im: Image.Image, fp: IO[bytes], tile: list[_Tile], bufsize: int = 0) -> None:
"""Helper to save image based on tile list
:param im: Image object.
:param fp: File object.
:param tile: Tile list.
:param bufsize: Optional buffer size
"""
im.load()
if not hasattr(im, "encoderconfig"):
im.encoderconfig = ()
tile.sort(key=_tilesort)
# FIXME: make MAXBLOCK a configuration parameter
# It would be great if we could have the encoder specify what it needs
# But, it would need at least the image size in most cases. RawEncode is
# a tricky case.
bufsize = max(MAXBLOCK, bufsize, im.size[0] * 4) # see RawEncode.c
try:
fh = fp.fileno()
fp.flush()
_encode_tile(im, fp, tile, bufsize, fh)
except (AttributeError, io.UnsupportedOperation) as exc:
_encode_tile(im, fp, tile, bufsize, None, exc)
if hasattr(fp, "flush"):
fp.flush()
def _encode_tile(
im: Image.Image,
fp: IO[bytes],
tile: list[_Tile],
bufsize: int,
fh: int | None,
exc: BaseException | None = None,
) -> None:
for encoder_name, extents, offset, args in tile:
if offset > 0:
fp.seek(offset)
encoder = Image._getencoder(im.mode, encoder_name, args, im.encoderconfig)
try:
encoder.setimage(im.im, extents)
if encoder.pushes_fd:
encoder.setfd(fp)
errcode = encoder.encode_to_pyfd()[1]
else:
if exc:
# compress to Python file-compatible object
while True:
errcode, data = encoder.encode(bufsize)[1:]
fp.write(data)
if errcode:
break
else:
# slight speedup: compress to real file object
assert fh is not None
errcode = encoder.encode_to_file(fh, bufsize)
if errcode < 0:
raise _get_oserror(errcode, encoder=True) from exc
finally:
encoder.cleanup()
def _safe_read(fp: IO[bytes], size: int) -> bytes:
"""
Reads large blocks in a safe way. Unlike fp.read(n), this function
doesn't trust the user. If the requested size is larger than
SAFEBLOCK, the file is read block by block.
:param fp: File handle. Must implement a <b>read</b> method.
:param size: Number of bytes to read.
:returns: A string containing <i>size</i> bytes of data.
Raises an OSError if the file is truncated and the read cannot be completed
"""
if size <= 0:
return b""
if size <= SAFEBLOCK:
data = fp.read(size)
if len(data) < size:
msg = "Truncated File Read"
raise OSError(msg)
return data
blocks: list[bytes] = []
remaining_size = size
while remaining_size > 0:
block = fp.read(min(remaining_size, SAFEBLOCK))
if not block:
break
blocks.append(block)
remaining_size -= len(block)
if sum(len(block) for block in blocks) < size:
msg = "Truncated File Read"
raise OSError(msg)
return b"".join(blocks)
class PyCodecState:
def __init__(self) -> None:
self.xsize = 0
self.ysize = 0
self.xoff = 0
self.yoff = 0
def extents(self) -> tuple[int, int, int, int]:
return self.xoff, self.yoff, self.xoff + self.xsize, self.yoff + self.ysize
class PyCodec:
fd: IO[bytes] | None
def __init__(self, mode: str, *args: Any) -> None:
self.im: Image.core.ImagingCore | None = None
self.state = PyCodecState()
self.fd = None
self.mode = mode
self.init(args)
def init(self, args: tuple[Any, ...]) -> None:
"""
Override to perform codec specific initialization
:param args: Tuple of arg items from the tile entry
:returns: None
"""
self.args = args
def cleanup(self) -> None:
"""
Override to perform codec specific cleanup
:returns: None
"""
pass
def setfd(self, fd: IO[bytes]) -> None:
"""
Called from ImageFile to set the Python file-like object
:param fd: A Python file-like object
:returns: None
"""
self.fd = fd
def setimage(
self,
im: Image.core.ImagingCore,
extents: tuple[int, int, int, int] | None = None,
) -> None:
"""
Called from ImageFile to set the core output image for the codec
:param im: A core image object
:param extents: a 4 tuple of (x0, y0, x1, y1) defining the rectangle
for this tile
:returns: None
"""
# following c code
self.im = im
if extents:
x0, y0, x1, y1 = extents
if x0 < 0 or y0 < 0 or x1 > self.im.size[0] or y1 > self.im.size[1]:
msg = "Tile cannot extend outside image"
raise ValueError(msg)
self.state.xoff = x0
self.state.yoff = y0
self.state.xsize = x1 - x0
self.state.ysize = y1 - y0
else:
self.state.xsize, self.state.ysize = self.im.size
if self.state.xsize <= 0 or self.state.ysize <= 0:
msg = "Size must be positive"
raise ValueError(msg)
class PyDecoder(PyCodec):
"""
Python implementation of a format decoder. Override this class and
add the decoding logic in the :meth:`decode` method.
See :ref:`Writing Your Own File Codec in Python<file-codecs-py>`
"""
_pulls_fd = False
@property
def pulls_fd(self) -> bool:
return self._pulls_fd
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
"""
Override to perform the decoding process.
:param buffer: A bytes object with the data to be decoded.
:returns: A tuple of ``(bytes consumed, errcode)``.
If finished with decoding return -1 for the bytes consumed.
Err codes are from :data:`.ImageFile.ERRORS`.
"""
msg = "unavailable in base decoder"
raise NotImplementedError(msg)
def set_as_raw(
self, data: bytes, rawmode: str | None = None, extra: tuple[Any, ...] = ()
) -> None:
"""
Convenience method to set the internal image from a stream of raw data
:param data: Bytes to be set
:param rawmode: The rawmode to be used for the decoder.
If not specified, it will default to the mode of the image
:param extra: Extra arguments for the decoder.
:returns: None
"""
if not rawmode:
rawmode = self.mode
d = Image._getdecoder(self.mode, "raw", rawmode, extra)
assert self.im is not None
d.setimage(self.im, self.state.extents())
s = d.decode(data)
if s[0] >= 0:
msg = "not enough image data"
raise ValueError(msg)
if s[1] != 0:
msg = "cannot decode image data"
raise ValueError(msg)
class PyEncoder(PyCodec):
"""
Python implementation of a format encoder. Override this class and
add the decoding logic in the :meth:`encode` method.
See :ref:`Writing Your Own File Codec in Python<file-codecs-py>`
"""
_pushes_fd = False
@property
def pushes_fd(self) -> bool:
return self._pushes_fd
def encode(self, bufsize: int) -> tuple[int, int, bytes]:
"""
Override to perform the encoding process.
:param bufsize: Buffer size.
:returns: A tuple of ``(bytes encoded, errcode, bytes)``.
If finished with encoding return 1 for the error code.
Err codes are from :data:`.ImageFile.ERRORS`.
"""
msg = "unavailable in base encoder"
raise NotImplementedError(msg)
def encode_to_pyfd(self) -> tuple[int, int]:
"""
If ``pushes_fd`` is ``True``, then this method will be used,
and ``encode()`` will only be called once.
:returns: A tuple of ``(bytes consumed, errcode)``.
Err codes are from :data:`.ImageFile.ERRORS`.
"""
if not self.pushes_fd:
return 0, -8 # bad configuration
bytes_consumed, errcode, data = self.encode(0)
if data:
assert self.fd is not None
self.fd.write(data)
return bytes_consumed, errcode
def encode_to_file(self, fh: int, bufsize: int) -> int:
"""
:param fh: File handle.
:param bufsize: Buffer size.
:returns: If finished successfully, return 0.
Otherwise, return an error code. Err codes are from
:data:`.ImageFile.ERRORS`.
"""
errcode = 0
while errcode == 0:
status, errcode, buf = self.encode(bufsize)
if status > 0:
os.write(fh, buf[status:])
return errcode