Loading Bitmaps / Images from host to Medley?

[RyanBurnside]

I was just about to write a PBM (P1 or P4 loader) today and then recalled that there was mention of some existing image loaders (or had been) to go from host to Medley.

I thought I'd ask here so hopefully people can see answers should they search for bitmaps images etc.

I suppose I'll ask a few questions since I have them.

0. Is there a current approach to load ANY standard type of raster from the host? (Hopefully something I can export from GIMP or ImageMagik)

1. There is some ambiguity in my mind between textures, bitmaps, and image streams.
   It seems to me that textures are used for repeating "fills" in polygons and brushes.
   Bitmaps seem to be stand alone images stored in (some opaque internal format?) in memory.
   Image Streams are dynamic things that provide a stateful environment around writing to an internal bitmap. (Let me know if I've misunderstood please)

2. If Medley images may be written to (Linux, Mac, Windows) host, is there a specification for Medley type Image files?
   Perhaps it might be worth me adding an export plugin to GIMP if so...

3. If I use something like Common Lisp's READ-LINE is there a char count I should be concerned into bumping into? (Roughly)

[pamoroso]

@phantomics wrote some tools for importing and exporting PBM images into and out of Medley but as far as I know the code is not public.

There is a useful discussion here on the file format used by the Interlisp functions PRINTBITMAP and READBITMAP.

[nbriggs]

There is some ambiguity in my mind between textures, bitmaps, and image streams.
It seems to me that textures are used for repeating "fills" in polygons and brushes.
Bitmaps seem to be stand alone images stored in (some opaque internal format?) in memory.
Image Streams are dynamic things that provide a stateful environment around writing to an internal bitmap. (Let me know if I've misunderstood please)

You're mostly correct:

IRM 27.1.4 Textures
A Texture denotes a pattern of gray which can be used to
[conceptually] tessellate the plane to form an infinite sheet of gray.
It is currently either a 4 by 4 pattern or a 16 by N (N <= 16) pattern.

The 4 by 4 pattern is represented as a small integer.
The easiest way into the documentation section is "(TEXTUREP ?"

Bitmaps are stored internally as a sequence of 16-bit words, with a header that
describes the width, height, number of bits per pixel, and the stride (in 16-bit words).
You can look at the source for PRINTBITMAP and \WRITEBITMAP
(in sources/ADISPLAY) to see what's going on underneath.

Image Streams are regular (output) streams which also have associated with them a
collection of methods that perform the various DSPxxx functions, and a block of storage
which is private to the particular methods associated with the stream type.
A bitmap is one possible target, Postscript and Interpress are others (and HTML when
Herb is done with that). Create a stream with OPENIMAGESTREAM and INSPECT it, look
at the IMAGEOPS field to see the methods required.

You might look at the READAIS and AISBLT packages in lispusers. Array of Intensity Samples was a format
used on the Alto - documentation of the file format (and lots more) is at https://xeroxparcarchive.computerhistory.org/_cd8_/altodocs/.AISMANUAL.PRESS!2.pdf

[RyanBurnside]

Thanks for the pointers, I'll take a peek!
Maybe I can get host image loading working by the time the Game Jam kicks off in a few days.

It's not a show stopper though. ;)
I think NetPBM "P1" is probably the lowest (and slowest) hanging fruit to try if the native format is squirrelly - I'll take a look though.

[RyanBurnside]

@nbriggs I yanked mine after seeing a bug or three to quickly fix in the P1.
Looks like you posted something as well and I wonder if mine took yours out too?
Didn't mean to if that's the case. :)

Anyway this is the P1 loader (I misstated earlier as a P4).
I'd not gotten to go from the bit array to an actual image yet. That was the second pass.

EDIT (for a few typos and mislabels)

[RyanBurnside]

Here is the code. Remember it only provides and associative list of attributes and the bits for a binary image via a BIT array at the end...

;;;; PBMP1READ.LISP
;;;; Ryan Burnside 2025
;;;; code is provide as-is

(defparameter *whitespace-bag* (list #\Space #\Newline #\Backspace #\Tab
				     #\Linefeed #\Page #\Return #\Rubout))

(defun bisect (object seq)
  "Splits an sequence once upon the first encounter of object, dropping object."
  (let ((n (position object seq)))
    (when n
      (list (subseq seq 0 n)
	    (subseq seq (1+ n))))))

(defun whitespace-trim (string)
  "Remove Space, Newline, Backspace, Tab, Linefeed, Page, Return, Rubout"
  (string-trim *whitespace-bag* string))

(defun read-header (stream)
  "Read the header, upcase it or return nil if invalid."
  (let ((header (string-upcase (whitespace-trim (read-line stream)))))
    (when (string= header "P1")
      header)))

(defun read-comments-and-size (stream)
  "Returns a list of (comment ... comment-n  (width height)) nil if invalid."
  (loop with comments = nil
	with sizes = nil
	;; Collect comments with #
	for l = (whitespace-trim (read-line stream))
	do (if (char= (char l 0) #\#)
	       (push l comments)
	     (progn
	       (setf sizes (mapcar #'parse-integer (bisect #\Space l)))
	       (if (and (every #'integerp sizes) (= 2 (length sizes)))
		   (return (list (reverse comments) sizes))
		 (return nil))))))

(defun read-body (stream num-pixels)
  "Hooo boy, this is going to be slow trash... *ahem* returns bit array or empty array if 0 size."
  (loop with buffer = (make-array num-pixels :element-type 'bit
     :initial-element 0)
	with cur-elt = 0
	for ch = (read-char stream nil nil)
	while ch
	do (case ch
		 ((#\0) (incf cur-elt))
		 ((#\1) (setf (aref buffer cur-elt) 1)
		  (incf cur-elt)))
	finally (return buffer)))

;;; Driver
(defun read-pbm-p1 (file)
  "Returns a list containing (comments width height num-pixels and bit array for pixels."
  (with-open-file (s file)
    (let* ((header (read-header s))
	   (coms-and-size (read-comments-and-size s))
	   (comments (first coms-and-size))
	   (width (first (second coms-and-size)))
	   (height (second (second coms-and-size)))
	   (pixel-count (* width height))
	   (body-bit-array (read-body s pixel-count)))
      (unless (and header width height)
	(error "Problem with file found"))
      (print (* width height))
      (return-from read-pbm-p1 `((comments ,comments)
				 (width . ,width)
				 (height . ,height)
				 (pixel-count . ,pixel-count)
				 (body-bit-array . ,body-bit-array))))))

[nbriggs]

No, I sent mine to only you in e-mail to avoid posting a spoiler if you saw it through the webpage.
Mine is buggy around reading the comments.

[RyanBurnside]

Ah, ok. I thought the email notification was pointing to this thread.
I've got a fairly full set of tabs open at the moment!

I'll finish mine and try yours.
These will be nice to have. PBM isn't an excellent format but for now it's a nice way to hoist simple images in.

[nbriggs]

Testing load speed on an 800x800 PBM "P4" bitmap on an old slow MacBook Air (1.4 GHz Dual-Core Intel Core i5):

[RyanBurnside]

I realised that I had the P1 version implemented.
The P4 is much better we should go with that if the comment thing can get patched later.

[nbriggs]

If you want -- it's in your e-mail, I can post it here too. It works for files that don't have extra whitespace (but it's OK with/without comments). Should be an easy fix for the whitespace.

[RyanBurnside]

Feel free, you've earned it. :)
It'll be nice to have for the Game Jam.

[RyanBurnside]

Guess I have the code :D

(LAMBDA (FILE)
                                                          (* ; "Edited 28-Oct-2025 16:38 by briggs")
       (CL:WITH-OPEN-FILE (S FILE :DIRECTION :INPUT)
              (SELECTQ (RATOM S READ-LINE-RDTBL)
                     (P4 (LET (LINE WIDTH HEIGHT)
                              (READC S READ-LINE-RDTBL)
                                                             (* ; "eat the eol")
                              (while (EQ 35 (CHCON1 (SETQ LINE (RSTRING S READ-LINE-RDTBL))))
                                     do
                                     (READC S READ-LINE-RDTBL)
                                                             (* ; " eat the eol")
                                     (SETQ LINE (RSTRING S READ-LINE-RDTBL)))
                              (SETQ WIDTH (RATOM S))
                              (SETQ HEIGHT (RATOM S))
                              (SETQ BM (BITMAPCREATE WIDTH HEIGHT))
                              (to HEIGHT bind (RBASE ← (fetch (BITMAP BITMAPBASE)
                                                              of BM))
                                  (RWIDTH ← (fetch (BITMAP BITMAPRASTERWIDTH)
                                                   of BM))
                                  (BYTEWIDTH ← (SUB1 (IQUOTIENT (IPLUS WIDTH 7)
                                                            8)))
                                  do
                                  (for OFFSET from 0 to BYTEWIDTH do (\PUTBASEBYTE RBASE OFFSET
                                                                            (BIN S)))
                                  (SETQ RBASE (\ADDBASE RBASE RWIDTH)))
                              BM))
                     (HELP "Not a PBM format we can read"))))

[nbriggs]

I fixed it a little -

(LAMBDA (FILE)
                                                          (* ; "Edited 28-Oct-2025 17:30 by briggs")
       (CL:WITH-OPEN-FILE (S FILE :DIRECTION :INPUT)
              (SELECTQ (RATOM S READ-LINE-RDTBL)
                     (P4 (LET (LINE WIDTH HEIGHT BM)
                              (READCCODE S READ-LINE-RDTBL)
                              (while (EQ 35 (PEEKCCODE S))
                                     do
                                     (RSTRING S READ-LINE-RDTBL)
                                     (READCCODE S READ-LINE-RDTBL))
                              (SETQ WIDTH (RATOM S))
                                                             (* ; "regular RDTBL ")
                              (SETQ HEIGHT (RATOM S))
                                                             (* ; "regular RDTBL")
                              (READCCODE S READ-LINE-RDTBL)
                                                             (* ; "another eol ")
                              (SETQ BM (BITMAPCREATE WIDTH HEIGHT))
                              (to HEIGHT bind (RBASE ← (fetch (BITMAP BITMAPBASE)
                                                              of BM))
                                  (RWIDTH ← (fetch (BITMAP BITMAPRASTERWIDTH)
                                                   of BM))
                                  (BYTEWIDTH ← (SUB1 (IQUOTIENT (IPLUS WIDTH 7)
                                                            8)))
                                  do
                                  (for OFFSET from 0 to BYTEWIDTH do (\PUTBASEBYTE RBASE OFFSET
                                                                            (BIN S)))
                                  (SETQ RBASE (\ADDBASE RBASE RWIDTH)))
                              BM))
                     (HELP "Not a PBM format we can read"))))

[RyanBurnside]

We had some discussion about pasting code directly into SEdit today.
Worth a mention to folks that one shouldn't paste commented IL lines with ; directly into SEdit as it'll confuse it and chop off the closing parens. I'm just posting this here so other folks can try the code.

[nbriggs]

It works correctly if you shift-select within SEdit (or between two SEdits), but the text it sends to a foreign destination is the Interlisp form where the ; comment is converted to (* ; "comment" ). Taking a quick look at the implementation -- It's handled in sedit::copy-selection-comment (on SEDIT-COMMENTS)

[RyanBurnside]

Working on my P1 loader but it turned into an Australian P1 loader.
(Medley uses lower left origin while most graphics stuff tends to use upper left).
But I thought it was funny.

[RyanBurnside]

Here is the P1 loader. Some polish to do yet but now we have P1 and P4 (via @nbriggs ) support.
Today was an excellent hacking day!

;;;; PBMP1READ.LISP
;;;; Ryan Burnside 2025
;;;; code is provided as-is

(defparameter *whitespace-bag* (list #\Space #\Newline #\Backspace #\Tab
				     #\Linefeed #\Page #\Return #\Rubout))

(defun bisect (object seq)
  "Splits an sequence once upon the first encounter of object, dropping object."
  (let ((n (position object seq)))
    (when n
      (list (subseq seq 0 n)
	    (subseq seq (1+ n))))))

(defun whitespace-trim (string)
  "Remove Space, Newline, Backspace, Tab, Linefeed, Page, Return, Rubout"
  (string-trim *whitespace-bag* string))

(defun read-header (stream)
  "Read the header, upcase it or return nil if invalid."
  (let ((header (string-upcase (whitespace-trim (read-line stream)))))
    (when (string= header "P1")
      header)))

(defun read-comments-and-size (stream)
  "Returns a list of ((comment ... comment-n) (width height)) nil if invalid."
  (loop with comments = nil
	with sizes = nil
	;; Collect comments with #
	for l = (whitespace-trim (read-line stream))
	do (if (char= (char l 0) #\#)
	       (push l comments)
	     (progn
	       (setf sizes (mapcar #'parse-integer (bisect #\Space l)))
	       (if (and (every #'integerp sizes) (= 2 (length sizes)))
		   (return (list (reverse comments) sizes))
		 (return nil))))))

(defun build-bitmap (stream width height)
  "Return a binary bitmap given PBM characters in stream"
  (let ((BM (IL:BITMAPCREATE width height)))
    (loop with x = 0
	  with y = (1- height)
	  for ch = (read-char stream nil nil)
	  while ch
	  do (case ch
		   ((#\0) (incf x)) 
		   ((#\1) (IL:BITMAPBIT bm x y 1) (incf x)))
	  (when (>= x width)
	    (setf x 0)
	    (decf y))
	  finally (return BM))))
 
;;; Driver
(defun read-pbm-p1 (file)
  "Returns a binary bitmap from a PBM P1 ASCII file"
  (with-open-file (s file)
    (let* ((header (read-header s))
	   (coms-and-size (read-comments-and-size s))
	   (comments (first coms-and-size)) ; not used for now
	   (width (first (second coms-and-size)))
	   (height (second (second coms-and-size))))
      (unless (and header width height)
	(error "Problem with file found"))
      (build-bitmap s width height))))

[nbriggs]

I believe P1 and P4 have the same header format, so if you allowed either P1 or P4 in the read-header code, you could trivially select the correct reader for the rest of the file, say, build-bitmap-p1 or build-buitmap-p4 where that would use the hack I used to read byte-at-a-time straight into the bitmap storage.

[RyanBurnside]

Yep, I think dispatching on the header is the correct approach.
Now that we've got both parts (and I want to rework mine a bit) it's probably The Right Thing.
It's nice that two people can hack comfortably in two dialects and still dispatch in the same environment.

[nbriggs]

I'd suggest you rewrite mine in Common Lisp, in good style, just using the knowledge of the internal structure of the bitmap memory it incorporates to IL:\\PUTBASEBYTE the data in the right place. Mine was just a one-of hack to show how to get the bytes in efficiently, not a piece of code i'm proud of!

[RyanBurnside]

OK, I can probably do that. I'll get sucked into the Autumn Lisp Game Jam here shortly however.