�ɲɾ�����ӯ�����һ��ˣ��������С���˴��ͣ�������P���ҹ��ñ˽��ά�Բ��������˸߸ԣ�������ơ��ҹ��ñ�����ά�Բ���ˡ���˳^�ӣ������ӡ� ���ͯj�ӣ��ƺ���ӣ� ? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!PK/F2]00rnrs.scmnu[;;; rnrs.scm --- The R6RS composite library ;; Copyright (C) 2010, 2011 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs (6)) (export ;; (rnrs arithmetic bitwise) bitwise-not bitwise-and bitwise-ior bitwise-xor bitwise-if bitwise-bit-count bitwise-length bitwise-first-bit-set bitwise-bit-set? bitwise-copy-bit bitwise-bit-field bitwise-copy-bit-field bitwise-arithmetic-shift bitwise-arithmetic-shift-left bitwise-arithmetic-shift-right bitwise-rotate-bit-field bitwise-reverse-bit-field ;; (rnrs arithmetic fixnums) fixnum? fixnum-width least-fixnum greatest-fixnum fx=? fx>? fx=? fx<=? fxzero? fxpositive? fxnegative? fxodd? fxeven? fxmax fxmin fx+ fx* fx- fxdiv-and-mod fxdiv fxmod fxdiv0-and-mod0 fxdiv0 fxmod0 fx+/carry fx-/carry fx*/carry fxnot fxand fxior fxxor fxif fxbit-count fxlength fxfirst-bit-set fxbit-set? fxcopy-bit fxbit-field fxcopy-bit-field fxarithmetic-shift fxarithmetic-shift-left fxarithmetic-shift-right fxrotate-bit-field fxreverse-bit-field ;; (rnrs arithmetic flonums) flonum? real->flonum fl=? fl? fl>=? flinteger? flzero? flpositive? flnegative? flodd? fleven? flfinite? flinfinite? flnan? flmax flmin fl+ fl* fl- fl/ flabs fldiv-and-mod fldiv flmod fldiv0-and-mod0 fldiv0 flmod0 flnumerator fldenominator flfloor flceiling fltruncate flround flexp fllog flsin flcos fltan flacos flasin flatan flsqrt flexpt &no-infinities make-no-infinities-violation no-infinities-violation? &no-nans make-no-nans-violation no-nans-violation? fixnum->flonum ;; (rnrs base) boolean? symbol? char? vector? null? pair? number? string? procedure? define define-syntax syntax-rules lambda let let* let-values let*-values letrec letrec* begin quote lambda if set! cond case or and not eqv? equal? eq? + - * / max min abs numerator denominator gcd lcm floor ceiling truncate round rationalize real-part imag-part make-rectangular angle div mod div-and-mod div0 mod0 div0-and-mod0 expt exact-integer-sqrt sqrt exp log sin cos tan asin acos atan make-polar magnitude angle complex? real? rational? integer? exact? inexact? real-valued? rational-valued? integer-valued? zero? positive? negative? odd? even? nan? finite? infinite? exact inexact = < > <= >= number->string string->number boolean=? cons car cdr caar cadr cdar cddr caaar caadr cadar cdaar caddr cdadr cddar cdddr caaaar caaadr caadar cadaar cdaaar cddaar cdadar cdaadr cadadr caaddr caddar cadddr cdaddr cddadr cdddar cddddr list? list length append reverse list-tail list-ref map for-each symbol->string string->symbol symbol=? char->integer integer->char char=? char? char<=? char>=? make-string string string-length string-ref string=? string? string<=? string>=? substring string-append string->list list->string string-for-each string-copy vector? make-vector vector vector-length vector-ref vector-set! vector->list list->vector vector-fill! vector-map vector-for-each error assertion-violation assert call-with-current-continuation call/cc call-with-values dynamic-wind values apply quasiquote unquote unquote-splicing let-syntax letrec-syntax syntax-rules identifier-syntax ;; (rnrs bytevectors) endianness native-endianness bytevector? make-bytevector bytevector-length bytevector=? bytevector-fill! bytevector-copy! bytevector-copy uniform-array->bytevector bytevector-u8-ref bytevector-s8-ref bytevector-u8-set! bytevector-s8-set! bytevector->u8-list u8-list->bytevector bytevector-uint-ref bytevector-uint-set! bytevector-sint-ref bytevector-sint-set! bytevector->sint-list bytevector->uint-list uint-list->bytevector sint-list->bytevector bytevector-u16-ref bytevector-s16-ref bytevector-u16-set! bytevector-s16-set! bytevector-u16-native-ref bytevector-s16-native-ref bytevector-u16-native-set! bytevector-s16-native-set! bytevector-u32-ref bytevector-s32-ref bytevector-u32-set! bytevector-s32-set! bytevector-u32-native-ref bytevector-s32-native-ref bytevector-u32-native-set! bytevector-s32-native-set! bytevector-u64-ref bytevector-s64-ref bytevector-u64-set! bytevector-s64-set! bytevector-u64-native-ref bytevector-s64-native-ref bytevector-u64-native-set! bytevector-s64-native-set! bytevector-ieee-single-ref bytevector-ieee-single-set! bytevector-ieee-single-native-ref bytevector-ieee-single-native-set! bytevector-ieee-double-ref bytevector-ieee-double-set! bytevector-ieee-double-native-ref bytevector-ieee-double-native-set! string->utf8 string->utf16 string->utf32 utf8->string utf16->string utf32->string ;; (rnrs conditions) &condition condition simple-conditions condition? condition-predicate condition-accessor define-condition-type &message make-message-condition message-condition? condition-message &warning make-warning warning? &serious make-serious-condition serious-condition? &error make-error error? &violation make-violation violation? &assertion make-assertion-violation assertion-violation? &irritants make-irritants-condition irritants-condition? condition-irritants &who make-who-condition who-condition? condition-who &non-continuable make-non-continuable-violation non-continuable-violation? &implementation-restriction make-implementation-restriction-violation implementation-restriction-violation? &lexical make-lexical-violation lexical-violation? &syntax make-syntax-violation syntax-violation? syntax-violation-form syntax-violation-subform &undefined make-undefined-violation undefined-violation? ;; (rnrs control) when unless do case-lambda ;; (rnrs enums) make-enumeration enum-set-universe enum-set-indexer enum-set-constructor enum-set->list enum-set-member? enum-set-subset? enum-set=? enum-set-union enum-set-intersection enum-set-difference enum-set-complement enum-set-projection define-enumeration ;; (rnrs exceptions) guard with-exception-handler raise raise-continuable ;; (rnrs files) file-exists? delete-file &i/o make-i/o-error i/o-error? &i/o-read make-i/o-read-error i/o-read-error? &i/o-write make-i/o-write-error i/o-write-error? &i/o-invalid-position make-i/o-invalid-position-error i/o-invalid-position-error? i/o-error-position &i/o-filename make-i/o-filename-error i/o-filename-error? i/o-error-filename &i/o-file-protection make-i/o-file-protection-error i/o-file-protection-error? &i/o-file-is-read-only make-i/o-file-is-read-only-error i/o-file-is-read-only-error? &i/o-file-already-exists make-i/o-file-already-exists-error i/o-file-already-exists-error? &i/o-file-does-not-exist make-i/o-file-does-not-exist-error i/o-file-does-not-exist-error? &i/o-port make-i/o-port-error i/o-port-error? i/o-error-port ;; (rnrs hashtables) make-eq-hashtable make-eqv-hashtable make-hashtable hashtable? hashtable-size hashtable-ref hashtable-set! hashtable-delete! hashtable-contains? hashtable-update! hashtable-copy hashtable-clear! hashtable-keys hashtable-entries hashtable-equivalence-function hashtable-hash-function hashtable-mutable? equal-hash string-hash string-ci-hash symbol-hash ;; (rnrs io ports) file-options buffer-mode buffer-mode? eol-style native-eol-style error-handling-mode make-transcoder transcoder-codec transcoder-eol-style transcoder-error-handling-mode native-transcoder latin-1-codec utf-8-codec utf-16-codec eof-object? port? input-port? output-port? eof-object port-eof? port-transcoder binary-port? textual-port? transcoded-port port-position set-port-position! port-has-port-position? port-has-set-port-position!? close-port call-with-port open-bytevector-input-port make-custom-binary-input-port get-u8 lookahead-u8 get-bytevector-n get-bytevector-n! get-bytevector-some get-bytevector-all open-bytevector-output-port make-custom-binary-output-port put-u8 put-bytevector open-string-input-port open-string-output-port call-with-bytevector-output-port call-with-string-output-port latin-1-codec utf-8-codec utf-16-codec open-file-input-port open-file-output-port open-file-input/output-port make-custom-textual-output-port call-with-string-output-port flush-output-port put-string get-char get-datum get-line get-string-all get-string-n get-string-n! lookahead-char put-char put-datum put-string standard-input-port standard-output-port standard-error-port ;; (rnrs io simple) call-with-input-file call-with-output-file current-input-port current-output-port current-error-port with-input-from-file with-output-to-file open-input-file open-output-file close-input-port close-output-port read-char peek-char read write-char newline display write ;; (rnrs lists) find for-all exists filter partition fold-left fold-right remp remove remv remq memp member memv memq assp assoc assv assq cons* ;; (rnrs programs) command-line exit ;; (rnrs records inspection) record? record-rtd record-type-name record-type-parent record-type-uid record-type-generative? record-type-sealed? record-type-opaque? record-type-field-names record-field-mutable? ;; (rnrs records procedural) make-record-type-descriptor record-type-descriptor? make-record-constructor-descriptor record-constructor record-predicate record-accessor record-mutator ;; (rnrs records syntactic) define-record-type record-type-descriptor record-constructor-descriptor ;; (rnrs sorting) list-sort vector-sort vector-sort! ;; (rnrs syntax-case) make-variable-transformer syntax ;; Until the deprecated support for a unified modules and ;; bindings namespace is removed, we need to manually resolve ;; a conflict between two bindings: that of the (rnrs ;; syntax-case) module, and the imported `syntax-case' ;; binding. We do so here and below by renaming the macro ;; import. (rename (syntax-case-hack syntax-case)) identifier? bound-identifier=? free-identifier=? syntax->datum datum->syntax generate-temporaries with-syntax quasisyntax unsyntax unsyntax-splicing syntax-violation ;; (rnrs unicode) char-upcase char-downcase char-titlecase char-foldcase char-ci=? char-ci? char-ci<=? char-ci>=? char-alphabetic? char-numeric? char-whitespace? char-upper-case? char-lower-case? char-title-case? char-general-category string-upcase string-downcase string-titlecase string-foldcase string-ci=? string-ci? string-ci<=? string-ci>=? string-normalize-nfd string-normalize-nfkd string-normalize-nfc string-normalize-nfkc) (import (rnrs arithmetic bitwise (6)) (rnrs arithmetic fixnums (6)) (rnrs arithmetic flonums (6)) (rnrs base (6)) (rnrs bytevectors (6)) (rnrs conditions (6)) (rnrs control (6)) (rnrs enums (6)) (rnrs exceptions (6)) (rnrs files (6)) (rnrs hashtables (6)) (rnrs io ports (6)) (rnrs io simple (6)) (rnrs lists (6)) (rnrs programs (6)) (rnrs records inspection (6)) (rnrs records procedural (6)) (rnrs records syntactic (6)) (rnrs sorting (6)) ;; See note above on exporting syntax-case. (rename (rnrs syntax-case (6)) (syntax-case syntax-case-hack)) (rnrs unicode (6)))) PK/F2]F=P;P;web/client.scmnu[;;; Web client ;; Copyright (C) 2011, 2012, 2013, 2014 Free Software Foundation, Inc. ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ;; 02110-1301 USA ;;; Commentary: ;;; ;;; (web client) is a simple HTTP URL fetcher for Guile. ;;; ;;; In its current incarnation, (web client) is synchronous. If you ;;; want to fetch a number of URLs at once, probably the best thing to ;;; do is to write an event-driven URL fetcher, similar in structure to ;;; the web server. ;;; ;;; Another option, good but not as performant, would be to use threads, ;;; possibly via a thread pool. ;;; ;;; Code: (define-module (web client) #:use-module (rnrs bytevectors) #:use-module (ice-9 binary-ports) #:use-module (ice-9 iconv) #:use-module (ice-9 rdelim) #:use-module (web request) #:use-module (web response) #:use-module (web uri) #:use-module (web http) #:use-module (srfi srfi-1) #:use-module (srfi srfi-9) #:use-module (srfi srfi-9 gnu) #:export (current-http-proxy open-socket-for-uri http-get http-get* http-head http-post http-put http-delete http-trace http-options)) (define current-http-proxy (make-parameter (let ((proxy (getenv "http_proxy"))) (and (not (equal? proxy "")) proxy)))) (define (ensure-uri uri-or-string) (cond ((string? uri-or-string) (string->uri uri-or-string)) ((uri? uri-or-string) uri-or-string) (else (error "Invalid URI" uri-or-string)))) (define (open-socket-for-uri uri-or-string) "Return an open input/output port for a connection to URI." (define http-proxy (current-http-proxy)) (define uri (ensure-uri (or http-proxy uri-or-string))) (define addresses (let ((port (uri-port uri))) (delete-duplicates (getaddrinfo (uri-host uri) (cond (port => number->string) (else (symbol->string (uri-scheme uri)))) (if port AI_NUMERICSERV 0)) (lambda (ai1 ai2) (equal? (addrinfo:addr ai1) (addrinfo:addr ai2)))))) (let loop ((addresses addresses)) (let* ((ai (car addresses)) (s (with-fluids ((%default-port-encoding #f)) ;; Restrict ourselves to TCP. (socket (addrinfo:fam ai) SOCK_STREAM IPPROTO_IP)))) (catch 'system-error (lambda () (connect s (addrinfo:addr ai)) ;; Buffer input and output on this port. (setvbuf s _IOFBF) ;; If we're using a proxy, make a note of that. (when http-proxy (set-http-proxy-port?! s #t)) s) (lambda args ;; Connection failed, so try one of the other addresses. (close s) (if (null? (cdr addresses)) (apply throw args) (loop (cdr addresses)))))))) (define (extend-request r k v . additional) (let ((r (set-field r (request-headers) (assoc-set! (copy-tree (request-headers r)) k v)))) (if (null? additional) r (apply extend-request r additional)))) ;; -> request body (define (sanitize-request request body) "\"Sanitize\" the given request and body, ensuring that they are complete and coherent. This method is most useful for methods that send data to the server, like POST, but can be used for any method. Return two values: a request and a bytevector, possibly the same ones that were passed as arguments. If BODY is a string, encodes the string to a bytevector, in an encoding appropriate for REQUEST. Adds a ‘content-length’ and ‘content-type’ header, as necessary. If BODY is a procedure, it is called with a port as an argument, and the output collected as a bytevector. In the future we might try to instead use a compressing, chunk-encoded port, and call this procedure later. Authors are advised not to rely on the procedure being called at any particular time. Note that we rely on the request itself already having been validated, as is the case by default with a request returned by `build-request'." (cond ((not body) (let ((length (request-content-length request))) (if length ;; FIXME make this stricter: content-length header should be ;; prohibited if there's no body, even if the content-length ;; is 0. (unless (zero? length) (error "content-length, but no body")) (when (assq 'transfer-encoding (request-headers request)) (error "transfer-encoding not allowed with no body"))) (values request #vu8()))) ((string? body) (let* ((type (request-content-type request '(text/plain))) (declared-charset (assq-ref (cdr type) 'charset)) (charset (or declared-charset "utf-8"))) (sanitize-request (if declared-charset request (extend-request request 'content-type `(,@type (charset . ,charset)))) (string->bytevector body charset)))) ((procedure? body) (let* ((type (request-content-type request '(text/plain))) (declared-charset (assq-ref (cdr type) 'charset)) (charset (or declared-charset "utf-8"))) (sanitize-request (if declared-charset request (extend-request request 'content-type `(,@type (charset . ,charset)))) (call-with-encoded-output-string charset body)))) ((not (bytevector? body)) (error "unexpected body type")) (else (values (let ((rlen (request-content-length request)) (blen (bytevector-length body))) (cond (rlen (if (= rlen blen) request (error "bad content-length" rlen blen))) (else (extend-request request 'content-length blen)))) body)))) (define (decode-response-body response body) ;; `body' is either #f or a bytevector. (cond ((not body) body) ((bytevector? body) (let ((rlen (response-content-length response)) (blen (bytevector-length body))) (cond ((and rlen (not (= rlen blen))) (error "bad content-length" rlen blen)) ((response-content-type response) => (lambda (type) (cond ((text-content-type? (car type)) ;; RFC 2616 3.7.1: "When no explicit charset parameter is ;; provided by the sender, media subtypes of the "text" ;; type are defined to have a default charset value of ;; "ISO-8859-1" when received via HTTP." (bytevector->string body (or (assq-ref (cdr type) 'charset) "iso-8859-1"))) (else body)))) (else body)))) (else (error "unexpected body type" body)))) ;; We could expose this to user code if there is demand. (define* (request uri #:key (body #f) (port (open-socket-for-uri uri)) (method 'GET) (version '(1 . 1)) (keep-alive? #f) (headers '()) (decode-body? #t) (streaming? #f) (request (build-request (ensure-uri uri) #:method method #:version version #:headers (if keep-alive? headers (cons '(connection close) headers)) #:port port))) (call-with-values (lambda () (sanitize-request request body)) (lambda (request body) (let ((request (write-request request port))) (when body (write-request-body request body)) (force-output (request-port request)) (let ((response (read-response port))) (cond ((eq? (request-method request) 'HEAD) (unless keep-alive? (close-port port)) (values response #f)) (streaming? (values response (response-body-port response #:keep-alive? keep-alive? #:decode? decode-body?))) (else (let ((body (read-response-body response))) (unless keep-alive? (close-port port)) (values response (if decode-body? (decode-response-body response body) body)))))))))) (define* (http-get uri #:key (body #f) (port (open-socket-for-uri uri)) (version '(1 . 1)) (keep-alive? #f) ;; #:headers is the new name of #:extra-headers. (extra-headers #f) (headers (or extra-headers '())) (decode-body? #t) (streaming? #f)) "Connect to the server corresponding to URI and ask for the resource, using the ‘GET’ method. If you already have a port open, pass it as PORT. The port will be closed at the end of the request unless KEEP-ALIVE? is true. Any extra headers in the alist HEADERS will be added to the request. If BODY is not ‘#f’, a message body will also be sent with the HTTP request. If BODY is a string, it is encoded according to the content-type in HEADERS, defaulting to UTF-8. Otherwise BODY should be a bytevector, or ‘#f’ for no body. Although it's allowed to send a message body along with any request, usually only POST and PUT requests have bodies. See ‘http-put’ and ‘http-post’ documentation, for more. If DECODE-BODY? is true, as is the default, the body of the response will be decoded to string, if it is a textual content-type. Otherwise it will be returned as a bytevector. However, if STREAMING? is true, instead of eagerly reading the response body from the server, this function only reads off the headers. The response body will be returned as a port on which the data may be read. Unless KEEP-ALIVE? is true, the port will be closed after the full response body has been read. Returns two values: the response read from the server, and the response body as a string, bytevector, #f value, or as a port (if STREAMING? is true)." (when extra-headers (issue-deprecation-warning "The #:extra-headers argument to http-get has been renamed to #:headers. " "Please update your code.")) (request uri #:method 'GET #:body body #:port port #:version version #:keep-alive? keep-alive? #:headers headers #:decode-body? decode-body? #:streaming? streaming?)) (define* (http-get* uri #:key (body #f) (port (open-socket-for-uri uri)) (version '(1 . 1)) (keep-alive? #f) ;; #:headers is the new name of #:extra-headers. (extra-headers #f) (headers (or extra-headers '())) (decode-body? #t)) "Deprecated in favor of (http-get #:streaming? #t)." (issue-deprecation-warning "`http-get*' has been deprecated. " "Instead, use `http-get' with the #:streaming? #t keyword argument.") (http-get uri #:body body #:port port #:version version #:keep-alive? keep-alive? #:headers headers #:decode-body? #t #:streaming? #t)) (define-syntax-rule (define-http-verb http-verb method doc) (define* (http-verb uri #:key (body #f) (port (open-socket-for-uri uri)) (version '(1 . 1)) (keep-alive? #f) (headers '()) (decode-body? #t) (streaming? #f)) doc (request uri #:body body #:method method #:port port #:version version #:keep-alive? keep-alive? #:headers headers #:decode-body? decode-body? #:streaming? streaming?))) (define-http-verb http-head 'HEAD "Fetch message headers for the given URI using the HTTP \"HEAD\" method. This function is similar to ‘http-get’, except it uses the \"HEAD\" method. See ‘http-get’ for full documentation on the various keyword arguments that are accepted by this function. Returns two values: the resulting response, and ‘#f’. Responses to HEAD requests do not have a body. The second value is only returned so that other procedures can treat all of the http-foo verbs identically.") (define-http-verb http-post 'POST "Post data to the given URI using the HTTP \"POST\" method. This function is similar to ‘http-get’, except it uses the \"POST\" method. See ‘http-get’ for full documentation on the various keyword arguments that are accepted by this function. Returns two values: the resulting response, and the response body.") (define-http-verb http-put 'PUT "Put data at the given URI using the HTTP \"PUT\" method. This function is similar to ‘http-get’, except it uses the \"PUT\" method. See ‘http-get’ for full documentation on the various keyword arguments that are accepted by this function. Returns two values: the resulting response, and the response body.") (define-http-verb http-delete 'DELETE "Delete data at the given URI using the HTTP \"DELETE\" method. This function is similar to ‘http-get’, except it uses the \"DELETE\" method. See ‘http-get’ for full documentation on the various keyword arguments that are accepted by this function. Returns two values: the resulting response, and the response body.") (define-http-verb http-trace 'TRACE "Send an HTTP \"TRACE\" request. This function is similar to ‘http-get’, except it uses the \"TRACE\" method. See ‘http-get’ for full documentation on the various keyword arguments that are accepted by this function. Returns two values: the resulting response, and the response body.") (define-http-verb http-options 'OPTIONS "Query characteristics of an HTTP resource using the HTTP \"OPTIONS\" method. This function is similar to ‘http-get’, except it uses the \"OPTIONS\" method. See ‘http-get’ for full documentation on the various keyword arguments that are accepted by this function. Returns two values: the resulting response, and the response body.") PK/F2] h < < web/uri.scmnu[;;;; (web uri) --- URI manipulation tools ;;;; ;;;; Copyright (C) 1997,2001,2002,2010,2011,2012,2013 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; A data type for Universal Resource Identifiers, as defined in RFC ;; 3986. ;;; Code: (define-module (web uri) #:use-module (srfi srfi-9) #:use-module (ice-9 regex) #:use-module (ice-9 rdelim) #:use-module (ice-9 control) #:use-module (rnrs bytevectors) #:use-module (ice-9 binary-ports) #:export (uri? uri-scheme uri-userinfo uri-host uri-port uri-path uri-query uri-fragment build-uri declare-default-port! string->uri uri->string uri-decode uri-encode split-and-decode-uri-path encode-and-join-uri-path)) (define-record-type (make-uri scheme userinfo host port path query fragment) uri? (scheme uri-scheme) (userinfo uri-userinfo) (host uri-host) (port uri-port) (path uri-path) (query uri-query) (fragment uri-fragment)) (define (absolute-uri? obj) (and (uri? obj) (uri-scheme obj) #t)) (define (uri-error message . args) (throw 'uri-error message args)) (define (positive-exact-integer? port) (and (number? port) (exact? port) (integer? port) (positive? port))) (define (validate-uri scheme userinfo host port path query fragment) (cond ((not (symbol? scheme)) (uri-error "Expected a symbol for the URI scheme: ~s" scheme)) ((and (or userinfo port) (not host)) (uri-error "Expected a host, given userinfo or port")) ((and port (not (positive-exact-integer? port))) (uri-error "Expected port to be an integer: ~s" port)) ((and host (or (not (string? host)) (not (valid-host? host)))) (uri-error "Expected valid host: ~s" host)) ((and userinfo (not (string? userinfo))) (uri-error "Expected string for userinfo: ~s" userinfo)) ((not (string? path)) (uri-error "Expected string for path: ~s" path)) ((and host (not (string-null? path)) (not (eqv? (string-ref path 0) #\/))) (uri-error "Expected path of absolute URI to start with a /: ~a" path)))) (define* (build-uri scheme #:key userinfo host port (path "") query fragment (validate? #t)) "Construct a URI object. SCHEME should be a symbol, PORT either a positive, exact integer or ‘#f’, and the rest of the fields are either strings or ‘#f’. If VALIDATE? is true, also run some consistency checks to make sure that the constructed URI is valid." (if validate? (validate-uri scheme userinfo host port path query fragment)) (make-uri scheme userinfo host port path query fragment)) ;; See RFC 3986 #3.2.2 for comments on percent-encodings, IDNA (RFC ;; 3490), and non-ASCII host names. ;; (define ipv4-regexp (make-regexp "^([0-9.]+)$")) (define ipv6-regexp (make-regexp "^([0-9a-fA-F:.]+)$")) (define domain-label-regexp (make-regexp "^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?$")) (define top-label-regexp (make-regexp "^[a-zA-Z]([a-zA-Z0-9-]*[a-zA-Z0-9])?$")) (define (valid-host? host) (cond ((regexp-exec ipv4-regexp host) (false-if-exception (inet-pton AF_INET host))) ((regexp-exec ipv6-regexp host) (false-if-exception (inet-pton AF_INET6 host))) (else (let lp ((start 0)) (let ((end (string-index host #\. start))) (if end (and (regexp-exec domain-label-regexp (substring host start end)) (lp (1+ end))) (regexp-exec top-label-regexp host start))))))) (define userinfo-pat "[a-zA-Z0-9_.!~*'();:&=+$,-]+") (define host-pat "[a-zA-Z0-9.-]+") (define ipv6-host-pat "[0-9a-fA-F:.]+") (define port-pat "[0-9]*") (define authority-regexp (make-regexp (format #f "^//((~a)@)?((~a)|(\\[(~a)\\]))(:(~a))?$" userinfo-pat host-pat ipv6-host-pat port-pat))) (define (parse-authority authority fail) (if (equal? authority "//") ;; Allow empty authorities: file:///etc/hosts is a synonym of ;; file:/etc/hosts. (values #f #f #f) (let ((m (regexp-exec authority-regexp authority))) (if (and m (valid-host? (or (match:substring m 4) (match:substring m 6)))) (values (match:substring m 2) (or (match:substring m 4) (match:substring m 6)) (let ((port (match:substring m 8))) (and port (not (string-null? port)) (string->number port)))) (fail))))) ;;; RFC 3986, #3. ;;; ;;; URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] ;;; ;;; hier-part = "//" authority path-abempty ;;; / path-absolute ;;; / path-rootless ;;; / path-empty (define scheme-pat "[a-zA-Z][a-zA-Z0-9+.-]*") (define authority-pat "[^/?#]*") (define path-pat "[^?#]*") (define query-pat "[^#]*") (define fragment-pat ".*") (define uri-pat (format #f "^((~a):)?(//~a)?(~a)(\\?(~a))?(#(~a))?$" scheme-pat authority-pat path-pat query-pat fragment-pat)) (define uri-regexp (make-regexp uri-pat)) (define (string->uri* string) "Parse STRING into a URI object. Return ‘#f’ if the string could not be parsed." (% (let ((m (regexp-exec uri-regexp string))) (if (not m) (abort)) (let ((scheme (let ((str (match:substring m 2))) (and str (string->symbol (string-downcase str))))) (authority (match:substring m 3)) (path (match:substring m 4)) (query (match:substring m 6)) (fragment (match:substring m 7))) (call-with-values (lambda () (if authority (parse-authority authority abort) (values #f #f #f))) (lambda (userinfo host port) (make-uri scheme userinfo host port path query fragment))))) (lambda (k) #f))) (define (string->uri string) "Parse STRING into a URI object. Return ‘#f’ if the string could not be parsed." (let ((uri (string->uri* string))) (and uri (uri-scheme uri) uri))) (define *default-ports* (make-hash-table)) (define (declare-default-port! scheme port) "Declare a default port for the given URI scheme." (hashq-set! *default-ports* scheme port)) (define (default-port? scheme port) (or (not port) (eqv? port (hashq-ref *default-ports* scheme)))) (declare-default-port! 'http 80) (declare-default-port! 'https 443) (define (uri->string uri) "Serialize URI to a string. If the URI has a port that is the default port for its scheme, the port is not included in the serialization." (let* ((scheme (uri-scheme uri)) (userinfo (uri-userinfo uri)) (host (uri-host uri)) (port (uri-port uri)) (path (uri-path uri)) (query (uri-query uri)) (fragment (uri-fragment uri))) (string-append (if scheme (string-append (symbol->string scheme) ":") "") (if host (string-append "//" (if userinfo (string-append userinfo "@") "") (if (string-index host #\:) (string-append "[" host "]") host) (if (default-port? (uri-scheme uri) port) "" (string-append ":" (number->string port)))) "") path (if query (string-append "?" query) "") (if fragment (string-append "#" fragment) "")))) ;; like call-with-output-string, but actually closes the port (doh) (define (call-with-output-string* proc) (let ((port (open-output-string))) (proc port) (let ((str (get-output-string port))) (close-port port) str))) (define (call-with-output-bytevector* proc) (call-with-values (lambda () (open-bytevector-output-port)) (lambda (port get-bytevector) (proc port) (let ((bv (get-bytevector))) (close-port port) bv)))) (define (call-with-encoded-output-string encoding proc) (if (string-ci=? encoding "utf-8") (string->utf8 (call-with-output-string* proc)) (call-with-output-bytevector* (lambda (port) (set-port-encoding! port encoding) (proc port))))) (define (encode-string str encoding) (if (string-ci=? encoding "utf-8") (string->utf8 str) (call-with-encoded-output-string encoding (lambda (port) (display str port))))) (define (decode-string bv encoding) (if (string-ci=? encoding "utf-8") (utf8->string bv) (let ((p (open-bytevector-input-port bv))) (set-port-encoding! p encoding) (let ((res (read-string p))) (close-port p) res)))) ;; A note on characters and bytes: URIs are defined to be sequences of ;; characters in a subset of ASCII. Those characters may encode a ;; sequence of bytes (octets), which in turn may encode sequences of ;; characters in other character sets. ;; ;; Return a new string made from uri-decoding STR. Specifically, ;; turn ‘+’ into space, and hex-encoded ‘%XX’ strings into ;; their eight-bit characters. ;; (define hex-chars (string->char-set "0123456789abcdefABCDEF")) (define* (uri-decode str #:key (encoding "utf-8") (decode-plus-to-space? #t)) "Percent-decode the given STR, according to ENCODING, which should be the name of a character encoding. Note that this function should not generally be applied to a full URI string. For paths, use ‘split-and-decode-uri-path’ instead. For query strings, split the query on ‘&’ and ‘=’ boundaries, and decode the components separately. Note also that percent-encoded strings encode _bytes_, not characters. There is no guarantee that a given byte sequence is a valid string encoding. Therefore this routine may signal an error if the decoded bytes are not valid for the given encoding. Pass ‘#f’ for ENCODING if you want decoded bytes as a bytevector directly. ‘set-port-encoding!’, for more information on character encodings. If DECODE-PLUS-TO-SPACE? is true, which is the default, also replace instances of the plus character (+) with a space character. This is needed when parsing application/x-www-form-urlencoded data. Returns a string of the decoded characters, or a bytevector if ENCODING was ‘#f’." (let* ((len (string-length str)) (bv (call-with-output-bytevector* (lambda (port) (let lp ((i 0)) (if (< i len) (let ((ch (string-ref str i))) (cond ((and (eqv? ch #\+) decode-plus-to-space?) (put-u8 port (char->integer #\space)) (lp (1+ i))) ((and (< (+ i 2) len) (eqv? ch #\%) (let ((a (string-ref str (+ i 1))) (b (string-ref str (+ i 2)))) (and (char-set-contains? hex-chars a) (char-set-contains? hex-chars b) (string->number (string a b) 16)))) => (lambda (u8) (put-u8 port u8) (lp (+ i 3)))) ((< (char->integer ch) 128) (put-u8 port (char->integer ch)) (lp (1+ i))) (else (uri-error "Invalid character in encoded URI ~a: ~s" str ch)))))))))) (if encoding (decode-string bv encoding) ;; Otherwise return raw bytevector bv))) (define ascii-alnum-chars (string->char-set "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")) ;; RFC 3986, #2.2. (define gen-delims (string->char-set ":/?#[]@")) (define sub-delims (string->char-set "!$&'()*+,l=")) (define reserved-chars (char-set-union gen-delims sub-delims)) ;; RFC 3986, #2.3 (define unreserved-chars (char-set-union ascii-alnum-chars (string->char-set "-._~"))) ;; Return a new string made from uri-encoding STR, unconditionally ;; transforming any characters not in UNESCAPED-CHARS. ;; (define* (uri-encode str #:key (encoding "utf-8") (unescaped-chars unreserved-chars)) "Percent-encode any character not in the character set, UNESCAPED-CHARS. The default character set includes alphanumerics from ASCII, as well as the special characters ‘-’, ‘.’, ‘_’, and ‘~’. Any other character will be percent-encoded, by writing out the character to a bytevector within the given ENCODING, then encoding each byte as ‘%HH’, where HH is the uppercase hexadecimal representation of the byte." (define (needs-escaped? ch) (not (char-set-contains? unescaped-chars ch))) (if (string-index str needs-escaped?) (call-with-output-string* (lambda (port) (string-for-each (lambda (ch) (if (char-set-contains? unescaped-chars ch) (display ch port) (let* ((bv (encode-string (string ch) encoding)) (len (bytevector-length bv))) (let lp ((i 0)) (if (< i len) (let ((byte (bytevector-u8-ref bv i))) (display #\% port) (when (< byte 16) (display #\0 port)) (display (string-upcase (number->string byte 16)) port) (lp (1+ i)))))))) str))) str)) (define (split-and-decode-uri-path path) "Split PATH into its components, and decode each component, removing empty components. For example, ‘\"/foo/bar%20baz/\"’ decodes to the two-element list, ‘(\"foo\" \"bar baz\")’." (filter (lambda (x) (not (string-null? x))) (map (lambda (s) (uri-decode s #:decode-plus-to-space? #f)) (string-split path #\/)))) (define (encode-and-join-uri-path parts) "URI-encode each element of PARTS, which should be a list of strings, and join the parts together with ‘/’ as a delimiter. For example, the list ‘(\"scrambled eggs\" \"biscuits&gravy\")’ encodes as ‘\"scrambled%20eggs/biscuits%26gravy\"’." (string-join (map uri-encode parts) "/")) PK/F2]Q:33web/response.scmnu[;;; HTTP response objects ;; Copyright (C) 2010, 2011, 2012, 2013, 2014, 2015 Free Software Foundation, Inc. ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ;; 02110-1301 USA ;;; Code: (define-module (web response) #:use-module (rnrs bytevectors) #:use-module (ice-9 binary-ports) #:use-module (ice-9 rdelim) #:use-module (ice-9 match) #:use-module (srfi srfi-9) #:use-module (web http) #:export (response? response-version response-code response-reason-phrase response-headers response-port read-response build-response adapt-response-version write-response response-must-not-include-body? response-body-port read-response-body write-response-body ;; General headers ;; response-cache-control response-connection response-date response-pragma response-trailer response-transfer-encoding response-upgrade response-via response-warning ;; Entity headers ;; response-allow response-content-encoding response-content-language response-content-length response-content-location response-content-md5 response-content-range response-content-type text-content-type? response-expires response-last-modified ;; Response headers ;; response-accept-ranges response-age response-etag response-location response-proxy-authenticate response-retry-after response-server response-vary response-www-authenticate)) (define-record-type (make-response version code reason-phrase headers port) response? (version response-version) (code response-code) (reason-phrase %response-reason-phrase) (headers response-headers) (port response-port)) (define (bad-response message . args) (throw 'bad-response message args)) (define (non-negative-integer? n) (and (number? n) (>= n 0) (exact? n) (integer? n))) (define (validate-headers headers) (if (pair? headers) (let ((h (car headers))) (if (pair? h) (let ((k (car h)) (v (cdr h))) (if (valid-header? k v) (validate-headers (cdr headers)) (bad-response "Bad value for header ~a: ~s" k v))) (bad-response "Header not a pair: ~a" h))) (if (not (null? headers)) (bad-response "Headers not a list: ~a" headers)))) (define* (build-response #:key (version '(1 . 1)) (code 200) reason-phrase (headers '()) port (validate-headers? #t)) "Construct an HTTP response object. If VALIDATE-HEADERS? is true, the headers are each run through their respective validators." (cond ((not (and (pair? version) (non-negative-integer? (car version)) (non-negative-integer? (cdr version)))) (bad-response "Bad version: ~a" version)) ((not (and (non-negative-integer? code) (< code 600))) (bad-response "Bad code: ~a" code)) ((and reason-phrase (not (string? reason-phrase))) (bad-response "Bad reason phrase" reason-phrase)) (else (if validate-headers? (validate-headers headers)))) (make-response version code reason-phrase headers port)) (define *reason-phrases* '((100 . "Continue") (101 . "Switching Protocols") (200 . "OK") (201 . "Created") (202 . "Accepted") (203 . "Non-Authoritative Information") (204 . "No Content") (205 . "Reset Content") (206 . "Partial Content") (300 . "Multiple Choices") (301 . "Moved Permanently") (302 . "Found") (303 . "See Other") (304 . "Not Modified") (305 . "Use Proxy") (307 . "Temporary Redirect") (400 . "Bad Request") (401 . "Unauthorized") (402 . "Payment Required") (403 . "Forbidden") (404 . "Not Found") (405 . "Method Not Allowed") (406 . "Not Acceptable") (407 . "Proxy Authentication Required") (408 . "Request Timeout") (409 . "Conflict") (410 . "Gone") (411 . "Length Required") (412 . "Precondition Failed") (413 . "Request Entity Too Large") (414 . "Request-URI Too Long") (415 . "Unsupported Media Type") (416 . "Requested Range Not Satisfiable") (417 . "Expectation Failed") (500 . "Internal Server Error") (501 . "Not Implemented") (502 . "Bad Gateway") (503 . "Service Unavailable") (504 . "Gateway Timeout") (505 . "HTTP Version Not Supported"))) (define (code->reason-phrase code) (or (assv-ref *reason-phrases* code) "(Unknown)")) (define (response-reason-phrase response) "Return the reason phrase given in RESPONSE, or the standard reason phrase for the response's code." (or (%response-reason-phrase response) (code->reason-phrase (response-code response)))) (define (text-content-type? type) "Return #t if TYPE, a symbol as returned by `response-content-type', represents a textual type such as `text/plain'." (let ((type (symbol->string type))) (or (string-prefix? "text/" type) (string-suffix? "/xml" type) (string-suffix? "+xml" type)))) (define (read-response port) "Read an HTTP response from PORT. As a side effect, sets the encoding on PORT to ISO-8859-1 (latin-1), so that reading one character reads one byte. See the discussion of character sets in \"HTTP Responses\" in the manual, for more information." (set-port-encoding! port "ISO-8859-1") (call-with-values (lambda () (read-response-line port)) (lambda (version code reason-phrase) (make-response version code reason-phrase (read-headers port) port)))) (define (adapt-response-version response version) "Adapt the given response to a different HTTP version. Returns a new HTTP response. The idea is that many applications might just build a response for the default HTTP version, and this method could handle a number of programmatic transformations to respond to older HTTP versions (0.9 and 1.0). But currently this function is a bit heavy-handed, just updating the version field." (build-response #:code (response-code response) #:version version #:headers (response-headers response) #:port (response-port response))) (define (write-response r port) "Write the given HTTP response to PORT. Returns a new response, whose ‘response-port’ will continue writing on PORT, perhaps using some transfer encoding." (write-response-line (response-version r) (response-code r) (response-reason-phrase r) port) (write-headers (response-headers r) port) (display "\r\n" port) (if (eq? port (response-port r)) r (make-response (response-version r) (response-code r) (response-reason-phrase r) (response-headers r) port))) (define (response-must-not-include-body? r) "Returns ‘#t’ if the response R is not permitted to have a body. This is true for some response types, like those with code 304." ;; RFC 2616, section 4.3. (or (<= 100 (response-code r) 199) (= (response-code r) 204) (= (response-code r) 304))) (define (make-delimited-input-port port len keep-alive?) "Return an input port that reads from PORT, and makes sure that exactly LEN bytes are available from PORT. Closing the returned port closes PORT, unless KEEP-ALIVE? is true." (define bytes-read 0) (define (fail) (bad-response "EOF while reading response body: ~a bytes of ~a" bytes-read len)) (define (read! bv start count) ;; Read at most LEN bytes in total. HTTP/1.1 doesn't say what to do ;; when a server provides more than the Content-Length, but it seems ;; wise to just stop reading at LEN. (let ((count (min count (- len bytes-read)))) (let loop ((ret (get-bytevector-n! port bv start count))) (cond ((eof-object? ret) (if (= bytes-read len) 0 ; EOF (fail))) ((and (zero? ret) (> count 0)) ;; Do not return zero since zero means EOF, so try again. (loop (get-bytevector-n! port bv start count))) (else (set! bytes-read (+ bytes-read ret)) ret))))) (define close (and (not keep-alive?) (lambda () (close-port port)))) (make-custom-binary-input-port "delimited input port" read! #f #f close)) (define* (response-body-port r #:key (decode? #t) (keep-alive? #t)) "Return an input port from which the body of R can be read. The encoding of the returned port is set according to R's ‘content-type’ header, when it's textual, except if DECODE? is ‘#f’. Return #f when no body is available. When KEEP-ALIVE? is ‘#f’, closing the returned port also closes R's response port." (define port (cond ((member '(chunked) (response-transfer-encoding r)) (make-chunked-input-port (response-port r) #:keep-alive? keep-alive?)) ((response-content-length r) => (lambda (len) (make-delimited-input-port (response-port r) len keep-alive?))) ((response-must-not-include-body? r) #f) ((or (memq 'close (response-connection r)) (and (equal? (response-version r) '(1 . 0)) (not (memq 'keep-alive (response-connection r))))) (response-port r)) (else ;; Here we have a message with no transfer encoding, no ;; content-length, and a response that won't necessarily be closed ;; by the server. Not much we can do; assume that the client ;; knows how to handle it. (response-port r)))) (when (and decode? port) (match (response-content-type r) (((? text-content-type?) . props) (set-port-encoding! port (or (assq-ref props 'charset) "ISO-8859-1"))) (_ #f))) port) (define (read-response-body r) "Reads the response body from R, as a bytevector. Returns ‘#f’ if there was no response body." (let ((body (and=> (response-body-port r #:decode? #f) get-bytevector-all))) ;; Reading a body of length 0 will result in get-bytevector-all ;; returning the EOF object. (if (eof-object? body) #vu8() body))) (define (write-response-body r bv) "Write BV, a bytevector, to the port corresponding to the HTTP response R." (put-bytevector (response-port r) bv)) (define-syntax define-response-accessor (lambda (x) (syntax-case x () ((_ field) #'(define-response-accessor field #f)) ((_ field def) (identifier? #'field) #`(define* (#,(datum->syntax #'field (symbol-append 'response- (syntax->datum #'field))) response #:optional (default def)) (cond ((assq 'field (response-headers response)) => cdr) (else default))))))) ;; General headers ;; (define-response-accessor cache-control '()) (define-response-accessor connection '()) (define-response-accessor date #f) (define-response-accessor pragma '()) (define-response-accessor trailer '()) (define-response-accessor transfer-encoding '()) (define-response-accessor upgrade '()) (define-response-accessor via '()) (define-response-accessor warning '()) ;; Entity headers ;; (define-response-accessor allow '()) (define-response-accessor content-encoding '()) (define-response-accessor content-language '()) (define-response-accessor content-length #f) (define-response-accessor content-location #f) (define-response-accessor content-md5 #f) (define-response-accessor content-range #f) (define-response-accessor content-type #f) (define-response-accessor expires #f) (define-response-accessor last-modified #f) ;; Response headers ;; (define-response-accessor accept-ranges #f) (define-response-accessor age #f) (define-response-accessor etag #f) (define-response-accessor location #f) (define-response-accessor proxy-authenticate #f) (define-response-accessor retry-after #f) (define-response-accessor server #f) (define-response-accessor vary '()) (define-response-accessor www-authenticate #f) PK/F2]oxqkkweb/server/http.scmnu[;;; Web I/O: HTTP ;; Copyright (C) 2010, 2011, 2012, 2015 Free Software Foundation, Inc. ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ;; 02110-1301 USA ;;; Commentary: ;;; ;;; This is the HTTP implementation of the (web server) interface. ;;; ;;; `read-request' sets the character encoding on the new port to ;;; latin-1. See the note in request.scm regarding character sets, ;;; strings, and bytevectors for more information. ;;; ;;; Code: (define-module (web server http) #:use-module ((srfi srfi-1) #:select (fold)) #:use-module (srfi srfi-9) #:use-module (rnrs bytevectors) #:use-module (web request) #:use-module (web response) #:use-module (web server) #:use-module (ice-9 poll) #:export (http)) (define (make-default-socket family addr port) (let ((sock (socket PF_INET SOCK_STREAM 0))) (setsockopt sock SOL_SOCKET SO_REUSEADDR 1) (bind sock family addr port) sock)) (define-record-type (make-http-server socket poll-idx poll-set) http-server? (socket http-socket) (poll-idx http-poll-idx set-http-poll-idx!) (poll-set http-poll-set)) (define *error-events* (logior POLLHUP POLLERR)) (define *read-events* POLLIN) (define *events* (logior *error-events* *read-events*)) ;; -> server (define* (http-open #:key (host #f) (family AF_INET) (addr (if host (inet-pton family host) INADDR_LOOPBACK)) (port 8080) (socket (make-default-socket family addr port))) (listen socket 128) (sigaction SIGPIPE SIG_IGN) (let ((poll-set (make-empty-poll-set))) (poll-set-add! poll-set socket *events*) (make-http-server socket 0 poll-set))) (define (bad-request port) (write-response (build-response #:version '(1 . 0) #:code 400 #:headers '((content-length . 0))) port)) ;; -> (client request body | #f #f #f) (define (http-read server) (let* ((poll-set (http-poll-set server))) (let lp ((idx (http-poll-idx server))) (let ((revents (poll-set-revents poll-set idx))) (cond ((zero? idx) ;; The server socket, and the end of our downward loop. (cond ((zero? revents) ;; No client ready, and no error; poll and loop. (poll poll-set) (lp (1- (poll-set-nfds poll-set)))) ((not (zero? (logand revents *error-events*))) ;; An error. (set-http-poll-idx! server idx) (throw 'interrupt)) (else ;; A new client. Add to set, poll, and loop. ;; ;; FIXME: preserve meta-info. (let ((client (accept (poll-set-port poll-set idx)))) ;; Buffer input and output on this port. (setvbuf (car client) _IOFBF) ;; From "HOP, A Fast Server for the Diffuse Web", Serrano. (setsockopt (car client) SOL_SOCKET SO_SNDBUF (* 12 1024)) (poll-set-add! poll-set (car client) *events*) (poll poll-set) (lp (1- (poll-set-nfds poll-set))))))) ((zero? revents) ;; Nothing on this port. (lp (1- idx))) ;; Otherwise, a client socket with some activity on ;; it. Remove it from the poll set. (else (let ((port (poll-set-remove! poll-set idx))) ;; Record the next index in all cases, in case the EOF check ;; throws an error. (set-http-poll-idx! server (1- idx)) (cond ((eof-object? (peek-char port)) ;; EOF. (close-port port) (lp (1- idx))) (else ;; Otherwise, try to read a request from this port. (with-throw-handler #t (lambda () (let ((req (read-request port))) (values port req (read-request-body req)))) (lambda (k . args) (define-syntax-rule (cleanup-catch statement) (catch #t (lambda () statement) (lambda (k . args) (format (current-error-port) "In ~a:\n" 'statement) (print-exception (current-error-port) #f k args)))) (cleanup-catch (bad-request port)) (cleanup-catch (close-port port))))))))))))) (define (keep-alive? response) (let ((v (response-version response))) (and (or (< (response-code response) 400) (= (response-code response) 404)) (case (car v) ((1) (case (cdr v) ((1) (not (memq 'close (response-connection response)))) ((0) (memq 'keep-alive (response-connection response))))) (else #f))))) ;; -> 0 values (define (http-write server client response body) (let* ((response (write-response response client)) (port (response-port response))) (cond ((not body)) ; pass ((bytevector? body) (write-response-body response body)) (else (error "Expected a bytevector for body" body))) (cond ((keep-alive? response) (force-output port) (poll-set-add! (http-poll-set server) port *events*)) (else (close-port port))) (values))) ;; -> unspecified values (define (http-close server) (let ((poll-set (http-poll-set server))) (let lp ((n (poll-set-nfds poll-set))) (if (positive? n) (begin (close-port (poll-set-remove! poll-set (1- n))) (lp (1- n))))))) (define-server-impl http http-open http-read http-write http-close) PK/F2] web/http.scmnu[;;; HTTP messages ;; Copyright (C) 2010-2016 Free Software Foundation, Inc. ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ;; 02110-1301 USA ;;; Commentary: ;;; ;;; This module has a number of routines to parse textual ;;; representations of HTTP data into native Scheme data structures. ;;; ;;; It tries to follow RFCs fairly strictly---the road to perdition ;;; being paved with compatibility hacks---though some allowances are ;;; made for not-too-divergent texts (like a quality of .2 which should ;;; be 0.2, etc). ;;; ;;; Code: (define-module (web http) #:use-module ((srfi srfi-1) #:select (append-map! map!)) #:use-module (srfi srfi-9) #:use-module (srfi srfi-19) #:use-module (ice-9 rdelim) #:use-module (ice-9 match) #:use-module (ice-9 q) #:use-module (ice-9 binary-ports) #:use-module (rnrs bytevectors) #:use-module (web uri) #:export (string->header header->string declare-header! declare-opaque-header! known-header? header-parser header-validator header-writer read-header parse-header valid-header? write-header read-headers write-headers parse-http-method parse-http-version parse-request-uri read-request-line write-request-line read-response-line write-response-line make-chunked-input-port make-chunked-output-port http-proxy-port? set-http-proxy-port?!)) (define (string->header name) "Parse NAME to a symbolic header name." (string->symbol (string-downcase name))) (define-record-type (make-header-decl name parser validator writer multiple?) header-decl? (name header-decl-name) (parser header-decl-parser) (validator header-decl-validator) (writer header-decl-writer) (multiple? header-decl-multiple?)) ;; sym -> header (define *declared-headers* (make-hash-table)) (define (lookup-header-decl sym) (hashq-ref *declared-headers* sym)) (define* (declare-header! name parser validator writer #:key multiple?) "Declare a parser, validator, and writer for a given header." (if (and (string? name) parser validator writer) (let ((decl (make-header-decl name parser validator writer multiple?))) (hashq-set! *declared-headers* (string->header name) decl) decl) (error "bad header decl" name parser validator writer multiple?))) (define (header->string sym) "Return the string form for the header named SYM." (let ((decl (lookup-header-decl sym))) (if decl (header-decl-name decl) (string-titlecase (symbol->string sym))))) (define (known-header? sym) "Return ‘#t’ iff SYM is a known header, with associated parsers and serialization procedures." (and (lookup-header-decl sym) #t)) (define (header-parser sym) "Return the value parser for headers named SYM. The result is a procedure that takes one argument, a string, and returns the parsed value. If the header isn't known to Guile, a default parser is returned that passes through the string unchanged." (let ((decl (lookup-header-decl sym))) (if decl (header-decl-parser decl) (lambda (x) x)))) (define (header-validator sym) "Return a predicate which returns ‘#t’ if the given value is valid for headers named SYM. The default validator for unknown headers is ‘string?’." (let ((decl (lookup-header-decl sym))) (if decl (header-decl-validator decl) string?))) (define (header-writer sym) "Return a procedure that writes values for headers named SYM to a port. The resulting procedure takes two arguments: a value and a port. The default writer is ‘display’." (let ((decl (lookup-header-decl sym))) (if decl (header-decl-writer decl) display))) (define (read-header-line port) "Read an HTTP header line and return it without its final CRLF or LF. Raise a 'bad-header' exception if the line does not end in CRLF or LF, or if EOF is reached." (match (%read-line port) (((? string? line) . #\newline) ;; '%read-line' does not consider #\return a delimiter; so if it's ;; there, remove it. We are more tolerant than the RFC in that we ;; tolerate LF-only endings. (if (string-suffix? "\r" line) (string-drop-right line 1) line)) ((line . _) ;EOF or missing delimiter (bad-header 'read-header-line line)))) (define (read-continuation-line port val) (if (or (eqv? (peek-char port) #\space) (eqv? (peek-char port) #\tab)) (read-continuation-line port (string-append val (read-header-line port))) val)) (define *eof* (call-with-input-string "" read)) (define (read-header port) "Read one HTTP header from PORT. Return two values: the header name and the parsed Scheme value. May raise an exception if the header was known but the value was invalid. Returns the end-of-file object for both values if the end of the message body was reached (i.e., a blank line)." (let ((line (read-header-line port))) (if (or (string-null? line) (string=? line "\r")) (values *eof* *eof*) (let* ((delim (or (string-index line #\:) (bad-header '%read line))) (sym (string->header (substring line 0 delim)))) (values sym (parse-header sym (read-continuation-line port (string-trim-both line char-set:whitespace (1+ delim))))))))) (define (parse-header sym val) "Parse VAL, a string, with the parser registered for the header named SYM. Returns the parsed value." ((header-parser sym) val)) (define (valid-header? sym val) "Returns a true value iff VAL is a valid Scheme value for the header with name SYM." (if (symbol? sym) ((header-validator sym) val) (error "header name not a symbol" sym))) (define (write-header sym val port) "Write the given header name and value to PORT, using the writer from ‘header-writer’." (display (header->string sym) port) (display ": " port) ((header-writer sym) val port) (display "\r\n" port)) (define (read-headers port) "Read the headers of an HTTP message from PORT, returning them as an ordered alist." (let lp ((headers '())) (call-with-values (lambda () (read-header port)) (lambda (k v) (if (eof-object? k) (reverse! headers) (lp (acons k v headers))))))) (define (write-headers headers port) "Write the given header alist to PORT. Doesn't write the final ‘\\r\\n’, as the user might want to add another header." (let lp ((headers headers)) (if (pair? headers) (begin (write-header (caar headers) (cdar headers) port) (lp (cdr headers)))))) ;;; ;;; Utilities ;;; (define (bad-header sym val) (throw 'bad-header sym val)) (define (bad-header-component sym val) (throw 'bad-header-component sym val)) (define (bad-header-printer port key args default-printer) (apply (case-lambda ((sym val) (format port "Bad ~a header: ~a\n" (header->string sym) val)) (_ (default-printer))) args)) (define (bad-header-component-printer port key args default-printer) (apply (case-lambda ((sym val) (format port "Bad ~a header component: ~a\n" sym val)) (_ (default-printer))) args)) (set-exception-printer! 'bad-header bad-header-printer) (set-exception-printer! 'bad-header-component bad-header-component-printer) (define (parse-opaque-string str) str) (define (validate-opaque-string val) (string? val)) (define (write-opaque-string val port) (display val port)) (define separators-without-slash (string->char-set "[^][()<>@,;:\\\"?= \t]")) (define (validate-media-type str) (let ((idx (string-index str #\/))) (and idx (= idx (string-rindex str #\/)) (not (string-index str separators-without-slash))))) (define (parse-media-type str) (if (validate-media-type str) (string->symbol str) (bad-header-component 'media-type str))) (define* (skip-whitespace str #:optional (start 0) (end (string-length str))) (let lp ((i start)) (if (and (< i end) (char-whitespace? (string-ref str i))) (lp (1+ i)) i))) (define* (trim-whitespace str #:optional (start 0) (end (string-length str))) (let lp ((i end)) (if (and (< start i) (char-whitespace? (string-ref str (1- i)))) (lp (1- i)) i))) (define* (split-and-trim str #:optional (delim #\,) (start 0) (end (string-length str))) (let lp ((i start)) (if (< i end) (let* ((idx (string-index str delim i end)) (tok (string-trim-both str char-set:whitespace i (or idx end)))) (cons tok (split-and-trim str delim (if idx (1+ idx) end) end))) '()))) (define (list-of-strings? val) (list-of? val string?)) (define (write-list-of-strings val port) (write-list val port display ", ")) (define (split-header-names str) (map string->header (split-and-trim str))) (define (list-of-header-names? val) (list-of? val symbol?)) (define (write-header-list val port) (write-list val port (lambda (x port) (display (header->string x) port)) ", ")) (define (collect-escaped-string from start len escapes) (let ((to (make-string len))) (let lp ((start start) (i 0) (escapes escapes)) (if (null? escapes) (begin (substring-move! from start (+ start (- len i)) to i) to) (let* ((e (car escapes)) (next-start (+ start (- e i) 2))) (substring-move! from start (- next-start 2) to i) (string-set! to e (string-ref from (- next-start 1))) (lp next-start (1+ e) (cdr escapes))))))) ;; in incremental mode, returns two values: the string, and the index at ;; which the string ended (define* (parse-qstring str #:optional (start 0) (end (trim-whitespace str start)) #:key incremental?) (if (and (< start end) (eqv? (string-ref str start) #\")) (let lp ((i (1+ start)) (qi 0) (escapes '())) (if (< i end) (case (string-ref str i) ((#\\) (lp (+ i 2) (1+ qi) (cons qi escapes))) ((#\") (let ((out (collect-escaped-string str (1+ start) qi escapes))) (if incremental? (values out (1+ i)) (if (= (1+ i) end) out (bad-header-component 'qstring str))))) (else (lp (1+ i) (1+ qi) escapes))) (bad-header-component 'qstring str))) (bad-header-component 'qstring str))) (define (write-list l port write-item delim) (if (pair? l) (let lp ((l l)) (write-item (car l) port) (if (pair? (cdr l)) (begin (display delim port) (lp (cdr l))))))) (define (write-qstring str port) (display #\" port) (if (string-index str #\") ;; optimize me (write-list (string-split str #\") port display "\\\"") (display str port)) (display #\" port)) (define* (parse-quality str #:optional (start 0) (end (string-length str))) (define (char->decimal c) (let ((i (- (char->integer c) (char->integer #\0)))) (if (and (<= 0 i) (< i 10)) i (bad-header-component 'quality str)))) (cond ((not (< start end)) (bad-header-component 'quality str)) ((eqv? (string-ref str start) #\1) (if (or (string= str "1" start end) (string= str "1." start end) (string= str "1.0" start end) (string= str "1.00" start end) (string= str "1.000" start end)) 1000 (bad-header-component 'quality str))) ((eqv? (string-ref str start) #\0) (if (or (string= str "0" start end) (string= str "0." start end)) 0 (if (< 2 (- end start) 6) (let lp ((place 1) (i (+ start 4)) (q 0)) (if (= i (1+ start)) (if (eqv? (string-ref str (1+ start)) #\.) q (bad-header-component 'quality str)) (lp (* 10 place) (1- i) (if (< i end) (+ q (* place (char->decimal (string-ref str i)))) q)))) (bad-header-component 'quality str)))) ;; Allow the nonstandard .2 instead of 0.2. ((and (eqv? (string-ref str start) #\.) (< 1 (- end start) 5)) (let lp ((place 1) (i (+ start 3)) (q 0)) (if (= i start) q (lp (* 10 place) (1- i) (if (< i end) (+ q (* place (char->decimal (string-ref str i)))) q))))) (else (bad-header-component 'quality str)))) (define (valid-quality? q) (and (non-negative-integer? q) (<= q 1000))) (define (write-quality q port) (define (digit->char d) (integer->char (+ (char->integer #\0) d))) (display (digit->char (modulo (quotient q 1000) 10)) port) (display #\. port) (display (digit->char (modulo (quotient q 100) 10)) port) (display (digit->char (modulo (quotient q 10) 10)) port) (display (digit->char (modulo q 10)) port)) (define (list-of? val pred) (or (null? val) (and (pair? val) (pred (car val)) (list-of? (cdr val) pred)))) (define* (parse-quality-list str) (map (lambda (part) (cond ((string-rindex part #\;) => (lambda (idx) (let ((qpart (string-trim-both part char-set:whitespace (1+ idx)))) (if (string-prefix? "q=" qpart) (cons (parse-quality qpart 2) (string-trim-both part char-set:whitespace 0 idx)) (bad-header-component 'quality qpart))))) (else (cons 1000 (string-trim-both part char-set:whitespace))))) (string-split str #\,))) (define (validate-quality-list l) (list-of? l (lambda (elt) (and (pair? elt) (valid-quality? (car elt)) (string? (cdr elt)))))) (define (write-quality-list l port) (write-list l port (lambda (x port) (let ((q (car x)) (str (cdr x))) (display str port) (if (< q 1000) (begin (display ";q=" port) (write-quality q port))))) ",")) (define* (parse-non-negative-integer val #:optional (start 0) (end (string-length val))) (define (char->decimal c) (let ((i (- (char->integer c) (char->integer #\0)))) (if (and (<= 0 i) (< i 10)) i (bad-header-component 'non-negative-integer val)))) (if (not (< start end)) (bad-header-component 'non-negative-integer val) (let lp ((i start) (out 0)) (if (< i end) (lp (1+ i) (+ (* out 10) (char->decimal (string-ref val i)))) out)))) (define (non-negative-integer? code) (and (number? code) (>= code 0) (exact? code) (integer? code))) (define (default-val-parser k val) val) (define (default-val-validator k val) (or (not val) (string? val))) (define (default-val-writer k val port) (if (or (string-index val #\;) (string-index val #\,) (string-index val #\")) (write-qstring val port) (display val port))) (define* (parse-key-value-list str #:optional (val-parser default-val-parser) (start 0) (end (string-length str))) (let lp ((i start) (out '())) (if (not (< i end)) (reverse! out) (let* ((i (skip-whitespace str i end)) (eq (string-index str #\= i end)) (comma (string-index str #\, i end)) (delim (min (or eq end) (or comma end))) (k (string->symbol (substring str i (trim-whitespace str i delim))))) (call-with-values (lambda () (if (and eq (or (not comma) (< eq comma))) (let ((i (skip-whitespace str (1+ eq) end))) (if (and (< i end) (eqv? (string-ref str i) #\")) (parse-qstring str i end #:incremental? #t) (values (substring str i (trim-whitespace str i (or comma end))) (or comma end)))) (values #f delim))) (lambda (v-str next-i) (let ((v (val-parser k v-str)) (i (skip-whitespace str next-i end))) (if (or (= i end) (eqv? (string-ref str i) #\,)) (lp (1+ i) (cons (if v (cons k v) k) out)) (bad-header-component 'key-value-list (substring str start end)))))))))) (define* (key-value-list? list #:optional (valid? default-val-validator)) (list-of? list (lambda (elt) (cond ((pair? elt) (let ((k (car elt)) (v (cdr elt))) (and (symbol? k) (valid? k v)))) ((symbol? elt) (valid? elt #f)) (else #f))))) (define* (write-key-value-list list port #:optional (val-writer default-val-writer) (delim ", ")) (write-list list port (lambda (x port) (let ((k (if (pair? x) (car x) x)) (v (if (pair? x) (cdr x) #f))) (display k port) (if v (begin (display #\= port) (val-writer k v port))))) delim)) ;; param-component = token [ "=" (token | quoted-string) ] \ ;; *(";" token [ "=" (token | quoted-string) ]) ;; (define param-delimiters (char-set #\, #\; #\=)) (define param-value-delimiters (char-set-adjoin char-set:whitespace #\, #\;)) (define* (parse-param-component str #:optional (val-parser default-val-parser) (start 0) (end (string-length str))) (let lp ((i start) (out '())) (if (not (< i end)) (values (reverse! out) end) (let ((delim (string-index str param-delimiters i))) (let ((k (string->symbol (substring str i (trim-whitespace str i (or delim end))))) (delimc (and delim (string-ref str delim)))) (case delimc ((#\=) (call-with-values (lambda () (let ((i (skip-whitespace str (1+ delim) end))) (if (and (< i end) (eqv? (string-ref str i) #\")) (parse-qstring str i end #:incremental? #t) (let ((delim (or (string-index str param-value-delimiters i end) end))) (values (substring str i delim) delim))))) (lambda (v-str next-i) (let* ((v (val-parser k v-str)) (x (if v (cons k v) k)) (i (skip-whitespace str next-i end))) (case (and (< i end) (string-ref str i)) ((#f) (values (reverse! (cons x out)) end)) ((#\;) (lp (skip-whitespace str (1+ i) end) (cons x out))) (else ; including #\, (values (reverse! (cons x out)) i))))))) ((#\;) (let ((v (val-parser k #f))) (lp (skip-whitespace str (1+ delim) end) (cons (if v (cons k v) k) out)))) (else ;; either the end of the string or a #\, (let ((v (val-parser k #f))) (values (reverse! (cons (if v (cons k v) k) out)) (or delim end)))))))))) (define* (parse-param-list str #:optional (val-parser default-val-parser) (start 0) (end (string-length str))) (let lp ((i start) (out '())) (call-with-values (lambda () (parse-param-component str val-parser i end)) (lambda (item i) (if (< i end) (if (eqv? (string-ref str i) #\,) (lp (skip-whitespace str (1+ i) end) (cons item out)) (bad-header-component 'param-list str)) (reverse! (cons item out))))))) (define* (validate-param-list list #:optional (valid? default-val-validator)) (list-of? list (lambda (elt) (key-value-list? elt valid?)))) (define* (write-param-list list port #:optional (val-writer default-val-writer)) (write-list list port (lambda (item port) (write-key-value-list item port val-writer ";")) ",")) (define-syntax string-match? (lambda (x) (syntax-case x () ((_ str pat) (string? (syntax->datum #'pat)) (let ((p (syntax->datum #'pat))) #`(let ((s str)) (and (= (string-length s) #,(string-length p)) #,@(let lp ((i 0) (tests '())) (if (< i (string-length p)) (let ((c (string-ref p i))) (lp (1+ i) (case c ((#\.) ; Whatever. tests) ((#\d) ; Digit. (cons #`(char-numeric? (string-ref s #,i)) tests)) ((#\a) ; Alphabetic. (cons #`(char-alphabetic? (string-ref s #,i)) tests)) (else ; Literal. (cons #`(eqv? (string-ref s #,i) #,c) tests))))) tests))))))))) ;; "Jan" | "Feb" | "Mar" | "Apr" | "May" | "Jun" ;; "Jul" | "Aug" | "Sep" | "Oct" | "Nov" | "Dec" (define (parse-month str start end) (define (bad) (bad-header-component 'month (substring str start end))) (if (not (= (- end start) 3)) (bad) (let ((a (string-ref str (+ start 0))) (b (string-ref str (+ start 1))) (c (string-ref str (+ start 2)))) (case a ((#\J) (case b ((#\a) (case c ((#\n) 1) (else (bad)))) ((#\u) (case c ((#\n) 6) ((#\l) 7) (else (bad)))) (else (bad)))) ((#\F) (case b ((#\e) (case c ((#\b) 2) (else (bad)))) (else (bad)))) ((#\M) (case b ((#\a) (case c ((#\r) 3) ((#\y) 5) (else (bad)))) (else (bad)))) ((#\A) (case b ((#\p) (case c ((#\r) 4) (else (bad)))) ((#\u) (case c ((#\g) 8) (else (bad)))) (else (bad)))) ((#\S) (case b ((#\e) (case c ((#\p) 9) (else (bad)))) (else (bad)))) ((#\O) (case b ((#\c) (case c ((#\t) 10) (else (bad)))) (else (bad)))) ((#\N) (case b ((#\o) (case c ((#\v) 11) (else (bad)))) (else (bad)))) ((#\D) (case b ((#\e) (case c ((#\c) 12) (else (bad)))) (else (bad)))) (else (bad)))))) ;; "GMT" | "+" 4DIGIT | "-" 4DIGIT ;; ;; RFC 2616 requires date values to use "GMT", but recommends accepting ;; the others as they are commonly generated by e.g. RFC 822 sources. (define (parse-zone-offset str start) (let ((s (substring str start))) (define (bad) (bad-header-component 'zone-offset s)) (cond ((string=? s "GMT") 0) ((string=? s "UTC") 0) ((string-match? s ".dddd") (let ((sign (case (string-ref s 0) ((#\+) +1) ((#\-) -1) (else (bad)))) (hours (parse-non-negative-integer s 1 3)) (minutes (parse-non-negative-integer s 3 5))) (* sign 60 (+ (* 60 hours) minutes)))) ; seconds east of Greenwich (else (bad))))) ;; RFC 822, updated by RFC 1123 ;; ;; Sun, 06 Nov 1994 08:49:37 GMT ;; 01234567890123456789012345678 ;; 0 1 2 (define (parse-rfc-822-date str space zone-offset) ;; We could verify the day of the week but we don't. (cond ((string-match? (substring str 0 space) "aaa, dd aaa dddd dd:dd:dd") (let ((date (parse-non-negative-integer str 5 7)) (month (parse-month str 8 11)) (year (parse-non-negative-integer str 12 16)) (hour (parse-non-negative-integer str 17 19)) (minute (parse-non-negative-integer str 20 22)) (second (parse-non-negative-integer str 23 25))) (make-date 0 second minute hour date month year zone-offset))) ((string-match? (substring str 0 space) "aaa, d aaa dddd dd:dd:dd") (let ((date (parse-non-negative-integer str 5 6)) (month (parse-month str 7 10)) (year (parse-non-negative-integer str 11 15)) (hour (parse-non-negative-integer str 16 18)) (minute (parse-non-negative-integer str 19 21)) (second (parse-non-negative-integer str 22 24))) (make-date 0 second minute hour date month year zone-offset))) ;; The next two clauses match dates that have a space instead of ;; a leading zero for hours, like " 8:49:37". ((string-match? (substring str 0 space) "aaa, dd aaa dddd d:dd:dd") (let ((date (parse-non-negative-integer str 5 7)) (month (parse-month str 8 11)) (year (parse-non-negative-integer str 12 16)) (hour (parse-non-negative-integer str 18 19)) (minute (parse-non-negative-integer str 20 22)) (second (parse-non-negative-integer str 23 25))) (make-date 0 second minute hour date month year zone-offset))) ((string-match? (substring str 0 space) "aaa, d aaa dddd d:dd:dd") (let ((date (parse-non-negative-integer str 5 6)) (month (parse-month str 7 10)) (year (parse-non-negative-integer str 11 15)) (hour (parse-non-negative-integer str 17 18)) (minute (parse-non-negative-integer str 19 21)) (second (parse-non-negative-integer str 22 24))) (make-date 0 second minute hour date month year zone-offset))) (else (bad-header 'date str) ; prevent tail call #f))) ;; RFC 850, updated by RFC 1036 ;; Sunday, 06-Nov-94 08:49:37 GMT ;; 0123456789012345678901 ;; 0 1 2 (define (parse-rfc-850-date str comma space zone-offset) ;; We could verify the day of the week but we don't. (let ((tail (substring str (1+ comma) space))) (if (not (string-match? tail " dd-aaa-dd dd:dd:dd")) (bad-header 'date str)) (let ((date (parse-non-negative-integer tail 1 3)) (month (parse-month tail 4 7)) (year (parse-non-negative-integer tail 8 10)) (hour (parse-non-negative-integer tail 11 13)) (minute (parse-non-negative-integer tail 14 16)) (second (parse-non-negative-integer tail 17 19))) (make-date 0 second minute hour date month (let* ((now (date-year (current-date))) (then (+ now year (- (modulo now 100))))) (cond ((< (+ then 50) now) (+ then 100)) ((< (+ now 50) then) (- then 100)) (else then))) zone-offset)))) ;; ANSI C's asctime() format ;; Sun Nov 6 08:49:37 1994 ;; 012345678901234567890123 ;; 0 1 2 (define (parse-asctime-date str) (if (not (string-match? str "aaa aaa .d dd:dd:dd dddd")) (bad-header 'date str)) (let ((date (parse-non-negative-integer str (if (eqv? (string-ref str 8) #\space) 9 8) 10)) (month (parse-month str 4 7)) (year (parse-non-negative-integer str 20 24)) (hour (parse-non-negative-integer str 11 13)) (minute (parse-non-negative-integer str 14 16)) (second (parse-non-negative-integer str 17 19))) (make-date 0 second minute hour date month year 0))) ;; Convert all date values to GMT time zone, as per RFC 2616 appendix C. (define (normalize-date date) (if (zero? (date-zone-offset date)) date (time-utc->date (date->time-utc date) 0))) (define (parse-date str) (let* ((space (string-rindex str #\space)) (zone-offset (and space (false-if-exception (parse-zone-offset str (1+ space)))))) (normalize-date (if zone-offset (let ((comma (string-index str #\,))) (cond ((not comma) (bad-header 'date str)) ((= comma 3) (parse-rfc-822-date str space zone-offset)) (else (parse-rfc-850-date str comma space zone-offset)))) (parse-asctime-date str))))) (define (write-date date port) (define (display-digits n digits port) (define zero (char->integer #\0)) (let lp ((tens (expt 10 (1- digits)))) (if (> tens 0) (begin (display (integer->char (+ zero (modulo (truncate/ n tens) 10))) port) (lp (floor/ tens 10)))))) (let ((date (if (zero? (date-zone-offset date)) date (time-tai->date (date->time-tai date) 0)))) (display (case (date-week-day date) ((0) "Sun, ") ((1) "Mon, ") ((2) "Tue, ") ((3) "Wed, ") ((4) "Thu, ") ((5) "Fri, ") ((6) "Sat, ") (else (error "bad date" date))) port) (display-digits (date-day date) 2 port) (display (case (date-month date) ((1) " Jan ") ((2) " Feb ") ((3) " Mar ") ((4) " Apr ") ((5) " May ") ((6) " Jun ") ((7) " Jul ") ((8) " Aug ") ((9) " Sep ") ((10) " Oct ") ((11) " Nov ") ((12) " Dec ") (else (error "bad date" date))) port) (display-digits (date-year date) 4 port) (display #\space port) (display-digits (date-hour date) 2 port) (display #\: port) (display-digits (date-minute date) 2 port) (display #\: port) (display-digits (date-second date) 2 port) (display " GMT" port))) ;; Following https://tools.ietf.org/html/rfc7232#section-2.3, an entity ;; tag should really be a qstring. However there are a number of ;; servers that emit etags as unquoted strings. Assume that if the ;; value doesn't start with a quote, it's an unquoted strong etag. (define (parse-entity-tag val) (cond ((string-prefix? "W/" val) (cons (parse-qstring val 2) #f)) ((string-prefix? "\"" val) (cons (parse-qstring val) #t)) (else (cons val #t)))) (define (entity-tag? val) (and (pair? val) (string? (car val)))) (define (write-entity-tag val port) (if (not (cdr val)) (display "W/" port)) (write-qstring (car val) port)) (define* (parse-entity-tag-list val #:optional (start 0) (end (string-length val))) (let ((strong? (not (string-prefix? "W/" val 0 2 start end)))) (call-with-values (lambda () (parse-qstring val (if strong? start (+ start 2)) end #:incremental? #t)) (lambda (tag next) (acons tag strong? (let ((next (skip-whitespace val next end))) (if (< next end) (if (eqv? (string-ref val next) #\,) (parse-entity-tag-list val (skip-whitespace val (1+ next) end) end) (bad-header-component 'entity-tag-list val)) '()))))))) (define (entity-tag-list? val) (list-of? val entity-tag?)) (define (write-entity-tag-list val port) (write-list val port write-entity-tag ", ")) ;; credentials = auth-scheme #auth-param ;; auth-scheme = token ;; auth-param = token "=" ( token | quoted-string ) ;; ;; That's what the spec says. In reality the Basic scheme doesn't have ;; k-v pairs, just one auth token, so we give that token as a string. ;; (define* (parse-credentials str #:optional (val-parser default-val-parser) (start 0) (end (string-length str))) (let* ((start (skip-whitespace str start end)) (delim (or (string-index str char-set:whitespace start end) end))) (if (= start end) (bad-header-component 'authorization str)) (let ((scheme (string->symbol (string-downcase (substring str start (or delim end)))))) (case scheme ((basic) (let* ((start (skip-whitespace str delim end))) (if (< start end) (cons scheme (substring str start end)) (bad-header-component 'credentials str)))) (else (cons scheme (parse-key-value-list str default-val-parser delim end))))))) (define (validate-credentials val) (and (pair? val) (symbol? (car val)) (case (car val) ((basic) (string? (cdr val))) (else (key-value-list? (cdr val)))))) (define (write-credentials val port) (display (car val) port) (display #\space port) (case (car val) ((basic) (display (cdr val) port)) (else (write-key-value-list (cdr val) port)))) ;; challenges = 1#challenge ;; challenge = auth-scheme 1*SP 1#auth-param ;; ;; A pain to parse, as both challenges and auth params are delimited by ;; commas, and qstrings can contain anything. We rely on auth params ;; necessarily having "=" in them. ;; (define* (parse-challenge str #:optional (start 0) (end (string-length str))) (let* ((start (skip-whitespace str start end)) (sp (string-index str #\space start end)) (scheme (if sp (string->symbol (string-downcase (substring str start sp))) (bad-header-component 'challenge str)))) (let lp ((i sp) (out (list scheme))) (if (not (< i end)) (values (reverse! out) end) (let* ((i (skip-whitespace str i end)) (eq (string-index str #\= i end)) (comma (string-index str #\, i end)) (delim (min (or eq end) (or comma end))) (token-end (trim-whitespace str i delim))) (if (string-index str #\space i token-end) (values (reverse! out) i) (let ((k (string->symbol (substring str i token-end)))) (call-with-values (lambda () (if (and eq (or (not comma) (< eq comma))) (let ((i (skip-whitespace str (1+ eq) end))) (if (and (< i end) (eqv? (string-ref str i) #\")) (parse-qstring str i end #:incremental? #t) (values (substring str i (trim-whitespace str i (or comma end))) (or comma end)))) (values #f delim))) (lambda (v next-i) (let ((i (skip-whitespace str next-i end))) (if (or (= i end) (eqv? (string-ref str i) #\,)) (lp (1+ i) (cons (if v (cons k v) k) out)) (bad-header-component 'challenge (substring str start end))))))))))))) (define* (parse-challenges str #:optional (val-parser default-val-parser) (start 0) (end (string-length str))) (let lp ((i start) (ret '())) (let ((i (skip-whitespace str i end))) (if (< i end) (call-with-values (lambda () (parse-challenge str i end)) (lambda (challenge i) (lp i (cons challenge ret)))) (reverse ret))))) (define (validate-challenges val) (list-of? val (lambda (x) (and (pair? x) (symbol? (car x)) (key-value-list? (cdr x)))))) (define (write-challenge val port) (display (car val) port) (display #\space port) (write-key-value-list (cdr val) port)) (define (write-challenges val port) (write-list val port write-challenge ", ")) ;;; ;;; Request-Line and Response-Line ;;; ;; Hmm. (define (bad-request message . args) (throw 'bad-request message args)) (define (bad-response message . args) (throw 'bad-response message args)) (define *known-versions* '()) (define* (parse-http-version str #:optional (start 0) (end (string-length str))) "Parse an HTTP version from STR, returning it as a major–minor pair. For example, ‘HTTP/1.1’ parses as the pair of integers, ‘(1 . 1)’." (or (let lp ((known *known-versions*)) (and (pair? known) (if (string= str (caar known) start end) (cdar known) (lp (cdr known))))) (let ((dot-idx (string-index str #\. start end))) (if (and (string-prefix? "HTTP/" str 0 5 start end) dot-idx (= dot-idx (string-rindex str #\. start end))) (cons (parse-non-negative-integer str (+ start 5) dot-idx) (parse-non-negative-integer str (1+ dot-idx) end)) (bad-header-component 'http-version (substring str start end)))))) (define (write-http-version val port) "Write the given major-minor version pair to PORT." (display "HTTP/" port) (display (car val) port) (display #\. port) (display (cdr val) port)) (for-each (lambda (v) (set! *known-versions* (acons v (parse-http-version v 0 (string-length v)) *known-versions*))) '("HTTP/1.0" "HTTP/1.1")) ;; Request-URI = "*" | absoluteURI | abs_path | authority ;; ;; The `authority' form is only permissible for the CONNECT method, so ;; because we don't expect people to implement CONNECT, we save ;; ourselves the trouble of that case, and disallow the CONNECT method. ;; (define* (parse-http-method str #:optional (start 0) (end (string-length str))) "Parse an HTTP method from STR. The result is an upper-case symbol, like ‘GET’." (cond ((string= str "GET" start end) 'GET) ((string= str "HEAD" start end) 'HEAD) ((string= str "POST" start end) 'POST) ((string= str "PUT" start end) 'PUT) ((string= str "DELETE" start end) 'DELETE) ((string= str "OPTIONS" start end) 'OPTIONS) ((string= str "TRACE" start end) 'TRACE) (else (bad-request "Invalid method: ~a" (substring str start end))))) (define* (parse-request-uri str #:optional (start 0) (end (string-length str))) "Parse a URI from an HTTP request line. Note that URIs in requests do not have to have a scheme or host name. The result is a URI object." (cond ((= start end) (bad-request "Missing Request-URI")) ((string= str "*" start end) #f) ((eqv? (string-ref str start) #\/) (let* ((q (string-index str #\? start end)) (f (string-index str #\# start end)) (q (and q (or (not f) (< q f)) q))) (build-uri 'http #:path (substring str start (or q f end)) #:query (and q (substring str (1+ q) (or f end))) #:fragment (and f (substring str (1+ f) end))))) (else (or (string->uri (substring str start end)) (bad-request "Invalid URI: ~a" (substring str start end)))))) (define (read-request-line port) "Read the first line of an HTTP request from PORT, returning three values: the method, the URI, and the version." (let* ((line (read-header-line port)) (d0 (string-index line char-set:whitespace)) ; "delimiter zero" (d1 (string-rindex line char-set:whitespace))) (if (and d0 d1 (< d0 d1)) (values (parse-http-method line 0 d0) (parse-request-uri line (skip-whitespace line (1+ d0) d1) d1) (parse-http-version line (1+ d1) (string-length line))) (bad-request "Bad Request-Line: ~s" line)))) (define (write-uri uri port) (if (uri-host uri) (begin (display (uri-scheme uri) port) (display "://" port) (if (uri-userinfo uri) (begin (display (uri-userinfo uri) port) (display #\@ port))) (display (uri-host uri) port) (let ((p (uri-port uri))) (if (and p (not (eqv? p 80))) (begin (display #\: port) (display p port)))))) (let* ((path (uri-path uri)) (len (string-length path))) (cond ((and (> len 0) (not (eqv? (string-ref path 0) #\/))) (bad-request "Non-absolute URI path: ~s" path)) ((and (zero? len) (not (uri-host uri))) (bad-request "Empty path and no host for URI: ~s" uri)) (else (display path port)))) (if (uri-query uri) (begin (display #\? port) (display (uri-query uri) port)))) (define (write-request-line method uri version port) "Write the first line of an HTTP request to PORT." (display method port) (display #\space port) (when (http-proxy-port? port) (let ((scheme (uri-scheme uri)) (host (uri-host uri)) (host-port (uri-port uri))) (when (and scheme host) (display scheme port) (display "://" port) (if (string-index host #\:) (begin (display #\[ port) (display host port) (display #\] port)) (display host port)) (unless ((@@ (web uri) default-port?) scheme host-port) (display #\: port) (display host-port port))))) (let ((path (uri-path uri)) (query (uri-query uri))) (if (string-null? path) (display "/" port) (display path port)) (if query (begin (display "?" port) (display query port)))) (display #\space port) (write-http-version version port) (display "\r\n" port)) (define (read-response-line port) "Read the first line of an HTTP response from PORT, returning three values: the HTTP version, the response code, and the (possibly empty) \"reason phrase\"." (let* ((line (read-header-line port)) (d0 (string-index line char-set:whitespace)) ; "delimiter zero" (d1 (and d0 (string-index line char-set:whitespace (skip-whitespace line d0))))) (if (and d0 d1) (values (parse-http-version line 0 d0) (parse-non-negative-integer line (skip-whitespace line d0 d1) d1) (string-trim-both line char-set:whitespace d1)) (bad-response "Bad Response-Line: ~s" line)))) (define (write-response-line version code reason-phrase port) "Write the first line of an HTTP response to PORT." (write-http-version version port) (display #\space port) (display code port) (display #\space port) (display reason-phrase port) (display "\r\n" port)) ;;; ;;; Helpers for declaring headers ;;; ;; emacs: (put 'declare-header! 'scheme-indent-function 1) ;; emacs: (put 'declare-opaque!-header 'scheme-indent-function 1) (define (declare-opaque-header! name) "Declares a given header as \"opaque\", meaning that its value is not treated specially, and is just returned as a plain string." (declare-header! name parse-opaque-string validate-opaque-string write-opaque-string)) ;; emacs: (put 'declare-date-header! 'scheme-indent-function 1) (define (declare-date-header! name) (declare-header! name parse-date date? write-date)) ;; emacs: (put 'declare-string-list-header! 'scheme-indent-function 1) (define (declare-string-list-header! name) (declare-header! name split-and-trim list-of-strings? write-list-of-strings)) ;; emacs: (put 'declare-symbol-list-header! 'scheme-indent-function 1) (define (declare-symbol-list-header! name) (declare-header! name (lambda (str) (map string->symbol (split-and-trim str))) (lambda (v) (list-of? v symbol?)) (lambda (v port) (write-list v port display ", ")))) ;; emacs: (put 'declare-header-list-header! 'scheme-indent-function 1) (define (declare-header-list-header! name) (declare-header! name split-header-names list-of-header-names? write-header-list)) ;; emacs: (put 'declare-integer-header! 'scheme-indent-function 1) (define (declare-integer-header! name) (declare-header! name parse-non-negative-integer non-negative-integer? display)) ;; emacs: (put 'declare-uri-header! 'scheme-indent-function 1) (define (declare-uri-header! name) (declare-header! name (lambda (str) (or (string->uri str) (bad-header-component 'uri str))) (@@ (web uri) absolute-uri?) write-uri)) ;; emacs: (put 'declare-relative-uri-header! 'scheme-indent-function 1) (define (declare-relative-uri-header! name) (declare-header! name (lambda (str) (or ((@@ (web uri) string->uri*) str) (bad-header-component 'uri str))) uri? write-uri)) ;; emacs: (put 'declare-quality-list-header! 'scheme-indent-function 1) (define (declare-quality-list-header! name) (declare-header! name parse-quality-list validate-quality-list write-quality-list)) ;; emacs: (put 'declare-param-list-header! 'scheme-indent-function 1) (define* (declare-param-list-header! name #:optional (val-parser default-val-parser) (val-validator default-val-validator) (val-writer default-val-writer)) (declare-header! name (lambda (str) (parse-param-list str val-parser)) (lambda (val) (validate-param-list val val-validator)) (lambda (val port) (write-param-list val port val-writer)))) ;; emacs: (put 'declare-key-value-list-header! 'scheme-indent-function 1) (define* (declare-key-value-list-header! name #:optional (val-parser default-val-parser) (val-validator default-val-validator) (val-writer default-val-writer)) (declare-header! name (lambda (str) (parse-key-value-list str val-parser)) (lambda (val) (key-value-list? val val-validator)) (lambda (val port) (write-key-value-list val port val-writer)))) ;; emacs: (put 'declare-entity-tag-list-header! 'scheme-indent-function 1) (define (declare-entity-tag-list-header! name) (declare-header! name (lambda (str) (if (string=? str "*") '* (parse-entity-tag-list str))) (lambda (val) (or (eq? val '*) (entity-tag-list? val))) (lambda (val port) (if (eq? val '*) (display "*" port) (write-entity-tag-list val port))))) ;; emacs: (put 'declare-credentials-header! 'scheme-indent-function 1) (define (declare-credentials-header! name) (declare-header! name parse-credentials validate-credentials write-credentials)) ;; emacs: (put 'declare-challenge-list-header! 'scheme-indent-function 1) (define (declare-challenge-list-header! name) (declare-header! name parse-challenges validate-challenges write-challenges)) ;;; ;;; General headers ;;; ;; Cache-Control = 1#(cache-directive) ;; cache-directive = cache-request-directive | cache-response-directive ;; cache-request-directive = ;; "no-cache" ; Section 14.9.1 ;; | "no-store" ; Section 14.9.2 ;; | "max-age" "=" delta-seconds ; Section 14.9.3, 14.9.4 ;; | "max-stale" [ "=" delta-seconds ] ; Section 14.9.3 ;; | "min-fresh" "=" delta-seconds ; Section 14.9.3 ;; | "no-transform" ; Section 14.9.5 ;; | "only-if-cached" ; Section 14.9.4 ;; | cache-extension ; Section 14.9.6 ;; cache-response-directive = ;; "public" ; Section 14.9.1 ;; | "private" [ "=" <"> 1#field-name <"> ] ; Section 14.9.1 ;; | "no-cache" [ "=" <"> 1#field-name <"> ]; Section 14.9.1 ;; | "no-store" ; Section 14.9.2 ;; | "no-transform" ; Section 14.9.5 ;; | "must-revalidate" ; Section 14.9.4 ;; | "proxy-revalidate" ; Section 14.9.4 ;; | "max-age" "=" delta-seconds ; Section 14.9.3 ;; | "s-maxage" "=" delta-seconds ; Section 14.9.3 ;; | cache-extension ; Section 14.9.6 ;; cache-extension = token [ "=" ( token | quoted-string ) ] ;; (declare-key-value-list-header! "Cache-Control" (lambda (k v-str) (case k ((max-age min-fresh s-maxage) (parse-non-negative-integer v-str)) ((max-stale) (and v-str (parse-non-negative-integer v-str))) ((private no-cache) (and v-str (split-header-names v-str))) (else v-str))) (lambda (k v) (case k ((max-age min-fresh s-maxage) (non-negative-integer? v)) ((max-stale) (or (not v) (non-negative-integer? v))) ((private no-cache) (or (not v) (list-of-header-names? v))) ((no-store no-transform only-if-cache must-revalidate proxy-revalidate) (not v)) (else (or (not v) (string? v))))) (lambda (k v port) (cond ((string? v) (default-val-writer k v port)) ((pair? v) (display #\" port) (write-header-list v port) (display #\" port)) ((integer? v) (display v port)) (else (bad-header-component 'cache-control v))))) ;; Connection = "Connection" ":" 1#(connection-token) ;; connection-token = token ;; e.g. ;; Connection: close, Foo-Header ;; (declare-header! "Connection" split-header-names list-of-header-names? (lambda (val port) (write-list val port (lambda (x port) (display (if (eq? x 'close) "close" (header->string x)) port)) ", "))) ;; Date = "Date" ":" HTTP-date ;; e.g. ;; Date: Tue, 15 Nov 1994 08:12:31 GMT ;; (declare-date-header! "Date") ;; Pragma = "Pragma" ":" 1#pragma-directive ;; pragma-directive = "no-cache" | extension-pragma ;; extension-pragma = token [ "=" ( token | quoted-string ) ] ;; (declare-key-value-list-header! "Pragma") ;; Trailer = "Trailer" ":" 1#field-name ;; (declare-header-list-header! "Trailer") ;; Transfer-Encoding = "Transfer-Encoding" ":" 1#transfer-coding ;; (declare-param-list-header! "Transfer-Encoding") ;; Upgrade = "Upgrade" ":" 1#product ;; (declare-string-list-header! "Upgrade") ;; Via = "Via" ":" 1#( received-protocol received-by [ comment ] ) ;; received-protocol = [ protocol-name "/" ] protocol-version ;; protocol-name = token ;; protocol-version = token ;; received-by = ( host [ ":" port ] ) | pseudonym ;; pseudonym = token ;; (declare-header! "Via" split-and-trim list-of-strings? write-list-of-strings #:multiple? #t) ;; Warning = "Warning" ":" 1#warning-value ;; ;; warning-value = warn-code SP warn-agent SP warn-text ;; [SP warn-date] ;; ;; warn-code = 3DIGIT ;; warn-agent = ( host [ ":" port ] ) | pseudonym ;; ; the name or pseudonym of the server adding ;; ; the Warning header, for use in debugging ;; warn-text = quoted-string ;; warn-date = <"> HTTP-date <"> (declare-header! "Warning" (lambda (str) (let ((len (string-length str))) (let lp ((i (skip-whitespace str 0))) (let* ((idx1 (string-index str #\space i)) (idx2 (string-index str #\space (1+ idx1)))) (if (and idx1 idx2) (let ((code (parse-non-negative-integer str i idx1)) (agent (substring str (1+ idx1) idx2))) (call-with-values (lambda () (parse-qstring str (1+ idx2) #:incremental? #t)) (lambda (text i) (call-with-values (lambda () (let ((c (and (< i len) (string-ref str i)))) (case c ((#\space) ;; we have a date. (call-with-values (lambda () (parse-qstring str (1+ i) #:incremental? #t)) (lambda (date i) (values text (parse-date date) i)))) (else (values text #f i))))) (lambda (text date i) (let ((w (list code agent text date)) (c (and (< i len) (string-ref str i)))) (case c ((#f) (list w)) ((#\,) (cons w (lp (skip-whitespace str (1+ i))))) (else (bad-header 'warning str)))))))))))))) (lambda (val) (list-of? val (lambda (elt) (and (list? elt) (= (length elt) 4) (apply (lambda (code host text date) (and (non-negative-integer? code) (< code 1000) (string? host) (string? text) (or (not date) (date? date)))) elt))))) (lambda (val port) (write-list val port (lambda (w port) (apply (lambda (code host text date) (display code port) (display #\space port) (display host port) (display #\space port) (write-qstring text port) (if date (begin (display #\space port) (write-date date port)))) w)) ", ")) #:multiple? #t) ;;; ;;; Entity headers ;;; ;; Allow = #Method ;; (declare-symbol-list-header! "Allow") ;; Content-Disposition = disposition-type *( ";" disposition-parm ) ;; disposition-type = "attachment" | disp-extension-token ;; disposition-parm = filename-parm | disp-extension-parm ;; filename-parm = "filename" "=" quoted-string ;; disp-extension-token = token ;; disp-extension-parm = token "=" ( token | quoted-string ) ;; (declare-header! "Content-Disposition" (lambda (str) (let ((disposition (parse-param-list str default-val-parser))) ;; Lazily reuse the param list parser. (unless (and (pair? disposition) (null? (cdr disposition))) (bad-header-component 'content-disposition str)) (car disposition))) (lambda (val) (and (pair? val) (symbol? (car val)) (list-of? (cdr val) (lambda (x) (and (pair? x) (symbol? (car x)) (string? (cdr x))))))) (lambda (val port) (write-param-list (list val) port))) ;; Content-Encoding = 1#content-coding ;; (declare-symbol-list-header! "Content-Encoding") ;; Content-Language = 1#language-tag ;; (declare-string-list-header! "Content-Language") ;; Content-Length = 1*DIGIT ;; (declare-integer-header! "Content-Length") ;; Content-Location = ( absoluteURI | relativeURI ) ;; (declare-relative-uri-header! "Content-Location") ;; Content-MD5 = ;; (declare-opaque-header! "Content-MD5") ;; Content-Range = content-range-spec ;; content-range-spec = byte-content-range-spec ;; byte-content-range-spec = bytes-unit SP ;; byte-range-resp-spec "/" ;; ( instance-length | "*" ) ;; byte-range-resp-spec = (first-byte-pos "-" last-byte-pos) ;; | "*" ;; instance-length = 1*DIGIT ;; (declare-header! "Content-Range" (lambda (str) (let ((dash (string-index str #\-)) (slash (string-index str #\/))) (if (and (string-prefix? "bytes " str) slash) (list 'bytes (cond (dash (cons (parse-non-negative-integer str 6 dash) (parse-non-negative-integer str (1+ dash) slash))) ((string= str "*" 6 slash) '*) (else (bad-header 'content-range str))) (if (string= str "*" (1+ slash)) '* (parse-non-negative-integer str (1+ slash)))) (bad-header 'content-range str)))) (lambda (val) (and (list? val) (= (length val) 3) (symbol? (car val)) (let ((x (cadr val))) (or (eq? x '*) (and (pair? x) (non-negative-integer? (car x)) (non-negative-integer? (cdr x))))) (let ((x (caddr val))) (or (eq? x '*) (non-negative-integer? x))))) (lambda (val port) (display (car val) port) (display #\space port) (if (eq? (cadr val) '*) (display #\* port) (begin (display (caadr val) port) (display #\- port) (display (caadr val) port))) (if (eq? (caddr val) '*) (display #\* port) (display (caddr val) port)))) ;; Content-Type = media-type ;; (declare-header! "Content-Type" (lambda (str) (let ((parts (string-split str #\;))) (cons (parse-media-type (car parts)) (map (lambda (x) (let ((eq (string-index x #\=))) (if (and eq (= eq (string-rindex x #\=))) (cons (string->symbol (string-trim x char-set:whitespace 0 eq)) (string-trim-right x char-set:whitespace (1+ eq))) (bad-header 'content-type str)))) (cdr parts))))) (lambda (val) (and (pair? val) (symbol? (car val)) (list-of? (cdr val) (lambda (x) (and (pair? x) (symbol? (car x)) (string? (cdr x))))))) (lambda (val port) (display (car val) port) (if (pair? (cdr val)) (begin (display ";" port) (write-list (cdr val) port (lambda (pair port) (display (car pair) port) (display #\= port) (display (cdr pair) port)) ";"))))) ;; Expires = HTTP-date ;; (define *date-in-the-past* (parse-date "Thu, 01 Jan 1970 00:00:00 GMT")) (declare-header! "Expires" (lambda (str) (if (member str '("0" "-1")) *date-in-the-past* (parse-date str))) date? write-date) ;; Last-Modified = HTTP-date ;; (declare-date-header! "Last-Modified") ;;; ;;; Request headers ;;; ;; Accept = #( media-range [ accept-params ] ) ;; media-range = ( "*/*" | ( type "/" "*" ) | ( type "/" subtype ) ) ;; *( ";" parameter ) ;; accept-params = ";" "q" "=" qvalue *( accept-extension ) ;; accept-extension = ";" token [ "=" ( token | quoted-string ) ] ;; (declare-param-list-header! "Accept" ;; -> (type/subtype (sym-prop . str-val) ...) ...) ;; ;; with the exception of prop `q', in which case the val will be a ;; valid quality value ;; (lambda (k v) (if (eq? k 'q) (parse-quality v) v)) (lambda (k v) (if (eq? k 'q) (valid-quality? v) (or (not v) (string? v)))) (lambda (k v port) (if (eq? k 'q) (write-quality v port) (default-val-writer k v port)))) ;; Accept-Charset = 1#( ( charset | "*" )[ ";" "q" "=" qvalue ] ) ;; (declare-quality-list-header! "Accept-Charset") ;; Accept-Encoding = 1#( codings [ ";" "q" "=" qvalue ] ) ;; codings = ( content-coding | "*" ) ;; (declare-quality-list-header! "Accept-Encoding") ;; Accept-Language = 1#( language-range [ ";" "q" "=" qvalue ] ) ;; language-range = ( ( 1*8ALPHA *( "-" 1*8ALPHA ) ) | "*" ) ;; (declare-quality-list-header! "Accept-Language") ;; Authorization = credentials ;; credentials = auth-scheme #auth-param ;; auth-scheme = token ;; auth-param = token "=" ( token | quoted-string ) ;; (declare-credentials-header! "Authorization") ;; Expect = 1#expectation ;; expectation = "100-continue" | expectation-extension ;; expectation-extension = token [ "=" ( token | quoted-string ) ;; *expect-params ] ;; expect-params = ";" token [ "=" ( token | quoted-string ) ] ;; (declare-param-list-header! "Expect") ;; From = mailbox ;; ;; Should be an email address; we just pass on the string as-is. ;; (declare-opaque-header! "From") ;; Host = host [ ":" port ] ;; (declare-header! "Host" (lambda (str) (let* ((rbracket (string-index str #\])) (colon (string-index str #\: (or rbracket 0))) (host (cond (rbracket (unless (eqv? (string-ref str 0) #\[) (bad-header 'host str)) (substring str 1 rbracket)) (colon (substring str 0 colon)) (else str))) (port (and colon (parse-non-negative-integer str (1+ colon))))) (cons host port))) (lambda (val) (and (pair? val) (string? (car val)) (or (not (cdr val)) (non-negative-integer? (cdr val))))) (lambda (val port) (if (string-index (car val) #\:) (begin (display #\[ port) (display (car val) port) (display #\] port)) (display (car val) port)) (if (cdr val) (begin (display #\: port) (display (cdr val) port))))) ;; If-Match = ( "*" | 1#entity-tag ) ;; (declare-entity-tag-list-header! "If-Match") ;; If-Modified-Since = HTTP-date ;; (declare-date-header! "If-Modified-Since") ;; If-None-Match = ( "*" | 1#entity-tag ) ;; (declare-entity-tag-list-header! "If-None-Match") ;; If-Range = ( entity-tag | HTTP-date ) ;; (declare-header! "If-Range" (lambda (str) (if (or (string-prefix? "\"" str) (string-prefix? "W/" str)) (parse-entity-tag str) (parse-date str))) (lambda (val) (or (date? val) (entity-tag? val))) (lambda (val port) (if (date? val) (write-date val port) (write-entity-tag val port)))) ;; If-Unmodified-Since = HTTP-date ;; (declare-date-header! "If-Unmodified-Since") ;; Max-Forwards = 1*DIGIT ;; (declare-integer-header! "Max-Forwards") ;; Proxy-Authorization = credentials ;; (declare-credentials-header! "Proxy-Authorization") ;; Range = "Range" ":" ranges-specifier ;; ranges-specifier = byte-ranges-specifier ;; byte-ranges-specifier = bytes-unit "=" byte-range-set ;; byte-range-set = 1#( byte-range-spec | suffix-byte-range-spec ) ;; byte-range-spec = first-byte-pos "-" [last-byte-pos] ;; first-byte-pos = 1*DIGIT ;; last-byte-pos = 1*DIGIT ;; suffix-byte-range-spec = "-" suffix-length ;; suffix-length = 1*DIGIT ;; (declare-header! "Range" (lambda (str) (if (string-prefix? "bytes=" str) (cons 'bytes (map (lambda (x) (let ((dash (string-index x #\-))) (cond ((not dash) (bad-header 'range str)) ((zero? dash) (cons #f (parse-non-negative-integer x 1))) ((= dash (1- (string-length x))) (cons (parse-non-negative-integer x 0 dash) #f)) (else (cons (parse-non-negative-integer x 0 dash) (parse-non-negative-integer x (1+ dash))))))) (string-split (substring str 6) #\,))) (bad-header 'range str))) (lambda (val) (and (pair? val) (symbol? (car val)) (list-of? (cdr val) (lambda (elt) (and (pair? elt) (let ((x (car elt)) (y (cdr elt))) (and (or x y) (or (not x) (non-negative-integer? x)) (or (not y) (non-negative-integer? y))))))))) (lambda (val port) (display (car val) port) (display #\= port) (write-list (cdr val) port (lambda (pair port) (if (car pair) (display (car pair) port)) (display #\- port) (if (cdr pair) (display (cdr pair) port))) ","))) ;; Referer = ( absoluteURI | relativeURI ) ;; (declare-relative-uri-header! "Referer") ;; TE = #( t-codings ) ;; t-codings = "trailers" | ( transfer-extension [ accept-params ] ) ;; (declare-param-list-header! "TE") ;; User-Agent = 1*( product | comment ) ;; (declare-opaque-header! "User-Agent") ;;; ;;; Reponse headers ;;; ;; Accept-Ranges = acceptable-ranges ;; acceptable-ranges = 1#range-unit | "none" ;; (declare-symbol-list-header! "Accept-Ranges") ;; Age = age-value ;; age-value = delta-seconds ;; (declare-integer-header! "Age") ;; ETag = entity-tag ;; (declare-header! "ETag" parse-entity-tag entity-tag? write-entity-tag) ;; Location = URI-reference ;; (declare-relative-uri-header! "Location") ;; Proxy-Authenticate = 1#challenge ;; (declare-challenge-list-header! "Proxy-Authenticate") ;; Retry-After = ( HTTP-date | delta-seconds ) ;; (declare-header! "Retry-After" (lambda (str) (if (and (not (string-null? str)) (char-numeric? (string-ref str 0))) (parse-non-negative-integer str) (parse-date str))) (lambda (val) (or (date? val) (non-negative-integer? val))) (lambda (val port) (if (date? val) (write-date val port) (display val port)))) ;; Server = 1*( product | comment ) ;; (declare-opaque-header! "Server") ;; Vary = ( "*" | 1#field-name ) ;; (declare-header! "Vary" (lambda (str) (if (equal? str "*") '* (split-header-names str))) (lambda (val) (or (eq? val '*) (list-of-header-names? val))) (lambda (val port) (if (eq? val '*) (display "*" port) (write-header-list val port)))) ;; WWW-Authenticate = 1#challenge ;; (declare-challenge-list-header! "WWW-Authenticate") ;; Chunked Responses (define (read-chunk-header port) "Read a chunk header from PORT and return the size in bytes of the upcoming chunk." (match (read-line port) ((? eof-object?) ;; Connection closed prematurely: there's nothing left to read. 0) (str (let ((extension-start (string-index str (lambda (c) (or (char=? c #\;) (char=? c #\return)))))) (string->number (if extension-start ; unnecessary? (substring str 0 extension-start) str) 16))))) (define* (make-chunked-input-port port #:key (keep-alive? #f)) "Returns a new port which translates HTTP chunked transfer encoded data from PORT into a non-encoded format. Returns eof when it has read the final chunk from PORT. This does not necessarily mean that there is no more data on PORT. When the returned port is closed it will also close PORT, unless the KEEP-ALIVE? is true." (define (close) (unless keep-alive? (close-port port))) (define chunk-size 0) ;size of the current chunk (define remaining 0) ;number of bytes left from the current chunk (define finished? #f) ;did we get all the chunks? (define (read! bv idx to-read) (define (loop to-read num-read) (cond ((or finished? (zero? to-read)) num-read) ((zero? remaining) ;get a new chunk (let ((size (read-chunk-header port))) (set! chunk-size size) (set! remaining size) (if (zero? size) (begin (set! finished? #t) num-read) (loop to-read num-read)))) (else ;read from the current chunk (let* ((ask-for (min to-read remaining)) (read (get-bytevector-n! port bv (+ idx num-read) ask-for))) (if (eof-object? read) (begin ;premature termination (set! finished? #t) num-read) (let ((left (- remaining read))) (set! remaining left) (when (zero? left) ;; We're done with this chunk; read CR and LF. (get-u8 port) (get-u8 port)) (loop (- to-read read) (+ num-read read)))))))) (loop to-read 0)) (make-custom-binary-input-port "chunked input port" read! #f #f close)) (define* (make-chunked-output-port port #:key (keep-alive? #f)) "Returns a new port which translates non-encoded data into a HTTP chunked transfer encoded data and writes this to PORT. Data written to this port is buffered until the port is flushed, at which point it is all sent as one chunk. Take care to close the port when done, as it will output the remaining data, and encode the final zero chunk. When the port is closed it will also close PORT, unless KEEP-ALIVE? is true." (define (q-for-each f q) (while (not (q-empty? q)) (f (deq! q)))) (define queue (make-q)) (define (put-char c) (enq! queue c)) (define (put-string s) (string-for-each (lambda (c) (enq! queue c)) s)) (define (flush) ;; It is important that we do _not_ write a chunk if the queue is ;; empty, since it will be treated as the final chunk. (unless (q-empty? queue) (let ((len (q-length queue))) (display (number->string len 16) port) (display "\r\n" port) (q-for-each (lambda (elem) (write-char elem port)) queue) (display "\r\n" port)))) (define (close) (flush) (display "0\r\n" port) (force-output port) (unless keep-alive? (close-port port))) (make-soft-port (vector put-char put-string flush #f close) "w")) (define %http-proxy-port? (make-object-property)) (define (http-proxy-port? port) (%http-proxy-port? port)) (define (set-http-proxy-port?! port flag) (set! (%http-proxy-port? port) flag)) PK/F2]:~i<i<web/server.scmnu[;;; Web server ;; Copyright (C) 2010, 2011, 2012, 2013, 2015 Free Software Foundation, Inc. ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ;; 02110-1301 USA ;;; Commentary: ;;; ;;; (web server) is a generic web server interface, along with a main ;;; loop implementation for web servers controlled by Guile. ;;; ;;; The lowest layer is the object, which defines a set of ;;; hooks to open a server, read a request from a client, write a ;;; response to a client, and close a server. These hooks -- open, ;;; read, write, and close, respectively -- are bound together in a ;;; object. Procedures in this module take a ;;; object, if needed. ;;; ;;; A may also be looked up by name. If you pass the ;;; `http' symbol to `run-server', Guile looks for a variable named ;;; `http' in the `(web server http)' module, which should be bound to a ;;; object. Such a binding is made by instantiation of ;;; the `define-server-impl' syntax. In this way the run-server loop can ;;; automatically load other backends if available. ;;; ;;; The life cycle of a server goes as follows: ;;; ;;; * The `open' hook is called, to open the server. `open' takes 0 or ;;; more arguments, depending on the backend, and returns an opaque ;;; server socket object, or signals an error. ;;; ;;; * The `read' hook is called, to read a request from a new client. ;;; The `read' hook takes one arguments, the server socket. It ;;; should return three values: an opaque client socket, the ;;; request, and the request body. The request should be a ;;; `' object, from `(web request)'. The body should be a ;;; string or a bytevector, or `#f' if there is no body. ;;; ;;; If the read failed, the `read' hook may return #f for the client ;;; socket, request, and body. ;;; ;;; * A user-provided handler procedure is called, with the request ;;; and body as its arguments. The handler should return two ;;; values: the response, as a `' record from `(web ;;; response)', and the response body as a string, bytevector, or ;;; `#f' if not present. We also allow the reponse to be simply an ;;; alist of headers, in which case a default response object is ;;; constructed with those headers. ;;; ;;; * The `write' hook is called with three arguments: the client ;;; socket, the response, and the body. The `write' hook returns no ;;; values. ;;; ;;; * At this point the request handling is complete. For a loop, we ;;; loop back and try to read a new request. ;;; ;;; * If the user interrupts the loop, the `close' hook is called on ;;; the server socket. ;;; ;;; Code: (define-module (web server) #:use-module (srfi srfi-9) #:use-module (srfi srfi-9 gnu) #:use-module (rnrs bytevectors) #:use-module (ice-9 binary-ports) #:use-module (web request) #:use-module (web response) #:use-module (system repl error-handling) #:use-module (ice-9 control) #:use-module (ice-9 iconv) #:export (define-server-impl lookup-server-impl make-server-impl server-impl? server-impl-name server-impl-open server-impl-read server-impl-write server-impl-close open-server read-client handle-request sanitize-response write-client close-server serve-one-client run-server)) (define *timer* (gettimeofday)) (define (print-elapsed who) (let ((t (gettimeofday))) (pk who (+ (* (- (car t) (car *timer*)) 1000000) (- (cdr t) (cdr *timer*)))) (set! *timer* t))) (eval-when (expand) (define *time-debug?* #f)) (define-syntax debug-elapsed (lambda (x) (syntax-case x () ((_ who) (if *time-debug?* #'(print-elapsed who) #'*unspecified*))))) (define-record-type server-impl (make-server-impl name open read write close) server-impl? (name server-impl-name) (open server-impl-open) (read server-impl-read) (write server-impl-write) (close server-impl-close)) (define-syntax-rule (define-server-impl name open read write close) (define name (make-server-impl 'name open read write close))) (define (lookup-server-impl impl) "Look up a server implementation. If IMPL is a server implementation already, it is returned directly. If it is a symbol, the binding named IMPL in the ‘(web server IMPL)’ module is looked up. Otherwise an error is signaled. Currently a server implementation is a somewhat opaque type, useful only for passing to other procedures in this module, like ‘read-client’." (cond ((server-impl? impl) impl) ((symbol? impl) (let ((impl (module-ref (resolve-module `(web server ,impl)) impl))) (if (server-impl? impl) impl (error "expected a server impl in module" `(web server ,impl))))) (else (error "expected a server-impl or a symbol" impl)))) ;; -> server (define (open-server impl open-params) "Open a server for the given implementation. Return one value, the new server object. The implementation's ‘open’ procedure is applied to OPEN-PARAMS, which should be a list." (apply (server-impl-open impl) open-params)) ;; -> (client request body | #f #f #f) (define (read-client impl server) "Read a new client from SERVER, by applying the implementation's ‘read’ procedure to the server. If successful, return three values: an object corresponding to the client, a request object, and the request body. If any exception occurs, return ‘#f’ for all three values." (call-with-error-handling (lambda () ((server-impl-read impl) server)) #:pass-keys '(quit interrupt) #:on-error (if (batch-mode?) 'backtrace 'debug) #:post-error (lambda _ (values #f #f #f)))) (define (extend-response r k v . additional) (let ((r (set-field r (response-headers) (assoc-set! (copy-tree (response-headers r)) k v)))) (if (null? additional) r (apply extend-response r additional)))) ;; -> response body (define (sanitize-response request response body) "\"Sanitize\" the given response and body, making them appropriate for the given request. As a convenience to web handler authors, RESPONSE may be given as an alist of headers, in which case it is used to construct a default response. Ensures that the response version corresponds to the request version. If BODY is a string, encodes the string to a bytevector, in an encoding appropriate for RESPONSE. Adds a ‘content-length’ and ‘content-type’ header, as necessary. If BODY is a procedure, it is called with a port as an argument, and the output collected as a bytevector. In the future we might try to instead use a compressing, chunk-encoded port, and call this procedure later, in the write-client procedure. Authors are advised not to rely on the procedure being called at any particular time." (cond ((list? response) (sanitize-response request (build-response #:version (request-version request) #:headers response) body)) ((not (equal? (request-version request) (response-version response))) (sanitize-response request (adapt-response-version response (request-version request)) body)) ((not body) (values response #vu8())) ((string? body) (let* ((type (response-content-type response '(text/plain))) (declared-charset (assq-ref (cdr type) 'charset)) (charset (or declared-charset "utf-8"))) (sanitize-response request (if declared-charset response (extend-response response 'content-type `(,@type (charset . ,charset)))) (string->bytevector body charset)))) ((procedure? body) (let* ((type (response-content-type response '(text/plain))) (declared-charset (assq-ref (cdr type) 'charset)) (charset (or declared-charset "utf-8"))) (sanitize-response request (if declared-charset response (extend-response response 'content-type `(,@type (charset . ,charset)))) (call-with-encoded-output-string charset body)))) ((not (bytevector? body)) (error "unexpected body type")) ((and (response-must-not-include-body? response) body ;; FIXME make this stricter: even an empty body should be prohibited. (not (zero? (bytevector-length body)))) (error "response with this status code must not include body" response)) (else ;; check length; assert type; add other required fields? (values (let ((rlen (response-content-length response)) (blen (bytevector-length body))) (cond (rlen (if (= rlen blen) response (error "bad content-length" rlen blen))) (else (extend-response response 'content-length blen)))) (if (eq? (request-method request) 'HEAD) ;; Responses to HEAD requests must not include bodies. ;; We could raise an error here, but it seems more ;; appropriate to just do something sensible. #f body))))) ;; -> response body state (define (handle-request handler request body state) "Handle a given request, returning the response and body. The response and response body are produced by calling the given HANDLER with REQUEST and BODY as arguments. The elements of STATE are also passed to HANDLER as arguments, and may be returned as additional values. The new STATE, collected from the HANDLER's return values, is then returned as a list. The idea is that a server loop receives a handler from the user, along with whatever state values the user is interested in, allowing the user's handler to explicitly manage its state." (call-with-error-handling (lambda () (call-with-values (lambda () (with-stack-and-prompt (lambda () (apply handler request body state)))) (lambda (response body . state) (call-with-values (lambda () (debug-elapsed 'handler) (sanitize-response request response body)) (lambda (response body) (debug-elapsed 'sanitize) (values response body state)))))) #:pass-keys '(quit interrupt) #:on-error (if (batch-mode?) 'backtrace 'debug) #:post-error (lambda _ (values (build-response #:code 500) #f state)))) ;; -> unspecified values (define (write-client impl server client response body) "Write an HTTP response and body to CLIENT. If the server and client support persistent connections, it is the implementation's responsibility to keep track of the client thereafter, presumably by attaching it to the SERVER argument somehow." (call-with-error-handling (lambda () ((server-impl-write impl) server client response body)) #:pass-keys '(quit interrupt) #:on-error (if (batch-mode?) 'backtrace 'debug) #:post-error (lambda _ (values)))) ;; -> unspecified values (define (close-server impl server) "Release resources allocated by a previous invocation of ‘open-server’." ((server-impl-close impl) server)) (define call-with-sigint (if (not (provided? 'posix)) (lambda (thunk handler-thunk) (thunk)) (lambda (thunk handler-thunk) (let ((handler #f)) (catch 'interrupt (lambda () (dynamic-wind (lambda () (set! handler (sigaction SIGINT (lambda (sig) (throw 'interrupt))))) thunk (lambda () (if handler ;; restore Scheme handler, SIG_IGN or SIG_DFL. (sigaction SIGINT (car handler) (cdr handler)) ;; restore original C handler. (sigaction SIGINT #f))))) (lambda (k . _) (handler-thunk))))))) (define (with-stack-and-prompt thunk) (call-with-prompt (default-prompt-tag) (lambda () (start-stack #t (thunk))) (lambda (k proc) (with-stack-and-prompt (lambda () (proc k)))))) ;; -> new-state (define (serve-one-client handler impl server state) "Read one request from SERVER, call HANDLER on the request and body, and write the response to the client. Return the new state produced by the handler procedure." (debug-elapsed 'serve-again) (call-with-values (lambda () (read-client impl server)) (lambda (client request body) (debug-elapsed 'read-client) (if client (call-with-values (lambda () (handle-request handler request body state)) (lambda (response body state) (debug-elapsed 'handle-request) (write-client impl server client response body) (debug-elapsed 'write-client) state)) state)))) (define* (run-server handler #:optional (impl 'http) (open-params '()) . state) "Run Guile's built-in web server. HANDLER should be a procedure that takes two or more arguments, the HTTP request and request body, and returns two or more values, the response and response body. For example, here is a simple \"Hello, World!\" server: @example (define (handler request body) (values '((content-type . (text/plain))) \"Hello, World!\")) (run-server handler) @end example The response and body will be run through ‘sanitize-response’ before sending back to the client. Additional arguments to HANDLER are taken from STATE. Additional return values are accumulated into a new STATE, which will be used for subsequent requests. In this way a handler can explicitly manage its state. The default server implementation is ‘http’, which accepts OPEN-PARAMS like ‘(#:port 8081)’, among others. See \"Web Server\" in the manual, for more information." (let* ((impl (lookup-server-impl impl)) (server (open-server impl open-params))) (call-with-sigint (lambda () (let lp ((state state)) (lp (serve-one-client handler impl server state)))) (lambda () (close-server impl server) (values))))) PK/F2] ,,web/request.scmnu[;;; HTTP request objects ;; Copyright (C) 2010, 2011, 2012 Free Software Foundation, Inc. ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ;; 02110-1301 USA ;;; Code: (define-module (web request) #:use-module (rnrs bytevectors) #:use-module (ice-9 binary-ports) #:use-module (ice-9 rdelim) #:use-module (srfi srfi-9) #:use-module (web uri) #:use-module (web http) #:export (request? request-method request-uri request-version request-headers request-meta request-port read-request build-request write-request read-request-body write-request-body ;; General headers ;; request-cache-control request-connection request-date request-pragma request-trailer request-transfer-encoding request-upgrade request-via request-warning ;; Entity headers ;; request-allow request-content-encoding request-content-language request-content-length request-content-location request-content-md5 request-content-range request-content-type request-expires request-last-modified ;; Request headers ;; request-accept request-accept-charset request-accept-encoding request-accept-language request-authorization request-expect request-from request-host request-if-match request-if-modified-since request-if-none-match request-if-range request-if-unmodified-since request-max-forwards request-proxy-authorization request-range request-referer request-te request-user-agent ;; Misc request-absolute-uri)) ;;; {Character Encodings, Strings, and Bytevectors} ;;; ;;; Requests are read from over the wire, and as such have to be treated ;;; very carefully. ;;; ;;; The header portion of the message is defined to be in a subset of ;;; ASCII, and may be processed either byte-wise (using bytevectors and ;;; binary I/O) or as characters in a single-byte ASCII-compatible ;;; encoding. ;;; ;;; We choose the latter, processing as strings in the latin-1 ;;; encoding. This allows us to use all the read-delimited machinery, ;;; character sets, and regular expressions, shared substrings, etc. ;;; ;;; The characters in the header values may themselves encode other ;;; bytes or characters -- basically each header has its own parser. We ;;; leave that as a header-specific topic. ;;; ;;; The body is present if the content-length header is present. Its ;;; format and, if textual, encoding is determined by the headers, but ;;; its length is encoded in bytes. So we just slurp that number of ;;; characters in latin-1, knowing that the number of characters ;;; corresponds to the number of bytes, and then convert to a ;;; bytevector, perhaps for later decoding. ;;; (define-record-type (make-request method uri version headers meta port) request? (method request-method) (uri request-uri) (version request-version) (headers request-headers) (meta request-meta) (port request-port)) (define (bad-request message . args) (throw 'bad-request message args)) (define (bad-request-printer port key args default-printer) (apply (case-lambda ((msg args) (display "Bad request: " port) (apply format port msg args) (newline port)) (_ (default-printer))) args)) (set-exception-printer! 'bad-request bad-request-printer) (define (non-negative-integer? n) (and (number? n) (>= n 0) (exact? n) (integer? n))) (define (validate-headers headers) (if (pair? headers) (let ((h (car headers))) (if (pair? h) (let ((k (car h)) (v (cdr h))) (if (valid-header? k v) (validate-headers (cdr headers)) (bad-request "Bad value for header ~a: ~s" k v))) (bad-request "Header not a pair: ~a" h))) (if (not (null? headers)) (bad-request "Headers not a list: ~a" headers)))) (define* (build-request uri #:key (method 'GET) (version '(1 . 1)) (headers '()) port (meta '()) (validate-headers? #t)) "Construct an HTTP request object. If VALIDATE-HEADERS? is true, the headers are each run through their respective validators." (let ((needs-host? (and (equal? version '(1 . 1)) (not (assq-ref headers 'host))))) (cond ((not (and (pair? version) (non-negative-integer? (car version)) (non-negative-integer? (cdr version)))) (bad-request "Bad version: ~a" version)) ((not (uri? uri)) (bad-request "Bad uri: ~a" uri)) ((and (not port) (memq method '(POST PUT))) (bad-request "Missing port for message ~a" method)) ((not (list? meta)) (bad-request "Bad metadata alist" meta)) ((and needs-host? (not (uri-host uri))) (bad-request "HTTP/1.1 request without Host header and no host in URI: ~a" uri)) (else (if validate-headers? (validate-headers headers)))) (make-request method uri version (if needs-host? (acons 'host (cons (uri-host uri) (uri-port uri)) headers) headers) meta port))) (define* (read-request port #:optional (meta '())) "Read an HTTP request from PORT, optionally attaching the given metadata, META. As a side effect, sets the encoding on PORT to ISO-8859-1 (latin-1), so that reading one character reads one byte. See the discussion of character sets in \"HTTP Requests\" in the manual, for more information. Note that the body is not part of the request. Once you have read a request, you may read the body separately, and likewise for writing requests." (set-port-encoding! port "ISO-8859-1") (call-with-values (lambda () (read-request-line port)) (lambda (method uri version) (make-request method uri version (read-headers port) meta port)))) ;; FIXME: really return a new request? (define (write-request r port) "Write the given HTTP request to PORT. Return a new request, whose ‘request-port’ will continue writing on PORT, perhaps using some transfer encoding." (write-request-line (request-method r) (request-uri r) (request-version r) port) (write-headers (request-headers r) port) (display "\r\n" port) (if (eq? port (request-port r)) r (make-request (request-method r) (request-uri r) (request-version r) (request-headers r) (request-meta r) port))) (define (read-request-body r) "Reads the request body from R, as a bytevector. Return ‘#f’ if there was no request body." (let ((nbytes (request-content-length r))) (and nbytes (let ((bv (get-bytevector-n (request-port r) nbytes))) (if (= (bytevector-length bv) nbytes) bv (bad-request "EOF while reading request body: ~a bytes of ~a" (bytevector-length bv) nbytes)))))) (define (write-request-body r bv) "Write BV, a bytevector, to the port corresponding to the HTTP request R." (put-bytevector (request-port r) bv)) (define-syntax define-request-accessor (lambda (x) (syntax-case x () ((_ field) #'(define-request-accessor field #f)) ((_ field def) (identifier? #'field) #`(define* (#,(datum->syntax #'field (symbol-append 'request- (syntax->datum #'field))) request #:optional (default def)) (cond ((assq 'field (request-headers request)) => cdr) (else default))))))) ;; General headers ;; (define-request-accessor cache-control '()) (define-request-accessor connection '()) (define-request-accessor date #f) (define-request-accessor pragma '()) (define-request-accessor trailer '()) (define-request-accessor transfer-encoding '()) (define-request-accessor upgrade '()) (define-request-accessor via '()) (define-request-accessor warning '()) ;; Entity headers ;; (define-request-accessor allow '()) (define-request-accessor content-encoding '()) (define-request-accessor content-language '()) (define-request-accessor content-length #f) (define-request-accessor content-location #f) (define-request-accessor content-md5 #f) (define-request-accessor content-range #f) (define-request-accessor content-type #f) (define-request-accessor expires #f) (define-request-accessor last-modified #f) ;; Request headers ;; (define-request-accessor accept '()) (define-request-accessor accept-charset '()) (define-request-accessor accept-encoding '()) (define-request-accessor accept-language '()) (define-request-accessor authorization #f) (define-request-accessor expect '()) (define-request-accessor from #f) (define-request-accessor host #f) ;; Absence of an if-directive appears to be different from `*'. (define-request-accessor if-match #f) (define-request-accessor if-modified-since #f) (define-request-accessor if-none-match #f) (define-request-accessor if-range #f) (define-request-accessor if-unmodified-since #f) (define-request-accessor max-forwards #f) (define-request-accessor proxy-authorization #f) (define-request-accessor range #f) (define-request-accessor referer #f) (define-request-accessor te '()) (define-request-accessor user-agent #f) ;; Misc accessors (define* (request-absolute-uri r #:optional default-host default-port) "A helper routine to determine the absolute URI of a request, using the ‘host’ header and the default host and port." (let ((uri (request-uri r))) (if (uri-host uri) uri (let ((host (or (request-host r) (if default-host (cons default-host default-port) (bad-request "URI not absolute, no Host header, and no default: ~s" uri))))) (build-uri (uri-scheme uri) #:host (car host) #:port (cdr host) #:path (uri-path uri) #:query (uri-query uri) #:fragment (uri-fragment uri)))))) PK/F2]tt statprof.scmnu[;;;; (statprof) -- a statistical profiler for Guile ;;;; -*-scheme-*- ;;;; ;;;; Copyright (C) 2009, 2010, 2011, 2015 Free Software Foundation, Inc. ;;;; Copyright (C) 2004, 2009 Andy Wingo ;;;; Copyright (C) 2001 Rob Browning ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;@code{(statprof)} is intended to be a fairly simple ;;statistical profiler for guile. It is in the early stages yet, so ;;consider its output still suspect, and please report any bugs to ;;@email{guile-devel at gnu.org}, or to me directly at @email{rlb at ;;defaultvalue.org}. ;; ;;A simple use of statprof would look like this: ;; ;;@example ;; (statprof-reset 0 50000 #t) ;; (statprof-start) ;; (do-something) ;; (statprof-stop) ;; (statprof-display) ;;@end example ;; ;;This would reset statprof, clearing all accumulated statistics, then ;;start profiling, run some code, stop profiling, and finally display a ;;gprof flat-style table of statistics which will look something like ;;this: ;; ;;@example ;; % cumulative self self total ;; time seconds seconds calls ms/call ms/call name ;; 35.29 0.23 0.23 2002 0.11 0.11 - ;; 23.53 0.15 0.15 2001 0.08 0.08 positive? ;; 23.53 0.15 0.15 2000 0.08 0.08 + ;; 11.76 0.23 0.08 2000 0.04 0.11 do-nothing ;; 5.88 0.64 0.04 2001 0.02 0.32 loop ;; 0.00 0.15 0.00 1 0.00 150.59 do-something ;; ... ;;@end example ;; ;;All of the numerical data with the exception of the calls column is ;;statistically approximate. In the following column descriptions, and ;;in all of statprof, "time" refers to execution time (both user and ;;system), not wall clock time. ;; ;;@table @asis ;;@item % time ;;The percent of the time spent inside the procedure itself ;;(not counting children). ;;@item cumulative seconds ;;The total number of seconds spent in the procedure, including ;;children. ;;@item self seconds ;;The total number of seconds spent in the procedure itself (not counting ;;children). ;;@item calls ;;The total number of times the procedure was called. ;;@item self ms/call ;;The average time taken by the procedure itself on each call, in ms. ;;@item total ms/call ;;The average time taken by each call to the procedure, including time ;;spent in child functions. ;;@item name ;;The name of the procedure. ;;@end table ;; ;;The profiler uses @code{eq?} and the procedure object itself to ;;identify the procedures, so it won't confuse different procedures with ;;the same name. They will show up as two different rows in the output. ;; ;;Right now the profiler is quite simplistic. I cannot provide ;;call-graphs or other higher level information. What you see in the ;;table is pretty much all there is. Patches are welcome :-) ;; ;;@section Implementation notes ;; ;;The profiler works by setting the unix profiling signal ;;@code{ITIMER_PROF} to go off after the interval you define in the call ;;to @code{statprof-reset}. When the signal fires, a sampling routine is ;;run which looks at the current procedure that's executing, and then ;;crawls up the stack, and for each procedure encountered, increments ;;that procedure's sample count. Note that if a procedure is encountered ;;multiple times on a given stack, it is only counted once. After the ;;sampling is complete, the profiler resets profiling timer to fire ;;again after the appropriate interval. ;; ;;Meanwhile, the profiler keeps track, via @code{get-internal-run-time}, ;;how much CPU time (system and user -- which is also what ;;@code{ITIMER_PROF} tracks), has elapsed while code has been executing ;;within a statprof-start/stop block. ;; ;;The profiler also tries to avoid counting or timing its own code as ;;much as possible. ;; ;;; Code: ;; When you add new features, please also add tests to ./tests/ if you ;; have time, and then add the new files to ./run-tests. Also, if ;; anyone's bored, there are a lot of existing API bits that don't ;; have tests yet. ;; TODO ;; ;; Check about profiling C functions -- does profiling primitives work? ;; Also look into stealing code from qprof so we can sample the C stack ;; Call graphs? (define-module (statprof) #:use-module (srfi srfi-1) #:autoload (ice-9 format) (format) #:use-module (system vm vm) #:use-module (system vm frame) #:use-module (system vm program) #:export (statprof-active? statprof-start statprof-stop statprof-reset statprof-accumulated-time statprof-sample-count statprof-fold-call-data statprof-proc-call-data statprof-call-data-name statprof-call-data-calls statprof-call-data-cum-samples statprof-call-data-self-samples statprof-call-data->stats statprof-stats-proc-name statprof-stats-%-time-in-proc statprof-stats-cum-secs-in-proc statprof-stats-self-secs-in-proc statprof-stats-calls statprof-stats-self-secs-per-call statprof-stats-cum-secs-per-call statprof-display statprof-display-anomolies statprof-fetch-stacks statprof-fetch-call-tree statprof with-statprof gcprof)) ;; This profiler tracks two numbers for every function called while ;; it's active. It tracks the total number of calls, and the number ;; of times the function was active when the sampler fired. ;; ;; Globally the profiler tracks the total time elapsed and the number ;; of times the sampler was fired. ;; ;; Right now, this profiler is not per-thread and is not thread safe. (define accumulated-time #f) ; total so far. (define last-start-time #f) ; start-time when timer is active. (define sample-count #f) ; total count of sampler calls. (define sampling-frequency #f) ; in (seconds . microseconds) (define remaining-prof-time #f) ; time remaining when prof suspended. (define profile-level 0) ; for user start/stop nesting. (define %count-calls? #t) ; whether to catch apply-frame. (define gc-time-taken 0) ; gc time between statprof-start and ; statprof-stop. (define record-full-stacks? #f) ; if #t, stash away the stacks ; for later analysis. (define stacks '()) ;; procedure-data will be a hash where the key is the function object ;; itself and the value is the data. The data will be a vector like ;; this: #(name call-count cum-sample-count self-sample-count) (define procedure-data #f) ;; If you change the call-data data structure, you need to also change ;; sample-uncount-frame. (define (make-call-data proc call-count cum-sample-count self-sample-count) (vector proc call-count cum-sample-count self-sample-count)) (define (call-data-proc cd) (vector-ref cd 0)) (define (call-data-name cd) (procedure-name (call-data-proc cd))) (define (call-data-printable cd) (or (call-data-name cd) (with-output-to-string (lambda () (write (call-data-proc cd)))))) (define (call-data-call-count cd) (vector-ref cd 1)) (define (call-data-cum-sample-count cd) (vector-ref cd 2)) (define (call-data-self-sample-count cd) (vector-ref cd 3)) (define (inc-call-data-call-count! cd) (vector-set! cd 1 (1+ (vector-ref cd 1)))) (define (inc-call-data-cum-sample-count! cd) (vector-set! cd 2 (1+ (vector-ref cd 2)))) (define (inc-call-data-self-sample-count! cd) (vector-set! cd 3 (1+ (vector-ref cd 3)))) (define-macro (accumulate-time stop-time) `(set! accumulated-time (+ accumulated-time 0.0 (- ,stop-time last-start-time)))) (define (get-call-data proc) (let ((k (if (or (not (program? proc)) (zero? (program-num-free-variables proc))) proc (program-objcode proc)))) (or (hashq-ref procedure-data k) (let ((call-data (make-call-data proc 0 0 0))) (hashq-set! procedure-data k call-data) call-data)))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; SIGPROF handler (define (sample-stack-procs stack) (let ((stacklen (stack-length stack)) (hit-count-call? #f)) (if record-full-stacks? (set! stacks (cons stack stacks))) (set! sample-count (+ sample-count 1)) ;; Now accumulate stats for the whole stack. (let loop ((frame (stack-ref stack 0)) (procs-seen (make-hash-table 13)) (self #f)) (cond ((not frame) (hash-fold (lambda (proc val accum) (inc-call-data-cum-sample-count! (get-call-data proc))) #f procs-seen) (and=> (and=> self get-call-data) inc-call-data-self-sample-count!)) ((frame-procedure frame) => (lambda (proc) (cond ((eq? proc count-call) ;; We're not supposed to be sampling count-call and ;; its sub-functions, so loop again with a clean ;; slate. (set! hit-count-call? #t) (loop (frame-previous frame) (make-hash-table 13) #f)) (else (hashq-set! procs-seen proc #t) (loop (frame-previous frame) procs-seen (or self proc)))))) (else (loop (frame-previous frame) procs-seen self)))) hit-count-call?)) (define inside-profiler? #f) (define (profile-signal-handler sig) (set! inside-profiler? #t) ;; FIXME: with-statprof should be able to set an outer frame for the ;; stack cut (if (positive? profile-level) (let* ((stop-time (get-internal-run-time)) ;; cut down to the signal handler. note that this will only ;; work if statprof.scm is compiled; otherwise we get ;; `eval' on the stack instead, because if it's not ;; compiled, profile-signal-handler is a thunk that ;; tail-calls eval. perhaps we should always compile the ;; signal handler instead... (stack (or (make-stack #t profile-signal-handler) (pk 'what! (make-stack #t)))) (inside-apply-trap? (sample-stack-procs stack))) (if (not inside-apply-trap?) (begin ;; disabling here is just a little more efficient, but ;; not necessary given inside-profiler?. We can't just ;; disable unconditionally at the top of this function ;; and eliminate inside-profiler? because it seems to ;; confuse guile wrt re-enabling the trap when ;; count-call finishes. (if %count-calls? (set-vm-trace-level! (the-vm) (1- (vm-trace-level (the-vm))))) (accumulate-time stop-time))) (setitimer ITIMER_PROF 0 0 (car sampling-frequency) (cdr sampling-frequency)) (if (not inside-apply-trap?) (begin (set! last-start-time (get-internal-run-time)) (if %count-calls? (set-vm-trace-level! (the-vm) (1+ (vm-trace-level (the-vm))))))))) (set! inside-profiler? #f)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; Count total calls. (define (count-call frame) (if (not inside-profiler?) (begin (accumulate-time (get-internal-run-time)) (and=> (frame-procedure frame) (lambda (proc) (inc-call-data-call-count! (get-call-data proc)))) (set! last-start-time (get-internal-run-time))))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (define (statprof-active?) "Returns @code{#t} if @code{statprof-start} has been called more times than @code{statprof-stop}, @code{#f} otherwise." (positive? profile-level)) ;; Do not call this from statprof internal functions -- user only. (define (statprof-start) "Start the profiler.@code{}" ;; After some head-scratching, I don't *think* I need to mask/unmask ;; signals here, but if I'm wrong, please let me know. (set! profile-level (+ profile-level 1)) (if (= profile-level 1) (let* ((rpt remaining-prof-time) (use-rpt? (and rpt (or (positive? (car rpt)) (positive? (cdr rpt)))))) (set! remaining-prof-time #f) (set! last-start-time (get-internal-run-time)) (set! gc-time-taken (cdr (assq 'gc-time-taken (gc-stats)))) (if use-rpt? (setitimer ITIMER_PROF 0 0 (car rpt) (cdr rpt)) (setitimer ITIMER_PROF 0 0 (car sampling-frequency) (cdr sampling-frequency))) (if %count-calls? (add-hook! (vm-apply-hook (the-vm)) count-call)) (set-vm-trace-level! (the-vm) (1+ (vm-trace-level (the-vm)))) #t))) ;; Do not call this from statprof internal functions -- user only. (define (statprof-stop) "Stop the profiler.@code{}" ;; After some head-scratching, I don't *think* I need to mask/unmask ;; signals here, but if I'm wrong, please let me know. (set! profile-level (- profile-level 1)) (if (zero? profile-level) (begin (set! gc-time-taken (- (cdr (assq 'gc-time-taken (gc-stats))) gc-time-taken)) (set-vm-trace-level! (the-vm) (1- (vm-trace-level (the-vm)))) (if %count-calls? (remove-hook! (vm-apply-hook (the-vm)) count-call)) ;; I believe that we need to do this before getting the time ;; (unless we want to make things even more complicated). (set! remaining-prof-time (setitimer ITIMER_PROF 0 0 0 0)) (accumulate-time (get-internal-run-time)) (set! last-start-time #f)))) (define* (statprof-reset sample-seconds sample-microseconds count-calls? #:optional full-stacks?) "Reset the statprof sampler interval to @var{sample-seconds} and @var{sample-microseconds}. If @var{count-calls?} is true, arrange to instrument procedure calls as well as collecting statistical profiling data. If @var{full-stacks?} is true, collect all sampled stacks into a list for later analysis. Enables traps and debugging as necessary." (if (positive? profile-level) (error "Can't reset profiler while profiler is running.")) (set! %count-calls? count-calls?) (set! accumulated-time 0) (set! last-start-time #f) (set! sample-count 0) (set! sampling-frequency (cons sample-seconds sample-microseconds)) (set! remaining-prof-time #f) (set! procedure-data (make-hash-table 131)) (set! record-full-stacks? full-stacks?) (set! stacks '()) (sigaction SIGPROF profile-signal-handler) #t) (define (statprof-fold-call-data proc init) "Fold @var{proc} over the call-data accumulated by statprof. Cannot be called while statprof is active. @var{proc} should take two arguments, @code{(@var{call-data} @var{prior-result})}. Note that a given proc-name may appear multiple times, but if it does, it represents different functions with the same name." (if (positive? profile-level) (error "Can't call statprof-fold-called while profiler is running.")) (hash-fold (lambda (key value prior-result) (proc value prior-result)) init procedure-data)) (define (statprof-proc-call-data proc) "Returns the call-data associated with @var{proc}, or @code{#f} if none is available." (if (positive? profile-level) (error "Can't call statprof-fold-called while profiler is running.")) (hashq-ref procedure-data proc)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; Stats (define (statprof-call-data->stats call-data) "Returns an object of type @code{statprof-stats}." ;; returns (vector proc-name ;; %-time-in-proc ;; cum-seconds-in-proc ;; self-seconds-in-proc ;; num-calls ;; self-secs-per-call ;; total-secs-per-call) (let* ((proc-name (call-data-printable call-data)) (self-samples (call-data-self-sample-count call-data)) (cum-samples (call-data-cum-sample-count call-data)) (all-samples (statprof-sample-count)) (secs-per-sample (/ (statprof-accumulated-time) (statprof-sample-count))) (num-calls (and %count-calls? (statprof-call-data-calls call-data)))) (vector proc-name (* (/ self-samples all-samples) 100.0) (* cum-samples secs-per-sample 1.0) (* self-samples secs-per-sample 1.0) num-calls (and num-calls ;; maybe we only sampled in children (if (zero? self-samples) 0.0 (/ (* self-samples secs-per-sample) 1.0 num-calls))) (and num-calls ;; cum-samples must be positive (/ (* cum-samples secs-per-sample) 1.0 ;; num-calls might be 0 if we entered statprof during the ;; dynamic extent of the call (max num-calls 1)))))) (define (statprof-stats-proc-name stats) (vector-ref stats 0)) (define (statprof-stats-%-time-in-proc stats) (vector-ref stats 1)) (define (statprof-stats-cum-secs-in-proc stats) (vector-ref stats 2)) (define (statprof-stats-self-secs-in-proc stats) (vector-ref stats 3)) (define (statprof-stats-calls stats) (vector-ref stats 4)) (define (statprof-stats-self-secs-per-call stats) (vector-ref stats 5)) (define (statprof-stats-cum-secs-per-call stats) (vector-ref stats 6)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (define (stats-sorter x y) (let ((diff (- (statprof-stats-self-secs-in-proc x) (statprof-stats-self-secs-in-proc y)))) (positive? (if (= diff 0) (- (statprof-stats-cum-secs-in-proc x) (statprof-stats-cum-secs-in-proc y)) diff)))) (define (statprof-display . port) "Displays a gprof-like summary of the statistics collected. Unless an optional @var{port} argument is passed, uses the current output port." (if (null? port) (set! port (current-output-port))) (cond ((zero? (statprof-sample-count)) (format port "No samples recorded.\n")) (else (let* ((stats-list (statprof-fold-call-data (lambda (data prior-value) (cons (statprof-call-data->stats data) prior-value)) '())) (sorted-stats (sort stats-list stats-sorter))) (define (display-stats-line stats) (if %count-calls? (format port "~6,2f ~9,2f ~9,2f ~7d ~8,2f ~8,2f " (statprof-stats-%-time-in-proc stats) (statprof-stats-cum-secs-in-proc stats) (statprof-stats-self-secs-in-proc stats) (statprof-stats-calls stats) (* 1000 (statprof-stats-self-secs-per-call stats)) (* 1000 (statprof-stats-cum-secs-per-call stats))) (format port "~6,2f ~9,2f ~9,2f " (statprof-stats-%-time-in-proc stats) (statprof-stats-cum-secs-in-proc stats) (statprof-stats-self-secs-in-proc stats))) (display (statprof-stats-proc-name stats) port) (newline port)) (if %count-calls? (begin (format port "~5a ~10a ~7a ~8a ~8a ~8a ~8@a\n" "% " "cumulative" "self" "" "self" "total" "") (format port "~5a ~9a ~8a ~8a ~8a ~8a ~8@a\n" "time" "seconds" "seconds" "calls" "ms/call" "ms/call" "name")) (begin (format port "~5a ~10a ~7a ~8@a\n" "%" "cumulative" "self" "") (format port "~5a ~10a ~7a ~8@a\n" "time" "seconds" "seconds" "name"))) (for-each display-stats-line sorted-stats) (display "---\n" port) (simple-format #t "Sample count: ~A\n" (statprof-sample-count)) (simple-format #t "Total time: ~A seconds (~A seconds in GC)\n" (statprof-accumulated-time) (/ gc-time-taken 1.0 internal-time-units-per-second)))))) (define (statprof-display-anomolies) "A sanity check that attempts to detect anomolies in statprof's statistics.@code{}" (statprof-fold-call-data (lambda (data prior-value) (if (and %count-calls? (zero? (call-data-call-count data)) (positive? (call-data-cum-sample-count data))) (simple-format #t "==[~A ~A ~A]\n" (call-data-name data) (call-data-call-count data) (call-data-cum-sample-count data)))) #f) (simple-format #t "Total time: ~A\n" (statprof-accumulated-time)) (simple-format #t "Sample count: ~A\n" (statprof-sample-count))) (define (statprof-accumulated-time) "Returns the time accumulated during the last statprof run.@code{}" (if (positive? profile-level) (error "Can't get accumulated time while profiler is running.")) (/ accumulated-time internal-time-units-per-second)) (define (statprof-sample-count) "Returns the number of samples taken during the last statprof run.@code{}" (if (positive? profile-level) (error "Can't get accumulated time while profiler is running.")) sample-count) (define statprof-call-data-name call-data-name) (define statprof-call-data-calls call-data-call-count) (define statprof-call-data-cum-samples call-data-cum-sample-count) (define statprof-call-data-self-samples call-data-self-sample-count) (define (statprof-fetch-stacks) "Returns a list of stacks, as they were captured since the last call to @code{statprof-reset}. Note that stacks are only collected if the @var{full-stacks?} argument to @code{statprof-reset} is true." stacks) (define procedure=? (lambda (a b) (cond ((eq? a b)) ((and (program? a) (program? b)) (eq? (program-objcode a) (program-objcode b))) (else #f)))) ;; tree ::= (car n . tree*) (define (lists->trees lists equal?) (let lp ((in lists) (n-terminal 0) (tails '())) (cond ((null? in) (let ((trees (map (lambda (tail) (cons (car tail) (lists->trees (cdr tail) equal?))) tails))) (cons (apply + n-terminal (map cadr trees)) (sort trees (lambda (a b) (> (cadr a) (cadr b))))))) ((null? (car in)) (lp (cdr in) (1+ n-terminal) tails)) ((find (lambda (x) (equal? (car x) (caar in))) tails) => (lambda (tail) (lp (cdr in) n-terminal (assq-set! tails (car tail) (cons (cdar in) (cdr tail)))))) (else (lp (cdr in) n-terminal (acons (caar in) (list (cdar in)) tails)))))) (define (stack->procedures stack) (filter identity (unfold-right (lambda (x) (not x)) frame-procedure frame-previous (stack-ref stack 0)))) (define (statprof-fetch-call-tree) "Return a call tree for the previous statprof run. The return value is a list of nodes, each of which is of the type: @code node ::= (@var{proc} @var{count} . @var{nodes}) @end code" (cons #t (lists->trees (map stack->procedures stacks) procedure=?))) (define* (statprof thunk #:key (loop 1) (hz 100) (count-calls? #f) (full-stacks? #f)) "Profile the execution of @var{thunk}, and return its return values. The stack will be sampled @var{hz} times per second, and the thunk itself will be called @var{loop} times. If @var{count-calls?} is true, all procedure calls will be recorded. This operation is somewhat expensive. If @var{full-stacks?} is true, at each sample, statprof will store away the whole call tree, for later analysis. Use @code{statprof-fetch-stacks} or @code{statprof-fetch-call-tree} to retrieve the last-stored stacks." (dynamic-wind (lambda () (statprof-reset (inexact->exact (floor (/ 1 hz))) (inexact->exact (* 1e6 (- (/ 1 hz) (floor (/ 1 hz))))) count-calls? full-stacks?) (statprof-start)) (lambda () (let lp ((i loop) (result '())) (if (zero? i) (apply values result) (call-with-values thunk (lambda result (lp (1- i) result)))))) (lambda () (statprof-stop) (statprof-display) (set! procedure-data #f)))) (define-macro (with-statprof . args) "Profile the expressions in the body, and return the body's return values. Keyword arguments: @table @code @item #:loop Execute the body @var{loop} number of times, or @code{#f} for no looping default: @code{#f} @item #:hz Sampling rate default: @code{20} @item #:count-calls? Whether to instrument each function call (expensive) default: @code{#f} @item #:full-stacks? Whether to collect away all sampled stacks into a list default: @code{#f} @end table" (define (kw-arg-ref kw args def) (cond ((null? args) (error "Invalid macro body")) ((keyword? (car args)) (if (eq? (car args) kw) (cadr args) (kw-arg-ref kw (cddr args) def))) ((eq? kw #f def) ;; asking for the body args) (else def))) ;; kw not found `((@ (statprof) statprof) (lambda () ,@(kw-arg-ref #f args #f)) #:loop ,(kw-arg-ref #:loop args 1) #:hz ,(kw-arg-ref #:hz args 100) #:count-calls? ,(kw-arg-ref #:count-calls? args #f) #:full-stacks? ,(kw-arg-ref #:full-stacks? args #f))) (define* (gcprof thunk #:key (loop 1) (full-stacks? #f)) "Do an allocation profile of the execution of @var{thunk}. The stack will be sampled soon after every garbage collection, yielding an approximate idea of what is causing allocation in your program. Since GC does not occur very frequently, you may need to use the @var{loop} parameter, to cause @var{thunk} to be called @var{loop} times. If @var{full-stacks?} is true, at each sample, statprof will store away the whole call tree, for later analysis. Use @code{statprof-fetch-stacks} or @code{statprof-fetch-call-tree} to retrieve the last-stored stacks." (define (reset) (if (positive? profile-level) (error "Can't reset profiler while profiler is running.")) (set! accumulated-time 0) (set! last-start-time #f) (set! sample-count 0) (set! %count-calls? #f) (set! procedure-data (make-hash-table 131)) (set! record-full-stacks? full-stacks?) (set! stacks '())) (define (gc-callback) (cond (inside-profiler?) (else (set! inside-profiler? #t) ;; FIXME: should be able to set an outer frame for the stack cut (let ((stop-time (get-internal-run-time)) ;; Cut down to gc-callback, and then one before (the ;; after-gc async). See the note in profile-signal-handler ;; also. (stack (or (make-stack #t gc-callback 0 1) (pk 'what! (make-stack #t))))) (sample-stack-procs stack) (accumulate-time stop-time) (set! last-start-time (get-internal-run-time))) (set! inside-profiler? #f)))) (define (start) (set! profile-level (+ profile-level 1)) (if (= profile-level 1) (begin (set! remaining-prof-time #f) (set! last-start-time (get-internal-run-time)) (set! gc-time-taken (cdr (assq 'gc-time-taken (gc-stats)))) (add-hook! after-gc-hook gc-callback) (set-vm-trace-level! (the-vm) (1+ (vm-trace-level (the-vm)))) #t))) (define (stop) (set! profile-level (- profile-level 1)) (if (zero? profile-level) (begin (set! gc-time-taken (- (cdr (assq 'gc-time-taken (gc-stats))) gc-time-taken)) (remove-hook! after-gc-hook gc-callback) (accumulate-time (get-internal-run-time)) (set! last-start-time #f)))) (dynamic-wind (lambda () (reset) (start)) (lambda () (let lp ((i loop)) (if (not (zero? i)) (begin (thunk) (lp (1- i)))))) (lambda () (stop) (statprof-display) (set! procedure-data #f)))) PK/F2]WFrnrs/mutable-pairs.scmnu[;;; mutable-pairs.scm --- The R6RS mutable pair library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs mutable-pairs (6)) (export set-car! set-cdr!) (import (only (guile) set-car! set-cdr!))) PK/F2])Jn7 7 rnrs/bytevectors.scmnu[;;;; bytevectors.scm --- R6RS bytevector API -*- coding: utf-8 -*- ;;;; Copyright (C) 2009, 2010 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Ludovic Courtès ;;; Commentary: ;;; ;;; A "bytevector" is a raw bit string. This module provides procedures to ;;; manipulate bytevectors and interpret their contents in a number of ways: ;;; bytevector contents can be accessed as signed or unsigned integer of ;;; various sizes and endianness, as IEEE-754 floating point numbers, or as ;;; strings. It is a useful tool to decode binary data. ;;; ;;; Code: (define-module (rnrs bytevectors) #:version (6) #:export-syntax (endianness) #:export (native-endianness bytevector? make-bytevector bytevector-length bytevector=? bytevector-fill! bytevector-copy! bytevector-copy uniform-array->bytevector bytevector-u8-ref bytevector-s8-ref bytevector-u8-set! bytevector-s8-set! bytevector->u8-list u8-list->bytevector bytevector-uint-ref bytevector-uint-set! bytevector-sint-ref bytevector-sint-set! bytevector->sint-list bytevector->uint-list uint-list->bytevector sint-list->bytevector bytevector-u16-ref bytevector-s16-ref bytevector-u16-set! bytevector-s16-set! bytevector-u16-native-ref bytevector-s16-native-ref bytevector-u16-native-set! bytevector-s16-native-set! bytevector-u32-ref bytevector-s32-ref bytevector-u32-set! bytevector-s32-set! bytevector-u32-native-ref bytevector-s32-native-ref bytevector-u32-native-set! bytevector-s32-native-set! bytevector-u64-ref bytevector-s64-ref bytevector-u64-set! bytevector-s64-set! bytevector-u64-native-ref bytevector-s64-native-ref bytevector-u64-native-set! bytevector-s64-native-set! bytevector-ieee-single-ref bytevector-ieee-single-set! bytevector-ieee-single-native-ref bytevector-ieee-single-native-set! bytevector-ieee-double-ref bytevector-ieee-double-set! bytevector-ieee-double-native-ref bytevector-ieee-double-native-set! string->utf8 string->utf16 string->utf32 utf8->string utf16->string utf32->string)) (load-extension (string-append "libguile-" (effective-version)) "scm_init_bytevectors") (define-macro (endianness sym) (if (memq sym '(big little)) `(quote ,sym) (error "unsupported endianness" sym))) ;;; bytevector.scm ends here PK/F2]rgrnrs/control.scmnu[;;; control.scm --- The R6RS control structures library ;; Copyright (C) 2010, 2012 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs control (6)) (export when unless do case-lambda) (import (only (guile) when unless do case-lambda))) PK/F2]dyBWWrnrs/enums.scmnu[;;; enums.scm --- The R6RS enumerations library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs enums (6)) (export make-enumeration enum-set-universe enum-set-indexer enum-set-constructor enum-set->list enum-set-member? enum-set-subset? enum-set=? enum-set-union enum-set-intersection enum-set-difference enum-set-complement enum-set-projection define-enumeration) (import (only (guile) and=>) (rnrs base (6)) (rnrs conditions (6)) (rnrs exceptions (6)) (rnrs records procedural (6)) (rnrs syntax-case (6)) (srfi :1)) (define enum-set-rtd (make-record-type-descriptor 'enum-set #f #f #f #f '#((mutable universe) (immutable set)))) (define make-enum-set (record-constructor (make-record-constructor-descriptor enum-set-rtd #f #f))) (define enum-set-universe-internal (record-accessor enum-set-rtd 0)) (define enum-set-universe-set! (record-mutator enum-set-rtd 0)) (define enum-set-set (record-accessor enum-set-rtd 1)) (define (make-enumeration symbol-list) (let ((es (make-enum-set #f symbol-list))) (enum-set-universe-set! es es))) (define (enum-set-universe enum-set) (or (enum-set-universe-internal enum-set) enum-set)) (define (enum-set-indexer enum-set) (let* ((symbols (enum-set->list (enum-set-universe enum-set))) (cardinality (length symbols))) (lambda (x) (and=> (memq x symbols) (lambda (probe) (- cardinality (length probe))))))) (define (enum-set-constructor enum-set) (lambda (symbol-list) (make-enum-set (enum-set-universe enum-set) (list-copy symbol-list)))) (define (enum-set->list enum-set) (lset-intersection eq? (enum-set-set (enum-set-universe enum-set)) (enum-set-set enum-set))) (define (enum-set-member? symbol enum-set) (and (memq symbol (enum-set-set enum-set)) #t)) (define (enum-set-subset? enum-set-1 enum-set-2) (and (lset<= eq? (enum-set-set (enum-set-universe enum-set-1)) (enum-set-set (enum-set-universe enum-set-2))) (lset<= eq? (enum-set-set enum-set-1) (enum-set-set enum-set-2)))) (define (enum-set=? enum-set-1 enum-set-2) (and (enum-set-subset? enum-set-1 enum-set-2) (enum-set-subset? enum-set-2 enum-set-1))) (define (enum-set-union enum-set-1 enum-set-2) (if (equal? (enum-set-universe enum-set-1) (enum-set-universe enum-set-2)) (make-enum-set (enum-set-universe enum-set-1) (lset-union eq? (enum-set-set enum-set-1) (enum-set-set enum-set-2))) (raise (make-assertion-violation)))) (define (enum-set-intersection enum-set-1 enum-set-2) (if (equal? (enum-set-universe enum-set-1) (enum-set-universe enum-set-2)) (make-enum-set (enum-set-universe enum-set-1) (lset-intersection eq? (enum-set-set enum-set-1) (enum-set-set enum-set-2))) (raise (make-assertion-violation)))) (define (enum-set-difference enum-set-1 enum-set-2) (if (equal? (enum-set-universe enum-set-1) (enum-set-universe enum-set-2)) (make-enum-set (enum-set-universe enum-set-1) (lset-difference eq? (enum-set-set enum-set-1) (enum-set-set enum-set-2))) (raise (make-assertion-violation)))) (define (enum-set-complement enum-set) (let ((universe (enum-set-universe enum-set))) (make-enum-set universe (lset-difference eq? (enum-set->list universe) (enum-set-set enum-set))))) (define (enum-set-projection enum-set-1 enum-set-2) (make-enum-set (enum-set-universe enum-set-2) (lset-intersection eq? (enum-set-set enum-set-1) (enum-set->list (enum-set-universe enum-set-2))))) (define-syntax define-enumeration (syntax-rules () ((_ type-name (symbol ...) constructor-syntax) (begin (define-syntax type-name (lambda (s) (syntax-case s () ((type-name sym) (if (memq (syntax->datum #'sym) '(symbol ...)) #'(quote sym) (syntax-violation (symbol->string 'type-name) "not a member of the set" #f)))))) (define-syntax constructor-syntax (lambda (s) (syntax-case s () ((_ sym (... ...)) (let* ((universe '(symbol ...)) (syms (syntax->datum #'(sym (... ...)))) (quoted-universe (datum->syntax s (list 'quote universe))) (quoted-syms (datum->syntax s (list 'quote syms)))) (or (every (lambda (x) (memq x universe)) syms) (syntax-violation (symbol->string 'constructor-syntax) "not a subset of the universe" #f)) #`((enum-set-constructor (make-enumeration #,quoted-universe)) #,quoted-syms)))))))))) ) PK/F2]{ +rnrs/lists.scmnu[;;; lists.scm --- The R6RS list utilities library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs lists (6)) (export find for-all exists filter partition fold-left fold-right remp remove remv remq memp member memv memq assp assoc assv assq cons*) (import (rnrs base (6)) (only (guile) filter member memv memq assoc assv assq cons*) (rename (only (srfi srfi-1) any every remove member assoc find partition fold-right filter-map) (any exists) (every for-all) (remove remp) (member memp-internal) (assoc assp-internal))) (define (fold-left combine nil list . lists) (define (fold nil lists) (if (exists null? lists) nil (fold (apply combine nil (map car lists)) (map cdr lists)))) (fold nil (cons list lists))) (define (remove obj list) (remp (lambda (elt) (equal? obj elt)) list)) (define (remv obj list) (remp (lambda (elt) (eqv? obj elt)) list)) (define (remq obj list) (remp (lambda (elt) (eq? obj elt)) list)) (define (memp pred list) (memp-internal #f list (lambda (x y) (pred y)))) (define (assp pred list) (assp-internal #f list (lambda (x y) (pred y)))) ) PK/F2] 2( ( rnrs/unicode.scmnu[;;; unicode.scm --- The R6RS Unicode library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs unicode (6)) (export char-upcase char-downcase char-titlecase char-foldcase char-ci=? char-ci? char-ci<=? char-ci>=? char-alphabetic? char-numeric? char-whitespace? char-upper-case? char-lower-case? char-title-case? char-general-category string-upcase string-downcase string-titlecase string-foldcase string-ci=? string-ci? string-ci<=? string-ci>=? string-normalize-nfd string-normalize-nfkd string-normalize-nfc string-normalize-nfkc) (import (only (guile) char-upcase char-downcase char-titlecase char-ci=? char-ci? char-ci<=? char-ci>=? char-alphabetic? char-numeric? char-whitespace? char-upper-case? char-lower-case? char-set-contains? char-set:title-case char-general-category char-upcase char-downcase char-titlecase string-upcase string-downcase string-titlecase string-ci=? string-ci? string-ci<=? string-ci>=? string-normalize-nfd string-normalize-nfkd string-normalize-nfc string-normalize-nfkc) (rnrs base (6))) (define (char-foldcase char) (if (or (eqv? char #\460) (eqv? char #\461)) char (char-downcase (char-upcase char)))) (define (char-title-case? char) (char-set-contains? char-set:title-case char)) (define (string-foldcase str) (string-downcase (string-upcase str))) ) PK/F2]6`E22rnrs/arithmetic/fixnums.scmnu[;;; fixnums.scm --- The R6RS fixnums arithmetic library ;; Copyright (C) 2010, 2011, 2013 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs arithmetic fixnums (6)) (export fixnum? fixnum-width least-fixnum greatest-fixnum fx=? fx>? fx=? fx<=? fxzero? fxpositive? fxnegative? fxodd? fxeven? fxmax fxmin fx+ fx* fx- fxdiv-and-mod fxdiv fxmod fxdiv0-and-mod0 fxdiv0 fxmod0 fx+/carry fx-/carry fx*/carry fxnot fxand fxior fxxor fxif fxbit-count fxlength fxfirst-bit-set fxbit-set? fxcopy-bit fxbit-field fxcopy-bit-field fxarithmetic-shift fxarithmetic-shift-left fxarithmetic-shift-right fxrotate-bit-field fxreverse-bit-field) (import (only (guile) ash cons* define-inlinable inexact->exact logand logbit? logcount logior lognot logxor most-positive-fixnum most-negative-fixnum object-address) (ice-9 optargs) (rnrs base (6)) (rnrs control (6)) (rnrs arithmetic bitwise (6)) (rnrs conditions (6)) (rnrs exceptions (6)) (rnrs lists (6))) (define fixnum-width (let ((w (do ((i 0 (+ 1 i)) (n 1 (* 2 n))) ((> n most-positive-fixnum) (+ 1 i))))) (lambda () w))) (define (greatest-fixnum) most-positive-fixnum) (define (least-fixnum) most-negative-fixnum) (define (fixnum? obj) (not (= 0 (logand 2 (object-address obj))))) (define-inlinable (inline-fixnum? obj) (not (= 0 (logand 2 (object-address obj))))) (define-syntax assert-fixnum (syntax-rules () ((_ arg ...) (or (and (inline-fixnum? arg) ...) (raise (make-assertion-violation)))))) (define (assert-fixnums args) (or (for-all inline-fixnum? args) (raise (make-assertion-violation)))) (define-syntax define-fxop* (syntax-rules () ((_ name op) (define name (case-lambda ((x y) (assert-fixnum x y) (op x y)) (args (assert-fixnums args) (apply op args))))))) ;; All these predicates don't check their arguments for fixnum-ness, ;; as this doesn't seem to be strictly required by R6RS. (define fx=? =) (define fx>? >) (define fx=? >=) (define fx<=? <=) (define fxzero? zero?) (define fxpositive? positive?) (define fxnegative? negative?) (define fxodd? odd?) (define fxeven? even?) (define-fxop* fxmax max) (define-fxop* fxmin min) (define (fx+ fx1 fx2) (assert-fixnum fx1 fx2) (let ((r (+ fx1 fx2))) (or (inline-fixnum? r) (raise (make-implementation-restriction-violation))) r)) (define (fx* fx1 fx2) (assert-fixnum fx1 fx2) (let ((r (* fx1 fx2))) (or (inline-fixnum? r) (raise (make-implementation-restriction-violation))) r)) (define* (fx- fx1 #:optional fx2) (assert-fixnum fx1) (if fx2 (begin (assert-fixnum fx2) (let ((r (- fx1 fx2))) (or (inline-fixnum? r) (raise (make-assertion-violation))) r)) (let ((r (- fx1))) (or (inline-fixnum? r) (raise (make-assertion-violation))) r))) (define (fxdiv fx1 fx2) (assert-fixnum fx1 fx2) (div fx1 fx2)) (define (fxmod fx1 fx2) (assert-fixnum fx1 fx2) (mod fx1 fx2)) (define (fxdiv-and-mod fx1 fx2) (assert-fixnum fx1 fx2) (div-and-mod fx1 fx2)) (define (fxdiv0 fx1 fx2) (assert-fixnum fx1 fx2) (div0 fx1 fx2)) (define (fxmod0 fx1 fx2) (assert-fixnum fx1 fx2) (mod0 fx1 fx2)) (define (fxdiv0-and-mod0 fx1 fx2) (assert-fixnum fx1 fx2) (div0-and-mod0 fx1 fx2)) (define (fx+/carry fx1 fx2 fx3) (assert-fixnum fx1 fx2 fx3) (let* ((s (+ fx1 fx2 fx3)) (s0 (mod0 s (expt 2 (fixnum-width)))) (s1 (div0 s (expt 2 (fixnum-width))))) (values s0 s1))) (define (fx-/carry fx1 fx2 fx3) (assert-fixnum fx1 fx2 fx3) (let* ((d (- fx1 fx2 fx3)) (d0 (mod0 d (expt 2 (fixnum-width)))) (d1 (div0 d (expt 2 (fixnum-width))))) (values d0 d1))) (define (fx*/carry fx1 fx2 fx3) (assert-fixnum fx1 fx2 fx3) (let* ((s (+ (* fx1 fx2) fx3)) (s0 (mod0 s (expt 2 (fixnum-width)))) (s1 (div0 s (expt 2 (fixnum-width))))) (values s0 s1))) (define (fxnot fx) (assert-fixnum fx) (lognot fx)) (define-fxop* fxand logand) (define-fxop* fxior logior) (define-fxop* fxxor logxor) (define (fxif fx1 fx2 fx3) (assert-fixnum fx1 fx2 fx3) (bitwise-if fx1 fx2 fx3)) (define (fxbit-count fx) (assert-fixnum fx) (if (negative? fx) (bitwise-not (logcount fx)) (logcount fx))) (define (fxlength fx) (assert-fixnum fx) (bitwise-length fx)) (define (fxfirst-bit-set fx) (assert-fixnum fx) (bitwise-first-bit-set fx)) (define (fxbit-set? fx1 fx2) (assert-fixnum fx1 fx2) (logbit? fx2 fx1)) (define (fxcopy-bit fx1 fx2 fx3) (assert-fixnum fx1 fx2 fx3) (bitwise-copy-bit fx1 fx2 fx3)) (define (fxbit-field fx1 fx2 fx3) (assert-fixnum fx1 fx2 fx3) (bitwise-bit-field fx1 fx2 fx3)) (define (fxcopy-bit-field fx1 fx2 fx3 fx4) (assert-fixnum fx1 fx2 fx3 fx4) (bitwise-copy-bit-field fx1 fx2 fx3 fx4)) (define (fxarithmetic-shift fx1 fx2) (assert-fixnum fx1 fx2) (ash fx1 fx2)) (define fxarithmetic-shift-left fxarithmetic-shift) (define (fxarithmetic-shift-right fx1 fx2) (assert-fixnum fx1 fx2) (ash fx1 (- fx2))) (define (fxrotate-bit-field fx1 fx2 fx3 fx4) (assert-fixnum fx1 fx2 fx3 fx4) (bitwise-rotate-bit-field fx1 fx2 fx3 fx4)) (define (fxreverse-bit-field fx1 fx2 fx3) (assert-fixnum fx1 fx2 fx3) (bitwise-reverse-bit-field fx1 fx2 fx3)) ) PK/F2]ZGrnrs/arithmetic/flonums.scmnu[;;; flonums.scm --- The R6RS flonums arithmetic library ;; Copyright (C) 2010, 2011, 2013 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs arithmetic flonums (6)) (export flonum? real->flonum fl=? fl? fl>=? flinteger? flzero? flpositive? flnegative? flodd? fleven? flfinite? flinfinite? flnan? flmax flmin fl+ fl* fl- fl/ flabs fldiv-and-mod fldiv flmod fldiv0-and-mod0 fldiv0 flmod0 flnumerator fldenominator flfloor flceiling fltruncate flround flexp fllog flsin flcos fltan flacos flasin flatan flsqrt flexpt &no-infinities make-no-infinities-violation no-infinities-violation? &no-nans make-no-nans-violation no-nans-violation? fixnum->flonum) (import (ice-9 optargs) (only (guile) inf?) (rnrs arithmetic fixnums (6)) (rnrs base (6)) (rnrs control (6)) (rnrs conditions (6)) (rnrs exceptions (6)) (rnrs lists (6)) (rnrs r5rs (6))) (define (flonum? obj) (and (real? obj) (inexact? obj))) (define (assert-flonum . args) (or (for-all flonum? args) (raise (make-assertion-violation)))) (define (assert-iflonum . args) (or (for-all (lambda (i) (and (flonum? i) (integer? i))) args) (raise (make-assertion-violation)))) (define (ensure-flonum z) (cond ((real? z) z) ((zero? (imag-part z)) (real-part z)) (else +nan.0))) (define (real->flonum x) (or (real? x) (raise (make-assertion-violation))) (exact->inexact x)) (define (fl=? . args) (apply assert-flonum args) (apply = args)) (define (fl? . args) (apply assert-flonum args) (apply > args)) (define (fl>=? . args) (apply assert-flonum args) (apply >= args)) (define (flinteger? fl) (assert-flonum fl) (integer? fl)) (define (flzero? fl) (assert-flonum fl) (zero? fl)) (define (flpositive? fl) (assert-flonum fl) (positive? fl)) (define (flnegative? fl) (assert-flonum fl) (negative? fl)) (define (flodd? ifl) (assert-iflonum ifl) (odd? ifl)) (define (fleven? ifl) (assert-iflonum ifl) (even? ifl)) (define (flfinite? fl) (assert-flonum fl) (not (or (inf? fl) (nan? fl)))) (define (flinfinite? fl) (assert-flonum fl) (inf? fl)) (define (flnan? fl) (assert-flonum fl) (nan? fl)) (define (flmax fl1 . args) (let ((flargs (cons fl1 args))) (apply assert-flonum flargs) (apply max flargs))) (define (flmin fl1 . args) (let ((flargs (cons fl1 args))) (apply assert-flonum flargs) (apply min flargs))) (define (fl+ . args) (apply assert-flonum args) (if (null? args) 0.0 (apply + args))) (define (fl* . args) (apply assert-flonum args) (if (null? args) 1.0 (apply * args))) (define (fl- fl1 . args) (let ((flargs (cons fl1 args))) (apply assert-flonum flargs) (apply - flargs))) (define (fl/ fl1 . args) (let ((flargs (cons fl1 args))) (apply assert-flonum flargs) (apply / flargs))) (define (flabs fl) (assert-flonum fl) (abs fl)) (define (fldiv-and-mod fl1 fl2) (assert-iflonum fl1 fl2) (div-and-mod fl1 fl2)) (define (fldiv fl1 fl2) (assert-iflonum fl1 fl2) (div fl1 fl2)) (define (flmod fl1 fl2) (assert-iflonum fl1 fl2) (mod fl1 fl2)) (define (fldiv0-and-mod0 fl1 fl2) (assert-iflonum fl1 fl2) (div0-and-mod0 fl1 fl2)) (define (fldiv0 fl1 fl2) (assert-iflonum fl1 fl2) (div0 fl1 fl2)) (define (flmod0 fl1 fl2) (assert-iflonum fl1 fl2) (mod0 fl1 fl2)) (define (flnumerator fl) (assert-flonum fl) (numerator fl)) (define (fldenominator fl) (assert-flonum fl) (denominator fl)) (define (flfloor fl) (assert-flonum fl) (floor fl)) (define (flceiling fl) (assert-flonum fl) (ceiling fl)) (define (fltruncate fl) (assert-flonum fl) (truncate fl)) (define (flround fl) (assert-flonum fl) (round fl)) (define (flexp fl) (assert-flonum fl) (exp fl)) (define fllog (case-lambda ((fl) (assert-flonum fl) ;; add 0.0 to fl, to change -0.0 to 0.0, ;; so that (fllog -0.0) will be -inf.0, not -inf.0+pi*i. (ensure-flonum (log (+ fl 0.0)))) ((fl fl2) (assert-flonum fl fl2) (ensure-flonum (/ (log (+ fl 0.0)) (log (+ fl2 0.0))))))) (define (flsin fl) (assert-flonum fl) (sin fl)) (define (flcos fl) (assert-flonum fl) (cos fl)) (define (fltan fl) (assert-flonum fl) (tan fl)) (define (flasin fl) (assert-flonum fl) (ensure-flonum (asin fl))) (define (flacos fl) (assert-flonum fl) (ensure-flonum (acos fl))) (define flatan (case-lambda ((fl) (assert-flonum fl) (atan fl)) ((fl fl2) (assert-flonum fl fl2) (atan fl fl2)))) (define (flsqrt fl) (assert-flonum fl) (ensure-flonum (sqrt fl))) (define (flexpt fl1 fl2) (assert-flonum fl1 fl2) (ensure-flonum (expt fl1 fl2))) (define-condition-type &no-infinities &implementation-restriction make-no-infinities-violation no-infinities-violation?) (define-condition-type &no-nans &implementation-restriction make-no-nans-violation no-nans-violation?) (define (fixnum->flonum fx) (or (fixnum? fx) (raise (make-assertion-violation))) (exact->inexact fx)) ) PK/F2]$M; ; rnrs/arithmetic/bitwise.scmnu[;;; bitwise.scm --- The R6RS bitwise arithmetic operations library ;; Copyright (C) 2010, 2013 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs arithmetic bitwise (6)) (export bitwise-not bitwise-and bitwise-ior bitwise-xor bitwise-if bitwise-bit-count bitwise-length bitwise-first-bit-set bitwise-bit-set? bitwise-copy-bit bitwise-bit-field bitwise-copy-bit-field bitwise-arithmetic-shift bitwise-arithmetic-shift-left bitwise-arithmetic-shift-right bitwise-rotate-bit-field bitwise-reverse-bit-field) (import (rnrs base (6)) (rnrs control (6)) (rename (only (srfi srfi-60) bitwise-if integer-length first-set-bit copy-bit bit-field copy-bit-field rotate-bit-field reverse-bit-field) (integer-length bitwise-length) (first-set-bit bitwise-first-bit-set) (bit-field bitwise-bit-field) (reverse-bit-field bitwise-reverse-bit-field)) (rename (only (guile) lognot logand logior logxor logcount logbit? modulo ash) (lognot bitwise-not) (logand bitwise-and) (logior bitwise-ior) (logxor bitwise-xor) (ash bitwise-arithmetic-shift))) (define (bitwise-bit-count ei) (if (negative? ei) (bitwise-not (logcount ei)) (logcount ei))) (define (bitwise-bit-set? ei1 ei2) (logbit? ei2 ei1)) (define (bitwise-copy-bit ei1 ei2 ei3) ;; The specification states that ei3 should be either 0 or 1. ;; However, other values have been tolerated by both Guile 2.0.x and ;; the sample implementation given the R6RS library document, so for ;; backward compatibility we continue to permit it. (copy-bit ei2 ei1 (logbit? 0 ei3))) (define (bitwise-copy-bit-field ei1 ei2 ei3 ei4) (copy-bit-field ei1 ei4 ei2 ei3)) (define (bitwise-rotate-bit-field ei1 ei2 ei3 ei4) (rotate-bit-field ei1 ei4 ei2 ei3)) (define bitwise-arithmetic-shift-left bitwise-arithmetic-shift) (define (bitwise-arithmetic-shift-right ei1 ei2) (bitwise-arithmetic-shift ei1 (- ei2)))) PK/F2]rnrs/programs.scmnu[;;; programs.scm --- The R6RS process management library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs programs (6)) (export command-line exit) (import (only (guile) command-line exit))) PK/F2]|K-(-(rnrs/records/procedural.scmnu[;;; procedural.scm --- Procedural interface to R6RS records ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs records procedural (6)) (export make-record-type-descriptor record-type-descriptor? make-record-constructor-descriptor record-constructor record-predicate record-accessor record-mutator) (import (rnrs base (6)) (only (guile) cons* logand logior ash and=> throw display make-struct make-vtable map simple-format string-append symbol-append struct? struct-layout struct-ref struct-set! struct-vtable vtable-index-layout make-hash-table hashq-ref hashq-set! vector->list) (ice-9 receive) (only (srfi :1) fold split-at take)) (define (record-internal? obj) (and (struct? obj) (record-type-descriptor? (struct-vtable obj)))) (define rtd-index-name 8) (define rtd-index-uid 9) (define rtd-index-parent 10) (define rtd-index-sealed? 11) (define rtd-index-opaque? 12) (define rtd-index-predicate 13) (define rtd-index-field-names 14) (define rtd-index-field-bit-field 15) (define rtd-index-field-binder 16) (define rctd-index-rtd 0) (define rctd-index-parent 1) (define rctd-index-protocol 2) (define vtable-base-layout (symbol->string (struct-layout (make-vtable "")))) (define record-type-vtable (make-vtable (string-append vtable-base-layout "prprprprprprprprprpr") (lambda (obj port) (simple-format port "#" (struct-ref obj rtd-index-name))))) (define record-constructor-vtable (make-vtable "prprpr" (lambda (obj port) (simple-format port "#" (struct-ref (struct-ref obj rctd-index-rtd) rtd-index-name))))) (define uid-table (make-hash-table)) (define (make-record-type-descriptor name parent uid sealed? opaque? fields) (define fields-pair (let loop ((field-list (vector->list fields)) (layout-sym 'pr) (layout-bit-field 0) (counter 0)) (if (null? field-list) (cons layout-sym layout-bit-field) (case (caar field-list) ((immutable) (loop (cdr field-list) (symbol-append layout-sym 'pr) layout-bit-field (+ counter 1))) ((mutable) (loop (cdr field-list) (symbol-append layout-sym 'pw) (logior layout-bit-field (ash 1 counter)) (+ counter 1))) (else (r6rs-raise (make-assertion-violation))))))) (define fields-layout (car fields-pair)) (define fields-bit-field (cdr fields-pair)) (define field-names (list->vector (map cadr (vector->list fields)))) (define late-rtd #f) (define (private-record-predicate obj) (and (record-internal? obj) (or (eq? (struct-vtable obj) late-rtd) (and=> (struct-ref obj 0) private-record-predicate)))) (define (field-binder parent-struct . args) (apply make-struct (cons* late-rtd 0 parent-struct args))) (if (and parent (struct-ref parent rtd-index-sealed?)) (r6rs-raise (make-assertion-violation))) (let ((matching-rtd (and uid (hashq-ref uid-table uid))) (opaque? (or opaque? (and parent (struct-ref parent rtd-index-opaque?))))) (if matching-rtd (if (equal? (list name parent sealed? opaque? field-names fields-bit-field) (list (struct-ref matching-rtd rtd-index-name) (struct-ref matching-rtd rtd-index-parent) (struct-ref matching-rtd rtd-index-sealed?) (struct-ref matching-rtd rtd-index-opaque?) (struct-ref matching-rtd rtd-index-field-names) (struct-ref matching-rtd rtd-index-field-bit-field))) matching-rtd (r6rs-raise (make-assertion-violation))) (let ((rtd (make-struct record-type-vtable 0 fields-layout (lambda (obj port) (simple-format port "#" name)) name uid parent sealed? opaque? private-record-predicate field-names fields-bit-field field-binder))) (set! late-rtd rtd) (if uid (hashq-set! uid-table uid rtd)) rtd)))) (define (record-type-descriptor? obj) (and (struct? obj) (eq? (struct-vtable obj) record-type-vtable))) (define (make-record-constructor-descriptor rtd parent-constructor-descriptor protocol) (define rtd-arity (vector-length (struct-ref rtd rtd-index-field-names))) (define (default-inherited-protocol n) (lambda args (receive (n-args p-args) (split-at args (- (length args) rtd-arity)) (let ((p (apply n n-args))) (apply p p-args))))) (define (default-protocol p) p) (let* ((prtd (struct-ref rtd rtd-index-parent)) (pcd (or parent-constructor-descriptor (and=> prtd (lambda (d) (make-record-constructor-descriptor prtd #f #f))))) (prot (or protocol (if pcd default-inherited-protocol default-protocol)))) (make-struct record-constructor-vtable 0 rtd pcd prot))) (define (record-constructor rctd) (let* ((rtd (struct-ref rctd rctd-index-rtd)) (parent-rctd (struct-ref rctd rctd-index-parent)) (protocol (struct-ref rctd rctd-index-protocol))) (protocol (if parent-rctd (let ((parent-record-constructor (record-constructor parent-rctd)) (parent-rtd (struct-ref parent-rctd rctd-index-rtd))) (lambda args (let ((struct (apply parent-record-constructor args))) (lambda args (apply (struct-ref rtd rtd-index-field-binder) (cons struct args)))))) (lambda args (apply (struct-ref rtd rtd-index-field-binder) (cons #f args))))))) (define (record-predicate rtd) (struct-ref rtd rtd-index-predicate)) (define (record-accessor rtd k) (define (record-accessor-inner obj) (if (eq? (struct-vtable obj) rtd) (struct-ref obj (+ k 1)) (and=> (struct-ref obj 0) record-accessor-inner))) (lambda (obj) (if (not (record-internal? obj)) (r6rs-raise (make-assertion-violation))) (record-accessor-inner obj))) (define (record-mutator rtd k) (define (record-mutator-inner obj val) (and obj (or (and (eq? (struct-vtable obj) rtd) (struct-set! obj (+ k 1) val)) (record-mutator-inner (struct-ref obj 0) val)))) (let ((bit-field (struct-ref rtd rtd-index-field-bit-field))) (if (zero? (logand bit-field (ash 1 k))) (r6rs-raise (make-assertion-violation)))) (lambda (obj val) (record-mutator-inner obj val))) ;; Condition types that are used in the current library. These are defined ;; here and not in (rnrs conditions) to avoid a circular dependency. (define &condition (make-record-type-descriptor '&condition #f #f #f #f '#())) (define &condition-constructor-descriptor (make-record-constructor-descriptor &condition #f #f)) (define &serious (make-record-type-descriptor '&serious &condition #f #f #f '#())) (define &serious-constructor-descriptor (make-record-constructor-descriptor &serious &condition-constructor-descriptor #f)) (define make-serious-condition (record-constructor &serious-constructor-descriptor)) (define &violation (make-record-type-descriptor '&violation &serious #f #f #f '#())) (define &violation-constructor-descriptor (make-record-constructor-descriptor &violation &serious-constructor-descriptor #f)) (define make-violation (record-constructor &violation-constructor-descriptor)) (define &assertion (make-record-type-descriptor '&assertion &violation #f #f #f '#())) (define make-assertion-violation (record-constructor (make-record-constructor-descriptor &assertion &violation-constructor-descriptor #f))) ;; Exception wrapper type, along with a wrapping `throw' implementation. ;; These are used in the current library, and so they are defined here and not ;; in (rnrs exceptions) to avoid a circular dependency. (define &raise-object-wrapper (make-record-type-descriptor '&raise-object-wrapper #f #f #f #f '#((immutable obj) (immutable continuation)))) (define make-raise-object-wrapper (record-constructor (make-record-constructor-descriptor &raise-object-wrapper #f #f))) (define raise-object-wrapper? (record-predicate &raise-object-wrapper)) (define raise-object-wrapper-obj (record-accessor &raise-object-wrapper 0)) (define raise-object-wrapper-continuation (record-accessor &raise-object-wrapper 1)) (define (r6rs-raise obj) (throw 'r6rs:exception (make-raise-object-wrapper obj #f))) (define (r6rs-raise-continuable obj) (define (r6rs-raise-continuable-internal continuation) (throw 'r6rs:exception (make-raise-object-wrapper obj continuation))) (call/cc r6rs-raise-continuable-internal)) ) PK/F2]~Oi i rnrs/records/inspection.scmnu[;;; inspection.scm --- Inspection support for R6RS records ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs records inspection (6)) (export record? record-rtd record-type-name record-type-parent record-type-uid record-type-generative? record-type-sealed? record-type-opaque? record-type-field-names record-field-mutable?) (import (rnrs arithmetic bitwise (6)) (rnrs base (6)) (rnrs records procedural (6)) (only (guile) struct-ref struct-vtable vtable-index-layout @@)) (define record-internal? (@@ (rnrs records procedural) record-internal?)) (define rtd-index-name (@@ (rnrs records procedural) rtd-index-name)) (define rtd-index-parent (@@ (rnrs records procedural) rtd-index-parent)) (define rtd-index-uid (@@ (rnrs records procedural) rtd-index-uid)) (define rtd-index-sealed? (@@ (rnrs records procedural) rtd-index-sealed?)) (define rtd-index-opaque? (@@ (rnrs records procedural) rtd-index-opaque?)) (define rtd-index-field-names (@@ (rnrs records procedural) rtd-index-field-names)) (define rtd-index-field-bit-field (@@ (rnrs records procedural) rtd-index-field-bit-field)) (define (record? obj) (and (record-internal? obj) (not (record-type-opaque? (struct-vtable obj))))) (define (record-rtd record) (or (and (record-internal? record) (let ((rtd (struct-vtable record))) (and (not (struct-ref rtd rtd-index-opaque?)) rtd))) (assertion-violation 'record-rtd "not a record" record))) (define (guarantee-rtd who rtd) (if (record-type-descriptor? rtd) rtd (assertion-violation who "not a record type descriptor" rtd))) (define (record-type-name rtd) (struct-ref (guarantee-rtd 'record-type-name rtd) rtd-index-name)) (define (record-type-parent rtd) (struct-ref (guarantee-rtd 'record-type-parent rtd) rtd-index-parent)) (define (record-type-uid rtd) (struct-ref (guarantee-rtd 'record-type-uid rtd) rtd-index-uid)) (define (record-type-generative? rtd) (not (record-type-uid (guarantee-rtd 'record-type-generative? rtd)))) (define (record-type-sealed? rtd) (struct-ref (guarantee-rtd 'record-type-sealed? rtd) rtd-index-sealed?)) (define (record-type-opaque? rtd) (struct-ref (guarantee-rtd 'record-type-opaque? rtd) rtd-index-opaque?)) (define (record-type-field-names rtd) (struct-ref (guarantee-rtd 'record-type-field-names rtd) rtd-index-field-names)) (define (record-field-mutable? rtd k) (bitwise-bit-set? (struct-ref (guarantee-rtd 'record-field-mutable? rtd) rtd-index-field-bit-field) k)) ) PK/F2]Y,,rnrs/records/syntactic.scmnu[;;; syntactic.scm --- Syntactic support for R6RS records ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs records syntactic (6)) (export define-record-type record-type-descriptor record-constructor-descriptor) (import (only (guile) and=> gensym) (rnrs base (6)) (rnrs conditions (6)) (rnrs exceptions (6)) (rnrs hashtables (6)) (rnrs lists (6)) (rnrs records procedural (6)) (rnrs syntax-case (6)) (only (srfi :1) take)) (define record-type-registry (make-eq-hashtable)) (define (guess-constructor-name record-name) (string->symbol (string-append "make-" (symbol->string record-name)))) (define (guess-predicate-name record-name) (string->symbol (string-append (symbol->string record-name) "?"))) (define (register-record-type name rtd rcd) (hashtable-set! record-type-registry name (cons rtd rcd))) (define (lookup-record-type-descriptor name) (and=> (hashtable-ref record-type-registry name #f) car)) (define (lookup-record-constructor-descriptor name) (and=> (hashtable-ref record-type-registry name #f) cdr)) (define-syntax define-record-type (lambda (stx) (syntax-case stx () ((_ (record-name constructor-name predicate-name) record-clause ...) #'(define-record-type0 (record-name constructor-name predicate-name) record-clause ...)) ((_ record-name record-clause ...) (let* ((record-name-sym (syntax->datum #'record-name)) (constructor-name (datum->syntax #'record-name (guess-constructor-name record-name-sym))) (predicate-name (datum->syntax #'record-name (guess-predicate-name record-name-sym)))) #`(define-record-type0 (record-name #,constructor-name #,predicate-name) record-clause ...)))))) (define (sequence n) (define (seq-inner n) (if (= n 0) '(0) (cons n (seq-inner (- n 1))))) (reverse (seq-inner n))) (define (number-fields fields) (define (number-fields-inner fields counter) (if (null? fields) '() (cons (cons fields counter) (number-fields-inner (cdr fields) (+ counter 1))))) (number-fields-inner fields 0)) (define (process-fields record-name fields) (define (wrap x) (datum->syntax record-name x)) (define (id->string x) (symbol->string (syntax->datum x))) (define record-name-str (id->string record-name)) (define (guess-accessor-name field-name) (wrap (string->symbol (string-append record-name-str "-" (id->string field-name))))) (define (guess-mutator-name field-name) (wrap (string->symbol (string-append record-name-str "-" (id->string field-name) "-set!")))) (define (f x) (syntax-case x (immutable mutable) [(immutable name) (list (wrap `(immutable ,(syntax->datum #'name))) (guess-accessor-name #'name) #f)] [(immutable name accessor) (list (wrap `(immutable ,(syntax->datum #'name))) #'accessor #f)] [(mutable name) (list (wrap `(mutable ,(syntax->datum #'name))) (guess-accessor-name #'name) (guess-mutator-name #'name))] [(mutable name accessor mutator) (list (wrap `(mutable ,(syntax->datum #'name))) #'accessor #'mutator)] [name (identifier? #'name) (list (wrap `(immutable ,(syntax->datum #'name))) (guess-accessor-name #'name) #f)] [else (syntax-violation 'define-record-type "invalid field specifier" x)])) (map f fields)) (define-syntax define-record-type0 (lambda (stx) (define *unspecified* (cons #f #f)) (define (unspecified? obj) (eq? *unspecified* obj)) (syntax-case stx () ((_ (record-name constructor-name predicate-name) record-clause ...) (let loop ((_fields *unspecified*) (_parent *unspecified*) (_protocol *unspecified*) (_sealed *unspecified*) (_opaque *unspecified*) (_nongenerative *unspecified*) (_constructor *unspecified*) (_parent-rtd *unspecified*) (record-clauses #'(record-clause ...))) (syntax-case record-clauses (fields parent protocol sealed opaque nongenerative constructor parent-rtd) [() (let* ((fields (if (unspecified? _fields) '() _fields)) (field-names (list->vector (map car fields))) (field-accessors (fold-left (lambda (lst x c) (cons #`(define #,(cadr x) (record-accessor record-name #,c)) lst)) '() fields (sequence (length fields)))) (field-mutators (fold-left (lambda (lst x c) (if (caddr x) (cons #`(define #,(caddr x) (record-mutator record-name #,c)) lst) lst)) '() fields (sequence (length fields)))) (parent-cd (cond ((not (unspecified? _parent)) #`(record-constructor-descriptor #,_parent)) ((not (unspecified? _parent-rtd)) (cadr _parent-rtd)) (else #f))) (parent-rtd (cond ((not (unspecified? _parent)) #`(record-type-descriptor #,_parent)) ((not (unspecified? _parent-rtd)) (car _parent-rtd)) (else #f))) (protocol (if (unspecified? _protocol) #f _protocol)) (uid (if (unspecified? _nongenerative) #f _nongenerative)) (sealed? (if (unspecified? _sealed) #f _sealed)) (opaque? (if (unspecified? _opaque) #f _opaque))) #`(begin (define record-name (make-record-type-descriptor (quote record-name) #,parent-rtd #,uid #,sealed? #,opaque? #,field-names)) (define constructor-name (record-constructor (make-record-constructor-descriptor record-name #,parent-cd #,protocol))) (define dummy (let () (register-record-type (quote record-name) record-name (make-record-constructor-descriptor record-name #,parent-cd #,protocol)) 'dummy)) (define predicate-name (record-predicate record-name)) #,@field-accessors #,@field-mutators))] [((fields record-fields ...) . rest) (if (unspecified? _fields) (loop (process-fields #'record-name #'(record-fields ...)) _parent _protocol _sealed _opaque _nongenerative _constructor _parent-rtd #'rest) (raise (make-assertion-violation)))] [((parent parent-name) . rest) (if (not (unspecified? _parent-rtd)) (raise (make-assertion-violation)) (if (unspecified? _parent) (loop _fields #'parent-name _protocol _sealed _opaque _nongenerative _constructor _parent-rtd #'rest) (raise (make-assertion-violation))))] [((protocol expression) . rest) (if (unspecified? _protocol) (loop _fields _parent #'expression _sealed _opaque _nongenerative _constructor _parent-rtd #'rest) (raise (make-assertion-violation)))] [((sealed sealed?) . rest) (if (unspecified? _sealed) (loop _fields _parent _protocol #'sealed? _opaque _nongenerative _constructor _parent-rtd #'rest) (raise (make-assertion-violation)))] [((opaque opaque?) . rest) (if (unspecified? _opaque) (loop _fields _parent _protocol _sealed #'opaque? _nongenerative _constructor _parent-rtd #'rest) (raise (make-assertion-violation)))] [((nongenerative) . rest) (if (unspecified? _nongenerative) (loop _fields _parent _protocol _sealed _opaque #`(quote #,(datum->syntax #'record-name (gensym))) _constructor _parent-rtd #'rest) (raise (make-assertion-violation)))] [((nongenerative uid) . rest) (if (unspecified? _nongenerative) (loop _fields _parent _protocol _sealed _opaque #''uid _constructor _parent-rtd #'rest) (raise (make-assertion-violation)))] [((parent-rtd rtd cd) . rest) (if (not (unspecified? _parent)) (raise (make-assertion-violation)) (if (unspecified? _parent-rtd) (loop _fields _parent _protocol _sealed _opaque _nongenerative _constructor #'(rtd cd) #'rest) (raise (make-assertion-violation))))])))))) (define-syntax record-type-descriptor (lambda (stx) (syntax-case stx () ((_ name) #`(lookup-record-type-descriptor #,(datum->syntax stx (list 'quote (syntax->datum #'name)))))))) (define-syntax record-constructor-descriptor (lambda (stx) (syntax-case stx () ((_ name) #`(lookup-record-constructor-descriptor #,(datum->syntax stx (list 'quote (syntax->datum #'name)))))))) ) PK/F2]P_(+hhrnrs/syntax-case.scmnu[;;; syntax-case.scm --- R6RS support for `syntax-case' macros ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs syntax-case (6)) (export make-variable-transformer syntax-case syntax identifier? bound-identifier=? free-identifier=? syntax->datum datum->syntax generate-temporaries with-syntax quasisyntax unsyntax unsyntax-splicing syntax-violation) (import (only (guile) make-variable-transformer syntax-case syntax identifier? bound-identifier=? free-identifier=? syntax->datum datum->syntax generate-temporaries with-syntax quasisyntax unsyntax unsyntax-splicing) (ice-9 optargs) (rnrs base (6)) (rnrs conditions (6)) (rnrs exceptions (6)) (rnrs records procedural (6))) (define* (syntax-violation who message form #:optional subform) (let* ((conditions (list (make-message-condition message) (make-syntax-violation form subform))) (conditions (if who (cons (make-who-condition who) conditions) conditions))) (raise (apply condition conditions)))) ) PK/F2]hЗN N rnrs/files.scmnu[;;; files.scm --- The R6RS file system library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs files (6)) (export file-exists? delete-file &i/o make-i/o-error i/o-error? &i/o-read make-i/o-read-error i/o-read-error? &i/o-write make-i/o-write-error i/o-write-error? &i/o-invalid-position make-i/o-invalid-position-error i/o-invalid-position-error? i/o-error-position &i/o-filename make-i/o-filename-error i/o-filename-error? i/o-error-filename &i/o-file-protection make-i/o-file-protection-error i/o-file-protection-error? &i/o-file-is-read-only make-i/o-file-is-read-only-error i/o-file-is-read-only-error? &i/o-file-already-exists make-i/o-file-already-exists-error i/o-file-already-exists-error? &i/o-file-does-not-exist make-i/o-file-does-not-exist-error i/o-file-does-not-exist-error? &i/o-port make-i/o-port-error i/o-port-error? i/o-error-port) (import (rename (only (guile) file-exists? delete-file catch @@) (delete-file delete-file-internal)) (rnrs base (6)) (rnrs conditions (6)) (rnrs exceptions (6))) (define (delete-file filename) (catch #t (lambda () (delete-file-internal filename)) (lambda (key . args) (raise (make-i/o-filename-error filename))))) ;; Condition types that are used by (rnrs files), (rnrs io ports), and ;; (rnrs io simple). These are defined here so as to be easily shareable by ;; these three libraries. (define-condition-type &i/o &error make-i/o-error i/o-error?) (define-condition-type &i/o-read &i/o make-i/o-read-error i/o-read-error?) (define-condition-type &i/o-write &i/o make-i/o-write-error i/o-write-error?) (define-condition-type &i/o-invalid-position &i/o make-i/o-invalid-position-error i/o-invalid-position-error? (position i/o-error-position)) (define-condition-type &i/o-filename &i/o make-i/o-filename-error i/o-filename-error? (filename i/o-error-filename)) (define-condition-type &i/o-file-protection &i/o-filename make-i/o-file-protection-error i/o-file-protection-error?) (define-condition-type &i/o-file-is-read-only &i/o-file-protection make-i/o-file-is-read-only-error i/o-file-is-read-only-error?) (define-condition-type &i/o-file-already-exists &i/o-filename make-i/o-file-already-exists-error i/o-file-already-exists-error?) (define-condition-type &i/o-file-does-not-exist &i/o-filename make-i/o-file-does-not-exist-error i/o-file-does-not-exist-error?) (define-condition-type &i/o-port &i/o make-i/o-port-error i/o-port-error? (port i/o-error-port)) ) PK/F2]ڒUrnrs/mutable-strings.scmnu[;;; mutable-strings.scm --- The R6RS mutable string library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs mutable-strings (6)) (export string-set! string-fill!) (import (only (guile) string-set! string-fill!))) PK/F2]3+AArnrs/io/ports.scmnu[;;;; ports.scm --- R6RS port API -*- coding: utf-8 -*- ;;;; Copyright (C) 2009, 2010, 2011, 2013 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Ludovic Courtès ;;; Commentary: ;;; ;;; The I/O port API of the R6RS is provided by this module. In many areas ;;; it complements or refines Guile's own historical port API. For instance, ;;; it allows for binary I/O with bytevectors. ;;; ;;; Code: (library (rnrs io ports (6)) (export eof-object eof-object? ;; auxiliary types file-options buffer-mode buffer-mode? eol-style native-eol-style error-handling-mode make-transcoder transcoder-codec transcoder-eol-style transcoder-error-handling-mode native-transcoder latin-1-codec utf-8-codec utf-16-codec ;; input & output ports port? input-port? output-port? port-eof? port-transcoder binary-port? textual-port? transcoded-port port-position set-port-position! port-has-port-position? port-has-set-port-position!? call-with-port close-port ;; input ports open-bytevector-input-port open-string-input-port open-file-input-port make-custom-binary-input-port ;; binary input get-u8 lookahead-u8 get-bytevector-n get-bytevector-n! get-bytevector-some get-bytevector-all ;; output ports open-bytevector-output-port open-string-output-port open-file-output-port make-custom-binary-output-port call-with-bytevector-output-port call-with-string-output-port make-custom-textual-output-port flush-output-port ;; input/output ports open-file-input/output-port ;; binary output put-u8 put-bytevector ;; textual input get-char get-datum get-line get-string-all get-string-n get-string-n! lookahead-char ;; textual output put-char put-datum put-string ;; standard ports standard-input-port standard-output-port standard-error-port current-input-port current-output-port current-error-port ;; condition types &i/o i/o-error? make-i/o-error &i/o-read i/o-read-error? make-i/o-read-error &i/o-write i/o-write-error? make-i/o-write-error &i/o-invalid-position i/o-invalid-position-error? make-i/o-invalid-position-error &i/o-filename i/o-filename-error? make-i/o-filename-error i/o-error-filename &i/o-file-protection i/o-file-protection-error? make-i/o-file-protection-error &i/o-file-is-read-only i/o-file-is-read-only-error? make-i/o-file-is-read-only-error &i/o-file-already-exists i/o-file-already-exists-error? make-i/o-file-already-exists-error &i/o-file-does-not-exist i/o-file-does-not-exist-error? make-i/o-file-does-not-exist-error &i/o-port i/o-port-error? make-i/o-port-error i/o-error-port &i/o-decoding-error i/o-decoding-error? make-i/o-decoding-error &i/o-encoding-error i/o-encoding-error? make-i/o-encoding-error i/o-encoding-error-char) (import (ice-9 binary-ports) (only (rnrs base) assertion-violation) (rnrs enums) (rnrs records syntactic) (rnrs exceptions) (rnrs conditions) (rnrs files) ;for the condition types (srfi srfi-8) (ice-9 rdelim) (except (guile) raise display) (prefix (only (guile) display) guile:)) ;;; ;;; Auxiliary types ;;; (define-enumeration file-option (no-create no-fail no-truncate) file-options) (define-enumeration buffer-mode (none line block) buffer-modes) (define (buffer-mode? symbol) (enum-set-member? symbol (enum-set-universe (buffer-modes)))) (define-enumeration eol-style (lf cr crlf nel crnel ls none) eol-styles) (define (native-eol-style) (eol-style none)) (define-enumeration error-handling-mode (ignore raise replace) error-handling-modes) (define-record-type (transcoder %make-transcoder transcoder?) (fields codec eol-style error-handling-mode)) (define* (make-transcoder codec #:optional (eol-style (native-eol-style)) (handling-mode (error-handling-mode replace))) (%make-transcoder codec eol-style handling-mode)) (define (native-transcoder) (make-transcoder (or (fluid-ref %default-port-encoding) (latin-1-codec)))) (define (latin-1-codec) "ISO-8859-1") (define (utf-8-codec) "UTF-8") (define (utf-16-codec) "UTF-16") ;;; ;;; Internal helpers ;;; (define (with-i/o-filename-conditions filename thunk) (with-throw-handler 'system-error thunk (lambda args (let ((errno (system-error-errno args))) (let ((construct-condition (cond ((= errno EACCES) make-i/o-file-protection-error) ((= errno EEXIST) make-i/o-file-already-exists-error) ((= errno ENOENT) make-i/o-file-does-not-exist-error) ((= errno EROFS) make-i/o-file-is-read-only-error) (else make-i/o-filename-error)))) (raise (construct-condition filename))))))) (define (with-i/o-port-error port make-primary-condition thunk) (with-throw-handler 'system-error thunk (lambda args (let ((errno (system-error-errno args))) (if (memv errno (list EIO EFBIG ENOSPC EPIPE)) (raise (condition (make-primary-condition) (make-i/o-port-error port))) (apply throw args)))))) (define-syntax with-textual-output-conditions (syntax-rules () ((_ port body0 body ...) (with-i/o-port-error port make-i/o-write-error (lambda () (with-i/o-encoding-error body0 body ...)))))) (define-syntax with-textual-input-conditions (syntax-rules () ((_ port body0 body ...) (with-i/o-port-error port make-i/o-read-error (lambda () (with-i/o-decoding-error body0 body ...)))))) ;;; ;;; Input and output ports. ;;; (define (port-transcoder port) "Return the transcoder object associated with @var{port}, or @code{#f} if the port has no transcoder." (cond ((port-encoding port) => (lambda (encoding) (make-transcoder encoding (native-eol-style) (case (port-conversion-strategy port) ((error) 'raise) ((substitute) 'replace) (else (assertion-violation 'port-transcoder "unsupported error handling mode")))))) (else #f))) (define (binary-port? port) "Returns @code{#t} if @var{port} does not have an associated encoding, @code{#f} otherwise." (not (port-encoding port))) (define (textual-port? port) "Always returns @code{#t}, as all ports can be used for textual I/O in Guile." #t) (define (port-eof? port) (eof-object? (if (binary-port? port) (lookahead-u8 port) (lookahead-char port)))) (define (transcoded-port port transcoder) "Return a new textual port based on @var{port}, using @var{transcoder} to encode and decode data written to or read from its underlying binary port @var{port}." ;; Hackily get at %make-transcoded-port. (let ((result ((@@ (ice-9 binary-ports) %make-transcoded-port) port))) (set-port-encoding! result (transcoder-codec transcoder)) (case (transcoder-error-handling-mode transcoder) ((raise) (set-port-conversion-strategy! result 'error)) ((replace) (set-port-conversion-strategy! result 'substitute)) (else (error "unsupported error handling mode" (transcoder-error-handling-mode transcoder)))) result)) (define (port-position port) "Return the offset (an integer) indicating where the next octet will be read from/written to in @var{port}." ;; FIXME: We should raise an `&assertion' error when not supported. (seek port 0 SEEK_CUR)) (define (set-port-position! port offset) "Set the position where the next octet will be read from/written to @var{port}." ;; FIXME: We should raise an `&assertion' error when not supported. (seek port offset SEEK_SET)) (define (port-has-port-position? port) "Return @code{#t} is @var{port} supports @code{port-position}." (and (false-if-exception (port-position port)) #t)) (define (port-has-set-port-position!? port) "Return @code{#t} is @var{port} supports @code{set-port-position!}." (and (false-if-exception (set-port-position! port (port-position port))) #t)) (define (call-with-port port proc) "Call @var{proc}, passing it @var{port} and closing @var{port} upon exit of @var{proc}. Return the return values of @var{proc}." (call-with-values (lambda () (proc port)) (lambda vals (close-port port) (apply values vals)))) (define* (call-with-bytevector-output-port proc #:optional (transcoder #f)) (receive (port extract) (open-bytevector-output-port transcoder) (call-with-port port proc) (extract))) (define (open-string-input-port str) "Open an input port that will read from @var{str}." (with-fluids ((%default-port-encoding "UTF-8")) (open-input-string str))) (define (r6rs-open filename mode buffer-mode transcoder) (let ((port (with-i/o-filename-conditions filename (lambda () (with-fluids ((%default-port-encoding #f)) (open filename mode)))))) (cond (transcoder (set-port-encoding! port (transcoder-codec transcoder)))) port)) (define (file-options->mode file-options base-mode) (logior base-mode (if (enum-set-member? 'no-create file-options) 0 O_CREAT) (if (enum-set-member? 'no-truncate file-options) 0 O_TRUNC) (if (enum-set-member? 'no-fail file-options) 0 O_EXCL))) (define* (open-file-input-port filename #:optional (file-options (file-options)) (buffer-mode (buffer-mode block)) transcoder) "Return an input port for reading from @var{filename}." (r6rs-open filename O_RDONLY buffer-mode transcoder)) (define* (open-file-input/output-port filename #:optional (file-options (file-options)) (buffer-mode (buffer-mode block)) transcoder) "Return a port for reading from and writing to @var{filename}." (r6rs-open filename (file-options->mode file-options O_RDWR) buffer-mode transcoder)) (define (open-string-output-port) "Return two values: an output port that will collect characters written to it as a string, and a thunk to retrieve the characters associated with that port." (let ((port (with-fluids ((%default-port-encoding "UTF-8")) (open-output-string)))) (values port (lambda () (get-output-string port))))) (define* (open-file-output-port filename #:optional (file-options (file-options)) (buffer-mode (buffer-mode block)) maybe-transcoder) "Return an output port for writing to @var{filename}." (r6rs-open filename (file-options->mode file-options O_WRONLY) buffer-mode maybe-transcoder)) (define (call-with-string-output-port proc) "Call @var{proc}, passing it a string output port. When @var{proc} returns, return the characters accumulated in that port." (let ((port (open-output-string))) (proc port) (get-output-string port))) (define (make-custom-textual-output-port id write! get-position set-position! close) (make-soft-port (vector (lambda (c) (write! (string c) 0 1)) (lambda (s) (write! s 0 (string-length s))) #f ;flush #f ;read character close) "w")) (define (flush-output-port port) (force-output port)) ;;; ;;; Textual output. ;;; (define-condition-type &i/o-encoding &i/o-port make-i/o-encoding-error i/o-encoding-error? (char i/o-encoding-error-char)) (define-syntax with-i/o-encoding-error (syntax-rules () "Convert Guile throws to `encoding-error' to `&i/o-encoding-error'." ((_ body ...) ;; XXX: This is heavyweight for small functions like `put-char'. (with-throw-handler 'encoding-error (lambda () (begin body ...)) (lambda (key subr message errno port chr) (raise (make-i/o-encoding-error port chr))))))) (define (put-char port char) (with-textual-output-conditions port (write-char char port))) (define (put-datum port datum) (with-textual-output-conditions port (write datum port))) (define* (put-string port s #:optional start count) (with-textual-output-conditions port (cond ((not (string? s)) (assertion-violation 'put-string "expected string" s)) ((and start count) (display (substring/shared s start (+ start count)) port)) (start (display (substring/shared s start (string-length s)) port)) (else (display s port))))) ;; Defined here to be able to make use of `with-i/o-encoding-error', but ;; not exported from here, but from `(rnrs io simple)'. (define* (display object #:optional (port (current-output-port))) (with-textual-output-conditions port (guile:display object port))) ;;; ;;; Textual input. ;;; (define-condition-type &i/o-decoding &i/o-port make-i/o-decoding-error i/o-decoding-error?) (define-syntax with-i/o-decoding-error (syntax-rules () "Convert Guile throws to `decoding-error' to `&i/o-decoding-error'." ((_ body ...) ;; XXX: This is heavyweight for small functions like `get-char' and ;; `lookahead-char'. (with-throw-handler 'decoding-error (lambda () (begin body ...)) (lambda (key subr message errno port) (raise (make-i/o-decoding-error port))))))) (define (get-char port) (with-textual-input-conditions port (read-char port))) (define (get-datum port) (with-textual-input-conditions port (read port))) (define (get-line port) (with-textual-input-conditions port (read-line port 'trim))) (define (get-string-all port) (with-textual-input-conditions port (read-string port))) (define (get-string-n port count) "Read up to @var{count} characters from @var{port}. If no characters could be read before encountering the end of file, return the end-of-file object, otherwise return a string containing the characters read." (let* ((s (make-string count)) (rv (get-string-n! port s 0 count))) (cond ((eof-object? rv) rv) ((= rv count) s) (else (substring/shared s 0 rv))))) (define (lookahead-char port) (with-textual-input-conditions port (peek-char port))) ;;; ;;; Standard ports. ;;; (define (standard-input-port) (with-fluids ((%default-port-encoding #f)) (dup->inport 0))) (define (standard-output-port) (with-fluids ((%default-port-encoding #f)) (dup->outport 1))) (define (standard-error-port) (with-fluids ((%default-port-encoding #f)) (dup->outport 2))) ) ;;; ports.scm ends here PK/F2]].ernrs/io/simple.scmnu[;;; simple.scm --- The R6RS simple I/O library ;; Copyright (C) 2010, 2011, 2014 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs io simple (6)) (export eof-object eof-object? call-with-input-file call-with-output-file input-port? output-port? current-input-port current-output-port current-error-port with-input-from-file with-output-to-file open-input-file open-output-file close-input-port close-output-port read-char peek-char read write-char newline display write &i/o make-i/o-error i/o-error? &i/o-read make-i/o-read-error i/o-read-error? &i/o-write make-i/o-write-error i/o-write-error? &i/o-invalid-position make-i/o-invalid-position-error i/o-invalid-position-error? i/o-error-position &i/o-filename make-i/o-filename-error i/o-filename-error? i/o-error-filename &i/o-file-protection make-i/o-file-protection-error i/o-file-protection-error? &i/o-file-is-read-only make-i/o-file-is-read-only-error i/o-file-is-read-only-error? &i/o-file-already-exists make-i/o-file-already-exists-error i/o-file-already-exists-error? &i/o-file-does-not-exist make-i/o-file-does-not-exist-error i/o-file-does-not-exist-error? &i/o-port make-i/o-port-error i/o-port-error? i/o-error-port) (import (only (rnrs io ports) call-with-port close-port open-file-input-port open-file-output-port eof-object eof-object? file-options buffer-mode native-transcoder get-char lookahead-char get-datum put-char put-datum input-port? output-port?) (only (guile) @@ current-input-port current-output-port current-error-port define* with-input-from-port with-output-to-port) (rnrs base (6)) (rnrs files (6)) ;for the condition types ) (define display (@@ (rnrs io ports) display)) (define (call-with-input-file filename proc) (call-with-port (open-file-input-port filename) proc)) (define (call-with-output-file filename proc) (call-with-port (open-file-output-port filename) proc)) (define (with-input-from-file filename thunk) (call-with-input-file filename (lambda (port) (with-input-from-port port thunk)))) (define (with-output-to-file filename thunk) (call-with-output-file filename (lambda (port) (with-output-to-port port thunk)))) (define (open-input-file filename) (open-file-input-port filename (file-options) (buffer-mode block) (native-transcoder))) (define (open-output-file filename) (open-file-output-port filename (file-options) (buffer-mode block) (native-transcoder))) (define close-input-port close-port) (define close-output-port close-port) (define* (read-char #:optional (port (current-input-port))) (get-char port)) (define* (peek-char #:optional (port (current-input-port))) (lookahead-char port)) (define* (read #:optional (port (current-input-port))) (get-datum port)) (define* (write-char char #:optional (port (current-output-port))) (put-char port char)) (define* (newline #:optional (port (current-output-port))) (put-char port #\newline)) (define* (write object #:optional (port (current-output-port))) (put-datum port object)) ) PK/F2]vȯ//rnrs/exceptions.scmnu[;;; exceptions.scm --- The R6RS exceptions library ;; Copyright (C) 2010, 2011, 2013 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs exceptions (6)) (export guard with-exception-handler raise raise-continuable) (import (rnrs base (6)) (rnrs control (6)) (rnrs conditions (6)) (rnrs records procedural (6)) (rnrs records inspection (6)) (only (guile) format newline display filter acons assv-ref throw set-exception-printer! with-throw-handler *unspecified* @@)) ;; When a native guile exception is caught by an R6RS exception ;; handler, we convert it to an R6RS compound condition that includes ;; not only the standard condition objects expected by R6RS code, but ;; also a special &guile condition that preserves the original KEY and ;; ARGS passed to the native Guile catch handler. (define-condition-type &guile &condition make-guile-condition guile-condition? (key guile-condition-key) (args guile-condition-args)) (define (default-guile-condition-converter key args) (condition (make-serious-condition) (guile-common-conditions key args))) (define (guile-common-conditions key args) (apply (case-lambda ((subr msg margs . _) (condition (make-who-condition subr) (make-message-condition msg) (make-irritants-condition margs))) (_ (make-irritants-condition args))) args)) (define (convert-guile-condition key args) (let ((converter (assv-ref guile-condition-converters key))) (condition (or (and converter (converter key args)) (default-guile-condition-converter key args)) ;; Preserve the original KEY and ARGS in the R6RS ;; condition object. (make-guile-condition key args)))) ;; If an R6RS exception handler chooses not to handle a given ;; condition, it will re-raise the condition to pass it on to the next ;; handler. If the condition was converted from a native Guile ;; exception, we must re-raise using the native Guile facilities and ;; the original exception KEY and ARGS. We arrange for this in ;; 'raise' so that native Guile exception handlers will continue to ;; work when mixed with R6RS code. (define (raise obj) (if (guile-condition? obj) (apply throw (guile-condition-key obj) (guile-condition-args obj)) ((@@ (rnrs records procedural) r6rs-raise) obj))) (define raise-continuable (@@ (rnrs records procedural) r6rs-raise-continuable)) (define raise-object-wrapper? (@@ (rnrs records procedural) raise-object-wrapper?)) (define raise-object-wrapper-obj (@@ (rnrs records procedural) raise-object-wrapper-obj)) (define raise-object-wrapper-continuation (@@ (rnrs records procedural) raise-object-wrapper-continuation)) (define (with-exception-handler handler thunk) (with-throw-handler #t thunk (lambda (key . args) (cond ((not (eq? key 'r6rs:exception)) (let ((obj (convert-guile-condition key args))) (handler obj) (raise (make-non-continuable-violation)))) ((and (not (null? args)) (raise-object-wrapper? (car args))) (let* ((cargs (car args)) (obj (raise-object-wrapper-obj cargs)) (continuation (raise-object-wrapper-continuation cargs)) (handler-return (handler obj))) (if continuation (continuation handler-return) (raise (make-non-continuable-violation))))))))) (define-syntax guard0 (syntax-rules () ((_ (variable cond-clause ...) . body) (call/cc (lambda (continuation) (with-exception-handler (lambda (variable) (continuation (cond cond-clause ...))) (lambda () . body))))))) (define-syntax guard (syntax-rules (else) ((_ (variable cond-clause ... . ((else else-clause ...))) . body) (guard0 (variable cond-clause ... (else else-clause ...)) . body)) ((_ (variable cond-clause ...) . body) (guard0 (variable cond-clause ... (else (raise variable))) . body)))) ;;; Exception printing (define (exception-printer port key args punt) (cond ((and (= 1 (length args)) (raise-object-wrapper? (car args))) (let ((obj (raise-object-wrapper-obj (car args)))) (cond ((condition? obj) (display "ERROR: R6RS exception:\n" port) (format-condition port obj)) (else (format port "ERROR: R6RS exception: `~s'" obj))))) (else (punt)))) (define (format-condition port condition) (let ((components (simple-conditions condition))) (if (null? components) (format port "Empty condition object") (let loop ((i 1) (components components)) (cond ((pair? components) (format port " ~a. " i) (format-simple-condition port (car components)) (when (pair? (cdr components)) (newline port)) (loop (+ i 1) (cdr components)))))))) (define (format-simple-condition port condition) (define (print-rtd-fields rtd field-names) (let ((n-fields (vector-length field-names))) (do ((i 0 (+ i 1))) ((>= i n-fields)) (format port " ~a: ~s" (vector-ref field-names i) ((record-accessor rtd i) condition)) (unless (= i (- n-fields 1)) (newline port))))) (let ((condition-name (record-type-name (record-rtd condition)))) (let loop ((rtd (record-rtd condition)) (rtd.fields-list '()) (n-fields 0)) (cond (rtd (let ((field-names (record-type-field-names rtd))) (loop (record-type-parent rtd) (cons (cons rtd field-names) rtd.fields-list) (+ n-fields (vector-length field-names))))) (else (let ((rtd.fields-list (filter (lambda (rtd.fields) (not (zero? (vector-length (cdr rtd.fields))))) (reverse rtd.fields-list)))) (case n-fields ((0) (format port "~a" condition-name)) ((1) (format port "~a: ~s" condition-name ((record-accessor (caar rtd.fields-list) 0) condition))) (else (format port "~a:\n" condition-name) (let loop ((lst rtd.fields-list)) (when (pair? lst) (let ((rtd.fields (car lst))) (print-rtd-fields (car rtd.fields) (cdr rtd.fields)) (when (pair? (cdr lst)) (newline port)) (loop (cdr lst))))))))))))) (set-exception-printer! 'r6rs:exception exception-printer) ;; Guile condition converters ;; ;; Each converter is a procedure (converter KEY ARGS) that returns ;; either an R6RS condition or #f. If #f is returned, ;; 'default-guile-condition-converter' will be used. (define (guile-syntax-violation-converter key args) (apply (case-lambda ((who what where form subform . extra) (condition (make-syntax-violation form subform) (make-who-condition who) (make-message-condition what))) (_ #f)) args)) (define (guile-lexical-violation-converter key args) (condition (make-lexical-violation) (guile-common-conditions key args))) (define (guile-assertion-violation-converter key args) (condition (make-assertion-violation) (guile-common-conditions key args))) (define (guile-undefined-violation-converter key args) (condition (make-undefined-violation) (guile-common-conditions key args))) (define (guile-implementation-restriction-converter key args) (condition (make-implementation-restriction-violation) (guile-common-conditions key args))) (define (guile-error-converter key args) (condition (make-error) (guile-common-conditions key args))) (define (guile-system-error-converter key args) (apply (case-lambda ((subr msg msg-args errno . rest) ;; XXX TODO we should return a more specific error ;; (usually an I/O error) as expected by R6RS programs. ;; Unfortunately this often requires the 'filename' (or ;; other?) which is not currently provided by the native ;; Guile exceptions. (condition (make-error) (guile-common-conditions key args))) (_ (guile-error-converter key args))) args)) ;; TODO: Arrange to have the needed information included in native ;; Guile I/O exceptions, and arrange here to convert them to the ;; proper conditions. Remove the earlier exception conversion ;; mechanism: search for 'with-throw-handler' in the 'rnrs' ;; tree, e.g. 'with-i/o-filename-conditions' and ;; 'with-i/o-port-error' in (rnrs io ports). ;; XXX TODO: How should we handle the 'misc-error', 'vm-error', and ;; 'signal' native Guile exceptions? ;; XXX TODO: Should we handle the 'quit' exception specially? ;; An alist mapping native Guile exception keys to converters. (define guile-condition-converters `((read-error . ,guile-lexical-violation-converter) (syntax-error . ,guile-syntax-violation-converter) (unbound-variable . ,guile-undefined-violation-converter) (wrong-number-of-args . ,guile-assertion-violation-converter) (wrong-type-arg . ,guile-assertion-violation-converter) (keyword-argument-error . ,guile-assertion-violation-converter) (out-of-range . ,guile-assertion-violation-converter) (regular-expression-syntax . ,guile-assertion-violation-converter) (program-error . ,guile-assertion-violation-converter) (goops-error . ,guile-assertion-violation-converter) (null-pointer-error . ,guile-assertion-violation-converter) (system-error . ,guile-system-error-converter) (host-not-found . ,guile-error-converter) (getaddrinfo-error . ,guile-error-converter) (no-data . ,guile-error-converter) (no-recovery . ,guile-error-converter) (try-again . ,guile-error-converter) (stack-overflow . ,guile-implementation-restriction-converter) (numerical-overflow . ,guile-implementation-restriction-converter) (memory-allocation-error . ,guile-implementation-restriction-converter))) (define (set-guile-condition-converter! key proc) (set! guile-condition-converters (acons key proc guile-condition-converters)))) PK/F2]}e rnrs/hashtables.scmnu[;;; hashtables.scm --- The R6RS hashtables library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs hashtables (6)) (export make-eq-hashtable make-eqv-hashtable make-hashtable hashtable? hashtable-size hashtable-ref hashtable-set! hashtable-delete! hashtable-contains? hashtable-update! hashtable-copy hashtable-clear! hashtable-keys hashtable-entries hashtable-equivalence-function hashtable-hash-function hashtable-mutable? equal-hash string-hash string-ci-hash symbol-hash) (import (rename (only (guile) string-hash-ci string-hash hashq hashv modulo *unspecified* @@) (string-hash-ci string-ci-hash)) (only (ice-9 optargs) define*) (rename (only (srfi :69) make-hash-table hash hash-by-identity hash-table-size hash-table-ref/default hash-table-set! hash-table-delete! hash-table-exists? hash-table-update!/default hash-table-copy hash-table-equivalence-function hash-table-hash-function hash-table-keys hash-table-fold) (hash equal-hash) (hash-by-identity symbol-hash)) (rnrs base (6)) (rnrs records procedural (6))) (define r6rs:hashtable (make-record-type-descriptor 'r6rs:hashtable #f #f #t #t '#((mutable wrapped-table) (immutable orig-hash-function) (immutable mutable)))) (define hashtable? (record-predicate r6rs:hashtable)) (define make-r6rs-hashtable (record-constructor (make-record-constructor-descriptor r6rs:hashtable #f #f))) (define r6rs:hashtable-wrapped-table (record-accessor r6rs:hashtable 0)) (define r6rs:hashtable-set-wrapped-table! (record-mutator r6rs:hashtable 0)) (define r6rs:hashtable-orig-hash-function (record-accessor r6rs:hashtable 1)) (define r6rs:hashtable-mutable? (record-accessor r6rs:hashtable 2)) (define hashtable-mutable? r6rs:hashtable-mutable?) (define hash-by-value ((@@ (srfi srfi-69) caller-with-default-size) hashv)) (define (wrap-hash-function proc) (lambda (key capacity) (modulo (proc key) capacity))) (define* (make-eq-hashtable #:optional k) (make-r6rs-hashtable (if k (make-hash-table eq? hashq k) (make-hash-table eq? symbol-hash)) symbol-hash #t)) (define* (make-eqv-hashtable #:optional k) (make-r6rs-hashtable (if k (make-hash-table eqv? hashv k) (make-hash-table eqv? hash-by-value)) hash-by-value #t)) (define* (make-hashtable hash-function equiv #:optional k) (let ((wrapped-hash-function (wrap-hash-function hash-function))) (make-r6rs-hashtable (if k (make-hash-table equiv wrapped-hash-function k) (make-hash-table equiv wrapped-hash-function)) hash-function #t))) (define (hashtable-size hashtable) (hash-table-size (r6rs:hashtable-wrapped-table hashtable))) (define (hashtable-ref hashtable key default) (hash-table-ref/default (r6rs:hashtable-wrapped-table hashtable) key default)) (define (hashtable-set! hashtable key obj) (if (r6rs:hashtable-mutable? hashtable) (hash-table-set! (r6rs:hashtable-wrapped-table hashtable) key obj)) *unspecified*) (define (hashtable-delete! hashtable key) (if (r6rs:hashtable-mutable? hashtable) (hash-table-delete! (r6rs:hashtable-wrapped-table hashtable) key)) *unspecified*) (define (hashtable-contains? hashtable key) (hash-table-exists? (r6rs:hashtable-wrapped-table hashtable) key)) (define (hashtable-update! hashtable key proc default) (if (r6rs:hashtable-mutable? hashtable) (hash-table-update!/default (r6rs:hashtable-wrapped-table hashtable) key proc default)) *unspecified*) (define* (hashtable-copy hashtable #:optional mutable) (make-r6rs-hashtable (hash-table-copy (r6rs:hashtable-wrapped-table hashtable)) (r6rs:hashtable-orig-hash-function hashtable) (and mutable #t))) (define* (hashtable-clear! hashtable #:optional k) (if (r6rs:hashtable-mutable? hashtable) (let* ((ht (r6rs:hashtable-wrapped-table hashtable)) (equiv (hash-table-equivalence-function ht)) (hash-function (r6rs:hashtable-orig-hash-function hashtable)) (wrapped-hash-function (wrap-hash-function hash-function))) (r6rs:hashtable-set-wrapped-table! hashtable (if k (make-hash-table equiv wrapped-hash-function k) (make-hash-table equiv wrapped-hash-function))))) *unspecified*) (define (hashtable-keys hashtable) (list->vector (hash-table-keys (r6rs:hashtable-wrapped-table hashtable)))) (define (hashtable-entries hashtable) (let* ((ht (r6rs:hashtable-wrapped-table hashtable)) (size (hash-table-size ht)) (keys (make-vector size)) (vals (make-vector size))) (hash-table-fold (r6rs:hashtable-wrapped-table hashtable) (lambda (k v i) (vector-set! keys i k) (vector-set! vals i v) (+ i 1)) 0) (values keys vals))) (define (hashtable-equivalence-function hashtable) (hash-table-equivalence-function (r6rs:hashtable-wrapped-table hashtable))) (define (hashtable-hash-function hashtable) (r6rs:hashtable-orig-hash-function hashtable))) PK/F2]d!VVrnrs/conditions.scmnu[;;; conditions.scm --- The R6RS conditions library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs conditions (6)) (export &condition condition simple-conditions condition? condition-predicate condition-accessor define-condition-type &message make-message-condition message-condition? condition-message &warning make-warning warning? &serious make-serious-condition serious-condition? &error make-error error? &violation make-violation violation? &assertion make-assertion-violation assertion-violation? &irritants make-irritants-condition irritants-condition? condition-irritants &who make-who-condition who-condition? condition-who &non-continuable make-non-continuable-violation non-continuable-violation? &implementation-restriction make-implementation-restriction-violation implementation-restriction-violation? &lexical make-lexical-violation lexical-violation? &syntax make-syntax-violation syntax-violation? syntax-violation-form syntax-violation-subform &undefined make-undefined-violation undefined-violation?) (import (only (guile) and=> @@) (rnrs base (6)) (rnrs lists (6)) (rnrs records procedural (6))) (define &compound-condition (make-record-type-descriptor '&compound-condition #f #f #f #f '#((immutable components)))) (define compound-condition? (record-predicate &compound-condition)) (define make-compound-condition (record-constructor (make-record-constructor-descriptor &compound-condition #f #f))) (define simple-conditions (let ((compound-ref (record-accessor &compound-condition 0))) (lambda (condition) (cond ((compound-condition? condition) (compound-ref condition)) ((condition-internal? condition) (list condition)) (else (assertion-violation 'simple-conditions "not a condition" condition)))))) (define (condition? obj) (or (compound-condition? obj) (condition-internal? obj))) (define condition (lambda conditions (define (flatten cond) (if (compound-condition? cond) (simple-conditions cond) (list cond))) (or (for-all condition? conditions) (assertion-violation 'condition "non-condition argument" conditions)) (if (or (null? conditions) (> (length conditions) 1)) (make-compound-condition (apply append (map flatten conditions))) (car conditions)))) (define-syntax define-condition-type (syntax-rules () ((_ condition-type supertype constructor predicate (field accessor) ...) (letrec-syntax ((transform-fields (syntax-rules () ((_ (f a) . rest) (cons '(immutable f a) (transform-fields . rest))) ((_) '()))) (generate-accessors (syntax-rules () ((_ counter (f a) . rest) (begin (define a (condition-accessor condition-type (record-accessor condition-type counter))) (generate-accessors (+ counter 1) . rest))) ((_ counter) (begin))))) (begin (define condition-type (make-record-type-descriptor 'condition-type supertype #f #f #f (list->vector (transform-fields (field accessor) ...)))) (define constructor (record-constructor (make-record-constructor-descriptor condition-type #f #f))) (define predicate (condition-predicate condition-type)) (generate-accessors 0 (field accessor) ...)))))) (define &condition (@@ (rnrs records procedural) &condition)) (define &condition-constructor-descriptor (make-record-constructor-descriptor &condition #f #f)) (define condition-internal? (record-predicate &condition)) (define (condition-predicate rtd) (let ((rtd-predicate (record-predicate rtd))) (lambda (obj) (cond ((compound-condition? obj) (exists rtd-predicate (simple-conditions obj))) ((condition-internal? obj) (rtd-predicate obj)) (else #f))))) (define (condition-accessor rtd proc) (let ((rtd-predicate (record-predicate rtd))) (lambda (obj) (cond ((rtd-predicate obj) (proc obj)) ((compound-condition? obj) (and=> (find rtd-predicate (simple-conditions obj)) proc)) (else #f))))) (define-condition-type &message &condition make-message-condition message-condition? (message condition-message)) (define-condition-type &warning &condition make-warning warning?) (define &serious (@@ (rnrs records procedural) &serious)) (define make-serious-condition (@@ (rnrs records procedural) make-serious-condition)) (define serious-condition? (condition-predicate &serious)) (define-condition-type &error &serious make-error error?) (define &violation (@@ (rnrs records procedural) &violation)) (define make-violation (@@ (rnrs records procedural) make-violation)) (define violation? (condition-predicate &violation)) (define &assertion (@@ (rnrs records procedural) &assertion)) (define make-assertion-violation (@@ (rnrs records procedural) make-assertion-violation)) (define assertion-violation? (condition-predicate &assertion)) (define-condition-type &irritants &condition make-irritants-condition irritants-condition? (irritants condition-irritants)) (define-condition-type &who &condition make-who-condition who-condition? (who condition-who)) (define-condition-type &non-continuable &violation make-non-continuable-violation non-continuable-violation?) (define-condition-type &implementation-restriction &violation make-implementation-restriction-violation implementation-restriction-violation?) (define-condition-type &lexical &violation make-lexical-violation lexical-violation?) (define-condition-type &syntax &violation make-syntax-violation syntax-violation? (form syntax-violation-form) (subform syntax-violation-subform)) (define-condition-type &undefined &violation make-undefined-violation undefined-violation?) ) PK/F2]7y6(6( rnrs/base.scmnu[;;; base.scm --- The R6RS base library ;; Copyright (C) 2010, 2011 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs base (6)) (export boolean? symbol? char? vector? null? pair? number? string? procedure? define define-syntax syntax-rules lambda let let* let-values let*-values letrec letrec* begin quote lambda if set! cond case or and not eqv? equal? eq? + - * / max min abs numerator denominator gcd lcm floor ceiling truncate round rationalize real-part imag-part make-rectangular angle div mod div-and-mod div0 mod0 div0-and-mod0 expt exact-integer-sqrt sqrt exp log sin cos tan asin acos atan make-polar magnitude angle complex? real? rational? integer? exact? inexact? real-valued? rational-valued? integer-valued? zero? positive? negative? odd? even? nan? finite? infinite? exact inexact = < > <= >= number->string string->number boolean=? cons car cdr caar cadr cdar cddr caaar caadr cadar cdaar caddr cdadr cddar cdddr caaaar caaadr caadar cadaar cdaaar cddaar cdadar cdaadr cadadr caaddr caddar cadddr cdaddr cddadr cdddar cddddr list? list length append reverse list-tail list-ref map for-each symbol->string string->symbol symbol=? char->integer integer->char char=? char? char<=? char>=? make-string string string-length string-ref string=? string? string<=? string>=? substring string-append string->list list->string string-for-each string-copy vector? make-vector vector vector-length vector-ref vector-set! vector->list list->vector vector-fill! vector-map vector-for-each error assertion-violation assert call-with-current-continuation call/cc call-with-values dynamic-wind values apply quasiquote unquote unquote-splicing let-syntax letrec-syntax syntax-rules identifier-syntax) (import (rename (except (guile) error raise map string-for-each) (log log-internal) (euclidean-quotient div) (euclidean-remainder mod) (euclidean/ div-and-mod) (centered-quotient div0) (centered-remainder mod0) (centered/ div0-and-mod0) (inf? infinite?) (exact->inexact inexact) (inexact->exact exact)) (srfi srfi-11)) (define string-for-each (case-lambda ((proc string) (let ((end (string-length string))) (let loop ((i 0)) (unless (= i end) (proc (string-ref string i)) (loop (+ i 1)))))) ((proc string1 string2) (let ((end1 (string-length string1)) (end2 (string-length string2))) (unless (= end1 end2) (assertion-violation 'string-for-each "string arguments must all have the same length" string1 string2)) (let loop ((i 0)) (unless (= i end1) (proc (string-ref string1 i) (string-ref string2 i)) (loop (+ i 1)))))) ((proc string . strings) (let ((end (string-length string)) (ends (map string-length strings))) (for-each (lambda (x) (unless (= end x) (apply assertion-violation 'string-for-each "string arguments must all have the same length" string strings))) ends) (let loop ((i 0)) (unless (= i end) (apply proc (string-ref string i) (map (lambda (s) (string-ref s i)) strings)) (loop (+ i 1)))))))) (define map (case-lambda ((f l) (let map1 ((hare l) (tortoise l) (move? #f) (out '())) (if (pair? hare) (if move? (if (eq? tortoise hare) (scm-error 'wrong-type-arg "map" "Circular list: ~S" (list l) #f) (map1 (cdr hare) (cdr tortoise) #f (cons (f (car hare)) out))) (map1 (cdr hare) tortoise #t (cons (f (car hare)) out))) (if (null? hare) (reverse out) (scm-error 'wrong-type-arg "map" "Not a list: ~S" (list l) #f))))) ((f l1 l2) (let map2 ((h1 l1) (h2 l2) (t1 l1) (t2 l2) (move? #f) (out '())) (cond ((pair? h1) (cond ((not (pair? h2)) (scm-error 'wrong-type-arg "map" (if (list? h2) "List of wrong length: ~S" "Not a list: ~S") (list l2) #f)) ((not move?) (map2 (cdr h1) (cdr h2) t1 t2 #t (cons (f (car h1) (car h2)) out))) ((eq? t1 h1) (scm-error 'wrong-type-arg "map" "Circular list: ~S" (list l1) #f)) ((eq? t2 h2) (scm-error 'wrong-type-arg "map" "Circular list: ~S" (list l2) #f)) (else (map2 (cdr h1) (cdr h2) (cdr t1) (cdr t2) #f (cons (f (car h1) (car h2)) out))))) ((and (null? h1) (null? h2)) (reverse out)) ((null? h1) (scm-error 'wrong-type-arg "map" (if (list? h2) "List of wrong length: ~S" "Not a list: ~S") (list l2) #f)) (else (scm-error 'wrong-type-arg "map" "Not a list: ~S" (list l1) #f))))) ((f l1 . rest) (let ((len (length l1))) (let mapn ((rest rest)) (or (null? rest) (if (= (length (car rest)) len) (mapn (cdr rest)) (scm-error 'wrong-type-arg "map" "List of wrong length: ~S" (list (car rest)) #f))))) (let mapn ((l1 l1) (rest rest) (out '())) (if (null? l1) (reverse out) (mapn (cdr l1) (map cdr rest) (cons (apply f (car l1) (map car rest)) out))))))) (define log (case-lambda ((n) (log-internal n)) ((n base) (/ (log n) (log base))))) (define (boolean=? . bools) (define (boolean=?-internal lst last) (or (null? lst) (let ((bool (car lst))) (and (eqv? bool last) (boolean=?-internal (cdr lst) bool))))) (or (null? bools) (let ((bool (car bools))) (and (boolean? bool) (boolean=?-internal (cdr bools) bool))))) (define (symbol=? . syms) (define (symbol=?-internal lst last) (or (null? lst) (let ((sym (car lst))) (and (eq? sym last) (symbol=?-internal (cdr lst) sym))))) (or (null? syms) (let ((sym (car syms))) (and (symbol? sym) (symbol=?-internal (cdr syms) sym))))) (define (real-valued? x) (and (complex? x) (zero? (imag-part x)))) (define (rational-valued? x) (and (real-valued? x) (rational? (real-part x)))) (define (integer-valued? x) (and (rational-valued? x) (= x (floor (real-part x))))) (define (vector-for-each proc . vecs) (apply for-each (cons proc (map vector->list vecs)))) (define (vector-map proc . vecs) (list->vector (apply map (cons proc (map vector->list vecs))))) (define-syntax define-proxy (syntax-rules (@) ;; Define BINDING to point to (@ MODULE ORIGINAL). This hack is to ;; make sure MODULE is loaded lazily, at run-time, when BINDING is ;; encountered, rather than being loaded while compiling and ;; loading (rnrs base). ;; This avoids circular dependencies among modules and makes ;; (rnrs base) more lightweight. ((_ binding (@ module original)) (define-syntax binding (identifier-syntax (module-ref (resolve-interface 'module) 'original)))))) (define-proxy raise (@ (rnrs exceptions) raise)) (define-proxy condition (@ (rnrs conditions) condition)) (define-proxy make-error (@ (rnrs conditions) make-error)) (define-proxy make-assertion-violation (@ (rnrs conditions) make-assertion-violation)) (define-proxy make-who-condition (@ (rnrs conditions) make-who-condition)) (define-proxy make-message-condition (@ (rnrs conditions) make-message-condition)) (define-proxy make-irritants-condition (@ (rnrs conditions) make-irritants-condition)) (define (error who message . irritants) (raise (apply condition (append (list (make-error)) (if who (list (make-who-condition who)) '()) (list (make-message-condition message) (make-irritants-condition irritants)))))) (define (assertion-violation who message . irritants) (raise (apply condition (append (list (make-assertion-violation)) (if who (list (make-who-condition who)) '()) (list (make-message-condition message) (make-irritants-condition irritants)))))) (define-syntax assert (syntax-rules () ((_ expression) (or expression (raise (condition (make-assertion-violation) (make-message-condition (format #f "assertion failed: ~s" 'expression)))))))) ) PK/F2]WG3rnrs/sorting.scmnu[;;; sorting.scm --- The R6RS sorting library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs sorting (6)) (export list-sort vector-sort vector-sort!) (import (rnrs base (6)) (only (guile) *unspecified* stable-sort sort!)) (define (list-sort proc list) (stable-sort list proc)) (define (vector-sort proc vector) (stable-sort vector proc)) (define (vector-sort! proc vector) (sort! vector proc) *unspecified*)) PK/F2]I rnrs/eval.scmnu[;;; eval.scm --- The R6RS `eval' library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs eval (6)) (export eval environment) (import (only (guile) eval make-module module-uses beautify-user-module! set-module-uses!) (rnrs base (6)) (rnrs io simple (6)) (rnrs lists (6))) (define (environment . import-specs) (let ((module (make-module)) (needs-purify? (not (member '(guile) import-specs)))) (beautify-user-module! module) (for-each (lambda (import-spec) (eval (list 'import import-spec) module)) import-specs) (if needs-purify? (set-module-uses! module (cdr (module-uses module)))) module)) ) PK/F2]Rb rnrs/r5rs.scmnu[;;; r5rs.scm --- The R6RS / R5RS compatibility library ;; Copyright (C) 2010 Free Software Foundation, Inc. ;; ;; This library is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public ;; License as published by the Free Software Foundation; either ;; version 3 of the License, or (at your option) any later version. ;; ;; This library is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this library; if not, write to the Free Software ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (library (rnrs r5rs (6)) (export exact->inexact inexact->exact quotient remainder modulo delay force null-environment scheme-report-environment) (import (only (guile) exact->inexact inexact->exact quotient remainder modulo delay force) (only (ice-9 r5rs) scheme-report-environment) (only (ice-9 safe-r5rs) null-environment))) PK/F2]o !!texinfo/docbook.scmnu[;;;; (texinfo docbook) -- translating sdocbook into stexinfo ;;;; ;;;; Copyright (C) 2009, 2010, 2012 Free Software Foundation, Inc. ;;;; Copyright (C) 2007, 2009 Andy Wingo ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;; @c ;; This module exports procedures for transforming a limited subset of ;; the SXML representation of docbook into stexi. It is not complete by ;; any means. The intention is to gather a number of routines and ;; stylesheets so that external modules can parse specific subsets of ;; docbook, for example that set generated by certain tools. ;; ;;; Code: (define-module (texinfo docbook) #:use-module (sxml fold) #:use-module ((srfi srfi-1) #:select (fold)) #:export (*sdocbook->stexi-rules* *sdocbook-block-commands* sdocbook-flatten filter-empty-elements replace-titles)) (define (identity . args) args) (define (identity-deattr tag . body) `(,tag ,@(if (and (pair? body) (pair? (car body)) (eq? (caar body) '@)) (cdr body) body))) (define (detag-one tag body) body) (define tag-replacements '((parameter var) (replaceable var) (type code) (function code) (literal samp) (emphasis emph) (simpara para) (programlisting example) (firstterm dfn) (filename file) (quote cite) (application cite) (symbol code) (note cartouche) (envar env))) (define ignore-list '()) (define (stringify exp) (with-output-to-string (lambda () (write exp)))) (define *sdocbook->stexi-rules* #; "A stylesheet for use with SSAX's @code{pre-post-order}, which defines a number of generic rules for transforming docbook into texinfo." `((@ *preorder* . ,identity) (% *preorder* . ,identity) (para . ,identity-deattr) (orderedlist ((listitem . ,(lambda (tag . body) `(item ,@body)))) . ,(lambda (tag . body) `(enumerate ,@body))) (itemizedlist ((listitem . ,(lambda (tag . body) `(item ,@body)))) . ,(lambda (tag . body) `(itemize ,@body))) (acronym . ,(lambda (tag . body) `(acronym (% (acronym . ,body))))) (term . ,detag-one) (informalexample . ,detag-one) (section . ,identity) (subsection . ,identity) (subsubsection . ,identity) (ulink . ,(lambda (tag attrs . body) (cond ((assq 'url (cdr attrs)) => (lambda (url) `(uref (% ,url (title ,@body))))) (else (car body))))) (*text* . ,detag-one) (*default* . ,(lambda (tag . body) (let ((subst (assq tag tag-replacements))) (cond (subst (if (and (pair? body) (pair? (car body)) (eq? (caar body) '@)) (begin (warn "Ignoring" tag "attributes" (car body)) (append (cdr subst) (cdr body))) (append (cdr subst) body))) ((memq tag ignore-list) #f) (else (warn "Don't know how to convert" tag "to stexi") `(c (% (all ,(stringify (cons tag body)))))))))))) ;; (variablelist ;; ((varlistentry ;; . ,(lambda (tag term . body) ;; `(entry (% (heading ,@(cdr term))) ,@body))) ;; (listitem ;; . ,(lambda (tag simpara) ;; simpara))) ;; . ,(lambda (tag attrs . body) ;; `(table (% (formatter (var))) ,@body))) (define *sdocbook-block-commands* #; "The set of sdocbook element tags that should not be nested inside each other. @xref{texinfo docbook sdocbook-flatten,,sdocbook-flatten}, for more information." '(para programlisting informalexample indexterm variablelist orderedlist refsect1 refsect2 refsect3 refsect4 title example note itemizedlist informaltable)) (define (inline-command? command) (not (memq command *sdocbook-block-commands*))) (define (sdocbook-flatten sdocbook) "\"Flatten\" a fragment of sdocbook so that block elements do not nest inside each other. Docbook is a nested format, where e.g. a @code{refsect2} normally appears inside a @code{refsect1}. Logical divisions in the document are represented via the tree topology; a @code{refsect2} element @emph{contains} all of the elements in its section. On the contrary, texinfo is a flat format, in which sections are marked off by standalone section headers like @code{@@chapter}, and block elements do not nest inside each other. This function takes a nested sdocbook fragment @var{sdocbook} and flattens all of the sections, such that e.g. @example (refsect1 (refsect2 (para \"Hello\"))) @end example becomes @example ((refsect1) (refsect2) (para \"Hello\")) @end example Oftentimes (always?) sectioning elements have @code{} as their first element child; users interested in processing the @code{refsect*} elements into proper sectioning elements like @code{chapter} might be interested in @code{replace-titles} and @code{filter-empty-elements}. @xref{texinfo docbook replace-titles,,replace-titles}, and @ref{texinfo docbook filter-empty-elements,,filter-empty-elements}. Returns a nodeset, as described in @ref{sxml xpath}. That is to say, this function returns an untagged list of stexi elements." (define (fhere str accum block cont) (values (cons str accum) block cont)) (define (fdown node accum block cont) (let ((command (car node)) (attrs (and (pair? (cdr node)) (pair? (cadr node)) (eq? (caadr node) '%) (cadr node)))) (values (if attrs (cddr node) (cdr node)) '() '() (lambda (accum block) (values `(,command ,@(if attrs (list attrs) '()) ,@(reverse accum)) block))))) (define (fup node paccum pblock pcont kaccum kblock kcont) (call-with-values (lambda () (kcont kaccum kblock)) (lambda (ret block) (if (inline-command? (car ret)) (values (cons ret paccum) (append kblock pblock) pcont) (values paccum (append kblock (cons ret pblock)) pcont))))) (call-with-values (lambda () (foldts*-values fdown fup fhere sdocbook '() '() #f)) (lambda (accum block cont) (reverse block)))) (define (filter-empty-elements sdocbook) "Filters out empty elements in an sdocbook nodeset. Mostly useful after running @code{sdocbook-flatten}." (reverse (fold (lambda (x rest) (if (and (pair? x) (null? (cdr x))) rest (cons x rest))) '() sdocbook))) (define (replace-titles sdocbook-fragment) "Iterate over the sdocbook nodeset @var{sdocbook-fragment}, transforming contiguous @code{refsect} and @code{title} elements into the appropriate texinfo sectioning command. Most useful after having run @code{sdocbook-flatten}. For example: @example (replace-titles '((refsect1) (title \"Foo\") (para \"Bar.\"))) @result{} '((chapter \"Foo\") (para \"Bar.\")) @end example " (define sections '((refsect1 . chapter) (refsect2 . section) (refsect3 . subsection) (refsect4 . subsubsection))) (let lp ((in sdocbook-fragment) (out '())) (cond ((null? in) (reverse out)) ((and (pair? (car in)) (assq (caar in) sections)) ;; pull out the title => (lambda (pair) (lp (cddr in) (cons `(,(cdr pair) ,@(cdadr in)) out)))) (else (lp (cdr in) (cons (car in) out)))))) PK�����/F2]D>t&��t&����texinfo/serialize.scmnu�[��������;;;; (texinfo serialize) -- rendering stexinfo as texinfo ;;;; ;;;; Copyright (C) 2009, 2012, 2013 Free Software Foundation, Inc. ;;;; Copyright (C) 2003,2004,2009 Andy Wingo <wingo at pobox dot com> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;Serialization of @code{stexi} to plain texinfo. ;; ;;; Code: (define-module (texinfo serialize) #:use-module (texinfo) #:use-module (texinfo string-utils) #:use-module (sxml transform) #:use-module (srfi srfi-1) #:use-module (srfi srfi-13) #:export (stexi->texi)) (define (list-intersperse src-l elem) (if (null? src-l) src-l (let loop ((l (cdr src-l)) (dest (cons (car src-l) '()))) (if (null? l) (reverse dest) (loop (cdr l) (cons (car l) (cons elem dest))))))) ;; converts improper lists to proper lists. (define (filter* pred l) (let lp ((in l) (out '())) (cond ((null? in) (reverse! out)) ((pair? in) (lp (cdr in) (if (pred (car in)) (cons (car in) out) out))) (else (lp '() (if (pred in) (cons in out) out)))))) ;; (list* 'a '(b c) 'd '(e f g)) => '(a b c d e f g) (define (list* . args) (let* ((args (reverse args)) (tail (car args))) (let lp ((in (cdr args)) (out tail)) (cond ((null? in) out) ((pair? (car in)) (lp (cdr in) (append (car in) out))) ((null? (car in)) (lp (cdr in) out)) (else (lp (cdr in) (cons (car in) out))))))) ;; Why? Well, because syntax-case defines `include', and carps about its ;; wrong usage below... (eval-when (expand load eval) (define (include exp lp command type formals args accum) (list* "\n" (list-intersperse args " ") " " command "@" accum))) (define (empty-command exp lp command type formals args accum) (list* " " command "@" accum)) (define (inline-text exp lp command type formals args accum) (if (not (string=? command "*braces*")) ;; fixme :( (list* "}" (append-map (lambda (x) (lp x '())) (reverse (cdr exp))) "{" command "@" accum) (list* "@}" (append-map (lambda (x) (lp x '())) (reverse (cdr exp))) "@{" accum))) (define (inline-args exp lp command type formals args accum) (list* "}" (if (not args) "" (list-intersperse (map (lambda (x) (cond ((not x) "") ((pair? x) (if (pair? (cdr x)) (warn "Strange inline-args!" args)) (car x)) (else (error "Invalid inline-args" args)))) (drop-while not (map (lambda (x) (assq-ref args x)) (reverse formals)))) ",")) "{" command "@" accum)) (define (inline-text-args exp lp command type formals args accum) (list* "}" (if (not args) "" (apply append (list-intersperse (map (lambda (x) (append-map (lambda (x) (lp x '())) (reverse x))) (drop-while not (map (lambda (x) (assq-ref args x)) (reverse formals)))) '(",")))) "{" command "@" accum)) (define (serialize-text-args lp formals args) (apply append (list-intersperse (map (lambda (arg) (append-map (lambda (x) (lp x '())) arg)) (map reverse (drop-while not (map (lambda (x) (assq-ref args x)) (reverse formals))))) '(" ")))) (define (eol-text-args exp lp command type formals args accum) (list* "\n" (serialize-text-args lp formals args) " " command "@" accum)) (define (eol-text exp lp command type formals args accum) (list* "\n" (append-map (lambda (x) (lp x '())) (reverse (if args (cddr exp) (cdr exp)))) " " command "@" accum)) (define (eol-args exp lp command type formals args accum) (list* "\n" (list-intersperse (apply append (drop-while not (map (lambda (x) (assq-ref args x)) (reverse formals)))) ", ") " " command "@" accum)) (define (environ exp lp command type formals args accum) (case (car exp) ((texinfo) (list* "@bye\n" (append-map (lambda (x) (lp x '())) (reverse (cddr exp))) "\n@c %**end of header\n\n" (reverse (assq-ref args 'title)) "@settitle " (or (and=> (assq-ref args 'filename) (lambda (filename) (cons "\n" (reverse (cons "@setfilename " filename))))) "") "\\input texinfo @c -*-texinfo-*-\n@c %**start of header\n" accum)) (else (list* "\n\n" command "@end " (let ((body (append-map (lambda (x) (lp x '())) (reverse (if args (cddr exp) (cdr exp)))))) (if (or (null? body) (eqv? (string-ref (car body) (1- (string-length (car body)))) #\newline)) body (cons "\n" body))) "\n" (serialize-text-args lp formals args) " " command "@" accum)))) (define (table-environ exp lp command type formals args accum) (list* "\n\n" command "@end " (append-map (lambda (x) (lp x '())) (reverse (if args (cddr exp) (cdr exp)))) "\n" (let* ((arg (if args (cadar args) ""))) ;; zero or one args (if (pair? arg) (list (symbol->string (car arg)) "@") arg)) " " command "@" accum)) (define (wrap strings) (fill-string (string-concatenate strings) #:line-width 72 #:break-long-words? #f)) (define (paragraph exp lp command type formals args accum) (list* "\n\n" (wrap (reverse (append-map (lambda (x) (lp x '())) (reverse (cdr exp))))) accum)) (define (item exp lp command type formals args accum) (list* (append-map (lambda (x) (lp x '())) (reverse (cdr exp))) "@item\n" accum)) (define (entry exp lp command type formals args accum) (list* (append-map (lambda (x) (lp x '())) (reverse (cddr exp))) "\n" (append-map (lambda (x) (lp x '())) (reverse (cdar args))) "@item " accum)) (define (fragment exp lp command type formals args accum) (list* "\n@c %end of fragment\n" (append-map (lambda (x) (lp x '())) (reverse (cdr exp))) "\n@c %start of fragment\n\n" accum)) (define serializers `((EMPTY-COMMAND . ,empty-command) (INLINE-TEXT . ,inline-text) (INLINE-ARGS . ,inline-args) (INLINE-TEXT-ARGS . ,inline-text-args) (EOL-TEXT . ,eol-text) (EOL-TEXT-ARGS . ,eol-text-args) (INDEX . ,eol-text-args) (EOL-ARGS . ,eol-args) (ENVIRON . ,environ) (TABLE-ENVIRON . ,table-environ) (ENTRY . ,entry) (ITEM . ,item) (PARAGRAPH . ,paragraph) (FRAGMENT . ,fragment) (#f . ,include))) ; support writing include statements (define (serialize exp lp command type formals args accum) ((or (assq-ref serializers type) (error "Unknown command type" exp type)) exp lp command type formals args accum)) (define escaped-chars '(#\} #\{ #\@)) (define (escape str) "Escapes any illegal texinfo characters (currently @{, @}, and @@)." (let loop ((in (string->list str)) (out '())) (if (null? in) (apply string (reverse out)) (if (memq (car in) escaped-chars) (loop (cdr in) (cons* (car in) #\@ out)) (loop (cdr in) (cons (car in) out)))))) (define (stexi->texi tree) "Serialize the stexi @var{tree} into plain texinfo." (string-concatenate-reverse (let lp ((in tree) (out '())) (cond ((or (not in) (null? in)) out) ((string? in) (cons (escape in) out)) ((pair? in) (let ((command-spec (assq (car in) texi-command-specs))) (if (not command-spec) (begin (warn "Unknown stexi command, not rendering" in) out) (serialize in lp (symbol->string (car in)) (cadr command-spec) (filter* symbol? (cddr command-spec)) (cond ((and (pair? (cdr in)) (pair? (cadr in)) (eq? (caadr in) '%)) (cdadr in)) ((not (cadr command-spec)) ;; include (cdr in)) (else #f)) out)))) (else (error "Invalid stexi" in)))))) ;;; arch-tag: d3fa16ea-0bf7-4ec5-ab9f-3f08490f77f5 PK�����/F2]Te"$%��$%����texinfo/plain-text.scmnu�[��������;;;; (texinfo plain-text) -- rendering stexinfo as plain text ;;;; ;;;; Copyright (C) 2009, 2010, 2011, 2013 Free Software Foundation, Inc. ;;;; Copyright (C) 2003,2004,2009 Andy Wingo <wingo at pobox dot com> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;Transformation from stexi to plain-text. Strives to re-create the ;;output from @code{info}; comes pretty damn close. ;; ;;; Code: (define-module (texinfo plain-text) #:use-module (texinfo) #:use-module (texinfo string-utils) #:use-module (sxml transform) #:use-module (srfi srfi-1) #:use-module (srfi srfi-13) #:use-module (ice-9 match) #:export (stexi->plain-text)) ;; The return value is a string. (define (arg-ref key %-args) (and=> (and=> (assq key (cdr %-args)) cdr) stexi->plain-text)) (define (arg-req key %-args) (or (arg-ref key %-args) (error "Missing argument:" key %-args))) (define (make-ticker str) (lambda () str)) (define (make-enumerator n) (lambda () (let ((last n)) (set! n (1+ n)) (format #f "~A. " last)))) (define *indent* (make-fluid "")) (define *itemizer* (make-fluid (make-ticker "* "))) (define-macro (with-indent n . body) `(with-fluids ((*indent* (string-append (fluid-ref *indent*) (make-string ,n #\space)))) ,@body)) (define (make-indenter n proc) (lambda args (with-indent n (apply proc args)))) (define (string-indent str) (string-append (fluid-ref *indent*) str "\n")) (define-macro (with-itemizer itemizer . body) `(with-fluids ((*itemizer* ,itemizer)) ,@body)) (define (wrap* . strings) (let ((indent (fluid-ref *indent*))) (fill-string (string-concatenate strings) #:line-width 72 #:initial-indent indent #:subsequent-indent indent))) (define (wrap . strings) (string-append (apply wrap* strings) "\n\n")) (define (wrap-heading . strings) (string-append (apply wrap* strings) "\n")) (define (ref tag args) (let* ((node (arg-req 'node args)) (name (or (arg-ref 'name args) node)) (manual (arg-ref 'manual args))) (string-concatenate (cons* (or (and=> (assq tag '((xref "See ") (pxref "see "))) cadr) "") name (if manual `(" in manual " ,manual) '()))))) (define (uref tag args) (let ((url (arg-req 'url args)) (title (arg-ref 'title args))) (if title (string-append title " (" url ")") (string-append "`" url "'")))) (define (def tag args . body) (define (first-line) (string-join (filter identity (map (lambda (x) (arg-ref x args)) '(data-type class name arguments))) " ")) (let* ((category (case tag ((defun) "Function") ((defspec) "Special Form") ((defvar) "Variable") (else (arg-req 'category args))))) (string-append (wrap-heading (string-append " - " category ": " (first-line))) (with-indent 5 (stexi->plain-text body))))) (define (enumerate tag . elts) (define (tonumber start) (let ((c (string-ref start 0))) (cond ((number? c) (string->number start)) (else (1+ (- (char->integer c) (char->integer (if (char-upper-case? c) #\A #\a)))))))) (let* ((args? (and (pair? elts) (pair? (car elts)) (eq? (caar elts) '%))) (start (and args? (arg-ref 'start (car elts))))) (with-itemizer (make-enumerator (if start (tonumber start) 1)) (with-indent 5 (stexi->plain-text (if start (cdr elts) elts)))))) (define (itemize tag args . elts) (with-itemizer (make-ticker "* ") (with-indent 5 (stexi->plain-text elts)))) (define (item tag . elts) (let* ((ret (stexi->plain-text elts)) (tick ((fluid-ref *itemizer*))) (tick-pos (- (string-length (fluid-ref *indent*)) (string-length tick)))) (if (and (not (string-null? ret)) (not (negative? tick-pos))) (string-copy! ret tick-pos tick)) ret)) (define (table tag args . body) (stexi->plain-text body)) (define (entry tag args . body) (let ((heading (wrap-heading (stexi->plain-text (arg-req 'heading args))))) (string-append heading (with-indent 5 (stexi->plain-text body))))) (define (make-underliner char) (lambda (tag . body) (let ((str (stexi->plain-text body))) (string-append "\n" (string-indent str) (string-indent (make-string (string-length str) char)) "\n")))) (define chapter (make-underliner #\*)) (define section (make-underliner #\=)) (define subsection (make-underliner #\-)) (define subsubsection (make-underliner #\.)) (define (example tag . body) (let ((ret (stexi->plain-text body))) (string-append (string-concatenate (with-indent 5 (map string-indent (string-split ret #\newline)))) "\n"))) (define (verbatim tag . body) (let ((ret (stexi->plain-text body))) (string-append (string-concatenate (map string-indent (string-split ret #\newline))) "\n"))) (define (fragment tag . body) (string-concatenate (map-in-order stexi->plain-text body))) (define (para tag . body) (wrap (stexi->plain-text body))) (define (make-surrounder str) (lambda (tag . body) (string-append str (stexi->plain-text body) str))) (define (code tag . body) (string-append "`" (stexi->plain-text body) "'")) (define (key tag . body) (string-append "<" (stexi->plain-text body) ">")) (define (var tag . body) (string-upcase (stexi->plain-text body))) (define (passthrough tag . body) (stexi->plain-text body)) (define (texinfo tag args . body) (let ((title (chapter 'foo (arg-req 'title args)))) (string-append title (stexi->plain-text body)))) (define ignore-list '(page setfilename setchapternewpage iftex ifinfo ifplaintext ifxml sp vskip menu ignore syncodeindex comment c % node anchor)) (define (ignored? tag) (memq tag ignore-list)) (define tag-handlers `((title ,chapter) (chapter ,chapter) (section ,section) (subsection ,subsection) (subsubsection ,subsubsection) (appendix ,chapter) (appendixsec ,section) (appendixsubsec ,subsection) (appendixsubsubsec ,subsubsection) (unnumbered ,chapter) (unnumberedsec ,section) (unnumberedsubsec ,subsection) (unnumberedsubsubsec ,subsubsection) (majorheading ,chapter) (chapheading ,chapter) (heading ,section) (subheading ,subsection) (subsubheading ,subsubsection) (strong ,(make-surrounder "*")) (sample ,code) (samp ,code) (code ,code) (math ,passthrough) (kbd ,code) (key ,key) (var ,var) (env ,code) (file ,code) (command ,code) (option ,code) (url ,code) (dfn ,(make-surrounder "\"")) (cite ,(make-surrounder "\"")) (acro ,passthrough) (email ,key) (emph ,(make-surrounder "_")) (sc ,var) (copyright ,(lambda args "(C)")) (result ,(lambda args "==>")) (dots ,(lambda args "...")) (xref ,ref) (ref ,ref) (pxref ,ref) (uref ,uref) (texinfo ,texinfo) (quotation ,(make-indenter 5 para)) (itemize ,itemize) (enumerate ,enumerate) (item ,item) (table ,table) (entry ,entry) (example ,example) (lisp ,example) (smallexample ,example) (smalllisp ,example) (verbatim ,verbatim) (*fragment* ,fragment) (deftp ,def) (defcv ,def) (defivar ,def) (deftypeivar ,def) (defop ,def) (deftypeop ,def) (defmethod ,def) (deftypemethod ,def) (defopt ,def) (defvr ,def) (defvar ,def) (deftypevr ,def) (deftypevar ,def) (deffn ,def) (deftypefn ,def) (defmac ,def) (defspec ,def) (defun ,def) (deftypefun ,def))) (define (stexi->plain-text tree) "Transform @var{tree} into plain text. Returns a string." (match tree (() "") ((? string?) tree) (((? symbol? tag) body ...) (let ((handler (and (not (ignored? tag)) (or (and=> (assq tag tag-handlers) cadr) para)))) (if handler (apply handler tree) ""))) ((tree ...) (string-concatenate (map-in-order stexi->plain-text tree))) (_ ""))) ;;; arch-tag: f966c3f6-3b46-4790-bbf9-3ad27e4917c2 PK�����/F2]Rx@��@����texinfo/string-utils.scmnu�[��������;;;; (texinfo string-utils) -- text filling and wrapping ;;;; ;;;; Copyright (C) 2009, 2013 Free Software Foundation, Inc. ;;;; Copyright (C) 2003 Richard Todd ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; Module @samp{(texinfo string-utils)} provides various string-related ;; functions useful to Guile's texinfo support. ;;; Code: (define-module (texinfo string-utils) #:use-module (srfi srfi-13) #:use-module (srfi srfi-14) #:export (escape-special-chars transform-string expand-tabs center-string left-justify-string right-justify-string collapse-repeated-chars make-text-wrapper fill-string string->wrapped-lines)) (define* (transform-string str match? replace #:optional (start #f) (end #f)) "Uses @var{match?} against each character in @var{str}, and performs a replacement on each character for which matches are found. @var{match?} may either be a function, a character, a string, or @code{#t}. If @var{match?} is a function, then it takes a single character as input, and should return @samp{#t} for matches. @var{match?} is a character, it is compared to each string character using @code{char=?}. If @var{match?} is a string, then any character in that string will be considered a match. @code{#t} will cause every character to be a match. If @var{replace} is a function, it is called with the matched character as an argument, and the returned value is sent to the output string via @samp{display}. If @var{replace} is anything else, it is sent through the output string via @samp{display}. Note that te replacement for the matched characters does not need to be a single character. That is what differentiates this function from @samp{string-map}, and what makes it useful for applications such as converting @samp{#\\&} to @samp{\"&\"} in web page text. Some other functions in this module are just wrappers around common uses of @samp{transform-string}. Transformations not possible with this function should probably be done with regular expressions. If @var{start} and @var{end} are given, they control which portion of the string undergoes transformation. The entire input string is still output, though. So, if @var{start} is @samp{5}, then the first five characters of @var{str} will still appear in the returned string. @lisp ; these two are equivalent... (transform-string str #\\space #\\-) ; change all spaces to -'s (transform-string str (lambda (c) (char=? #\\space c)) #\\-) @end lisp" ;; I had implemented this with string-fold, but it was ;; slower... (let* ((os (open-output-string)) (matcher (cond ((char? match?) (lambda (c) (char=? match? c))) ((procedure? match?) match?) ((string? match?) (lambda (c) (string-index match? c))) ((boolean? match?) (lambda (c) match?)) (else (throw 'bad-type "expected #t, char, string, or procedure")))) (replacer (if (procedure? replace) (lambda (c) (display (replace c) os)) (lambda (c) (display replace os))))) ;; put the first part in, un-transformed if they asked for it... (if (and start (<= start (string-length str))) (display (substring str 0 start) os)) ;; process the portion they want processed.... (string-for-each (lambda (c) (if (matcher c) ;; we have a match! replace the char as directed... (replacer c) ;; not a match, just insert the character itself... (write-char c os))) str (or start 0) (or end (string-length str))) ;; if there was any at the end, tack it on... (if (and end (< end (string-length str))) (display (substring str end) os)) (get-output-string os))) (define* (expand-tabs str #:optional (tab-size 8)) "Returns a copy of @var{str} with all tabs expanded to spaces. @var{tab-size} defaults to 8. Assuming tab size of 8, this is equivalent to: @lisp (transform-string str #\\tab \" \") @end lisp" (transform-string str #\tab (make-string tab-size #\space))) (define (escape-special-chars str special-chars escape-char) "Returns a copy of @var{str} with all given special characters preceded by the given @var{escape-char}. @var{special-chars} can either be a single character, or a string consisting of all the special characters. @lisp ;; make a string regexp-safe... (escape-special-chars \"***(Example String)***\" \"[]()/*.\" #\\\\) => \"\\\\*\\\\*\\\\*\\\\(Example String\\\\)\\\\*\\\\*\\\\*\" ;; also can escape a singe char... (escape-special-chars \"richardt@@vzavenue.net\" #\\@@ #\\@@) => \"richardt@@@@vzavenue.net\" @end lisp" (transform-string str (if (char? special-chars) ;; if they gave us a char, use char=? (lambda (c) (char=? c special-chars)) ;; if they gave us a string, see if our character is in it (lambda (c) (string-index special-chars c))) ;; replace matches with the character preceded by the escape character (lambda (c) (string escape-char c)))) (define* (center-string str #:optional (width 80) (chr #\space) (rchr #f)) "Returns a copy of @var{str} centered in a field of @var{width} characters. Any needed padding is done by character @var{chr}, which defaults to @samp{#\\space}. If @var{rchr} is provided, then the padding to the right will use it instead. See the examples below. left and @var{rchr} on the right. The default @var{width} is 80. The default @var{chr} and @var{rchr} is @samp{#\\space}. The string is never truncated. @lisp (center-string \"Richard Todd\" 24) => \" Richard Todd \" (center-string \" Richard Todd \" 24 #\\=) => \"===== Richard Todd =====\" (center-string \" Richard Todd \" 24 #\\< #\\>) => \"<<<<< Richard Todd >>>>>\" @end lisp" (let* ((len (string-length str)) (lpad (make-string (max (quotient (- width len) 2) 0) chr)) ;; right-char == char unless it has been provided by the user (right-chr (or rchr chr)) (rpad (if (char=? right-chr chr) lpad (make-string (max (quotient (- width len) 2) 0) right-chr)))) (if (>= len width) str (string-append lpad str rpad (if (odd? (- width len)) (string right-chr) ""))))) (define* (left-justify-string str #:optional (width 80) (chr #\space)) "@code{left-justify-string str [width chr]}. Returns a copy of @var{str} padded with @var{chr} such that it is left justified in a field of @var{width} characters. The default @var{width} is 80. Unlike @samp{string-pad} from srfi-13, the string is never truncated." (let* ((len (string-length str)) (pad (make-string (max (- width len) 0) chr))) (if (>= len width) str (string-append str pad)))) (define* (right-justify-string str #:optional (width 80) (chr #\space)) "Returns a copy of @var{str} padded with @var{chr} such that it is right justified in a field of @var{width} characters. The default @var{width} is 80. The default @var{chr} is @samp{#\\space}. Unlike @samp{string-pad} from srfi-13, the string is never truncated." (let* ((len (string-length str)) (pad (make-string (max (- width len) 0) chr))) (if (>= len width) str (string-append pad str)))) (define* (collapse-repeated-chars str #:optional (chr #\space) (num 1)) "Returns a copy of @var{str} with all repeated instances of @var{chr} collapsed down to at most @var{num} instances. The default value for @var{chr} is @samp{#\\space}, and the default value for @var{num} is 1. @lisp (collapse-repeated-chars \"H e l l o\") => \"H e l l o\" (collapse-repeated-chars \"H--e--l--l--o\" #\\-) => \"H-e-l-l-o\" (collapse-repeated-chars \"H-e--l---l----o\" #\\- 2) => \"H-e--l--l--o\" @end lisp" ;; define repeat-locator as a stateful match? function which remembers ;; the last character it had seen. (let ((repeat-locator ;; initialize prev-chr to something other than what we're seeking... (let ((prev-chr (if (char=? chr #\space) #\A #\space)) (match-count 0)) (lambda (c) (if (and (char=? c prev-chr) (char=? prev-chr chr)) ;; found enough duplicates if the match-count is high enough (begin (set! match-count (+ 1 match-count)) (>= match-count num)) ;; did not find a duplicate (begin (set! match-count 0) (set! prev-chr c) #f)))))) ;; transform the string with our stateful matcher... ;; deleting matches... (transform-string str repeat-locator ""))) ;; split a text string into segments that have the form... ;; <ws non-ws> <ws non-ws> etc.. (define (split-by-single-words str) (let ((non-wschars (char-set-complement char-set:whitespace))) (let loop ((ans '()) (index 0)) (let ((next-non-ws (string-index str non-wschars index))) (if next-non-ws ;; found non-ws...look for ws following... (let ((next-ws (string-index str char-set:whitespace next-non-ws))) (if next-ws ;; found the ws following... (loop (cons (substring str index next-ws) ans) next-ws) ;; did not find ws...must be the end... (reverse (cons (substring str index) ans)))) ;; did not find non-ws... only ws at end of the string... (reverse ans)))))) (define (end-of-sentence? str) "Return #t when STR likely denotes the end of sentence." (let ((len (string-length str))) (and (> len 1) (eqv? #\. (string-ref str (- len 1))) (not (eqv? #\. (string-ref str (- len 2))))))) (define* (make-text-wrapper #:key (line-width 80) (expand-tabs? #t) (tab-width 8) (collapse-whitespace? #t) (subsequent-indent "") (initial-indent "") (break-long-words? #t)) "Returns a procedure that will split a string into lines according to the given parameters. @table @code @item #:line-width This is the target length used when deciding where to wrap lines. Default is 80. @item #:expand-tabs? Boolean describing whether tabs in the input should be expanded. Default is #t. @item #:tab-width If tabs are expanded, this will be the number of spaces to which they expand. Default is 8. @item #:collapse-whitespace? Boolean describing whether the whitespace inside the existing text should be removed or not. Default is #t. If text is already well-formatted, and is just being wrapped to fit in a different width, then set this to @samp{#f}. This way, many common text conventions (such as two spaces between sentences) can be preserved if in the original text. If the input text spacing cannot be trusted, then leave this setting at the default, and all repeated whitespace will be collapsed down to a single space. @item #:initial-indent Defines a string that will be put in front of the first line of wrapped text. Default is the empty string, ``''. @item #:subsequent-indent Defines a string that will be put in front of all lines of wrapped text, except the first one. Default is the empty string, ``''. @item #:break-long-words? If a single word is too big to fit on a line, this setting tells the wrapper what to do. Defaults to #t, which will break up long words. When set to #f, the line will be allowed, even though it is longer than the defined @code{#:line-width}. @end table The return value is a procedure of one argument, the input string, which returns a list of strings, where each element of the list is one line." (lambda (str) ;; replace newlines with spaces (set! str (transform-string str (lambda (c) (char=? c #\nl)) #\space)) ;; expand tabs if they wanted us to... (if expand-tabs? (set! str (expand-tabs str tab-width))) ;; collapse whitespace if they wanted us to... (if collapse-whitespace? (set! str (collapse-repeated-chars str))) ;; drop any whitespace from the front... (set! str (string-trim str)) ;; now start breaking the text into lines... (let loop ((ans '()) (words (split-by-single-words str)) (line initial-indent) (count 0)) (if (null? words) ;; out of words? ...done! (reverse (if (> count 0) (cons line ans) ans)) ;; not out of words...keep going... (let ((length-left (- line-width (string-length line))) (next-word (if (= count 0) (string-trim (car words)) (car words)))) (cond ;; does the next entry fit? ((<= (string-length next-word) length-left) (loop ans (cdr words) (if (and collapse-whitespace? (end-of-sentence? line)) ;; Add an extra space after the period. (string-append line " " next-word) (string-append line next-word)) (+ count 1))) ;; ok, it didn't fit...is there already at least one word on the line? ((> count 0) ;; try to use it for the next line, then... (loop (cons line ans) words subsequent-indent 0)) ;; ok, it didn't fit...and it's the first word. ;; were we told to break up long words? (break-long-words? ;; break the like at the limit, since the user wants us to... (loop (cons (string-append line (substring next-word 0 length-left)) ans) (cons (substring next-word length-left) (cdr words)) subsequent-indent 0)) ;; well, then is it the first word and we *shouldn't* break long words, then... (else (loop (cons (string-append line next-word) ans) (cdr words) subsequent-indent 0)))))))) (define (string->wrapped-lines str . kwargs) "@code{string->wrapped-lines str keywds ...}. Wraps the text given in string @var{str} according to the parameters provided in @var{keywds}, or the default setting if they are not given. Returns a list of strings representing the formatted lines. Valid keyword arguments are discussed in @code{make-text-wrapper}." ((apply make-text-wrapper kwargs) str)) (define (fill-string str . kwargs) "Wraps the text given in string @var{str} according to the parameters provided in @var{kwargs}, or the default setting if they are not given. Returns a single string with the wrapped text. Valid keyword arguments are discussed in @code{make-text-wrapper}." (string-join (apply string->wrapped-lines str kwargs) "\n" 'infix)) PK�����/F2]V��V����texinfo/reflection.scmnu�[��������;;;; (texinfo reflection) -- documenting Scheme as stexinfo ;;;; ;;;; Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc. ;;;; Copyright (C) 2003,2004,2009 Andy Wingo <wingo at pobox dot com> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;Routines to generare @code{stexi} documentation for objects and ;;modules. ;; ;;Note that in this context, an @dfn{object} is just a value associated ;;with a location. It has nothing to do with GOOPS. ;; ;;; Code: (define-module (texinfo reflection) #:use-module ((srfi srfi-1) #:select (append-map)) #:use-module (oop goops) #:use-module (texinfo) #:use-module (texinfo plain-text) #:use-module (srfi srfi-13) #:use-module (ice-9 session) #:use-module (ice-9 documentation) #:use-module (ice-9 optargs) #:use-module ((sxml transform) #:select (pre-post-order)) #:export (module-stexi-documentation script-stexi-documentation object-stexi-documentation package-stexi-standard-copying package-stexi-standard-titlepage package-stexi-generic-menu package-stexi-standard-menu package-stexi-extended-menu package-stexi-standard-prologue package-stexi-documentation package-stexi-documentation-for-include)) ;; List for sorting the definitions in a module (define defs '(deftp defcv defivar deftypeivar defop deftypeop defmethod deftypemethod defopt defvr defvar deftypevr deftypevar deffn deftypefn defmac defspec defun deftypefun)) (define (sort-defs ordering a b) (define (def x) ;; a and b are lists of the form ((anchor ...) (def* ...)...) (cadr x)) (define (name x) (cadr (assq 'name (cdadr (def x))))) (define (priority x) (list-index defs (car (def x)))) (define (order x) (or (list-index ordering (string->symbol (name x))) ;; if the def is not in the list, a big number 1234567890)) (define (compare-in-order proc eq? < . args) (if (not (eq? (proc a) (proc b))) (< (proc a) (proc b)) (or (null? args) (apply compare-in-order args)))) (compare-in-order order = < priority = < name string=? string<=?)) (define (list*-join l infix restfix) (let lp ((in l) (out '())) (cond ((null? in) (reverse! out)) ((symbol? in) (reverse! (cons* in restfix out))) (else (lp (cdr in) (if (null? out) (list (car in)) (cons* (car in) infix out))))))) (define (process-args args) (map (lambda (x) (if (string? x) x (object->string x))) (list*-join (or args '()) " " " . "))) (define (get-proc-args proc) (cond ((procedure-arguments proc) => (lambda (args) (let ((required-args (assq-ref args 'required)) (optional-args (assq-ref args 'optional)) (keyword-args (assq-ref args 'keyword)) (rest-arg (assq-ref args 'rest))) (process-args (append ;; start with the required args... (map symbol->string required-args) ;; add any optional args if needed... (map (lambda (a) (if (list? a) (format #f "[~a = ~s]" (car a) (cadr a)) (format #f "[~a]" a))) optional-args) ;; now the keyword args.. (map (lambda (a) (if (pair? a) (format #f "[~a]" (car a)) (format #f "[#:~a]" a))) keyword-args) ;; now the rest arg... (if rest-arg (list "." (symbol->string rest-arg)) '())))))))) (define (macro-arguments name type transformer) (process-args (case type ((syntax-rules) (let ((patterns (procedure-property transformer 'patterns))) (if (pair? patterns) (car patterns) '()))) ((identifier-syntax) '()) ((defmacro) (or (procedure-property transformer 'defmacro-args) '())) (else ;; a procedural (syntax-case) macro. how to document these? '())))) (define (macro-additional-stexi name type transformer) (case type ((syntax-rules) (let ((patterns (procedure-property transformer 'patterns))) (if (pair? patterns) (map (lambda (x) `(defspecx (% (name ,name) (arguments ,@(process-args x))))) (cdr patterns)) '()))) (else '()))) (define many-space? (make-regexp "[[:space:]][[:space:]][[:space:]]")) (define initial-space? (make-regexp "^[[:space:]]")) (define (string->stexi str) (or (and (or (not str) (string-null? str)) '(*fragment*)) (and (or (string-index str #\@) (and (not (regexp-exec many-space? str)) (not (regexp-exec initial-space? str)))) (false-if-exception (texi-fragment->stexi str))) `(*fragment* (verbatim ,str)))) (define method-formals (and (defined? 'method-formals) method-formals)) (define (method-stexi-arguments method) (cond (method-formals (let lp ((formals (method-formals method)) (specializers (method-specializers method)) (out '())) (define (arg-texinfo formal specializer) `(" (" (var ,(symbol->string formal)) " " (code ,(symbol->string (class-name specializer))) ")")) (cond ((null? formals) (reverse out)) ((pair? formals) (lp (cdr formals) (cdr specializers) (append (reverse (arg-texinfo (car formals) (car specializers))) out))) (else (append (reverse out) (arg-texinfo formals specializers) (list "...")))))) ((method-source method) (let lp ((bindings (cadr (method-source method))) (out '())) (define (arg-texinfo arg) `(" (" (var ,(symbol->string (car arg))) " " (code ,(symbol->string (cadr arg))) ")")) (cond ((null? bindings) (reverse out)) ((not (pair? (car bindings))) (append (reverse out) (arg-texinfo bindings) (list "..."))) (else (lp (cdr bindings) (append (reverse (arg-texinfo (car bindings))) out)))))) (else (warn method) '()))) (define* (object-stexi-documentation object #:optional (name "[unknown]") #:key (force #f)) (if (symbol? name) (set! name (symbol->string name))) (let ((stexi ((lambda (x) (cond ((string? x) (string->stexi x)) ((and (pair? x) (eq? (car x) '*fragment*)) x) (force `(*fragment*)) (else #f))) (object-documentation (if (is-a? object <method>) (method-procedure object) object))))) (define (make-def type args) `(,type (% ,@args) ,@(cdr stexi))) (cond ((not stexi) #f) ;; stexi is now a list, headed by *fragment*. ((and (pair? (cdr stexi)) (pair? (cadr stexi)) (memq (caadr stexi) defs)) ;; it's already a deffoo. stexi) ((is-a? object <class>) (make-def 'deftp `((name ,name) (category "Class")))) ((is-a? object <macro>) (let* ((proc (macro-transformer object)) (type (and proc (procedure-property proc 'macro-type)))) `(defspec (% (name ,name) (arguments ,@(macro-arguments name type proc))) ,@(macro-additional-stexi name type proc) ,@(cdr stexi)))) ((is-a? object <procedure>) (make-def 'defun `((name ,name) (arguments ,@(get-proc-args object))))) ((is-a? object <method>) (make-def 'deffn `((category "Method") (name ,name) (arguments ,@(method-stexi-arguments object))))) ((is-a? object <generic>) `(*fragment* ,(make-def 'deffn `((name ,name) (category "Generic"))) ,@(map (lambda (method) (object-stexi-documentation method name #:force force)) (generic-function-methods object)))) (else (make-def 'defvar `((name ,name))))))) (define (module-name->node-name sym-name) (string-join (map symbol->string sym-name) " ")) ;; this copied from (ice-9 session); need to find a better way (define (module-filename name) (let* ((name (map symbol->string name)) (reverse-name (reverse name)) (leaf (car reverse-name)) (dir-hint-module-name (reverse (cdr reverse-name))) (dir-hint (apply string-append (map (lambda (elt) (string-append elt "/")) dir-hint-module-name)))) (%search-load-path (in-vicinity dir-hint leaf)))) (define (read-module name) (let ((filename (module-filename name))) (if filename (let ((port (open-input-file filename))) (let lp ((out '()) (form (read port))) (if (eof-object? form) (reverse out) (lp (cons form out) (read port))))) '()))) (define (module-export-list sym-name) (define (module-form-export-list form) (and (pair? form) (eq? (car form) 'define-module) (equal? (cadr form) sym-name) (and=> (memq #:export (cddr form)) cadr))) (let lp ((forms (read-module sym-name))) (cond ((null? forms) '()) ((module-form-export-list (car forms)) => identity) (else (lp (cdr forms)))))) (define* (module-stexi-documentation sym-name #:optional %docs-resolver #:key (docs-resolver (or %docs-resolver (lambda (name def) def)))) "Return documentation for the module named @var{sym-name}. The documentation will be formatted as @code{stexi} (@pxref{texinfo,texinfo})." (if %docs-resolver (issue-deprecation-warning "module-stexi-documentation: use #:docs-resolver instead of a positional argument.")) (let* ((commentary (and=> (module-commentary sym-name) (lambda (x) (string-trim-both x #\newline)))) (stexi (string->stexi commentary)) (node-name (module-name->node-name sym-name)) (name-str (with-output-to-string (lambda () (display sym-name)))) (module (resolve-interface sym-name)) (export-list (module-export-list sym-name))) (define (anchor-name sym) (string-append node-name " " (symbol->string sym))) (define (make-defs) (sort! (module-map (lambda (sym var) `((anchor (% (name ,(anchor-name sym)))) ,@((lambda (x) (if (eq? (car x) '*fragment*) (cdr x) (list x))) (if (variable-bound? var) (docs-resolver sym (object-stexi-documentation (variable-ref var) sym #:force #t)) (begin (warn "variable unbound!" sym) `(defvar (% (name ,(symbol->string sym))) "[unbound!]")))))) module) (lambda (a b) (sort-defs export-list a b)))) `(texinfo (% (title ,name-str)) (node (% (name ,node-name))) (section "Overview") ,@(cdr stexi) (section "Usage") ,@(apply append! (make-defs))))) (define (script-stexi-documentation scriptpath) "Return documentation for given script. The documentation will be taken from the script's commentary, and will be returned in the @code{stexi} format (@pxref{texinfo,texinfo})." (let ((commentary (file-commentary scriptpath))) `(texinfo (% (title ,(basename scriptpath))) (node (% (name ,(basename scriptpath)))) ,@(if commentary (cdr (string->stexi (string-trim-both commentary #\newline))) '())))) (cond ((defined? 'add-value-help-handler!) (add-value-help-handler! (lambda (name value) (stexi->plain-text (object-stexi-documentation value name #:force #t)))) (add-name-help-handler! (lambda (name) (and (list? name) (and-map symbol? name) (stexi->plain-text (module-stexi-documentation name))))))) ;; we could be dealing with an old (ice-9 session); fondle it to get ;; module-commentary (define module-commentary (@@ (ice-9 session) module-commentary)) (define (package-stexi-standard-copying name version updated years copyright-holder permissions) "Create a standard texinfo @code{copying} section. @var{years} is a list of years (as integers) in which the modules being documented were released. All other arguments are strings." `(copying (para "This manual is for " ,name " (version " ,version ", updated " ,updated ")") (para "Copyright " ,(string-join (map number->string years) ",") " " ,copyright-holder) (quotation (para ,permissions)))) (define (package-stexi-standard-titlepage name version updated authors) "Create a standard GNU title page. @var{authors} is a list of @code{(@var{name} . @var{email})} pairs. All other arguments are strings. Here is an example of the usage of this procedure: @smallexample (package-stexi-standard-titlepage \"Foolib\" \"3.2\" \"26 September 2006\" '((\"Alyssa P Hacker\" . \"alyssa@@example.com\")) '(2004 2005 2006) \"Free Software Foundation, Inc.\" \"Standard GPL permissions blurb goes here\") @end smallexample " `(;(setchapternewpage (% (all "odd"))) makes manuals too long (titlepage (title ,name) (subtitle "version " ,version ", updated " ,updated) ,@(map (lambda (pair) `(author ,(car pair) " (" (email ,(cdr pair)) ")")) authors) (page) (vskip (% (all "0pt plus 1filll"))) (insertcopying)))) (define (package-stexi-generic-menu name entries) "Create a menu from a generic alist of entries, the car of which should be the node name, and the cdr the description. As an exception, an entry of @code{#f} will produce a separator." (define (make-entry node description) `("* " ,node "::" ,(make-string (max (- 21 (string-length node)) 2) #\space) ,@description "\n")) `((ifnottex (node (% (name "Top"))) (top (% (title ,name))) (insertcopying) (menu ,@(apply append (map (lambda (entry) (if entry (make-entry (car entry) (cdr entry)) '("\n"))) entries)))) (iftex (shortcontents)))) (define (package-stexi-standard-menu name modules module-descriptions extra-entries) "Create a standard top node and menu, suitable for processing by makeinfo." (package-stexi-generic-menu name (let ((module-entries (map cons (map module-name->node-name modules) module-descriptions)) (separate-sections (lambda (x) (if (null? x) x (cons #f x))))) `(,@module-entries ,@(separate-sections extra-entries))))) (define (package-stexi-extended-menu name module-pairs script-pairs extra-entries) "Create an \"extended\" menu, like the standard menu but with a section for scripts." (package-stexi-generic-menu name (let ((module-entries (map cons (map module-name->node-name (map car module-pairs)) (map cdr module-pairs))) (script-entries (map cons (map basename (map car script-pairs)) (map cdr script-pairs))) (separate-sections (lambda (x) (if (null? x) x (cons #f x))))) `(,@module-entries ,@(separate-sections script-entries) ,@(separate-sections extra-entries))))) (define (package-stexi-standard-prologue name filename category description copying titlepage menu) "Create a standard prologue, suitable for later serialization to texinfo and .info creation with makeinfo. Returns a list of stexinfo forms suitable for passing to @code{package-stexi-documentation} as the prologue. @xref{texinfo reflection package-stexi-documentation}, @ref{texinfo reflection package-stexi-standard-titlepage,package-stexi-standard-titlepage}, @ref{texinfo reflection package-stexi-standard-copying,package-stexi-standard-copying}, and @ref{texinfo reflection package-stexi-standard-menu,package-stexi-standard-menu}." `(,copying (dircategory (% (category ,category))) (direntry "* " ,name ": (" ,filename "). " ,description ".") ,@titlepage ,@menu)) (define (stexi->chapter stexi) (pre-post-order stexi `((texinfo . ,(lambda (tag attrs node . body) `(,node (chapter ,@(assq-ref (cdr attrs) 'title)) ,@body))) (*text* . ,(lambda (tag text) text)) (*default* . ,(lambda args args))))) (define* (package-stexi-documentation modules name filename prologue epilogue #:key (module-stexi-documentation-args '()) (scripts '())) "Create stexi documentation for a @dfn{package}, where a package is a set of modules that is released together. @var{modules} is expected to be a list of module names, where a module name is a list of symbols. The stexi that is returned will be titled @var{name} and a texinfo filename of @var{filename}. @var{prologue} and @var{epilogue} are lists of stexi forms that will be spliced into the output document before and after the generated modules documentation, respectively. @xref{texinfo reflection package-stexi-standard-prologue}, to create a conventional GNU texinfo prologue. @var{module-stexi-documentation-args} is an optional argument that, if given, will be added to the argument list when @code{module-texi-documentation} is called. For example, it might be useful to define a @code{#:docs-resolver} argument." (define (verify-modules-list l) (define (all pred l) (and (pred (car l)) (or (null? (cdr l)) (all pred (cdr l))))) (false-if-exception (all (lambda (x) (all symbol? x)) modules))) (if (not (verify-modules-list modules)) (error "expected modules to be a list of a list of symbols" modules)) `(texinfo (% (title ,name) (filename ,filename)) ,@prologue ,@(append-map (lambda (mod) (stexi->chapter (apply module-stexi-documentation mod module-stexi-documentation-args))) modules) ,@(append-map (lambda (script) (stexi->chapter (script-stexi-documentation script))) scripts) ,@epilogue)) (define* (package-stexi-documentation-for-include modules module-descriptions #:key (module-stexi-documentation-args '())) "Create stexi documentation for a @dfn{package}, where a package is a set of modules that is released together. @var{modules} is expected to be a list of module names, where a module name is a list of symbols. Returns an stexinfo fragment. Unlike @code{package-stexi-documentation}, this function simply produces a menu and the module documentations instead of producing a full texinfo document. This can be useful if you write part of your manual by hand, and just use @code{@@include} to pull in the automatically generated parts. @var{module-stexi-documentation-args} is an optional argument that, if given, will be added to the argument list when @code{module-texi-documentation} is called. For example, it might be useful to define a @code{#:docs-resolver} argument." (define (make-entry node description) `("* " ,node "::" ,(make-string (max (- 21 (string-length node)) 2) #\space) ,@description "\n")) `(*fragment* (menu ,@(append-map (lambda (modname desc) (make-entry (module-name->node-name modname) desc)) modules module-descriptions)) ,@(append-map (lambda (modname) (stexi->chapter (apply module-stexi-documentation modname module-stexi-documentation-args))) modules))) ;;; arch-tag: bbe2bc03-e16d-4a9e-87b9-55225dc9836c PK�����/F2]a �� ����texinfo/indexing.scmnu�[��������;;;; (texinfo indexing) -- indexing stexinfo ;;;; ;;;; Copyright (C) 2009, 2010 Free Software Foundation, Inc. ;;;; Copyright (C) 2003,2004,2009 Andy Wingo <wingo at pobox dot com> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;@c texinfo formatting ;;Given a piece of stexi, return an index of a specified variety. ;; ;;Note that currently, @code{stexi-extract-index} doesn't differentiate ;;between different kinds of index entries. That's a bug ;) ;;; Code: (define-module (texinfo indexing) #:use-module (sxml simple) #:use-module (srfi srfi-13) #:export (stexi-extract-index)) (define defines '(deftp defcv defivar deftypeivar defop deftypeop defmethod deftypemethod defopt defvr defvar deftypevr deftypevar deffn deftypefn defspec defmac defun deftypefun)) (define indices '(cindex findex vindex kindex pindex tindex)) (define (stexi-extract-index tree manual-name kind) "Given an stexi tree @var{tree}, index all of the entries of type @var{kind}. @var{kind} can be one of the predefined texinfo indices (@code{concept}, @code{variable}, @code{function}, @code{key}, @code{program}, @code{type}) or one of the special symbols @code{auto} or @code{all}. @code{auto} will scan the stext for a @code{(printindex)} statement, and @code{all} will generate an index from all entries, regardless of type. The returned index is a list of pairs, the @sc{car} of which is the entry (a string) and the @sc{cdr} of which is a node name (a string)." (let loop ((in tree) (entries '())) (cond ((null? in) entries) ((pair? (car in)) (cond ((and (pair? (cdr in)) (pair? (cadr in)) (eq? (caar in) 'anchor) (memq (caadr in) defines)) (loop (cddr in) (acons (cadr (assq 'name (cdr (cadadr in)))) (cadr (assq 'name (cdadar in))) entries))) ((and (pair? (cdr in)) (pair? (cadr in)) (eq? (caar in) 'anchor) (memq (caadr in) indices)) (loop (cddr in) (acons (sxml->string (cadr in)) (cadr (assq 'name (cdadar in))) entries))) (else (loop (cdr in) (loop (car in) entries))))) (else (loop (cdr in) entries))))) ;;; arch-tag: 216d29d3-1ed9-433f-9c19-0dc4d6b439b6 PK�����/F2]tE0S(��S(����texinfo/html.scmnu�[��������;;;; (texinfo html) -- translating stexinfo into shtml ;;;; ;;;; Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc. ;;;; Copyright (C) 2003,2004,2009 Andy Wingo <wingo at pobox dot com> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;This module implements transformation from @code{stexi} to HTML. Note ;;that the output of @code{stexi->shtml} is actually SXML with the HTML ;;vocabulary. This means that the output can be further processed, and ;;that it must eventually be serialized by ;;@ref{sxml simple sxml->xml,sxml->xml}. ;; ;;References (i.e., the @code{@@ref} family of commands) are resolved by ;;a @dfn{ref-resolver}. ;;@xref{texinfo html add-ref-resolver!,add-ref-resolver!}, for more ;;information. ;; ;;; Code: ;; TODO: nice ref resolving API, default CSS stylesheet (esp. to remove ;; margin-top on dd > p) (define-module (texinfo html) #:use-module (texinfo) #:use-module (sxml transform) #:use-module (ice-9 match) #:use-module (srfi srfi-13) #:export (stexi->shtml add-ref-resolver! urlify)) ;; The caller is responsible for carring the returned list. (define (arg-ref key %-args) (and=> (assq key (cdr %-args)) (lambda (x) (stexi->shtml (cdr x))))) (define (arg-req key %-args) (or (arg-ref key %-args) (error "Missing argument:" key %-args))) (define (car* x) (and x (car x))) (define (urlify str) (string-downcase (string-map (lambda (c) (case c ((#\space #\/ #\:) #\-) (else c))) str))) (define ref-resolvers (list (lambda (node-name manual-name) ;; the default (urlify (string-append (or manual-name "") "#" node-name))))) (define (add-ref-resolver! proc) "Add @var{proc} to the head of the list of ref-resolvers. @var{proc} will be expected to take the name of a node and the name of a manual and return the URL of the referent, or @code{#f} to pass control to the next ref-resolver in the list. The default ref-resolver will return the concatenation of the manual name, @code{#}, and the node name." (set! ref-resolvers (cons proc ref-resolvers))) (define (resolve-ref node manual) (or (or-map (lambda (x) (x node manual)) ref-resolvers) (error "Could not resolve reference" node manual))) (define (ref tag args) (let* ((node (car (arg-req 'node args))) (section (or (car* (arg-ref 'section args)) node)) (manual (car* (arg-ref 'manual args))) (target (resolve-ref node manual))) `(span ,(and=> (assq tag '((xref "See ") (pxref "see "))) cdr) (a (@ (href ,target)) ,section)))) (define (uref tag args) (let ((url (car (arg-req 'url args)))) `(a (@ (href ,url)) ,(or (car* (arg-ref 'title args)) url)))) ;; @!*&%( Mozilla gets confused at an empty ("<a .. />") a tag. Put an ;; empty string here to placate the reptile. (define (node tag args) `(a (@ (name ,(urlify (car (arg-req 'name args))))) "")) (define (def tag args . body) (define (code x) (and x (cons 'code x))) (define (var x) (and x (cons 'var x))) (define (b x) (and x (cons 'b x))) (define (list/spaces . elts) (let lp ((in elts) (out '())) (cond ((null? in) (reverse! out)) ((null? (car in)) (lp (cdr in) out)) (else (lp (cdr in) (cons (car in) (if (null? out) out (cons " " out)))))))) (define (left-td-contents) (list/spaces (code (arg-ref 'data-type args)) (b (list (code (arg-ref 'class args)))) ;; is this right? (b (list (code (arg-ref 'name args)))) (if (memq tag '(deftypeop deftypefn deftypefun)) (code (arg-ref 'arguments args)) (var (list (code (arg-ref 'arguments args))))))) (let* ((category (case tag ((defun) "Function") ((defspec) "Special Form") ((defvar) "Variable") (else (car (arg-req 'category args)))))) `(div (table (@ (cellpadding "0") (cellspacing "0") (width "100%") (class "def")) (tr (td ,@(left-td-contents)) (td (div (@ (class "right")) "[" ,category "]")))) (div (@ (class "description")) ,@body)))) (define (enumerate tag . elts) (define (tonumber start) (let ((c (string-ref start 0))) (cond ((number? c) (string->number start)) (else (1+ (- (char->integer c) (char->integer (if (char-upper-case? c) #\A #\a)))))))) `(ol ,@(if (and (pair? elts) (pair? (car elts)) (eq? (caar elts) '%)) (cons `(@ (start ,@(tonumber (arg-req 'start (car elts))))) ;; (type ,(type (arg-ref 'start (car elts))))) (cdr elts)) elts))) (define (itemize tag . elts) `(ul ,@(match elts ;; Strip `bullet' attribute. ((('% . attrs) . elts) elts) (elts elts)))) (define (acronym tag . elts) (match elts ;; FIXME: Need attribute matcher that doesn't depend on attribute ;; order. ((('% ('acronym text) . _)) `(acronym ,text)))) (define (table tag args . body) (let ((formatter (caar (arg-req 'formatter args)))) (cons 'dl (map (lambda (x) (cond ((and (pair? x) (eq? (car x) 'dt)) (list (car x) (cons formatter (cdr x)))) (else x))) (apply append body))))) (define (entry tag args . body) (let lp ((out `((dt ,@(arg-req 'heading args)))) (body body)) (if (and (pair? body) (pair? (car body)) (eq? (caar body) 'itemx)) (lp (append out `(dt ,@(map stexi->shtml (cdar body)))) (cdr body)) (append out `((dd ,@(map stexi->shtml body))))))) (define tag-replacements '((titlepage div (@ (class "titlepage"))) (title h2 (@ (class "title"))) (subtitle h3 (@ (class "subtitle"))) (author h3 (@ (class "author"))) (example pre) (lisp pre) (smallexample pre (@ (class "smaller"))) (smalllisp pre (@ (class "smaller"))) (cartouche div (@ (class "cartouche"))) (verbatim pre (@ (class "verbatim"))) (chapter h2) (section h3) (subsection h4) (subsubsection h5) (appendix h2) (appendixsec h3) (appendixsubsec h4) (appendixsubsubsec h5) (unnumbered h2) (unnumberedsec h3) (unnumberedsubsec h4) (unnumberedsubsubsec h5) (majorheading h2) (chapheading h2) (heading h3) (subheading h4) (subsubheading h5) (quotation blockquote) (item li) ;; itemx ? (para p) (*fragment* div) ;; should be ok (asis span) (bold b) (sample samp) (samp samp) (code code) (kbd kbd) (key code (@ (class "key"))) (var var) (env code (@ (class "env"))) (file code (@ (class "file"))) (command code (@ (class "command"))) (option code (@ (class "option"))) (url code (@ (class "url"))) (dfn dfn) (cite cite) (acro acronym) (email code (@ (class "email"))) (emph em) (strong strong) (sc span (@ (class "small-caps"))))) (define ignore-list '(page setfilename setchapternewpage iftex ifinfo ifplaintext ifxml sp vskip menu ignore syncodeindex comment c dircategory direntry top shortcontents cindex printindex)) (define rules `((% *preorder* . ,(lambda args args)) ;; Keep these around... (texinfo . ,(lambda (tag args . body) (pre-post-order `(html (@ (xmlns "http://www.w3.org/1999/xhtml")) (head (title ,(car (arg-req 'title args)))) (body ,@body)) `((% *preorder* . ,(lambda args #f)) ;; ... filter out. (*text* . ,(lambda (tag x) x)) (*default* . ,(lambda (tag . body) (cons tag body))))))) (copyright . ,(lambda args '(*ENTITY* "copy"))) (result . ,(lambda args '(*ENTITY* "rArr"))) (xref . ,ref) (ref . ,ref) (pxref . ,ref) (uref . ,uref) (node . ,node) (anchor . ,node) (table . ,table) (enumerate . ,enumerate) (itemize . ,itemize) (acronym . ,acronym) (entry *preorder* . ,entry) (deftp . ,def) (defcv . ,def) (defivar . ,def) (deftypeivar . ,def) (defop . ,def) (deftypeop . ,def) (defmethod . ,def) (deftypemethod . ,def) (defopt . ,def) (defvr . ,def) (defvar . ,def) (deftypevr . ,def) (deftypevar . ,def) (deffn . ,def) (deftypefn . ,def) (defmac . ,def) (defspec . ,def) (defun . ,def) (deftypefun . ,def) (ifnottex . ,(lambda (tag . body) body)) (*text* . ,(lambda (tag x) x)) (*default* . ,(lambda (tag . body) (let ((subst (assq tag tag-replacements))) (cond (subst (append (cdr subst) body)) ((memq tag ignore-list) #f) (else (warn "Don't know how to convert" tag "to HTML") body))))))) (define (stexi->shtml tree) "Transform the stexi @var{tree} into shtml, resolving references via ref-resolvers. See the module commentary for more details." (pre-post-order tree rules)) ;;; arch-tag: ab05f3fe-9981-4a78-b64c-48efcd9983a6 PK�����/F2] '��'��'��scripts/snarf-check-and-output-texi.scmnu�[��������;;; snarf-check-and-output-texi --- called by the doc snarfer. ;; Copyright (C) 2001, 2002, 2006, 2011, 2014 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Michael Livshin ;;; Code: (define-module (scripts snarf-check-and-output-texi) :use-module (ice-9 streams) :use-module (ice-9 match) :export (snarf-check-and-output-texi)) (define %include-in-guild-list #f) (define %summary "Transform snarfed .doc files into texinfo documentation.") ;;; why aren't these in some module? (define-macro (when cond . body) `(if ,cond (begin ,@body))) (define-macro (unless cond . body) `(if (not ,cond) (begin ,@body))) (define *manual-flag* #f) (define (snarf-check-and-output-texi . flags) (if (member "--manual" flags) (set! *manual-flag* #t)) (process-stream (current-input-port))) (define (process-stream port) (let loop ((input (stream-map (match-lambda (('id . s) (cons 'id (string->symbol s))) (('int_dec . s) (cons 'int (string->number s))) (('int_oct . s) (cons 'int (string->number s 8))) (('int_hex . s) (cons 'int (string->number s 16))) ((and x (? symbol?)) (cons x x)) ((and x (? string?)) (cons 'string x)) (x x)) (make-stream (lambda (s) (let loop ((s s)) (cond ((stream-null? s) #t) ((memq (stream-car s) '(eol hash)) (loop (stream-cdr s))) (else (cons (stream-car s) (stream-cdr s)))))) (port->stream port read))))) (unless (stream-null? input) (let ((token (stream-car input))) (if (eq? (car token) 'snarf_cookie) (dispatch-top-cookie (stream-cdr input) loop) (loop (stream-cdr input))))))) (define (dispatch-top-cookie input cont) (when (stream-null? input) (error 'syntax "premature end of file")) (let ((token (stream-car input))) (cond ((eq? (car token) 'brace_open) (consume-multiline (stream-cdr input) cont)) (else (consume-upto-cookie process-singleline input cont))))) (define (consume-upto-cookie process input cont) (let loop ((acc '()) (input input)) (when (stream-null? input) (error 'syntax "premature end of file in directive context")) (let ((token (stream-car input))) (cond ((eq? (car token) 'snarf_cookie) (process (reverse! acc)) (cont (stream-cdr input))) (else (loop (cons token acc) (stream-cdr input))))))) (define (consume-multiline input cont) (begin-multiline) (let loop ((input input)) (when (stream-null? input) (error 'syntax "premature end of file in multiline context")) (let ((token (stream-car input))) (cond ((eq? (car token) 'brace_close) (end-multiline) (cont (stream-cdr input))) (else (consume-upto-cookie process-multiline-directive input loop)))))) (define *file* #f) (define *line* #f) (define *c-function-name* #f) (define *function-name* #f) (define *snarf-type* #f) (define *args* #f) (define *sig* #f) (define *docstring* #f) (define (begin-multiline) (set! *file* #f) (set! *line* #f) (set! *c-function-name* #f) (set! *function-name* #f) (set! *snarf-type* #f) (set! *args* #f) (set! *sig* #f) (set! *docstring* #f)) (define *primitive-deffnx-signature* "@deffnx {Scheme Procedure} ") (define *primitive-deffnx-sig-length* (string-length *primitive-deffnx-signature*)) (define (end-multiline) (let* ((req (car *sig*)) (opt (cadr *sig*)) (var (caddr *sig*)) (all (+ req opt var))) (if (and (not (eqv? *snarf-type* 'register)) (not (= (length *args*) all))) (error (format #f "~A:~A: ~A's C implementation takes ~A args (should take ~A)" *file* *line* *function-name* (length *args*) all))) (let ((nice-sig (if (eq? *snarf-type* 'register) *function-name* (with-output-to-string (lambda () (format #t "~A" *function-name*) (let loop-req ((args *args*) (r 0)) (if (< r req) (begin (format #t " ~A" (car args)) (loop-req (cdr args) (+ 1 r))) (let loop-opt ((o 0) (args args) (tail '())) (if (< o opt) (begin (format #t " [~A" (car args)) (loop-opt (+ 1 o) (cdr args) (cons #\] tail))) (begin (if (> var 0) (format #t " . ~A" (car args))) (let loop-tail ((tail tail)) (if (not (null? tail)) (begin (format #t "~A" (car tail)) (loop-tail (cdr tail)))))))))))))) (scm-deffnx (if (and *manual-flag* (eq? *snarf-type* 'primitive)) (with-output-to-string (lambda () (format #t "@deffnx {C Function} ~A (" *c-function-name*) (unless (null? *args*) (format #t "~A" (car *args*)) (let loop ((args (cdr *args*))) (unless (null? args) (format #t ", ~A" (car args)) (loop (cdr args))))) (format #t ")\n"))) #f))) (format #t "\n ~A\n" *function-name*) (format #t "@c snarfed from ~A:~A\n" *file* *line*) (format #t "@deffn {Scheme Procedure} ~A\n" nice-sig) (let loop ((strings *docstring*) (scm-deffnx scm-deffnx)) (cond ((null? strings)) ((or (not scm-deffnx) (and (>= (string-length (car strings)) *primitive-deffnx-sig-length*) (string=? (substring (car strings) 0 *primitive-deffnx-sig-length*) *primitive-deffnx-signature*))) (display (car strings)) (loop (cdr strings) scm-deffnx)) (else (display scm-deffnx) (loop strings #f)))) (display "\n") (display "@end deffn\n")))) (define (texi-quote s) (let rec ((i 0)) (if (= i (string-length s)) "" (string-append (let ((ss (substring s i (+ i 1)))) (if (string=? ss "@") "@@" ss)) (rec (+ i 1)))))) (define (process-multiline-directive l) (define do-args (match-lambda (('(paren_close . paren_close)) '()) (('(comma . comma) rest ...) (do-args rest)) (('(id . SCM) ('id . name) rest ...) (cons name (do-args rest))) (x (error (format #f "invalid argument syntax: ~A" (map cdr x)))))) (define do-arglist (match-lambda (('(paren_open . paren_open) '(id . void) '(paren_close . paren_close)) '()) (('(paren_open . paren_open) rest ...) (do-args rest)) (x (error (format #f "invalid arglist syntax: ~A" (map cdr x)))))) (define do-command (match-lambda (('cname ('id . name)) (set! *c-function-name* (texi-quote (symbol->string name)))) (('fname ('string . name) ...) (set! *function-name* (texi-quote (apply string-append name)))) (('type ('id . type)) (set! *snarf-type* type)) (('type ('int . num)) (set! *snarf-type* num)) (('location ('string . file) ('int . line)) (set! *file* file) (set! *line* line)) (('arglist rest ...) (set! *args* (do-arglist rest))) (('argsig ('int . req) ('int . opt) ('int . var)) (set! *sig* (list req opt var))) (x (error (format #f "unknown doc attribute: ~A" x))))) (define do-directive (match-lambda ((('id . command) rest ...) (do-command (cons command rest))) ((('string . string) ...) (set! *docstring* string)) (x (error (format #f "unknown doc attribute syntax: ~A" x))))) (do-directive l)) (define (process-singleline l) (define do-argpos (match-lambda ((('id . name) ('int . pos) ('int . line)) (let ((idx (list-index *args* name))) (when idx (unless (= (+ idx 1) pos) (display (format #f "~A:~A: wrong position for argument ~A: ~A (should be ~A)\n" *file* line name pos (+ idx 1)) (current-error-port)))))) (x #f))) (define do-command (match-lambda (('(id . argpos) rest ...) (do-argpos rest)) (x (error (format #f "unknown check: ~A" x))))) (when *function-name* (do-command l))) (define main snarf-check-and-output-texi) PK�����/F2]-������scripts/help.scmnu�[��������;;; Help --- Show help on guild commands ;;;; Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free ;;;; Software Foundation, Inc., 51 Franklin Street, Fifth Floor, ;;;; Boston, MA 02110-1301 USA ;;; Commentary: ;; Usage: help ;; ;; Show help for Guild scripts. ;;; Code: (define-module (scripts help) #:use-module (ice-9 format) #:use-module (ice-9 documentation) #:use-module ((srfi srfi-1) #:select (fold append-map)) #:export (show-help show-summary show-usage main)) (define %summary "Show a brief help message.") (define %synopsis "help\nhelp --all\nhelp COMMAND") (define %help " Show help on guild commands. With --all, show arcane incantations as well. With COMMAND, show more detailed help for a particular command. ") (define (directory-files dir) (if (and (file-exists? dir) (file-is-directory? dir)) (let ((dir-stream (opendir dir))) (let loop ((new (readdir dir-stream)) (acc '())) (if (eof-object? new) (begin (closedir dir-stream) acc) (loop (readdir dir-stream) (if (or (string=? "." new) ; ignore (string=? ".." new)) ; ignore acc (cons new acc)))))) '())) (define (strip-extensions path) (or-map (lambda (ext) (and (string-suffix? ext path) ;; We really can't be adding e.g. ChangeLog-2008 to the set ;; of runnable scripts, just because "" is a valid ;; extension, by default. So hack around that here. (not (string-null? ext)) (substring path 0 (- (string-length path) (string-length ext))))) (append %load-compiled-extensions %load-extensions))) (define (unique l) (cond ((null? l) l) ((null? (cdr l)) l) ((equal? (car l) (cadr l)) (unique (cdr l))) (else (cons (car l) (unique (cdr l)))))) (define (find-submodules head) (let ((shead (map symbol->string head))) (unique (sort (append-map (lambda (path) (fold (lambda (x rest) (let ((stripped (strip-extensions x))) (if stripped (cons stripped rest) rest))) '() (directory-files (fold (lambda (x y) (in-vicinity y x)) path shead)))) %load-path) string<?)))) (define (list-commands all?) (display "\ Usage: guild COMMAND [ARGS] Run command-line scripts provided by GNU Guile and related programs. Commands: ") (for-each (lambda (name) (let* ((modname `(scripts ,(string->symbol name))) (mod (resolve-module modname #:ensure #f)) (summary (and mod (and=> (module-variable mod '%summary) variable-ref)))) (if (and mod (or all? (let ((v (module-variable mod '%include-in-guild-list))) (if v (variable-ref v) #t)))) (if summary (format #t " ~A ~23t~a\n" name summary) (format #t " ~A\n" name))))) (find-submodules '(scripts))) (format #t " For help on a specific command, try \"guild help COMMAND\". Report guild bugs to ~a GNU Guile home page: <http://www.gnu.org/software/guile/> General help using GNU software: <http://www.gnu.org/gethelp/> For complete documentation, run: info guile 'Using Guile Tools' " %guile-bug-report-address)) (define (module-commentary mod) (file-commentary (%search-load-path (module-filename mod)))) (define (module-command-name mod) (symbol->string (car (last-pair (module-name mod))))) (define* (show-usage mod #:optional (port (current-output-port))) (let ((usages (string-split (let ((var (module-variable mod '%synopsis))) (if var (variable-ref var) (string-append (module-command-name mod) " OPTION..."))) #\newline))) (display "Usage: guild " port) (display (car usages)) (newline port) (for-each (lambda (u) (display " guild " port) (display u port) (newline port)) (cdr usages)))) (define* (show-summary mod #:optional (port (current-output-port))) (let ((var (module-variable mod '%summary))) (if var (begin (display (variable-ref var) port) (newline port))))) (define* (show-help mod #:optional (port (current-output-port))) (show-usage mod port) (show-summary mod port) (cond ((module-variable mod '%help) => (lambda (var) (display (variable-ref var) port) (newline port))) ((module-commentary mod) => (lambda (commentary) (newline port) (display commentary port))) (else (format #t "No documentation found for command \"~a\".\n" (module-command-name mod))))) (define %mod (current-module)) (define (main . args) (cond ((null? args) (list-commands #f)) ((or (equal? args '("--all")) (equal? args '("-a"))) (list-commands #t)) ((and (null? (cdr args)) (not (string-prefix? "-" (car args)))) ;; help for particular command (let ((name (car args))) (cond ((resolve-module `(scripts ,(string->symbol name)) #:ensure #f) => (lambda (mod) (show-help mod) (exit 0))) (else (format #t "No command named \"~a\".\n" name) (exit 1))))) (else (show-help %mod (current-error-port)) (exit 1)))) PK�����/F2]l.��.����scripts/read-scheme-source.scmnu�[��������;;; read-scheme-source --- Read a file, recognizing scheme forms and comments ;; Copyright (C) 2001, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen ;;; Commentary: ;; Usage: read-scheme-source FILE1 FILE2 ... ;; ;; This program parses each FILE and writes to stdout sexps that describe the ;; top-level structures of the file: scheme forms, single-line comments, and ;; hash-bang comments. You can further process these (to associate comments ;; w/ scheme forms as a kind of documentation, for example). ;; ;; The output sexps have one of these forms: ;; ;; (quote (filename FILENAME)) ;; ;; (quote (comment :leading-semicolons N ;; :text LINE)) ;; ;; (quote (whitespace :text LINE)) ;; ;; (quote (hash-bang-comment :line LINUM ;; :line-count N ;; :text-list (LINE1 LINE2 ...))) ;; ;; (quote (following-form-properties :line LINUM ;; :line-count N) ;; :type TYPE ;; :signature SIGNATURE ;; :std-int-doc DOCSTRING)) ;; ;; SEXP ;; ;; The first four are straightforward (both FILENAME and LINE are strings sans ;; newline, while LINUM and N are integers). The last two always go together, ;; in that order. SEXP is scheme code processed only by `read' and then ;; `write'. ;; ;; The :type field may be omitted if the form is not recognized. Otherwise, ;; TYPE may be one of: procedure, alias, define-module, variable. ;; ;; The :signature field may be omitted if the form is not a procedure. ;; Otherwise, SIGNATURE is a list showing the procedure's signature. ;; ;; If the type is `procedure' and the form has a standard internal docstring ;; (first body form a string), that is extracted in full -- including any ;; embedded newlines -- and recorded by field :std-int-doc. ;; ;; ;; Usage from a program: The output list of sexps can be retrieved by scheme ;; programs w/o having to capture stdout, like so: ;; ;; (use-modules (scripts read-scheme-source)) ;; (define source-forms (read-scheme-source-silently "FILE1" "FILE2" ...)) ;; ;; There are also two convenience procs exported for use by Scheme programs: ;; ;; (clump FORMS) --- filter FORMS combining contiguous comment forms that ;; have the same number of leading semicolons. ;; ;; (quoted? SYM FORM) --- see if FORM looks like: "(quote (SYM ...))", parse ;; the ":tags", and return alist of (TAG . VAL) elems. ;; ;; TODO: Add option "--clump-comments", maybe w/ different clumping styles. ;; Make `annotate!' extensible. ;;; Code: (define-module (scripts read-scheme-source) :use-module (ice-9 rdelim) :export (read-scheme-source read-scheme-source-silently quoted? clump)) (define %include-in-guild-list #f) (define %summary "Print a parsed representation of a Scheme file.") ;; Try to figure out what FORM is and its various attributes. ;; Call proc NOTE! with key (a symbol) and value. ;; (define (annotate! form note!) (cond ((and (list? form) (< 2 (length form)) (eq? 'define (car form)) (pair? (cadr form)) (symbol? (caadr form))) (note! ':type 'procedure) (note! ':signature (cadr form)) (and (< 3 (length form)) (string? (caddr form)) (note! ':std-int-doc (caddr form)))) ((and (list? form) (< 2 (length form)) (eq? 'define (car form)) (symbol? (cadr form)) (list? (caddr form)) (< 3 (length (caddr form))) (eq? 'lambda (car (caddr form))) (string? (caddr (caddr form)))) (note! ':type 'procedure) (note! ':signature (cons (cadr form) (cadr (caddr form)))) (note! ':std-int-doc (caddr (caddr form)))) ((and (list? form) (= 3 (length form)) (eq? 'define (car form)) (symbol? (cadr form)) (symbol? (caddr form))) (note! ':type 'alias)) ((and (list? form) (eq? 'define-module (car form))) (note! ':type 'define-module)) ;; Add other types here. (else (note! ':type 'variable)))) ;; Process FILE, calling NB! on parsed top-level elements. ;; Recognized: #!-!# and regular comments in addition to normal forms. ;; (define (process file nb!) (nb! `'(filename ,file)) (let ((hash-bang-rx (make-regexp "^#!")) (bang-hash-rx (make-regexp "^!#")) (all-comment-rx (make-regexp "^[ \t]*(;+)")) (all-whitespace-rx (make-regexp "^[ \t]*$")) (p (open-input-file file))) (let loop ((n (1+ (port-line p))) (line (read-line p))) (or (not n) (eof-object? line) (begin (cond ((regexp-exec hash-bang-rx line) (let loop ((line (read-line p)) (text (list line))) (if (or (eof-object? line) (regexp-exec bang-hash-rx line)) (nb! `'(hash-bang-comment :line ,n :line-count ,(1+ (length text)) :text-list ,(reverse (cons line text)))) (loop (read-line p) (cons line text))))) ((regexp-exec all-whitespace-rx line) (nb! `'(whitespace :text ,line))) ((regexp-exec all-comment-rx line) => (lambda (m) (nb! `'(comment :leading-semicolons ,(let ((m1 (vector-ref m 1))) (- (cdr m1) (car m1))) :text ,line)))) (else (unread-string line p) (let* ((form (read p)) (count (- (port-line p) n)) (props (let* ((props '()) (prop+ (lambda args (set! props (append props args))))) (annotate! form prop+) props))) (or (= count 1) ; ugh (begin (read-line p) (set! count (1+ count)))) (nb! `'(following-form-properties :line ,n :line-count ,count ,@props)) (nb! form)))) (loop (1+ (port-line p)) (read-line p))))))) ;;; entry points (define (read-scheme-source-silently . files) "See commentary in module (scripts read-scheme-source)." (let* ((res '())) (for-each (lambda (file) (process file (lambda (e) (set! res (cons e res))))) files) (reverse res))) (define (read-scheme-source . files) "See commentary in module (scripts read-scheme-source)." (for-each (lambda (file) (process file (lambda (e) (write e) (newline)))) files)) ;; Recognize: (quote (SYM :TAG1 VAL1 :TAG2 VAL2 ...)) ;; and return alist: ((TAG1 . VAL1) (TAG2 . VAL2) ...) ;; where the tags are symbols. ;; (define (quoted? sym form) (and (list? form) (= 2 (length form)) (eq? 'quote (car form)) (let ((inside (cadr form))) (and (list? inside) (< 0 (length inside)) (eq? sym (car inside)) (let loop ((ls (cdr inside)) (alist '())) (if (null? ls) alist ; retval (let ((first (car ls))) (or (symbol? first) (error "bad list!")) (loop (cddr ls) (acons (string->symbol (substring (symbol->string first) 1)) (cadr ls) alist))))))))) ;; Filter FORMS, combining contiguous comment forms that have the same number ;; of leading semicolons. Do not include in them whitespace lines. ;; Whitespace lines outside of such comment groupings are ignored, as are ;; hash-bang comments. All other forms are passed through unchanged. ;; (define (clump forms) (let loop ((forms forms) (acc '()) (pass-this-one-through? #f)) (if (null? forms) (reverse acc) ; retval (let ((form (car forms))) (cond (pass-this-one-through? (loop (cdr forms) (cons form acc) #f)) ((quoted? 'following-form-properties form) (loop (cdr forms) (cons form acc) #t)) ((quoted? 'whitespace form) ;;; ignore (loop (cdr forms) acc #f)) ((quoted? 'hash-bang-comment form) ;;; ignore for now (loop (cdr forms) acc #f)) ((quoted? 'comment form) => (lambda (alist) (let cloop ((inner-forms (cdr forms)) (level (assq-ref alist 'leading-semicolons)) (text (list (assq-ref alist 'text)))) (let ((up (lambda () (loop inner-forms (cons (cons level (reverse text)) acc) #f)))) (if (null? inner-forms) (up) (let ((inner-form (car inner-forms))) (cond ((quoted? 'comment inner-form) => (lambda (inner-alist) (let ((new-level (assq-ref inner-alist 'leading-semicolons))) (if (= new-level level) (cloop (cdr inner-forms) level (cons (assq-ref inner-alist 'text) text)) (up))))) (else (up))))))))) (else (loop (cdr forms) (cons form acc) #f))))))) ;;; script entry point (define main read-scheme-source) ;;; read-scheme-source ends here PK�����/F2]J������scripts/read-rfc822.scmnu�[��������;;; read-rfc822 --- Validate RFC822 file by displaying it to stdout ;; Copyright (C) 2002, 2004, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: read-rfc822 FILE ;; ;; Read FILE, assumed to be in RFC822 format, and display it to stdout. ;; This is not very interesting, admittedly. ;; ;; For Scheme programming, this module exports two procs: ;; (read-rfc822 . args) ; only first arg used ;; (read-rfc822-silently port) ;; ;; Parse FILE (a string) or PORT, respectively, and return a query proc that ;; takes a symbol COMP, and returns the message component COMP. Supported ;; values for COMP (and the associated query return values) are: ;; from -- #f (reserved for future mbox support) ;; headers -- alist of (HEADER-SYMBOL . "VALUE-STRING") pairs, in order ;; body -- rest of the mail message, a string ;; body-lines -- rest of the mail message, as a list of lines ;; Any other query results in a "bad component" error. ;; ;; TODO: Add "-m" option (mbox support). ;;; Code: (define-module (scripts read-rfc822) :use-module (ice-9 regex) :use-module (ice-9 rdelim) :autoload (srfi srfi-13) (string-join) :export (read-rfc822 read-rfc822-silently)) (define %include-in-guild-list #f) (define %summary "Validate an RFC822-style file.") (define from-line-rx (make-regexp "^From ")) (define header-name-rx (make-regexp "^([^:]+):[ \t]*")) (define header-cont-rx (make-regexp "^[ \t]+")) (define option #f) ; for future "-m" (define (drain-message port) (let loop ((line (read-line port)) (acc '())) (cond ((eof-object? line) (reverse acc)) ((and option (regexp-exec from-line-rx line)) (for-each (lambda (c) (unread-char c port)) (cons #\newline (reverse (string->list line)))) (reverse acc)) (else (loop (read-line port) (cons line acc)))))) (define (parse-message port) (let* ((from (and option (match:suffix (regexp-exec from-line-rx (read-line port))))) (body-lines #f) (body #f) (headers '()) (add-header! (lambda (reversed-hlines) (let* ((hlines (reverse reversed-hlines)) (first (car hlines)) (m (regexp-exec header-name-rx first)) (name (string->symbol (match:substring m 1))) (data (string-join (cons (substring first (match:end m)) (cdr hlines)) " "))) (set! headers (acons name data headers)))))) ;; "From " is only one line (let loop ((line (read-line port)) (current-header #f)) (cond ((string-null? line) (and current-header (add-header! current-header)) (set! body-lines (drain-message port))) ((regexp-exec header-cont-rx line) => (lambda (m) (loop (read-line port) (cons (match:suffix m) current-header)))) (else (and current-header (add-header! current-header)) (loop (read-line port) (list line))))) (set! headers (reverse headers)) (lambda (component) (case component ((from) from) ((body-lines) body-lines) ((headers) headers) ((body) (or body (begin (set! body (string-join body-lines "\n" 'suffix)) body))) (else (error "bad component:" component)))))) (define (read-rfc822-silently port) (parse-message port)) (define (display-rfc822 parse) (cond ((parse 'from) => (lambda (from) (format #t "From ~A\n" from)))) (for-each (lambda (header) (format #t "~A: ~A\n" (car header) (cdr header))) (parse 'headers)) (format #t "\n~A" (parse 'body))) (define (read-rfc822 . args) (let ((parse (read-rfc822-silently (open-file (car args) OPEN_READ)))) (display-rfc822 parse)) #t) (define main read-rfc822) ;;; read-rfc822 ends here PK�����/F2]*%4������scripts/generate-autoload.scmnu�[��������;;; generate-autoload --- Display define-module form with autoload info ;; Copyright (C) 2001, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen ;;; Commentary: ;; Usage: generate-autoload [OPTIONS] FILE1 FILE2 ... ;; ;; The autoload form is displayed to standard output: ;; ;; (define-module (guile-user) ;; :autoload (ZAR FOO) (FOO-1 FOO-2 ...) ;; : ;; : ;; :autoload (ZAR BAR) (BAR-1 BAR-2 ...)) ;; ;; For each file, a symbol triggers an autoload if it is found in one ;; of these situations: ;; - in the `:export' clause of a `define-module' form ;; - in a top-level `export' or `export-syntax' form ;; - in a `define-public' form ;; - in a `defmacro-public' form ;; ;; The module name is inferred from the `define-module' form. If either the ;; module name or the exports list cannot be determined, no autoload entry is ;; generated for that file. ;; ;; Options: ;; --target MODULE-NAME -- Use MODULE-NAME instead of `(guile-user)'. ;; Note that some shells may require you to ;; quote the argument to handle parentheses ;; and spaces. ;; ;; Usage examples from Scheme code as a module: ;; (use-modules (scripts generate-autoload)) ;; (generate-autoload "generate-autoload") ;; (generate-autoload "--target" "(my module)" "generate-autoload") ;; (apply generate-autoload "--target" "(my module)" '("foo" "bar" "baz")) ;;; Code: (define-module (scripts generate-autoload) :export (generate-autoload)) (define %include-in-guild-list #f) (define %summary "Generate #:autoload clauses for a module.") (define (autoload-info file) (let ((p (open-input-file file))) (let loop ((form (read p)) (module-name #f) (exports '())) (if (eof-object? form) (and module-name (not (null? exports)) (list module-name exports)) ; ret (cond ((and (list? form) (< 1 (length form)) (eq? 'define-module (car form))) (loop (read p) (cadr form) (cond ((member ':export form) => (lambda (val) (append (cadr val) exports))) (else exports)))) ((and (list? form) (< 1 (length form)) (memq (car form) '(export export-syntax))) (loop (read p) module-name (append (cdr form) exports))) ((and (list? form) (< 2 (length form)) (eq? 'define-public (car form)) (list? (cadr form)) (symbol? (caadr form))) (loop (read p) module-name (cons (caadr form) exports))) ((and (list? form) (< 2 (length form)) (eq? 'define-public (car form)) (symbol? (cadr form))) (loop (read p) module-name (cons (cadr form) exports))) ((and (list? form) (< 3 (length form)) (eq? 'defmacro-public (car form)) (symbol? (cadr form))) (loop (read p) module-name (cons (cadr form) exports))) (else (loop (read p) module-name exports))))))) (define (generate-autoload . args) (let* ((module-count 0) (syms-count 0) (target-override (cond ((member "--target" args) => cadr) (else #f))) (files (if target-override (cddr args) (cdr args)))) (display ";;; do not edit --- generated ") (display (strftime "%Y-%m-%d %H:%M:%S" (localtime (current-time)))) (newline) (display "(define-module ") (display (or target-override "(guile-user)")) (for-each (lambda (file) (cond ((autoload-info file) => (lambda (info) (and info (apply (lambda (module-name exports) (set! module-count (1+ module-count)) (set! syms-count (+ (length exports) syms-count)) (for-each display (list "\n :autoload " module-name " " exports))) info)))))) files) (display ")") (newline) (for-each display (list " ;;; " syms-count " symbols in " module-count " modules\n")))) (define main generate-autoload) ;;; generate-autoload ends here PK�����/F2]#q-��-����scripts/frisk.scmnu�[��������;;; frisk --- Grok the module interfaces of a body of files ;; Copyright (C) 2002, 2006, 2010, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: frisk [options] file ... ;; ;; Analyze FILE... module interfaces in aggregate (as a "body"), ;; and display a summary. Modules that are `define-module'd are ;; considered "internal" (and those not, "external"). When module X ;; uses module Y, X is said to be "(a) downstream of" Y, and Y is ;; "(an) upstream of" X. ;; ;; Normally, the summary displays external modules and their internal ;; downstreams, as this is the usual question asked by a body. There ;; are several options that modify this output. ;; ;; -u, --upstream show upstream edges ;; -d, --downstream show downstream edges (default) ;; -i, --internal show internal modules ;; -x, --external show external modules (default) ;; ;; If given both `upstream' and `downstream' options ("frisk -ud"), the ;; output is formatted: "C MODULE --- UP-LS --- DOWN-LS", where C is ;; either `i' or `x', and each element of UP-LS and DOWN-LS is (TYPE ;; MODULE-NAME ...). ;; ;; In all other cases, the "C MODULE" occupies its own line, and ;; subsequent lines list the up- or downstream edges, respectively, ;; indented by some non-zero amount of whitespace. ;; ;; Top-level `use-modules' (or `load' or 'primitive-load') forms in a ;; file that do not follow a `define-module' result an edge where the ;; downstream is the "default module", normally `(guile-user)'. This ;; can be set to another value by using: ;; ;; -m, --default-module MOD set MOD as the default module ;; Usage from a Scheme Program: (use-modules (scripts frisk)) ;; ;; Module export list: ;; (frisk . args) ;; (make-frisker . options) => (lambda (files) ...) [see below] ;; (mod-up-ls module) => upstream edges ;; (mod-down-ls module) => downstream edges ;; (mod-int? module) => is the module internal? ;; (edge-type edge) => symbol: {regular,autoload,computed} ;; (edge-up edge) => upstream module ;; (edge-down edge) => downstream module ;; ;; OPTIONS is an alist. Recognized keys are: ;; default-module ;; ;; `make-frisker' returns a procedure that takes a list of files, the ;; FRISKER. FRISKER returns a closure, REPORT, that takes one of the ;; keys: ;; modules -- entire list of modules ;; internal -- list of internal modules ;; external -- list of external modules ;; i-up -- list of modules upstream of internal modules ;; x-up -- list of modules upstream of external modules ;; i-down -- list of modules downstream of internal modules ;; x-down -- list of modules downstream of external modules ;; edges -- list of edges ;; Note that `x-up' should always be null, since by (lack of!) ;; definition, we only know external modules by reference. ;; ;; The module and edge objects managed by REPORT can be examined in ;; detail by using the other (self-explanatory) procedures. Be careful ;; not to confuse a freshly consed list of symbols, like `(a b c)' with ;; the module `(a b c)'. If you want to find the module by that name, ;; try: (cond ((member '(a b c) (REPORT 'modules)) => car)). ;; TODO: Make "frisk -ud" output less ugly. ;; Consider default module as internal; add option to invert. ;; Support `edge-misc' data. ;;; Code: (define-module (scripts frisk) :autoload (ice-9 getopt-long) (getopt-long) :use-module ((srfi srfi-1) :select (filter remove)) :export (frisk make-frisker mod-up-ls mod-down-ls mod-int? edge-type edge-up edge-down)) (define %include-in-guild-list #f) (define %summary "Show dependency information for a module.") (define *default-module* '(guile-user)) (define (grok-proc default-module note-use!) (lambda (filename) (let* ((p (open-file filename "r")) (next (lambda () (read p))) (ferret (lambda (use) ;;; handle "((foo bar) :select ...)" (let ((maybe (car use))) (if (list? maybe) maybe use)))) (curmod #f)) (let loop ((form (next))) (cond ((eof-object? form)) ((not (list? form)) (loop (next))) (else (case (car form) ((define-module) (let ((module (cadr form))) (set! curmod module) (note-use! 'def module #f) (let loop ((ls form)) (or (null? ls) (case (car ls) ((#:use-module :use-module) (note-use! 'regular module (ferret (cadr ls))) (loop (cddr ls))) ((#:autoload :autoload) (note-use! 'autoload module (cadr ls)) (loop (cdddr ls))) (else (loop (cdr ls)))))))) ((use-modules) (for-each (lambda (use) (note-use! 'regular (or curmod default-module) (ferret use))) (cdr form))) ((load primitive-load) (note-use! 'computed (or curmod default-module) (let ((file (cadr form))) (if (string? file) file (format #f "[computed in ~A]" filename)))))) (loop (next)))))))) (define up-ls (make-object-property)) ; list (define dn-ls (make-object-property)) ; list (define int? (make-object-property)) ; defined via `define-module' (define mod-up-ls up-ls) (define mod-down-ls dn-ls) (define mod-int? int?) (define (i-or-x module) (if (int? module) 'i 'x)) (define edge-type (make-object-property)) ; symbol (define (make-edge type up down) (let ((new (cons up down))) (set! (edge-type new) type) new)) (define edge-up car) (define edge-down cdr) (define (up-ls+! m new) (set! (up-ls m) (cons new (up-ls m)))) (define (dn-ls+! m new) (set! (dn-ls m) (cons new (dn-ls m)))) (define (make-body alist) (lambda (key) (assq-ref alist key))) (define (scan default-module files) (let* ((modules (list)) (edges (list)) (intern (lambda (module) (cond ((member module modules) => car) (else (set! (up-ls module) (list)) (set! (dn-ls module) (list)) (set! modules (cons module modules)) module)))) (grok (grok-proc default-module (lambda (type d u) (let ((d (intern d))) (if (eq? type 'def) (set! (int? d) #t) (let* ((u (intern u)) (edge (make-edge type u d))) (set! edges (cons edge edges)) (up-ls+! d edge) (dn-ls+! u edge)))))))) (for-each grok files) (make-body `((modules . ,modules) (internal . ,(filter int? modules)) (external . ,(remove int? modules)) (i-up . ,(filter int? (map edge-down edges))) (x-up . ,(remove int? (map edge-down edges))) (i-down . ,(filter int? (map edge-up edges))) (x-down . ,(remove int? (map edge-up edges))) (edges . ,edges))))) (define (make-frisker . options) (let ((default-module (or (assq-ref options 'default-module) *default-module*))) (lambda (files) (scan default-module files)))) (define (dump-updown modules) (for-each (lambda (m) (format #t "~A ~A --- ~A --- ~A\n" (i-or-x m) m (map (lambda (edge) (cons (edge-type edge) (edge-up edge))) (up-ls m)) (map (lambda (edge) (cons (edge-type edge) (edge-down edge))) (dn-ls m)))) modules)) (define (dump-up modules) (for-each (lambda (m) (format #t "~A ~A\n" (i-or-x m) m) (for-each (lambda (edge) (format #t "\t\t\t ~A\t~A\n" (edge-type edge) (edge-up edge))) (up-ls m))) modules)) (define (dump-down modules) (for-each (lambda (m) (format #t "~A ~A\n" (i-or-x m) m) (for-each (lambda (edge) (format #t "\t\t\t ~A\t~A\n" (edge-type edge) (edge-down edge))) (dn-ls m))) modules)) (define (frisk . args) (let* ((parsed-opts (getopt-long (cons "frisk" args) ;;; kludge '((upstream (single-char #\u)) (downstream (single-char #\d)) (internal (single-char #\i)) (external (single-char #\x)) (default-module (single-char #\m) (value #t))))) (=u (option-ref parsed-opts 'upstream #f)) (=d (option-ref parsed-opts 'downstream #f)) (=i (option-ref parsed-opts 'internal #f)) (=x (option-ref parsed-opts 'external #f)) (files (option-ref parsed-opts '() (list))) (report ((make-frisker `(default-module . ,(option-ref parsed-opts 'default-module *default-module*))) files)) (modules (report 'modules)) (internal (report 'internal)) (external (report 'external)) (edges (report 'edges))) (format #t "~A ~A, ~A ~A (~A ~A, ~A ~A), ~A ~A\n\n" (length files) "files" (length modules) "modules" (length internal) "internal" (length external) "external" (length edges) "edges") ((cond ((and =u =d) dump-updown) (=u dump-up) (else dump-down)) (cond ((and =i =x) modules) (=i internal) (else external))))) (define main frisk) ;;; frisk ends here PK�����/F2]hF������scripts/api-diff.scmnu�[��������;;; api-diff --- diff guile-api.alist files ;; Copyright (C) 2002, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: api-diff [-d GROUPS] ALIST-FILE-A ALIST-FILE-B ;; ;; Read in the alists from files ALIST-FILE-A and ALIST-FILE-B ;; and display a (count) summary of the groups defined therein. ;; Optional arg "--details" (or "-d") specifies a comma-separated ;; list of groups, in which case api-diff displays instead the ;; elements added and deleted for each of the specified groups. ;; ;; For scheme programming, this module exports the proc: ;; (api-diff A-file B-file) ;; ;; Note that the convention is that the "older" alist/file is ;; specified first. ;; ;; TODO: Develop scheme interface. ;;; Code: (define-module (scripts api-diff) :use-module (ice-9 common-list) :use-module (ice-9 format) :use-module (ice-9 getopt-long) :autoload (srfi srfi-13) (string-tokenize) :export (api-diff)) (define %include-in-guild-list #f) (define %summary "Show differences between two scan-api files.") (define (read-alist-file file) (with-input-from-file file (lambda () (read)))) (define put set-object-property!) (define get object-property) (define (read-api-alist-file file) (let* ((alist (read-alist-file file)) (meta (assq-ref alist 'meta)) (interface (assq-ref alist 'interface))) (put interface 'meta meta) (put interface 'groups (let ((ht (make-hash-table 31))) (for-each (lambda (group) (hashq-set! ht group '())) (assq-ref meta 'groups)) ht)) interface)) (define (hang-by-the-roots interface) (let ((ht (get interface 'groups))) (for-each (lambda (x) (for-each (lambda (group) (hashq-set! ht group (cons (car x) (hashq-ref ht group)))) (assq-ref x 'groups))) interface)) interface) (define (diff? a b) (let ((result (set-difference a b))) (if (null? result) #f ; CL weenies bite me result))) (define (diff+note! a b note-removals note-additions note-same) (let ((same? #t)) (cond ((diff? a b) => (lambda (x) (note-removals x) (set! same? #f)))) (cond ((diff? b a) => (lambda (x) (note-additions x) (set! same? #f)))) (and same? (note-same)))) (define (group-diff i-old i-new . options) (let* ((i-old (hang-by-the-roots i-old)) (g-old (hash-fold acons '() (get i-old 'groups))) (g-old-names (map car g-old)) (i-new (hang-by-the-roots i-new)) (g-new (hash-fold acons '() (get i-new 'groups))) (g-new-names (map car g-new))) (cond ((null? options) (diff+note! g-old-names g-new-names (lambda (removals) (format #t "groups-removed: ~A\n" removals)) (lambda (additions) (format #t "groups-added: ~A\n" additions)) (lambda () #t)) (for-each (lambda (group) (let* ((old (assq-ref g-old group)) (new (assq-ref g-new group)) (old-count (and old (length old))) (new-count (and new (length new))) (delta (and old new (- new-count old-count)))) (format #t " ~5@A ~5@A : " (or old-count "-") (or new-count "-")) (cond ((and old new) (let ((add-count 0) (sub-count 0)) (diff+note! old new (lambda (subs) (set! sub-count (length subs))) (lambda (adds) (set! add-count (length adds))) (lambda () #t)) (format #t "~5@D ~5@D : ~5@D" add-count (- sub-count) delta))) (else (format #t "~5@A ~5@A : ~5@A" "-" "-" "-"))) (format #t " ~A\n" group))) (sort (union g-old-names g-new-names) (lambda (a b) (string<? (symbol->string a) (symbol->string b)))))) ((assq-ref options 'details) => (lambda (groups) (for-each (lambda (group) (let* ((old (or (assq-ref g-old group) '())) (new (or (assq-ref g-new group) '())) (>>! (lambda (label ls) (format #t "~A ~A:\n" group label) (for-each (lambda (x) (format #t " ~A\n" x)) ls)))) (diff+note! old new (lambda (removals) (>>! 'removals removals)) (lambda (additions) (>>! 'additions additions)) (lambda () (format #t "~A: no changes\n" group))))) groups))) (else (error "api-diff: group-diff: bad options"))))) (define (api-diff . args) (let* ((p (getopt-long (cons 'api-diff args) '((details (single-char #\d) (value #t)) ;; Add options here. ))) (rest (option-ref p '() '("/dev/null" "/dev/null"))) (i-old (read-api-alist-file (car rest))) (i-new (read-api-alist-file (cadr rest))) (options '())) (cond ((option-ref p 'details #f) => (lambda (groups) (set! options (cons (cons 'details (map string->symbol (string-tokenize groups #\,))) options))))) (apply group-diff i-old i-new options))) (define main api-diff) ;;; api-diff ends here PK�����/F2]v#o!��!�� ��scripts/summarize-guile-TODO.scmnu�[��������;;; summarize-guile-TODO --- Display Guile TODO list in various ways ;; Copyright (C) 2002, 2006, 2010, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: summarize-guile-TODO TODOFILE ;; ;; The TODOFILE is typically Guile's (see workbook/tasks/README) ;; presumed to serve as our signal to ourselves (lest we want real ;; bosses hassling us) wrt to the overt message "items to do" as well as ;; the messages that can be inferred from its structure. ;; ;; This program reads TODOFILE and displays interpretations on its ;; structure, including registered markers and ownership, in various ;; ways. ;; ;; A primary interest in any task is its parent task. The output ;; summarization by default lists every item and its parent chain. ;; Top-level parents are not items. You can use these command-line ;; options to modify the selection and display (selection criteria ;; are ANDed together): ;; ;; -i, --involved USER -- select USER-involved items ;; -p, --personal USER -- select USER-responsible items ;; -t, --todo -- select unfinished items (status "-") ;; -d, --done -- select finished items (status "+") ;; -r, --review -- select review items (marker "R") ;; ;; -w, --who -- also show who is associated w/ the item ;; -n, --no-parent -- do not show parent chain ;; ;; ;; Usage from a Scheme program: ;; (summarize-guile-TODO . args) ; uses first arg only ;; ;; ;; Bugs: (1) Markers are scanned in sequence: D R X N%. This means "XD" ;; and the like are completely dropped. However, such strings ;; are unlikely to be used if the markers are chosen to be ;; somewhat exclusive, which is currently the case for D R X. ;; N% used w/ these needs to be something like: "D25%" (this ;; means discussion accounts for 1/4 of the task). ;; ;; TODO: Implement more various ways. (Patches welcome.) ;; Add support for ORing criteria. ;;; Code: (debug-enable 'backtrace) (define-module (scripts summarize-guile-TODO) :use-module (scripts read-text-outline) :use-module (ice-9 getopt-long) :autoload (srfi srfi-13) (string-tokenize) ; string library :autoload (srfi srfi-14) (char-set) ; string library :autoload (ice-9 common-list) (remove-if-not) :export (summarize-guile-TODO)) (define %include-in-guild-list #f) (define %summary "A quaint relic of the past.") (define put set-object-property!) (define get object-property) (define (as-leaf x) (cond ((get x 'who) => (lambda (who) (put x 'who (map string->symbol (string-tokenize who (char-set #\:))))))) (cond ((get x 'pct-done) => (lambda (pct-done) (put x 'pct-done (string->number pct-done))))) x) (define (hang-by-the-leaves trees) (let ((leaves '())) (letrec ((hang (lambda (tree parent) (if (list? tree) (begin (put (car tree) 'parent parent) (for-each (lambda (child) (hang child (car tree))) (cdr tree))) (begin (put tree 'parent parent) (set! leaves (cons (as-leaf tree) leaves))))))) (for-each (lambda (tree) (hang tree #f)) trees)) leaves)) (define (read-TODO file) (hang-by-the-leaves ((make-text-outline-reader "(([ ][ ])*)([-+])(D*)(R*)(X*)(([0-9]+)%)* *([^[]*)(\\[(.*)\\])*" '((level-substring-divisor . 2) (body-submatch-number . 9) (extra-fields . ((status . 3) (design? . 4) (review? . 5) (extblock? . 6) (pct-done . 8) (who . 11))))) (open-file file "r")))) (define (select-items p items) (let ((sub '())) (cond ((option-ref p 'involved #f) => (lambda (u) (let ((u (string->symbol u))) (set! sub (cons (lambda (x) (and (get x 'who) (memq u (get x 'who)))) sub)))))) (cond ((option-ref p 'personal #f) => (lambda (u) (let ((u (string->symbol u))) (set! sub (cons (lambda (x) (cond ((get x 'who) => (lambda (ls) (eq? (car (reverse ls)) u))) (else #f))) sub)))))) (for-each (lambda (pair) (cond ((option-ref p (car pair) #f) (set! sub (cons (cdr pair) sub))))) `((todo . ,(lambda (x) (string=? (get x 'status) "-"))) (done . ,(lambda (x) (string=? (get x 'status) "+"))) (review . ,(lambda (x) (get x 'review?))))) (let loop ((sub (reverse sub)) (items items)) (if (null? sub) (reverse items) (loop (cdr sub) (remove-if-not (car sub) items)))))) (define (make-display-item show-who? show-parent?) (let ((show-who (if show-who? (lambda (item) (cond ((get item 'who) => (lambda (who) (format #f " ~A" who))) (else ""))) (lambda (item) ""))) (show-parents (if show-parent? (lambda (item) (let loop ((parent (get item 'parent)) (indent 2)) (and parent (begin (format #t "under : ~A~A\n" (make-string indent #\space) parent) (loop (get parent 'parent) (+ 2 indent)))))) (lambda (item) #t)))) (lambda (item) (format #t "status: ~A~A~A~A~A~A\nitem : ~A\n" (get item 'status) (if (get item 'design?) "D" "") (if (get item 'review?) "R" "") (if (get item 'extblock?) "X" "") (cond ((get item 'pct-done) => (lambda (pct-done) (format #f " ~A%" pct-done))) (else "")) (show-who item) item) (show-parents item)))) (define (display-items p items) (let ((display-item (make-display-item (option-ref p 'who #f) (not (option-ref p 'no-parent #f)) ))) (for-each display-item items))) (define (summarize-guile-TODO . args) (let ((p (getopt-long (cons "summarize-guile-TODO" args) '((who (single-char #\w)) (no-parent (single-char #\n)) (involved (single-char #\i) (value #t)) (personal (single-char #\p) (value #t)) (todo (single-char #\t)) (done (single-char #\d)) (review (single-char #\r)) ;; Add options here. )))) (display-items p (select-items p (read-TODO (car (option-ref p '() #f)))))) #t) ; exit val (define main summarize-guile-TODO) ;;; summarize-guile-TODO ends here PK�����/F2]uTn ��n ����scripts/snarf-guile-m4-docs.scmnu�[��������;;; snarf-guile-m4-docs --- Parse guile.m4 comments for texi documentation ;; Copyright (C) 2002, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: snarf-guile-m4-docs FILE ;; ;; Grep FILE for comments preceding macro definitions, massage ;; them into valid texi, and display to stdout. For each comment, ;; lines preceding "^# Usage:" are discarded. ;; ;; TODO: Generalize. ;;; Code: (define-module (scripts snarf-guile-m4-docs) :use-module (ice-9 rdelim) :export (snarf-guile-m4-docs)) (define %include-in-guild-list #f) (define %summary "Snarf out texinfo documentation from .m4 files.") (define (display-texi lines) (display "@deffn {Autoconf Macro}") (for-each (lambda (line) (display (cond ((and (>= (string-length line) 2) (string=? "# " (substring line 0 2))) (substring line 2)) ((string=? "#" (substring line 0 1)) (substring line 1)) (else line))) (newline)) lines) (display "@end deffn") (newline) (newline)) (define (prefix? line sub) (false-if-exception (string=? sub (substring line 0 (string-length sub))))) (define (massage-usage line) (let loop ((line (string->list line)) (acc '())) (if (null? line) (list (list->string (reverse acc))) (loop (cdr line) (cons (case (car line) ((#\( #\) #\,) #\space) (else (car line))) acc))))) (define (snarf-guile-m4-docs . args) (let* ((p (open-file (car args) "r")) (next (lambda () (read-line p)))) (let loop ((line (next)) (acc #f)) (or (eof-object? line) (cond ((prefix? line "# Usage:") (loop (next) (massage-usage (substring line 8)))) ((prefix? line "AC_DEFUN") (display-texi (reverse acc)) (loop (next) #f)) ((and acc (prefix? line "#")) (loop (next) (cons line acc))) (else (loop (next) #f))))))) (define main snarf-guile-m4-docs) ;;; snarf-guile-m4-docs ends here PK�����/F2]?y[Z;��Z;����scripts/doc-snarf.scmnu�[��������;;; doc-snarf --- Extract documentation from source files ;; Copyright (C) 2001, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Martin Grabmueller ;;; Commentary: ;; Usage: doc-snarf FILE ;; ;; This program reads in a Scheme source file and extracts docstrings ;; in the format specified below. Additionally, a procedure protoype ;; is infered from the procedure definition line starting with ;; (define... ). ;; ;; Currently, two output modi are implemented: texinfo and plaintext. ;; Default is plaintext, texinfo can be switched on with the ;; `--texinfo, -t' command line option. ;; ;; Format: A docstring can span multiple lines and a docstring line ;; begins with `;; ' (two semicoli and a space). A docstring is ended ;; by either a line beginning with (define ...) or one or more lines ;; beginning with `;;-' (two semicoli and a dash). These lines are ;; called `options' and begin with a keyword, followed by a colon and ;; a string. ;; ;; Additionally, "standard internal docstrings" (for Scheme source) are ;; recognized and output as "options". The output formatting is likely ;; to change in the future. ;; ;; Example: ;; This procedure foos, or bars, depending on the argument @var{braz}. ;;-Author: Martin Grabmueller (define (foo/bar braz) (if braz 'foo 'bar)) ;;; Which results in the following docstring if texinfo output is ;;; enabled: #! foo/bar @deffn procedure foo/bar braz This procedure foos, or bars, depending on the argument @var{braz}. @c Author: Martin Grabmueller @end deffn !# ;;; Or in this if plaintext output is used: #! Procedure: foo/bar braz This procedure foos, or bars, depending on the argument @var{braz}. ;; Author: Martin Grabmueller ^L !# ;; TODO: Convert option lines to alist. ;; More parameterization. ;; (maybe) Use in Guile build itself. (define doc-snarf-version "0.0.2") ; please update before publishing! ;;; Code: (define-module (scripts doc-snarf) :use-module (ice-9 getopt-long) :use-module (ice-9 regex) :use-module (ice-9 string-fun) :use-module (ice-9 rdelim) :export (doc-snarf)) (define %summary "Snarf out documentation from a file.") (define command-synopsis '((version (single-char #\v) (value #f)) (help (single-char #\h) (value #f)) (output (single-char #\o) (value #t)) (texinfo (single-char #\t) (value #f)) (lang (single-char #\l) (value #t)))) ;; Display version information and exit. ;;-ttn-mod: use var (define (display-version) (display "doc-snarf ") (display doc-snarf-version) (newline)) ;; Display the usage help message and exit. ;;-ttn-mod: change option "source" to "lang" (define (display-help) (display "Usage: doc-snarf [options...] inputfile\n") (display " --help, -h Show this usage information\n") (display " --version, -v Show version information\n") (display " --output=FILE, -o Specify output file [default=stdout]\n") (display " --texinfo, -t Format output as texinfo\n") (display " --lang=[c,scheme], -l Specify the input language\n")) ;; Main program. ;;-ttn-mod: canonicalize lang (define (doc-snarf . args) (let ((options (getopt-long (cons "doc-snarf" args) command-synopsis))) (let ((help-wanted (option-ref options 'help #f)) (version-wanted (option-ref options 'version #f)) (texinfo-wanted (option-ref options 'texinfo #f)) (lang (string->symbol (string-downcase (option-ref options 'lang "scheme"))))) (cond (version-wanted (display-version)) (help-wanted (display-help)) (else (let ((input (option-ref options '() #f)) (output (option-ref options 'output #f))) (if ;; Bonard B. Timmons III says `(pair? input)' alone is sufficient. ;; (and input (pair? input)) (pair? input) (snarf-file (car input) output texinfo-wanted lang) (display-help)))))))) (define main doc-snarf) ;; Supported languages and their parameters. Each element has form: ;; (LANG DOC-START DOC-END DOC-PREFIX OPT-PREFIX SIG-START STD-INT-DOC?) ;; LANG is a symbol, STD-INT-DOC? is a boolean indicating whether or not ;; LANG supports "standard internal docstring" (a string after the formals), ;; everything else is a string specifying a regexp. ;;-ttn-mod: new var (define supported-languages '((c "^/\\*(.*)" "^ \\*/" "^ \\* (.*)" "^ \\*-(.*)" "NOTHING AT THIS TIME!!!" #f ) (scheme "^;; (.*)" "^;;\\." "^;; (.*)" "^;;-(.*)" "^\\(define" #t ))) ;; Get @var{lang}'s @var{parameter}. Both args are symbols. ;;-ttn-mod: new proc (define (lang-parm lang parm) (list-ref (assq-ref supported-languages lang) (case parm ((docstring-start) 0) ((docstring-end) 1) ((docstring-prefix) 2) ((option-prefix) 3) ((signature-start) 4) ((std-int-doc?) 5)))) ;; Snarf all docstrings from the file @var{input} and write them to ;; file @var{output}. Use texinfo format for the output if ;; @var{texinfo?} is true. ;;-ttn-mod: don't use string comparison, consult table instead (define (snarf-file input output texinfo? lang) (or (memq lang (map car supported-languages)) (error "doc-snarf: input language must be c or scheme.")) (write-output (snarf input lang) output (if texinfo? format-texinfo format-plain))) ;; fixme: this comment is required to trigger standard internal ;; docstring snarfing... ideally, it wouldn't be necessary. ;;-ttn-mod: new proc, from snarf-docs (aren't these names fun?) (define (find-std-int-doc line input-port) "Unread @var{line} from @var{input-port}, then read in the entire form and return the standard internal docstring if found. Return #f if not." (unread-string line input-port) ; ugh (let ((form (read input-port))) (cond ((and (list? form) ; (define (PROC ARGS) "DOC" ...) (< 3 (length form)) (eq? 'define (car form)) (pair? (cadr form)) (symbol? (caadr form)) (string? (caddr form))) (caddr form)) ((and (list? form) ; (define VAR (lambda ARGS "DOC" ...)) (< 2 (length form)) (eq? 'define (car form)) (symbol? (cadr form)) (list? (caddr form)) (< 3 (length (caddr form))) (eq? 'lambda (car (caddr form))) (string? (caddr (caddr form)))) (caddr (caddr form))) (else #f)))) ;; Split @var{string} into lines, adding @var{prefix} to each. ;;-ttn-mod: new proc (define (split-prefixed string prefix) (separate-fields-discarding-char #\newline string (lambda lines (map (lambda (line) (string-append prefix line)) lines)))) ;; snarf input-file output-file ;; Extract docstrings from the input file @var{input}, presumed ;; to be written in language @var{lang}. ;;-Author: Martin Grabmueller <mgrabmue@cs.tu-berlin.de> ;;-Created: 2001-02-17 ;;-ttn-mod: regluarize lang parm lookup, add "std int doc" snarfing (2 places) (define (snarf input-file lang) (let* ((i-p (open-input-file input-file)) (parm-regexp (lambda (parm) (make-regexp (lang-parm lang parm)))) (docstring-start (parm-regexp 'docstring-start)) (docstring-end (parm-regexp 'docstring-end)) (docstring-prefix (parm-regexp 'docstring-prefix)) (option-prefix (parm-regexp 'option-prefix)) (signature-start (parm-regexp 'signature-start)) (augmented-options (lambda (line i-p options) (let ((int-doc (and (lang-parm lang 'std-int-doc?) (let ((d (find-std-int-doc line i-p))) (and d (split-prefixed d "internal: ")))))) (if int-doc (append (reverse int-doc) options) options))))) (let lp ((line (read-line i-p)) (state 'neutral) (doc-strings '()) (options '()) (entries '()) (lno 0)) (cond ((eof-object? line) (close-input-port i-p) (reverse entries)) ;; State 'neutral: we're currently not within a docstring or ;; option section ((eq? state 'neutral) (let ((m (regexp-exec docstring-start line))) (if m (lp (read-line i-p) 'doc-string (list (match:substring m 1)) '() entries (+ lno 1)) (lp (read-line i-p) state '() '() entries (+ lno 1))))) ;; State 'doc-string: we have started reading a docstring and ;; are waiting for more, for options or for a define. ((eq? state 'doc-string) (let ((m0 (regexp-exec docstring-prefix line)) (m1 (regexp-exec option-prefix line)) (m2 (regexp-exec signature-start line)) (m3 (regexp-exec docstring-end line))) (cond (m0 (lp (read-line i-p) 'doc-string (cons (match:substring m0 1) doc-strings) '() entries (+ lno 1))) (m1 (lp (read-line i-p) 'options doc-strings (cons (match:substring m1 1) options) entries (+ lno 1))) (m2 (let ((options (augmented-options line i-p options))) ; ttn-mod (lp (read-line i-p) 'neutral '() '() (cons (parse-entry doc-strings options line input-file lno) entries) (+ lno 1)))) (m3 (lp (read-line i-p) 'neutral '() '() (cons (parse-entry doc-strings options #f input-file lno) entries) (+ lno 1))) (else (lp (read-line i-p) 'neutral '() '() entries (+ lno 1)))))) ;; State 'options: We're waiting for more options or for a ;; define. ((eq? state 'options) (let ((m1 (regexp-exec option-prefix line)) (m2 (regexp-exec signature-start line)) (m3 (regexp-exec docstring-end line))) (cond (m1 (lp (read-line i-p) 'options doc-strings (cons (match:substring m1 1) options) entries (+ lno 1))) (m2 (let ((options (augmented-options line i-p options))) ; ttn-mod (lp (read-line i-p) 'neutral '() '() (cons (parse-entry doc-strings options line input-file lno) entries) (+ lno 1)))) (m3 (lp (read-line i-p) 'neutral '() '() (cons (parse-entry doc-strings options #f input-file lno) entries) (+ lno 1))) (else (lp (read-line i-p) 'neutral '() '() entries (+ lno 1)))))))))) (define (make-entry symbol signature docstrings options filename line) (vector 'entry symbol signature docstrings options filename line)) (define (entry-symbol e) (vector-ref e 1)) (define (entry-signature e) (vector-ref e 2)) (define (entry-docstrings e) (vector-ref e 3)) (define (entry-options e) (vector-ref e 4)) (define (entry-filename e) (vector-ref e 5)) (define (entry-line e) "This docstring will not be snarfed, unfortunately..." (vector-ref e 6)) ;; Create a docstring entry from the docstring line list ;; @var{doc-strings}, the option line list @var{options} and the ;; define line @var{def-line} (define (parse-entry docstrings options def-line filename line-no) ; (write-line docstrings) (cond (def-line (make-entry (get-symbol def-line) (make-prototype def-line) (reverse docstrings) (reverse options) filename (+ (- line-no (length docstrings) (length options)) 1))) ((> (length docstrings) 0) (make-entry (string->symbol (car (reverse docstrings))) (car (reverse docstrings)) (cdr (reverse docstrings)) (reverse options) filename (+ (- line-no (length docstrings) (length options)) 1))) (else (make-entry 'foo "" (reverse docstrings) (reverse options) filename (+ (- line-no (length docstrings) (length options)) 1))))) ;; Create a string which is a procedure prototype. The necessary ;; information for constructing the prototype is taken from the line ;; @var{def-line}, which is a line starting with @code{(define...}. (define (make-prototype def-line) (call-with-input-string def-line (lambda (s-p) (let* ((paren (read-char s-p)) (keyword (read s-p)) (tmp (read s-p))) (cond ((pair? tmp) (join-symbols tmp)) ((symbol? tmp) (symbol->string tmp)) (else "")))))) (define (get-symbol def-line) (call-with-input-string def-line (lambda (s-p) (let* ((paren (read-char s-p)) (keyword (read s-p)) (tmp (read s-p))) (cond ((pair? tmp) (car tmp)) ((symbol? tmp) tmp) (else 'foo)))))) ;; Append the symbols in the string list @var{s}, separated with a ;; space character. (define (join-symbols s) (cond ((null? s) "") ((symbol? s) (string-append ". " (symbol->string s))) ((null? (cdr s)) (symbol->string (car s))) (else (string-append (symbol->string (car s)) " " (join-symbols (cdr s)))))) ;; Write @var{entries} to @var{output-file} using @var{writer}. ;; @var{writer} is a proc that takes one entry. ;; If @var{output-file} is #f, write to stdout. ;;-ttn-mod: new proc (define (write-output entries output-file writer) (with-output-to-port (cond (output-file (open-output-file output-file)) (else (current-output-port))) (lambda () (for-each writer entries)))) ;; Write an @var{entry} using texinfo format. ;;-ttn-mod: renamed from `texinfo-output', distilled (define (format-texinfo entry) (display "\n\f") (display (entry-symbol entry)) (newline) (display "@c snarfed from ") (display (entry-filename entry)) (display ":") (display (entry-line entry)) (newline) (display "@deffn procedure ") (display (entry-signature entry)) (newline) (for-each (lambda (s) (write-line s)) (entry-docstrings entry)) (for-each (lambda (s) (display "@c ") (write-line s)) (entry-options entry)) (write-line "@end deffn")) ;; Write an @var{entry} using plain format. ;;-ttn-mod: renamed from `texinfo-output', distilled (define (format-plain entry) (display "Procedure: ") (display (entry-signature entry)) (newline) (for-each (lambda (s) (write-line s)) (entry-docstrings entry)) (for-each (lambda (s) (display ";; ") (write-line s)) (entry-options entry)) (display "Snarfed from ") (display (entry-filename entry)) (display ":") (display (entry-line entry)) (newline) (write-line "\f")) ;;; doc-snarf ends here PK�����/F2] )Ex&��x&����scripts/lint.scmnu�[��������;;; lint --- Preemptive checks for coding errors in Guile Scheme code ;; Copyright (C) 2002, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Neil Jerram ;;; Commentary: ;; Usage: lint FILE1 FILE2 ... ;; ;; Perform various preemptive checks for coding errors in Guile Scheme ;; code. ;; ;; Right now, there is only one check available, for unresolved free ;; variables. The intention is that future lint-like checks will be ;; implemented by adding to this script file. ;; ;; Unresolved free variables ;; ------------------------- ;; ;; Free variables are those whose definitions come from outside the ;; module under investigation. In Guile, these definitions are ;; imported from other modules using `#:use-module' forms. ;; ;; This tool scans the specified files for unresolved free variables - ;; i.e. variables for which you may have forgotten the appropriate ;; `#:use-module', or for which the module that is supposed to export ;; them forgot to. ;; ;; It isn't guaranteed that the scan will find absolutely all such ;; errors. Quoted (and quasiquoted) expressions are skipped, since ;; they are most commonly used to describe constant data, not code, so ;; code that is explicitly evaluated using `eval' will not be checked. ;; For example, the `unresolved-var' in `(eval 'unresolved-var ;; (current-module))' would be missed. ;; ;; False positives are also possible. Firstly, the tool doesn't ;; understand all possible forms of implicit quoting; in particular, ;; it doesn't detect and expand uses of macros. Secondly, it picks up ;; explicit compatibility code like `(if (defined? 'x) (define y x))'. ;; Thirdly, there are occasional oddities like `next-method'. ;; However, the number of false positives for realistic code is ;; hopefully small enough that they can be individually considered and ;; ignored. ;; ;; Example ;; ------- ;; ;; Note: most of the unresolved variables found in this example are ;; false positives, as you would hope. => scope for improvement. ;; ;; $ guild lint `guild` ;; No unresolved free variables in PROGRAM ;; No unresolved free variables in autofrisk ;; No unresolved free variables in display-commentary ;; Unresolved free variables in doc-snarf: ;; doc-snarf-version ;; No unresolved free variables in frisk ;; No unresolved free variables in generate-autoload ;; No unresolved free variables in lint ;; No unresolved free variables in punify ;; No unresolved free variables in read-scheme-source ;; Unresolved free variables in snarf-check-and-output-texi: ;; name ;; pos ;; line ;; x ;; rest ;; ... ;; do-argpos ;; do-command ;; do-args ;; type ;; num ;; file ;; do-arglist ;; req ;; opt ;; var ;; command ;; do-directive ;; s ;; ? ;; No unresolved free variables in use2dot ;;; Code: (define-module (scripts lint) #:use-module (ice-9 common-list) #:use-module (ice-9 format) #:export (lint)) (define %include-in-guild-list #f) (define %summary "Check for bugs and style errors in a Scheme file.") (define (lint filename) (let ((module-name (scan-file-for-module-name filename)) (free-vars (uniq (scan-file-for-free-variables filename)))) (let ((module (resolve-module module-name)) (all-resolved? #t)) (format #t "Resolved module: ~S\n" module) (let loop ((free-vars free-vars)) (or (null? free-vars) (begin (catch #t (lambda () (eval (car free-vars) module)) (lambda args (if all-resolved? (format #t "Unresolved free variables in ~A:\n" filename)) (write-char #\tab) (write (car free-vars)) (newline) (set! all-resolved? #f))) (loop (cdr free-vars))))) (if all-resolved? (format #t "No unresolved free variables in ~A\n" filename))))) (define (scan-file-for-module-name filename) (with-input-from-file filename (lambda () (let loop ((x (read))) (cond ((eof-object? x) #f) ((and (pair? x) (eq? (car x) 'define-module)) (cadr x)) (else (loop (read)))))))) (define (scan-file-for-free-variables filename) (with-input-from-file filename (lambda () (let loop ((x (read)) (fvlists '())) (if (eof-object? x) (apply append fvlists) (loop (read) (cons (detect-free-variables x '()) fvlists))))))) ; guile> (detect-free-variables '(let ((a 1)) a) '()) ; () ; guile> (detect-free-variables '(let ((a 1)) b) '()) ; (b) ; guile> (detect-free-variables '(let ((a 1) (b a)) b) '()) ; (a) ; guile> (detect-free-variables '(let* ((a 1) (b a)) b) '()) ; () ; guile> (detect-free-variables '(define a 1) '()) ; () ; guile> (detect-free-variables '(define a b) '()) ; (b) ; guile> (detect-free-variables '(define (a b c) b) '()) ; () ; guile> (detect-free-variables '(define (a b c) e) '()) ; (e) (define (detect-free-variables x locals) ;; Given an expression @var{x} and a list @var{locals} of local ;; variables (symbols) that are in scope for @var{x}, return a list ;; of free variable symbols. (cond ((symbol? x) (if (memq x locals) '() (list x))) ((pair? x) (case (car x) ((define-module define-generic quote quasiquote) ;; No code of interest in these expressions. '()) ((let letrec) ;; Check for named let. If there is a name, transform the ;; expression so that it looks like an unnamed let with ;; the name as one of the bindings. (if (symbol? (cadr x)) (set-cdr! x (cons (cons (list (cadr x) #f) (caddr x)) (cdddr x)))) ;; Unnamed let processing. (let ((letrec? (eq? (car x) 'letrec)) (locals-for-let-body (append locals (map car (cadr x))))) (append (apply append (map (lambda (binding) (detect-free-variables (cadr binding) (if letrec? locals-for-let-body locals))) (cadr x))) (apply append (map (lambda (bodyform) (detect-free-variables bodyform locals-for-let-body)) (cddr x)))))) ((let* and-let*) ;; Handle bindings recursively. (if (null? (cadr x)) (apply append (map (lambda (bodyform) (detect-free-variables bodyform locals)) (cddr x))) (append (detect-free-variables (cadr (caadr x)) locals) (detect-free-variables `(let* ,(cdadr x) ,@(cddr x)) (cons (caaadr x) locals))))) ((define define-public define-macro) (if (pair? (cadr x)) (begin (set! locals (cons (caadr x) locals)) (detect-free-variables `(lambda ,(cdadr x) ,@(cddr x)) locals)) (begin (set! locals (cons (cadr x) locals)) (detect-free-variables (caddr x) locals)))) ((lambda lambda*) (let ((locals-for-lambda-body (let loop ((locals locals) (args (cadr x))) (cond ((null? args) locals) ((pair? args) (loop (cons (car args) locals) (cdr args))) (else (cons args locals)))))) (apply append (map (lambda (bodyform) (detect-free-variables bodyform locals-for-lambda-body)) (cddr x))))) ((receive) (let ((locals-for-receive-body (append locals (cadr x)))) (apply append (detect-free-variables (caddr x) locals) (map (lambda (bodyform) (detect-free-variables bodyform locals-for-receive-body)) (cdddr x))))) ((define-method define*) (let ((locals-for-method-body (let loop ((locals locals) (args (cdadr x))) (cond ((null? args) locals) ((pair? args) (loop (cons (if (pair? (car args)) (caar args) (car args)) locals) (cdr args))) (else (cons args locals)))))) (apply append (map (lambda (bodyform) (detect-free-variables bodyform locals-for-method-body)) (cddr x))))) ((define-class) ;; Avoid picking up slot names at the start of slot ;; definitions. (apply append (map (lambda (slot/option) (detect-free-variables-noncar (if (pair? slot/option) (cdr slot/option) slot/option) locals)) (cdddr x)))) ((case) (apply append (detect-free-variables (cadr x) locals) (map (lambda (case) (detect-free-variables (cdr case) locals)) (cddr x)))) ((unquote unquote-splicing else =>) (detect-free-variables-noncar (cdr x) locals)) (else (append (detect-free-variables (car x) locals) (detect-free-variables-noncar (cdr x) locals))))) (else '()))) (define (detect-free-variables-noncar x locals) ;; Given an expression @var{x} and a list @var{locals} of local ;; variables (symbols) that are in scope for @var{x}, return a list ;; of free variable symbols. (cond ((symbol? x) (if (memq x locals) '() (list x))) ((pair? x) (case (car x) ((=>) (detect-free-variables-noncar (cdr x) locals)) (else (append (detect-free-variables (car x) locals) (detect-free-variables-noncar (cdr x) locals))))) (else '()))) (define (main . files) (for-each lint files)) ;;; lint ends here PK�����/F2]�U< �� ����scripts/punify.scmnu�[��������;;; punify --- Display Scheme code w/o unnecessary comments / whitespace ;; Copyright (C) 2001, 2006 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen ;;; Commentary: ;; Usage: punify FILE1 FILE2 ... ;; ;; Each file's forms are read and written to stdout. ;; The effect is to remove comments and much non-essential whitespace. ;; This is useful when installing Scheme source to space-limited media. ;; ;; Example: ;; $ wc ./punify ; ./punify ./punify | wc ;; 89 384 3031 ./punify ;; 0 42 920 ;; ;; TODO: Read from stdin. ;; Handle vectors. ;; Identifier punification. ;;; Code: (define-module (scripts punify) :export (punify)) (define %include-in-guild-list #f) (define %summary "Strip comments and whitespace from a Scheme file.") (define (write-punily form) (cond ((and (list? form) (not (null? form))) (let ((first (car form))) (display "(") (write-punily first) (let loop ((ls (cdr form)) (last-was-list? (list? first))) (if (null? ls) (display ")") (let* ((new-first (car ls)) (this-is-list? (list? new-first))) (and (not last-was-list?) (not this-is-list?) (display " ")) (write-punily new-first) (loop (cdr ls) this-is-list?)))))) ((and (symbol? form) (let ((ls (string->list (symbol->string form)))) (and (char=? (car ls) #\:) (not (memq #\space ls)) (list->string (cdr ls))))) => (lambda (symbol-name-after-colon) (display #\:) (display symbol-name-after-colon))) (else (write form)))) (define (punify-one file) (with-input-from-file file (lambda () (let ((toke (lambda () (read (current-input-port))))) (let loop ((form (toke))) (or (eof-object? form) (begin (write-punily form) (loop (toke))))))))) (define (punify . args) (for-each punify-one args)) (define main punify) ;;; punify ends here PK�����/F2]!z������scripts/disassemble.scmnu�[��������;;; Disassemble --- Disassemble .go files into something human-readable ;; Copyright 2005, 2008, 2009, 2011, 2014 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Ludovic Courtès <ludo@gnu.org> ;;; Author: Andy Wingo <wingo@pobox.com> ;;; Commentary: ;; Usage: disassemble [ARGS] ;;; Code: (define-module (scripts disassemble) #:use-module (system vm objcode) #:use-module ((language assembly disassemble) #:prefix asm:) #:export (disassemble)) (define %summary "Disassemble a compiled .go file.") (define (disassemble . files) (for-each (lambda (file) (asm:disassemble (load-objcode file))) files)) (define main disassemble) PK�����/F2]�%[ ��[ ����scripts/scan-api.scmnu�[��������;;; scan-api --- Scan and group interpreter and libguile interface elements ;; Copyright (C) 2002, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: scan-api GUILE SOFILE [GROUPINGS ...] ;; ;; Invoke GUILE, an executable guile interpreter, and use nm(1) on SOFILE, a ;; shared-object library, to determine available interface elements, and ;; display them to stdout as an alist: ;; ;; ((meta ...) (interface ...)) ;; ;; The meta fields are `GUILE_LOAD_PATH', `LTDL_LIBRARY_PATH', `guile' ;; `libguileinterface', `sofile' and `groups'. The interface elements are in ;; turn sub-alists w/ keys `groups' and `scan-data'. Interface elements ;; initially belong in one of two groups `Scheme' or `C' (but not both -- ;; signal error if that happens). ;; ;; Optional GROUPINGS ... are files each containing a single "grouping ;; definition" alist with each entry of the form: ;; ;; (NAME (description "DESCRIPTION") (members SYM...)) ;; ;; All of the SYM... should be proper subsets of the interface. In addition ;; to `description' and `members' forms, the entry may optionally include: ;; ;; (grok USE-MODULES (lambda (x) CODE)) ;; ;; where CODE implements a group-membership predicate to be applied to `x', a ;; symbol. [When evaluated, CODE can assume (use-modules MODULE) has been ;; executed where MODULE is an element of USE-MODULES, a list. [NOT YET ;; IMPLEMENTED!]] ;; ;; Currently, there are two convenience predicates that operate on `x': ;; (in-group? x GROUP) ;; (name-prefix? x PREFIX) ;; ;; TODO: Allow for concurrent Scheme/C membership. ;; Completely separate reporting. ;;; Code: (define-module (scripts scan-api) :use-module (ice-9 popen) :use-module (ice-9 rdelim) :use-module (ice-9 regex) :export (scan-api)) (define %include-in-guild-list #f) (define %summary "Generate an API description for a Guile extension.") (define put set-object-property!) (define get object-property) (define (add-props object . args) (let loop ((args args)) (if (null? args) object ; retval (let ((key (car args)) (value (cadr args))) (put object key value) (loop (cddr args)))))) (define (scan re command match) (let ((rx (make-regexp re)) (port (open-pipe command OPEN_READ))) (let loop ((line (read-line port))) (or (eof-object? line) (begin (cond ((regexp-exec rx line) => match)) (loop (read-line port))))))) (define (scan-Scheme! ht guile) (scan "^.guile.+: ([^ \t]+)([ \t]+(.+))*$" (format #f "~A -c '~S ~S'" guile '(use-modules (ice-9 session)) '(apropos ".")) (lambda (m) (let ((x (string->symbol (match:substring m 1)))) (put x 'Scheme (or (match:substring m 3) "")) (hashq-set! ht x #t))))) (define (scan-C! ht sofile) (scan "^[0-9a-fA-F]+ ([B-TV-Z]) (.+)$" (format #f "nm ~A" sofile) (lambda (m) (let ((x (string->symbol (match:substring m 2)))) (put x 'C (string->symbol (match:substring m 1))) (and (hashq-get-handle ht x) (error "both Scheme and C:" x)) (hashq-set! ht x #t))))) (define THIS-MODULE (current-module)) (define (in-group? x group) (memq group (get x 'groups))) (define (name-prefix? x prefix) (string-match (string-append "^" prefix) (symbol->string x))) (define (add-group-name! x name) (put x 'groups (cons name (get x 'groups)))) (define (make-grok-proc name form) (let* ((predicate? (eval form THIS-MODULE)) (p (lambda (x) (and (predicate? x) (add-group-name! x name))))) (put p 'name name) p)) (define (make-members-proc name members) (let ((p (lambda (x) (and (memq x members) (add-group-name! x name))))) (put p 'name name) p)) (define (make-grouper files) ; \/^^^o/ . o (let ((hook (make-hook 1))) ; /\____\ (for-each (lambda (file) (for-each (lambda (gdef) (let ((name (car gdef)) (members (assq-ref gdef 'members)) (grok (assq-ref gdef 'grok))) (or members grok (error "bad grouping, must have `members' or `grok'")) (add-hook! hook (if grok (add-props (make-grok-proc name (cadr grok)) 'description (assq-ref gdef 'description)) (make-members-proc name members)) #t))) ; append (read (open-file file OPEN_READ)))) files) hook)) (define (scan-api . args) (let ((guile (list-ref args 0)) (sofile (list-ref args 1)) (grouper (false-if-exception (make-grouper (cddr args)))) (ht (make-hash-table 3331))) (scan-Scheme! ht guile) (scan-C! ht sofile) (let ((all (sort (hash-fold (lambda (key value prior-result) (add-props key 'string (symbol->string key) 'scan-data (or (get key 'Scheme) (get key 'C)) 'groups (if (get key 'Scheme) '(Scheme) '(C))) (and grouper (run-hook grouper key)) (cons key prior-result)) '() ht) (lambda (a b) (string<? (get a 'string) (get b 'string)))))) (format #t ";;; generated by scan-api -- do not edit!\n\n") (format #t "(\n") (format #t "(meta\n") (format #t " (GUILE_LOAD_PATH . ~S)\n" (or (getenv "GUILE_LOAD_PATH") "")) (format #t " (LTDL_LIBRARY_PATH . ~S)\n" (or (getenv "LTDL_LIBRARY_PATH") "")) (format #t " (guile . ~S)\n" guile) (format #t " (libguileinterface . ~S)\n" (let ((i #f)) (scan "(.+)" (format #f "~A -c '(display ~A)'" guile '(assq-ref %guile-build-info 'libguileinterface)) (lambda (m) (set! i (match:substring m 1)))) i)) (format #t " (sofile . ~S)\n" sofile) (format #t " ~A\n" (cons 'groups (append (if grouper (map (lambda (p) (get p 'name)) (hook->list grouper)) '()) '(Scheme C)))) (format #t ") ;; end of meta\n") (format #t "(interface\n") (for-each (lambda (x) (format #t "(~A ~A (scan-data ~S))\n" x (cons 'groups (get x 'groups)) (get x 'scan-data))) all) (format #t ") ;; end of interface\n") (format #t ") ;; eof\n"))) #t) (define main scan-api) ;;; scan-api ends here PK�����/F2],������scripts/use2dot.scmnu�[��������;;; use2dot --- Display module dependencies as a DOT specification ;; Copyright (C) 2001, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen ;;; Commentary: ;; Usage: use2dot [OPTIONS] [FILE ...] ;; Display to stdout a DOT specification that describes module dependencies ;; in FILEs. ;; ;; A top-level `use-modules' form or a `:use-module' `define-module'-component ;; results in a "solid" style edge. ;; ;; An `:autoload' `define-module'-component results in a "dotted" style edge ;; with label "N" indicating that N names are responsible for triggering the ;; autoload. [The "N" label is not implemented.] ;; ;; A top-level `load' or `primitive-load' form results in a a "bold" style ;; edge to a node named with either the file name if the `load' argument is a ;; string, or "[computed in FILE]" otherwise. ;; ;; Options: ;; -m, --default-module MOD -- Set MOD as the default module (for top-level ;; `use-modules' forms that do not follow some ;; `define-module' form in a file). MOD should be ;; be a list or `#f', in which case such top-level ;; `use-modules' forms are effectively ignored. ;; Default value: `(guile-user)'. ;;; Code: (define-module (scripts use2dot) :autoload (ice-9 getopt-long) (getopt-long) :use-module ((srfi srfi-13) :select (string-join)) :use-module ((scripts frisk) :select (make-frisker edge-type edge-up edge-down)) :export (use2dot)) (define %summary "Print a module's dependencies in graphviz format.") (define *default-module* '(guile-user)) (define (q s) ; quote (format #f "~S" s)) (define (vv pairs) ; => ("var=val" ...) (map (lambda (pair) (format #f "~A=~A" (car pair) (cdr pair))) pairs)) (define (>>header) (format #t "digraph use2dot {\n") (for-each (lambda (s) (format #t " ~A;\n" s)) (vv `((label . ,(q "Guile Module Dependencies")) ;;(rankdir . LR) ;;(size . ,(q "7.5,10")) (ratio . fill) ;;(nodesep . ,(q "0.05")) )))) (define (>>body edges) (for-each (lambda (edge) (format #t " \"~A\" -> \"~A\"" (edge-down edge) (edge-up edge)) (cond ((case (edge-type edge) ((autoload) '((style . dotted) (fontsize . 5))) ((computed) '((style . bold))) (else #f)) => (lambda (etc) (format #t " [~A]" (string-join (vv etc) ","))))) (format #t ";\n")) edges)) (define (>>footer) (format #t "}")) (define (>> edges) (>>header) (>>body edges) (>>footer)) (define (use2dot . args) (let* ((parsed-args (getopt-long (cons "use2dot" args) ;;; kludge '((default-module (single-char #\m) (value #t))))) (=m (option-ref parsed-args 'default-module *default-module*)) (scan (make-frisker `(default-module . ,=m))) (files (option-ref parsed-args '() '()))) (>> (reverse ((scan files) 'edges))))) (define main use2dot) ;;; use2dot ends here PK�����/F2]HV������scripts/autofrisk.scmnu�[��������;;; autofrisk --- Generate module checks for use with auto* tools ;; Copyright (C) 2002, 2006, 2009, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: autofrisk [file] ;; ;; This program looks for the file modules.af in the current directory ;; and writes out modules.af.m4 containing autoconf definitions. ;; If given, look for FILE instead of modules.af and output to FILE.m4. ;; ;; After running autofrisk, you should add to configure.ac the lines: ;; AUTOFRISK_CHECKS ;; AUTOFRISK_SUMMARY ;; Then run "aclocal -I ." to update aclocal.m4, and finally autoconf. ;; ;; The modules.af file consists of a series of configuration forms (Scheme ;; lists), which have one of the following formats: ;; (files-glob PATTERN ...) ;; (non-critical-external MODULE ...) ;; (non-critical-internal MODULE ...) ;; (programs (MODULE PROG ...) ...) ;; (pww-varname VARNAME) ;; PATTERN is a string that may contain "*" and "?" characters to be ;; expanded into filenames. MODULE is a list of symbols naming a ;; module, such as `(srfi srfi-1)'. VARNAME is a shell-safe name to use ;; instead of "probably_wont_work", the default. This var is passed to ;; `AC_SUBST'. PROG is a string. ;; ;; Only the `files-glob' form is required. ;; ;; TODO: Write better commentary. ;; Make "please see README" configurable. ;;; Code: (define-module (scripts autofrisk) :autoload (ice-9 popen) (open-input-pipe) :use-module (srfi srfi-1) :use-module (srfi srfi-8) :use-module (srfi srfi-13) :use-module (srfi srfi-14) :use-module (scripts read-scheme-source) :use-module (scripts frisk) :export (autofrisk)) (define %include-in-guild-list #f) (define %summary "Generate snippets for use in configure.ac files.") (define *recognized-keys* '(files-glob non-critical-external non-critical-internal programs pww-varname)) (define (canonical-configuration forms) (let ((chk (lambda (condition . x) (or condition (apply error "syntax error:" x))))) (chk (list? forms) "input not a list") (chk (every list? forms) "non-list element") (chk (every (lambda (form) (< 1 (length form))) forms) "list too short") (let ((un #f)) (chk (every (lambda (form) (let ((key (car form))) (and (symbol? key) (or (eq? 'quote key) (memq key *recognized-keys*) (begin (set! un key) #f))))) forms) "unrecognized key:" un)) (let ((bunched (map (lambda (key) (fold (lambda (form so-far) (or (and (eq? (car form) key) (cdr form) (append so-far (cdr form))) so-far)) (list key) forms)) *recognized-keys*))) (lambda (key) (assq-ref bunched key))))) (define (>>strong modules) (for-each (lambda (module) (format #t "GUILE_MODULE_REQUIRED~A\n" module)) modules)) (define (safe-name module) (let ((var (object->string module))) (string-map! (lambda (c) (if (char-set-contains? char-set:letter+digit c) c #\_)) var) var)) (define *pww* "probably_wont_work") (define (>>weak weak-edges) (for-each (lambda (edge) (let* ((up (edge-up edge)) (down (edge-down edge)) (var (format #f "have_guile_module~A" (safe-name up)))) (format #t "GUILE_MODULE_AVAILABLE(~A, ~A)\n" var up) (format #t "test \"$~A\" = no &&\n ~A=\"~A $~A\"~A" var *pww* down *pww* "\n\n"))) weak-edges)) (define (>>program module progs) (let ((vars (map (lambda (prog) (format #f "guile_module~Asupport_~A" (safe-name module) prog)) progs))) (for-each (lambda (var prog) (format #t "AC_PATH_PROG(~A, ~A)\n" var prog)) vars progs) (format #t "test \\\n") (for-each (lambda (var) (format #t " \"$~A\" = \"\" -o \\\n" var)) vars) (format #t "~A &&\n~A=\"~A $~A\"\n\n" (list-ref (list "war = peace" "freedom = slavery" "ignorance = strength") (random 3)) *pww* module *pww*))) (define (>>programs programs) (for-each (lambda (form) (>>program (car form) (cdr form))) programs)) (define (unglob pattern) (let ((p (open-input-pipe (format #f "echo '(' ~A ')'" pattern)))) (map symbol->string (read p)))) (define (>>checks forms) (let* ((cfg (canonical-configuration forms)) (files (apply append (map unglob (cfg 'files-glob)))) (ncx (cfg 'non-critical-external)) (nci (cfg 'non-critical-internal)) (report ((make-frisker) files)) (external (report 'external))) (let ((pww-varname (cfg 'pww-varname))) (or (null? pww-varname) (set! *pww* (car pww-varname)))) (receive (weak strong) (partition (lambda (module) (or (member module ncx) (every (lambda (i) (member i nci)) (map edge-down (mod-down-ls module))))) external) (format #t "AC_DEFUN([AUTOFRISK_CHECKS],[\n\n") (>>strong strong) (format #t "\n~A=~S\n\n" *pww* "") (>>weak (fold (lambda (module so-far) (append so-far (mod-down-ls module))) (list) weak)) (>>programs (cfg 'programs)) (format #t "AC_SUBST(~A)\n])\n\n" *pww*)))) (define (>>summary) (format #t (symbol->string '#{ AC_DEFUN([AUTOFRISK_SUMMARY],[ if test ! "$~A" = "" ; then p=" ***" echo "$p" echo "$p NOTE:" echo "$p The following modules probably won't work:" echo "$p $~A" echo "$p They can be installed anyway, and will work if their" echo "$p dependencies are installed later. Please see README." echo "$p" fi ]) }#) *pww* *pww*)) (define (autofrisk . args) (let ((file (if (null? args) "modules.af" (car args)))) (or (file-exists? file) (error "could not find input file:" file)) (with-output-to-file (format #f "~A.m4" file) (lambda () (>>checks (read-scheme-source-silently file)) (>>summary))))) (define main autofrisk) ;; Local variables: ;; eval: (put 'receive 'scheme-indent-function 2) ;; End: ;;; autofrisk ends here PK�����/F2]=Ǟ �� ����scripts/list.scmnu�[��������;;; List --- List scripts that can be invoked by guild -*- coding: iso-8859-1 -*- ;;;; Copyright (C) 2009, 2010, 2011, 2012 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free ;;;; Software Foundation, Inc., 51 Franklin Street, Fifth Floor, ;;;; Boston, MA 02110-1301 USA ;;; Commentary: ;; Usage: list ;; ;; List scripts that can be invoked by guild. ;;; Code: (define-module (scripts list) #:use-module (srfi srfi-1) #:export (list-scripts)) (define %include-in-guild-list #f) (define %summary "An alias for \"help\".") (define (directory-files dir) (if (and (file-exists? dir) (file-is-directory? dir)) (let ((dir-stream (opendir dir))) (let loop ((new (readdir dir-stream)) (acc '())) (if (eof-object? new) (begin (closedir dir-stream) acc) (loop (readdir dir-stream) (if (or (string=? "." new) ; ignore (string=? ".." new)) ; ignore acc (cons new acc)))))) '())) (define (strip-extensions path) (or-map (lambda (ext) (and (string-suffix? ext path) ;; We really can't be adding e.g. ChangeLog-2008 to the set ;; of runnable scripts, just because "" is a valid ;; extension, by default. So hack around that here. (not (string-null? ext)) (substring path 0 (- (string-length path) (string-length ext))))) (append %load-compiled-extensions %load-extensions))) (define (unique l) (cond ((null? l) l) ((null? (cdr l)) l) ((equal? (car l) (cadr l)) (unique (cdr l))) (else (cons (car l) (unique (cdr l)))))) (define (find-submodules head) (let ((shead (map symbol->string head))) (unique (sort (append-map (lambda (path) (fold (lambda (x rest) (let ((stripped (strip-extensions x))) (if stripped (cons stripped rest) rest))) '() (directory-files (fold (lambda (x y) (in-vicinity y x)) path shead)))) %load-path) string<?)))) (define (list-scripts . args) (for-each (lambda (x) ;; would be nice to show a summary. (format #t "~A\n" x)) (find-submodules '(scripts)))) (define (main . args) (apply (@@ (scripts help) main) args)) PK�����/F2]8������scripts/compile.scmnu�[��������;;; Compile --- Command-line Guile Scheme compiler -*- coding: iso-8859-1 -*- ;; Copyright 2005, 2008, 2009, 2010, 2011, 2014 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Ludovic Courts <ludo@gnu.org> ;;; Author: Andy Wingo <wingo@pobox.com> ;;; Commentary: ;; Usage: compile [ARGS] ;; ;; A command-line interface to the Guile compiler. ;;; Code: (define-module (scripts compile) #:use-module ((system base compile) #:select (compile-file)) #:use-module (system base target) #:use-module (system base message) #:use-module (srfi srfi-1) #:use-module (srfi srfi-13) #:use-module (srfi srfi-37) #:use-module (ice-9 format) #:export (compile)) (define %summary "Compile a file.") (define (fail . messages) (format (current-error-port) "error: ~{~a~}~%" messages) (exit 1)) (define %options ;; Specifications of the command-line options. (list (option '(#\h "help") #f #f (lambda (opt name arg result) (alist-cons 'help? #t result))) (option '("version") #f #f (lambda (opt name arg result) (show-version) (exit 0))) (option '(#\L "load-path") #t #f (lambda (opt name arg result) (let ((load-path (assoc-ref result 'load-path))) (alist-cons 'load-path (cons arg load-path) result)))) (option '(#\o "output") #t #f (lambda (opt name arg result) (if (assoc-ref result 'output-file) (fail "`-o' option cannot be specified more than once") (alist-cons 'output-file arg result)))) (option '(#\W "warn") #t #f (lambda (opt name arg result) (if (string=? arg "help") (begin (show-warning-help) (exit 0)) (let ((warnings (assoc-ref result 'warnings))) (alist-cons 'warnings (cons (string->symbol arg) warnings) (alist-delete 'warnings result)))))) (option '(#\O "optimize") #f #f (lambda (opt name arg result) (alist-cons 'optimize? #t result))) (option '(#\f "from") #t #f (lambda (opt name arg result) (if (assoc-ref result 'from) (fail "`--from' option cannot be specified more than once") (alist-cons 'from (string->symbol arg) result)))) (option '(#\t "to") #t #f (lambda (opt name arg result) (if (assoc-ref result 'to) (fail "`--to' option cannot be specified more than once") (alist-cons 'to (string->symbol arg) result)))) (option '(#\T "target") #t #f (lambda (opt name arg result) (if (assoc-ref result 'target) (fail "`--target' option cannot be specified more than once") (alist-cons 'target arg result)))))) (define (parse-args args) "Parse argument list @var{args} and return an alist with all the relevant options." (args-fold args %options (lambda (opt name arg result) (format (current-error-port) "~A: unrecognized option" name) (exit 1)) (lambda (file result) (let ((input-files (assoc-ref result 'input-files))) (alist-cons 'input-files (cons file input-files) result))) ;; default option values '((input-files) (load-path) (warnings unsupported-warning)))) (define (show-version) (format #t "compile (GNU Guile) ~A~%" (version)) (format #t "Copyright (C) 2009, 2011 Free Software Foundation, Inc. License LGPLv3+: GNU LGPL version 3 or later <http://gnu.org/licenses/lgpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.~%")) (define (show-warning-help) (format #t "The available warning types are:~%~%") (for-each (lambda (wt) (format #t " ~22A ~A~%" (format #f "`~A'" (warning-type-name wt)) (warning-type-description wt))) %warning-types) (format #t "~%")) (define (compile . args) (let* ((options (parse-args args)) (help? (assoc-ref options 'help?)) (compile-opts (let ((o `(#:warnings ,(assoc-ref options 'warnings)))) (if (assoc-ref options 'optimize?) (cons #:O o) o))) (from (or (assoc-ref options 'from) 'scheme)) (to (or (assoc-ref options 'to) 'objcode)) (target (or (assoc-ref options 'target) %host-type)) (input-files (assoc-ref options 'input-files)) (output-file (assoc-ref options 'output-file)) (load-path (assoc-ref options 'load-path))) (if (or help? (null? input-files)) (begin (format #t "Usage: compile [OPTION] FILE... Compile each Guile source file FILE into a Guile object. -h, --help print this help message -L, --load-path=DIR add DIR to the front of the module load path -o, --output=OFILE write output to OFILE -W, --warn=WARNING emit warnings of type WARNING; use `--warn=help' for a list of available warnings -f, --from=LANG specify a source language other than `scheme' -t, --to=LANG specify a target language other than `objcode' -T, --target=TRIPLET produce bytecode for host TRIPLET Note that auto-compilation will be turned off. Report bugs to <~A>.~%" %guile-bug-report-address) (exit 0))) (set! %load-path (append load-path %load-path)) (set! %load-should-auto-compile #f) (if (and output-file (or (null? input-files) (not (null? (cdr input-files))))) (fail "`-o' option can only be specified " "when compiling a single file")) ;; Install a SIGINT handler. As a side effect, this gives unwind ;; handlers an opportunity to run upon SIGINT; this includes that of ;; 'call-with-output-file/atomic', called by 'compile-file', which ;; removes the temporary output file. (sigaction SIGINT (lambda args (fail "interrupted by the user"))) (for-each (lambda (file) (format #t "wrote `~A'\n" (with-fluids ((*current-warning-prefix* "")) (with-target target (lambda () (compile-file file #:output-file output-file #:from from #:to to #:opts compile-opts)))))) input-files))) (define main compile) PK�����/F2]8 �� ����scripts/display-commentary.scmnu�[��������;;; display-commentary --- As advertized ;; Copyright (C) 2001, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen ;;; Commentary: ;; Usage: display-commentary REF1 REF2 ... ;; ;; Display Commentary section from REF1, REF2 and so on. ;; Each REF may be a filename or module name (list of symbols). ;; In the latter case, a filename is computed by searching `%load-path'. ;;; Code: (define-module (scripts display-commentary) :use-module (ice-9 documentation) :export (display-commentary)) (define %summary "Display the Commentary section from a file or module.") (define (display-commentary-one file) (format #t "~A commentary:\n~A" file (file-commentary file))) (define (module-name->filename-frag ls) ; todo: export or move (let ((ls (map symbol->string ls))) (let loop ((ls (cdr ls)) (acc (car ls))) (if (null? ls) acc (loop (cdr ls) (string-append acc "/" (car ls))))))) (define (display-module-commentary module-name) (cond ((%search-load-path (module-name->filename-frag module-name)) => (lambda (file) (format #t "module ~A\n" module-name) (display-commentary-one file))))) (define (display-commentary . refs) (for-each (lambda (ref) (cond ((string? ref) (if (equal? 0 (string-index ref #\()) (display-module-commentary (with-input-from-string ref read)) (display-commentary-one ref))) ((list? ref) (display-module-commentary ref)))) refs)) (define main display-commentary) ;;; display-commentary ends here PK�����/F2] !)��)����scripts/read-text-outline.scmnu�[��������;;; read-text-outline --- Read a text outline and display it as a sexp ;; Copyright (C) 2002, 2006, 2011 Free Software Foundation, Inc. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU Lesser General Public License ;; as published by the Free Software Foundation; either version 3, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; Lesser General Public License for more details. ;; ;; You should have received a copy of the GNU Lesser General Public ;; License along with this software; see the file COPYING.LESSER. If ;; not, write to the Free Software Foundation, Inc., 51 Franklin ;; Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Author: Thien-Thi Nguyen <ttn@gnu.org> ;;; Commentary: ;; Usage: read-text-outline OUTLINE ;; ;; Scan OUTLINE file and display a list of trees, the structure of ;; each reflecting the "levels" in OUTLINE. The recognized outline ;; format (used to indicate outline headings) is zero or more pairs of ;; leading spaces followed by "-". Something like: ;; ;; - a 0 ;; - b 1 ;; - c 2 ;; - d 1 ;; - e 0 ;; - f 1 ;; - g 2 ;; - h 1 ;; ;; In this example the levels are shown to the right. The output for ;; such a file would be the single line: ;; ;; (("a" ("b" "c") "d") ("e" ("f" "g") "h")) ;; ;; Basically, anything at the beginning of a list is a parent, and the ;; remaining elements of that list are its children. ;; ;; ;; Usage from a Scheme program: These two procs are exported: ;; ;; (read-text-outline . args) ; only first arg is used ;; (read-text-outline-silently port) ;; (make-text-outline-reader re specs) ;; ;; `make-text-outline-reader' returns a proc that reads from PORT and ;; returns a list of trees (similar to `read-text-outline-silently'). ;; ;; RE is a regular expression (string) that is used to identify a header ;; line of the outline (as opposed to a whitespace line or intervening ;; text). RE must begin w/ a sub-expression to match the "level prefix" ;; of the line. You can use `level-submatch-number' in SPECS (explained ;; below) to specify a number other than 1, the default. ;; ;; Normally, the level of the line is taken directly as the length of ;; its level prefix. This often results in adjacent levels not mapping ;; to adjacent numbers, which confuses the tree-building portion of the ;; program, which expects top-level to be 0, first sub-level to be 1, ;; etc. You can use `level-substring-divisor' or `compute-level' in ;; SPECS to specify a constant scaling factor or specify a completely ;; alternative procedure, respectively. ;; ;; SPECS is an alist which may contain the following key/value pairs: ;; ;; - level-submatch-number NUMBER ;; - level-substring-divisor NUMBER ;; - compute-level PROC ;; - body-submatch-number NUMBER ;; - extra-fields ((FIELD-1 . SUBMATCH-1) (FIELD-2 . SUBMATCH-2) ...) ;; ;; The PROC value associated with key `compute-level' should take a ;; Scheme match structure (as returned by `regexp-exec') and return a ;; number, the normalized level for that line. If this is specified, ;; it takes precedence over other level-computation methods. ;; ;; Use `body-submatch-number' if RE specifies the whole body, or if you ;; want to make use of the extra fields parsing. The `extra-fields' ;; value is a sub-alist, whose keys name additional fields that are to ;; be recognized. These fields along with `level' are set as object ;; properties of the final string ("body") that is consed into the tree. ;; If a field name ends in "?" the field value is set to be #t if there ;; is a match and the result is not an empty string, and #f otherwise. ;; ;; ;; Bugs and caveats: ;; ;; (1) Only the first file specified on the command line is scanned. ;; (2) TAB characters at the beginnings of lines are not recognized. ;; (3) Outlines that "skip" levels signal an error. In other words, ;; this will fail: ;; ;; - a 0 ;; - b 1 ;; - c 3 <-- skipped 2 -- error! ;; - d 1 ;; ;; ;; TODO: Determine what's the right thing to do for skips. ;; Handle TABs. ;; Make line format customizable via longopts. ;;; Code: (define-module (scripts read-text-outline) :export (read-text-outline read-text-outline-silently make-text-outline-reader) :use-module (ice-9 regex) :autoload (ice-9 rdelim) (read-line) :autoload (ice-9 getopt-long) (getopt-long)) (define %include-in-guild-list #f) (define %summary "Convert textual outlines to s-expressions.") (define (?? symbol) (let ((name (symbol->string symbol))) (string=? "?" (substring name (1- (string-length name)))))) (define (msub n) (lambda (m) (match:substring m n))) (define (??-predicates pair) (cons (car pair) (if (?? (car pair)) (lambda (m) (not (string=? "" (match:substring m (cdr pair))))) (msub (cdr pair))))) (define (make-line-parser re specs) (let* ((rx (let ((fc (substring re 0 1))) (make-regexp (if (string=? "^" fc) re (string-append "^" re))))) (check (lambda (key) (assq-ref specs key))) (level-substring (msub (or (check 'level-submatch-number) 1))) (extract-level (cond ((check 'compute-level) => (lambda (proc) (lambda (m) (proc m)))) ((check 'level-substring-divisor) => (lambda (n) (lambda (m) (/ (string-length (level-substring m)) n)))) (else (lambda (m) (string-length (level-substring m)))))) (extract-body (cond ((check 'body-submatch-number) => msub) (else (lambda (m) (match:suffix m))))) (misc-props! (cond ((check 'extra-fields) => (lambda (alist) (let ((new (map ??-predicates alist))) (lambda (obj m) (for-each (lambda (pair) (set-object-property! obj (car pair) ((cdr pair) m))) new))))) (else (lambda (obj m) #t))))) ;; retval (lambda (line) (cond ((regexp-exec rx line) => (lambda (m) (let ((level (extract-level m)) (body (extract-body m))) (set-object-property! body 'level level) (misc-props! body m) body))) (else #f))))) (define (make-text-outline-reader re specs) (let ((parse-line (make-line-parser re specs))) ;; retval (lambda (port) (let* ((all '(start)) (pchain (list))) ; parents chain (let loop ((line (read-line port)) (prev-level -1) ; how this relates to the first input ; level determines whether or not we ; start in "sibling" or "child" mode. ; in the end, `start' is ignored and ; it's much easier to ignore parents ; than siblings (sometimes). this is ; not to encourage ignorance, however. (tp all)) ; tail pointer (or (eof-object? line) (cond ((parse-line line) => (lambda (w) (let* ((words (list w)) (level (object-property w 'level)) (diff (- level prev-level))) (cond ;; sibling ((zero? diff) ;; just extend the chain (set-cdr! tp words)) ;; child ((positive? diff) (or (= 1 diff) (error "unhandled diff not 1:" diff line)) ;; parent may be contacted by uncle later (kids ;; these days!) so save its level (set-object-property! tp 'level prev-level) (set! pchain (cons tp pchain)) ;; "push down" car into hierarchy (set-car! tp (cons (car tp) words))) ;; uncle ((negative? diff) ;; prune back to where levels match (do ((p pchain (cdr p))) ((= level (object-property (car p) 'level)) (set! pchain p))) ;; resume at this level (set-cdr! (car pchain) words) (set! pchain (cdr pchain)))) (loop (read-line port) level words)))) (else (loop (read-line port) prev-level tp))))) (set! all (car all)) (if (eq? 'start all) '() ; wasteland (cdr all)))))) (define read-text-outline-silently (make-text-outline-reader "(([ ][ ])*)- *" '((level-substring-divisor . 2)))) (define (read-text-outline . args) (write (read-text-outline-silently (open-file (car args) "r"))) (newline) #t) ; exit val (define main read-text-outline) ;;; read-text-outline ends here PK�����/F2]g[S\��\�� ��texinfo.scmnu�[��������;;;; (texinfo) -- parsing of texinfo into SXML ;;;; ;;;; Copyright (C) 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc. ;;;; Copyright (C) 2004, 2009 Andy Wingo <wingo at pobox dot com> ;;;; Copyright (C) 2001,2002 Oleg Kiselyov <oleg at pobox dot com> ;;;; ;;;; This file is based on SSAX's SSAX.scm. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;; Commentary: ;; ;; @subheading Texinfo processing in scheme ;; ;; This module parses texinfo into SXML. TeX will always be the ;; processor of choice for print output, of course. However, although ;; @code{makeinfo} works well for info, its output in other formats is ;; not very customizable, and the program is not extensible as a whole. ;; This module aims to provide an extensible framework for texinfo ;; processing that integrates texinfo into the constellation of SXML ;; processing tools. ;; ;; @subheading Notes on the SXML vocabulary ;; ;; Consider the following texinfo fragment: ;; ;;@example ;; @@deffn Primitive set-car! pair value ;; This function... ;; @@end deffn ;;@end example ;; ;; Logically, the category (Primitive), name (set-car!), and arguments ;; (pair value) are ``attributes'' of the deffn, with the description as ;; the content. However, texinfo allows for @@-commands within the ;; arguments to an environment, like @code{@@deffn}, which means that ;; texinfo ``attributes'' are PCDATA. XML attributes, on the other hand, ;; are CDATA. For this reason, ``attributes'' of texinfo @@-commands are ;; called ``arguments'', and are grouped under the special element, `%'. ;; ;; Because `%' is not a valid NCName, stexinfo is a superset of SXML. In ;; the interests of interoperability, this module provides a conversion ;; function to replace the `%' with `texinfo-arguments'. ;; ;;; Code: ;; Comparison to xml output of texinfo (which is rather undocumented): ;; Doesn't conform to texinfo dtd ;; No DTD at all, in fact :-/ ;; Actually outputs valid xml, after transforming % ;; Slower (although with caching the SXML that problem can go away) ;; Doesn't parse menus (although menus are shite) ;; Args go in a dedicated element, FBOFW ;; Definitions are handled a lot better ;; Does parse comments ;; Outputs only significant line breaks (a biggie!) ;; Nodes are treated as anchors, rather than content organizers (a biggie) ;; (more book-like, less info-like) ;; TODO ;; Integration: help, indexing, plain text (define-module (texinfo) #:use-module (sxml simple) #:use-module (sxml transform) #:use-module (sxml ssax input-parse) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11) #:use-module (srfi srfi-13) #:export (call-with-file-and-dir texi-command-specs texi-command-depth texi-fragment->stexi texi->stexi stexi->sxml)) ;; Some utilities (define (parser-error port message . rest) (apply throw 'parser-error port message rest)) (define (call-with-file-and-dir filename proc) "Call the one-argument procedure @var{proc} with an input port that reads from @var{filename}. During the dynamic extent of @var{proc}'s execution, the current directory will be @code{(dirname @var{filename})}. This is useful for parsing documents that can include files by relative path name." (let ((current-dir (getcwd))) (dynamic-wind (lambda () (chdir (dirname filename))) (lambda () (call-with-input-file (basename filename) proc)) (lambda () (chdir current-dir))))) ;;======================================================================== ;; Reflection on the XML vocabulary (define texi-command-specs #; "A list of (@var{name} @var{content-model} . @var{args}) @table @var @item name The name of an @@-command, as a symbol. @item content-model A symbol indicating the syntactic type of the @@-command: @table @code @item EMPTY-COMMAND No content, and no @code{@@end} is coming @item EOL-ARGS Unparsed arguments until end of line @item EOL-TEXT Parsed arguments until end of line @item INLINE-ARGS Unparsed arguments ending with @code{#\\@}} @item INLINE-TEXT Parsed arguments ending with @code{#\\@}} @item INLINE-TEXT-ARGS Parsed arguments ending with @code{#\\@}} @item ENVIRON The tag is an environment tag, expect @code{@@end foo}. @item TABLE-ENVIRON Like ENVIRON, but with special parsing rules for its arguments. @item FRAGMENT For @code{*fragment*}, the command used for parsing fragments of texinfo documents. @end table @code{INLINE-TEXT} commands will receive their arguments within their bodies, whereas the @code{-ARGS} commands will receive them in their attribute list. @code{EOF-TEXT} receives its arguments in its body. @code{ENVIRON} commands have both: parsed arguments until the end of line, received through their attribute list, and parsed text until the @code{@@end}, received in their bodies. @code{EOF-TEXT-ARGS} receives its arguments in its attribute list, as in @code{ENVIRON}. In addition, @code{ALIAS} can alias one command to another. The alias will never be seen in parsed stexinfo. There are four @@-commands that are treated specially. @code{@@include} is a low-level token that will not be seen by higher-level parsers, so it has no content-model. @code{@@para} is the paragraph command, which is only implicit in the texinfo source. @code{@@item} has special syntax, as noted above, and @code{@@entry} is how this parser treats @code{@@item} commands within @code{@@table}, @code{@@ftable}, and @code{@@vtable}. Also, indexing commands (@code{@@cindex}, etc.) are treated specially. Their arguments are parsed, but they are needed before entering the element so that an anchor can be inserted into the text before the index entry. @item args Named arguments to the command, in the same format as the formals for a lambda. Only present for @code{INLINE-ARGS}, @code{EOL-ARGS}, @code{INLINE-TEXT-ARGS}, @code{ENVIRON}, @code{TABLE-ENVIRON} commands. @end table" '(;; Special commands (include #f) ;; this is a low-level token (para PARAGRAPH) (item ITEM) (entry ENTRY . heading) (noindent EMPTY-COMMAND) (*fragment* FRAGMENT) ;; Inline text commands (*braces* INLINE-TEXT) ;; FIXME: make me irrelevant (bold INLINE-TEXT) (sample INLINE-TEXT) (samp INLINE-TEXT) (code INLINE-TEXT) (math INLINE-TEXT) (kbd INLINE-TEXT) (key INLINE-TEXT) (var INLINE-TEXT) (env INLINE-TEXT) (file INLINE-TEXT) (command INLINE-TEXT) (option INLINE-TEXT) (dfn INLINE-TEXT) (cite INLINE-TEXT) (acro INLINE-TEXT) (email INLINE-TEXT) (emph INLINE-TEXT) (strong INLINE-TEXT) (sample INLINE-TEXT) (sc INLINE-TEXT) (titlefont INLINE-TEXT) (asis INLINE-TEXT) (b INLINE-TEXT) (i INLINE-TEXT) (r INLINE-TEXT) (sansserif INLINE-TEXT) (slanted INLINE-TEXT) (t INLINE-TEXT) ;; Inline args commands (value INLINE-ARGS . (key)) (ref INLINE-ARGS . (node #:opt name section info-file manual)) (xref INLINE-ARGS . (node #:opt name section info-file manual)) (pxref INLINE-TEXT-ARGS . (node #:opt name section info-file manual)) (url ALIAS . uref) (uref INLINE-ARGS . (url #:opt title replacement)) (anchor INLINE-ARGS . (name)) (dots INLINE-ARGS . ()) (result INLINE-ARGS . ()) (bullet INLINE-ARGS . ()) (copyright INLINE-ARGS . ()) (tie INLINE-ARGS . ()) (image INLINE-ARGS . (file #:opt width height alt-text extension)) ;; Inline parsed args commands (acronym INLINE-TEXT-ARGS . (acronym #:opt meaning)) ;; EOL args elements (node EOL-ARGS . (name #:opt next previous up)) (c EOL-ARGS . all) (comment EOL-ARGS . all) (setchapternewpage EOL-ARGS . all) (sp EOL-ARGS . all) (page EOL-ARGS . ()) (vskip EOL-ARGS . all) (syncodeindex EOL-ARGS . all) (contents EOL-ARGS . ()) (shortcontents EOL-ARGS . ()) (summarycontents EOL-ARGS . ()) (insertcopying EOL-ARGS . ()) (dircategory EOL-ARGS . (category)) (top EOL-ARGS . (title)) (printindex EOL-ARGS . (type)) (paragraphindent EOL-ARGS . (indent)) ;; EOL text commands (*ENVIRON-ARGS* EOL-TEXT) (itemx EOL-TEXT) (set EOL-TEXT) (center EOL-TEXT) (title EOL-TEXT) (subtitle EOL-TEXT) (author EOL-TEXT) (chapter EOL-TEXT) (section EOL-TEXT) (appendix EOL-TEXT) (appendixsec EOL-TEXT) (unnumbered EOL-TEXT) (unnumberedsec EOL-TEXT) (subsection EOL-TEXT) (subsubsection EOL-TEXT) (appendixsubsec EOL-TEXT) (appendixsubsubsec EOL-TEXT) (unnumberedsubsec EOL-TEXT) (unnumberedsubsubsec EOL-TEXT) (chapheading EOL-TEXT) (majorheading EOL-TEXT) (heading EOL-TEXT) (subheading EOL-TEXT) (subsubheading EOL-TEXT) (deftpx EOL-TEXT-ARGS . (category name . attributes)) (defcvx EOL-TEXT-ARGS . (category class name)) (defivarx EOL-TEXT-ARGS . (class name)) (deftypeivarx EOL-TEXT-ARGS . (class data-type name)) (defopx EOL-TEXT-ARGS . (category class name . arguments)) (deftypeopx EOL-TEXT-ARGS . (category class data-type name . arguments)) (defmethodx EOL-TEXT-ARGS . (class name . arguments)) (deftypemethodx EOL-TEXT-ARGS . (class data-type name . arguments)) (defoptx EOL-TEXT-ARGS . (name)) (defvrx EOL-TEXT-ARGS . (category name)) (defvarx EOL-TEXT-ARGS . (name)) (deftypevrx EOL-TEXT-ARGS . (category data-type name)) (deftypevarx EOL-TEXT-ARGS . (data-type name)) (deffnx EOL-TEXT-ARGS . (category name . arguments)) (deftypefnx EOL-TEXT-ARGS . (category data-type name . arguments)) (defspecx EOL-TEXT-ARGS . (name . arguments)) (defmacx EOL-TEXT-ARGS . (name . arguments)) (defunx EOL-TEXT-ARGS . (name . arguments)) (deftypefunx EOL-TEXT-ARGS . (data-type name . arguments)) ;; Indexing commands (cindex INDEX . entry) (findex INDEX . entry) (vindex INDEX . entry) (kindex INDEX . entry) (pindex INDEX . entry) (tindex INDEX . entry) ;; Environment commands (those that need @end) (texinfo ENVIRON . title) (ignore ENVIRON . ()) (ifinfo ENVIRON . ()) (iftex ENVIRON . ()) (ifhtml ENVIRON . ()) (ifxml ENVIRON . ()) (ifplaintext ENVIRON . ()) (ifnotinfo ENVIRON . ()) (ifnottex ENVIRON . ()) (ifnothtml ENVIRON . ()) (ifnotxml ENVIRON . ()) (ifnotplaintext ENVIRON . ()) (titlepage ENVIRON . ()) (menu ENVIRON . ()) (direntry ENVIRON . ()) (copying ENVIRON . ()) (example ENVIRON . ()) (smallexample ENVIRON . ()) (display ENVIRON . ()) (smalldisplay ENVIRON . ()) (verbatim ENVIRON . ()) (format ENVIRON . ()) (smallformat ENVIRON . ()) (lisp ENVIRON . ()) (smalllisp ENVIRON . ()) (cartouche ENVIRON . ()) (quotation ENVIRON . ()) (deftp ENVIRON . (category name . attributes)) (defcv ENVIRON . (category class name)) (defivar ENVIRON . (class name)) (deftypeivar ENVIRON . (class data-type name)) (defop ENVIRON . (category class name . arguments)) (deftypeop ENVIRON . (category class data-type name . arguments)) (defmethod ENVIRON . (class name . arguments)) (deftypemethod ENVIRON . (class data-type name . arguments)) (defopt ENVIRON . (name)) (defvr ENVIRON . (category name)) (defvar ENVIRON . (name)) (deftypevr ENVIRON . (category data-type name)) (deftypevar ENVIRON . (data-type name)) (deffn ENVIRON . (category name . arguments)) (deftypefn ENVIRON . (category data-type name . arguments)) (defspec ENVIRON . (name . arguments)) (defmac ENVIRON . (name . arguments)) (defun ENVIRON . (name . arguments)) (deftypefun ENVIRON . (data-type name . arguments)) (table TABLE-ENVIRON . (formatter)) (itemize TABLE-ENVIRON . (formatter)) (enumerate TABLE-ENVIRON . (start)) (ftable TABLE-ENVIRON . (formatter)) (vtable TABLE-ENVIRON . (formatter)))) (define command-depths '((chapter . 1) (section . 2) (subsection . 3) (subsubsection . 4) (top . 0) (unnumbered . 1) (unnumberedsec . 2) (unnumberedsubsec . 3) (unnumberedsubsubsec . 4) (appendix . 1) (appendixsec . 2) (appendixsection . 2) (appendixsubsec . 3) (appendixsubsubsec . 4))) (define (texi-command-depth command max-depth) "Given the texinfo command @var{command}, return its nesting level, or @code{#f} if it nests too deep for @var{max-depth}. Examples: @example (texi-command-depth 'chapter 4) @result{} 1 (texi-command-depth 'top 4) @result{} 0 (texi-command-depth 'subsection 4) @result{} 3 (texi-command-depth 'appendixsubsec 4) @result{} 3 (texi-command-depth 'subsection 2) @result{} #f @end example" (let ((depth (and=> (assq command command-depths) cdr))) (and depth (<= depth max-depth) depth))) ;; The % is for arguments (define (space-significant? command) (memq command '(example smallexample verbatim lisp smalllisp menu %))) ;; Like a DTD for texinfo (define (command-spec command) (let ((spec (assq command texi-command-specs))) (cond ((not spec) (parser-error #f "Unknown command" command)) ((eq? (cadr spec) 'ALIAS) (command-spec (cddr spec))) (else spec)))) (define (inline-content? content) (case content ((INLINE-TEXT INLINE-ARGS INLINE-TEXT-ARGS) #t) (else #f))) ;;======================================================================== ;; Lower-level parsers and scanners ;; ;; They deal with primitive lexical units (Names, whitespaces, tags) and ;; with pieces of more generic productions. Most of these parsers must ;; be called in appropriate context. For example, complete-start-command ;; must be called only when the @-command start has been detected and ;; its name token has been read. ;; Test if a string is made of only whitespace ;; An empty string is considered made of whitespace as well (define (string-whitespace? str) (or (string-null? str) (string-every char-whitespace? str))) ;; Like read-text-line, but allows EOF. (define read-eof-breaks '(*eof* #\return #\newline)) (define (read-eof-line port) (if (eof-object? (peek-char port)) (peek-char port) (let* ((line (next-token '() read-eof-breaks "reading a line" port)) (c (read-char port))) ; must be either \n or \r or EOF (if (and (eq? c #\return) (eq? (peek-char port) #\newline)) (read-char port)) ; skip \n that follows \r line))) (define (skip-whitespace port) (skip-while '(#\space #\tab #\return #\newline) port)) (define (skip-horizontal-whitespace port) (skip-while '(#\space #\tab) port)) ;; command ::= Letter+ ;; procedure: read-command PORT ;; ;; Read a command starting from the current position in the PORT and ;; return it as a symbol. (define (read-command port) (let ((first-char (peek-char port))) (or (char-alphabetic? first-char) (parser-error port "Nonalphabetic @-command char: '" first-char "'"))) (string->symbol (next-token-of (lambda (c) (cond ((eof-object? c) #f) ((char-alphabetic? c) c) (else #f))) port))) ;; A token is a primitive lexical unit. It is a record with two fields, ;; token-head and token-kind. ;; ;; Token types: ;; END The end of a texinfo command. If the command is ended by }, ;; token-head will be #f. Otherwise if the command is ended by ;; @end COMMAND, token-head will be COMMAND. As a special case, ;; @bye is the end of a special @texinfo command. ;; START The start of a texinfo command. The token-head will be a ;; symbol of the @-command name. ;; INCLUDE An @include directive. The token-head will be empty -- the ;; caller is responsible for reading the include file name. ;; ITEM @item commands have an irregular syntax. They end at the ;; next @item, or at the end of the environment. For that ;; read-command-token treats them specially. (define (make-token kind head) (cons kind head)) (define token? pair?) (define token-kind car) (define token-head cdr) ;; procedure: read-command-token PORT ;; ;; This procedure starts parsing of a command token. The current ;; position in the stream must be #\@. This procedure scans enough of ;; the input stream to figure out what kind of a command token it is ;; seeing. The procedure returns a token structure describing the token. (define (read-command-token port) (assert-curr-char '(#\@) "start of the command" port) (let ((peeked (peek-char port))) (cond ((memq peeked '(#\! #\: #\. #\? #\@ #\\ #\{ #\})) ;; @-commands that escape characters (make-token 'STRING (string (read-char port)))) (else (let ((name (read-command port))) (case name ((end) ;; got an ending tag (let ((command (string-trim-both (read-eof-line port)))) (or (and (not (string-null? command)) (string-every char-alphabetic? command)) (parser-error port "malformed @end" command)) (make-token 'END (string->symbol command)))) ((bye) ;; the end of the top (make-token 'END 'texinfo)) ((item) (make-token 'ITEM 'item)) ((include) (make-token 'INCLUDE #f)) (else (make-token 'START name)))))))) ;; procedure+: read-verbatim-body PORT STR-HANDLER SEED ;; ;; This procedure must be called after we have read a string ;; "@verbatim\n" that begins a verbatim section. The current position ;; must be the first position of the verbatim body. This function reads ;; _lines_ of the verbatim body and passes them to a STR-HANDLER, a ;; character data consumer. ;; ;; The str-handler is a STR-HANDLER, a procedure STRING1 STRING2 SEED. ;; The first STRING1 argument to STR-HANDLER never contains a newline. ;; The second STRING2 argument often will. On the first invocation of the ;; STR-HANDLER, the seed is the one passed to read-verbatim-body ;; as the third argument. The result of this first invocation will be ;; passed as the seed argument to the second invocation of the line ;; consumer, and so on. The result of the last invocation of the ;; STR-HANDLER is returned by the read-verbatim-body. Note a ;; similarity to the fundamental 'fold' iterator. ;; ;; Within a verbatim section all characters are taken at their face ;; value. It ends with "\n@end verbatim(\r)?\n". ;; Must be called right after the newline after @verbatim. (define (read-verbatim-body port str-handler seed) (let loop ((seed seed)) (let ((fragment (next-token '() '(#\newline) "reading verbatim" port))) ;; We're reading the char after the 'fragment', which is ;; #\newline. (read-char port) (if (string=? fragment "@end verbatim") seed (loop (str-handler fragment "\n" seed)))))) ;; procedure+: read-arguments PORT ;; ;; This procedure reads and parses a production ArgumentList. ;; ArgumentList ::= S* Argument (S* , S* Argument)* S* ;; Argument ::= ([^@{},])* ;; ;; Arguments are the things in braces, i.e @ref{my node} has one ;; argument, "my node". Most commands taking braces actually don't have ;; arguments, they process text. For example, in ;; @emph{@strong{emphasized}}, the emph takes text, because the parse ;; continues into the braces. ;; ;; Any whitespace within Argument is replaced with a single space. ;; Whitespace around an Argument is trimmed. ;; ;; The procedure returns a list of arguments. Afterwards the current ;; character will be after the final #\}. (define (read-arguments port stop-char) (define (split str) (read-char port) ;; eat the delimiter (let ((ret (map (lambda (x) (if (string-null? x) #f x)) (map string-trim-both (string-split str #\,))))) (if (and (pair? ret) (eq? (car ret) #f) (null? (cdr ret))) '() ret))) (split (next-token '() (list stop-char) "arguments of @-command" port))) ;; procedure+: complete-start-command COMMAND PORT ;; ;; This procedure is to complete parsing of an @-command. The procedure ;; must be called after the command token has been read. COMMAND is a ;; TAG-NAME. ;; ;; This procedure returns several values: ;; COMMAND: a symbol. ;; ARGUMENTS: command's arguments, as an alist. ;; CONTENT-MODEL: the content model of the command. ;; ;; On exit, the current position in PORT will depend on the CONTENT-MODEL. ;; ;; Content model Port position ;; ============= ============= ;; INLINE-TEXT One character after the #\{. ;; INLINE-TEXT-ARGS One character after the #\{. ;; INLINE-ARGS The first character after the #\}. ;; EOL-TEXT The first non-whitespace character after the command. ;; ENVIRON, TABLE-ENVIRON, EOL-ARGS, EOL-TEXT ;; The first character on the next line. ;; PARAGRAPH, ITEM, EMPTY-COMMAND ;; The first character after the command. (define (arguments->attlist port args arg-names) (let loop ((in args) (names arg-names) (opt? #f) (out '())) (cond ((symbol? names) ;; a rest arg (reverse (if (null? in) out (acons names in out)))) ((and (not (null? names)) (eq? (car names) #:opt)) (loop in (cdr names) #t out)) ((null? in) (if (or (null? names) opt?) (reverse out) (parser-error port "@-command expected more arguments:" args arg-names names))) ((null? names) (parser-error port "@-command didn't expect more arguments:" in)) ((not (car in)) (or (and opt? (loop (cdr in) (cdr names) opt? out)) (parser-error "@-command missing required argument" (car names)))) (else (loop (cdr in) (cdr names) opt? (acons (car names) (if (list? (car in)) (car in) (list (car in))) out)))))) (define (parse-table-args command port) (let* ((line (string-trim-both (read-text-line port))) (length (string-length line))) (define (get-formatter) (or (and (not (zero? length)) (eq? (string-ref line 0) #\@) (let ((f (string->symbol (substring line 1)))) (or (inline-content? (cadr (command-spec f))) (parser-error port "@item formatter must be INLINE" f)) f)) (parser-error port "Invalid @item formatter" line))) (case command ((enumerate) (if (zero? length) '() `((start ,(if (or (and (eq? length 1) (char-alphabetic? (string-ref line 0))) (string-every char-numeric? line)) line (parser-error port "Invalid enumerate start" line)))))) ((itemize) `((bullet ,(or (and (eq? length 1) line) (and (string-null? line) '(bullet)) (list (get-formatter)))))) (else ;; tables of various varieties `((formatter (,(get-formatter)))))))) (define (complete-start-command command port) (define (get-arguments type arg-names stop-char) (arguments->attlist port (read-arguments port stop-char) arg-names)) (let* ((spec (command-spec command)) (command (car spec)) (type (cadr spec)) (arg-names (cddr spec))) (case type ((INLINE-TEXT) (assert-curr-char '(#\{) "Inline element lacks {" port) (values command '() type)) ((INLINE-ARGS) (assert-curr-char '(#\{) "Inline element lacks {" port) (values command (get-arguments type arg-names #\}) type)) ((INLINE-TEXT-ARGS) (assert-curr-char '(#\{) "Inline element lacks {" port) (values command '() type)) ((EOL-ARGS) (values command (get-arguments type arg-names #\newline) type)) ((ENVIRON ENTRY INDEX) (skip-horizontal-whitespace port) (values command (parse-environment-args command port) type)) ((TABLE-ENVIRON) (skip-horizontal-whitespace port) (values command (parse-table-args command port) type)) ((EOL-TEXT) (skip-horizontal-whitespace port) (values command '() type)) ((EOL-TEXT-ARGS) (skip-horizontal-whitespace port) (values command (parse-eol-text-args command port) type)) ((PARAGRAPH EMPTY-COMMAND ITEM FRAGMENT) (values command '() type)) (else ;; INCLUDE shouldn't get here (parser-error port "can't happen"))))) ;;----------------------------------------------------------------------------- ;; Higher-level parsers and scanners ;; ;; They parse productions corresponding entire @-commands. ;; Only reads @settitle, leaves it to the command parser to finish ;; reading the title. (define (take-until-settitle port) (or (find-string-from-port? "\n@settitle " port) (parser-error port "No \\n@settitle found")) (skip-horizontal-whitespace port) (and (eq? (peek-char port) #\newline) (parser-error port "You have a @settitle, but no title"))) ;; procedure+: read-char-data PORT EXPECT-EOF? STR-HANDLER SEED ;; ;; This procedure is to read the CharData of a texinfo document. ;; ;; text ::= (CharData | Command)* ;; ;; The procedure reads CharData and stops at @-commands (or ;; environments). It also stops at an open or close brace. ;; ;; port ;; a PORT to read ;; expect-eof? ;; a boolean indicating if EOF is normal, i.e., the character ;; data may be terminated by the EOF. EOF is normal ;; while processing the main document. ;; preserve-ws? ;; a boolean indicating if we are within a whitespace-preserving ;; environment. If #t, suppress paragraph detection. ;; str-handler ;; a STR-HANDLER, see read-verbatim-body ;; seed ;; an argument passed to the first invocation of STR-HANDLER. ;; ;; The procedure returns two results: SEED and TOKEN. The SEED is the ;; result of the last invocation of STR-HANDLER, or the original seed if ;; STR-HANDLER was never called. ;; ;; TOKEN can be either an eof-object (this can happen only if expect-eof? ;; was #t), or a texinfo token denoting the start or end of a tag. ;; read-char-data port expect-eof? preserve-ws? str-handler seed (define read-char-data (let* ((end-chars-eof '(*eof* #\{ #\} #\@ #\newline))) (define (handle str-handler str1 str2 seed) (if (and (string-null? str1) (string-null? str2)) seed (str-handler str1 str2 seed))) (lambda (port expect-eof? preserve-ws? str-handler seed) (let ((end-chars ((if expect-eof? identity cdr) end-chars-eof))) (let loop ((seed seed)) (let* ((fragment (next-token '() end-chars "reading char data" port)) (term-char (peek-char port))) ; one of end-chars (cond ((eof-object? term-char) ; only if expect-eof? (values (handle str-handler fragment "" seed) term-char)) ((memq term-char '(#\@ #\{ #\})) (values (handle str-handler fragment "" seed) (case term-char ((#\@) (read-command-token port)) ((#\{) (make-token 'START '*braces*)) ((#\}) (read-char port) (make-token 'END #f))))) ((eq? term-char #\newline) ;; Always significant, unless directly before an end token. (let ((c (peek-next-char port))) (cond ((eof-object? c) (or expect-eof? (parser-error port "EOF while reading char data")) (values (handle str-handler fragment "" seed) c)) ((eq? c #\@) (let* ((token (read-command-token port)) (end? (eq? (token-kind token) 'END))) (values (handle str-handler fragment (if end? "" (if preserve-ws? "\n" " ")) seed) token))) ((and (not preserve-ws?) (eq? c #\newline)) ;; paragraph-separator ::= #\newline #\newline+ (skip-while '(#\newline) port) (skip-horizontal-whitespace port) (values (handle str-handler fragment "" seed) (make-token 'PARA 'para))) (else (loop (handle str-handler fragment (if preserve-ws? "\n" " ") seed))))))))))))) ; procedure+: assert-token TOKEN KIND NAME ; Make sure that TOKEN is of anticipated KIND and has anticipated NAME (define (assert-token token kind name) (or (and (token? token) (eq? kind (token-kind token)) (equal? name (token-head token))) (parser-error #f "Expecting @end for " name ", got " token))) ;;======================================================================== ;; Highest-level parsers: Texinfo to SXML ;; These parsers are a set of syntactic forms to instantiate a SSAX ;; parser. The user tells what to do with the parsed character and ;; element data. These latter handlers determine if the parsing follows a ;; SAX or a DOM model. ;; syntax: make-command-parser fdown fup str-handler ;; Create a parser to parse and process one element, including its ;; character content or children elements. The parser is typically ;; applied to the root element of a document. ;; fdown ;; procedure COMMAND ARGUMENTS EXPECTED-CONTENT SEED ;; ;; This procedure is to generate the seed to be passed to handlers ;; that process the content of the element. This is the function ;; identified as 'fdown' in the denotational semantics of the XML ;; parser given in the title comments to (sxml ssax). ;; ;; fup ;; procedure COMMAND ARGUMENTS PARENT-SEED SEED ;; ;; This procedure is called when parsing of COMMAND is finished. ;; The SEED is the result from the last content parser (or from ;; fdown if the element has the empty content). PARENT-SEED is the ;; same seed as was passed to fdown. The procedure is to generate a ;; seed that will be the result of the element parser. This is the ;; function identified as 'fup' in the denotational semantics of ;; the XML parser given in the title comments to (sxml ssax). ;; ;; str-handler ;; A STR-HANDLER, see read-verbatim-body ;; ;; The generated parser is a ;; procedure COMMAND PORT SEED ;; ;; The procedure must be called *after* the command token has been read. (define (read-include-file-name port) (let ((x (string-trim-both (read-eof-line port)))) (if (string-null? x) (error "no file listed") x))) ;; fixme: should expand @value{} references (define (sxml->node-name sxml) "Turn some sxml string into a valid node name." (let loop ((in (string->list (sxml->string sxml))) (out '())) (if (null? in) (apply string (reverse out)) (if (memq (car in) '(#\{ #\} #\@ #\,)) (loop (cdr in) out) (loop (cdr in) (cons (car in) out)))))) (define (index command arguments fdown fup parent-seed) (case command ((deftp defcv defivar deftypeivar defop deftypeop defmethod deftypemethod defopt defvr defvar deftypevr deftypevar deffn deftypefn defspec defmac defun deftypefun) (let ((args `((name ,(string-append (symbol->string command) "-" (cadr (assq 'name arguments))))))) (fup 'anchor args parent-seed (fdown 'anchor args 'INLINE-ARGS '())))) ((cindex findex vindex kindex pindex tindex) (let ((args `((name ,(string-append (symbol->string command) "-" (sxml->node-name (assq 'entry arguments))))))) (fup 'anchor args parent-seed (fdown 'anchor args 'INLINE-ARGS '())))) (else parent-seed))) (define (make-command-parser fdown fup str-handler) (lambda (command port seed) (let visit ((command command) (port port) (sig-ws? #f) (parent-seed seed)) (let*-values (((command arguments expected-content) (complete-start-command command port))) (let* ((parent-seed (index command arguments fdown fup parent-seed)) (seed (fdown command arguments expected-content parent-seed)) (eof-closes? (or (memq command '(texinfo para *fragment*)) (eq? expected-content 'EOL-TEXT))) (sig-ws? (or sig-ws? (space-significant? command))) (up (lambda (s) (fup command arguments parent-seed s))) (new-para (lambda (s) (fdown 'para '() 'PARAGRAPH s))) (make-end-para (lambda (p) (lambda (s) (fup 'para '() p s))))) (define (port-for-content) (if (eq? expected-content 'EOL-TEXT) (call-with-input-string (read-text-line port) identity) port)) (cond ((memq expected-content '(EMPTY-COMMAND INLINE-ARGS EOL-ARGS INDEX EOL-TEXT-ARGS)) ;; empty or finished by complete-start-command (up seed)) ((eq? command 'verbatim) (up (read-verbatim-body port str-handler seed))) (else (let loop ((port (port-for-content)) (expect-eof? eof-closes?) (end-para identity) (need-break? (and (not sig-ws?) (memq expected-content '(ENVIRON TABLE-ENVIRON ENTRY ITEM FRAGMENT)))) (seed seed)) (cond ((and need-break? (or sig-ws? (skip-whitespace port)) (not (memq (peek-char port) '(#\@ #\}))) (not (eof-object? (peek-char port)))) ;; Even if we have an @, it might be inline -- check ;; that later (let ((seed (end-para seed))) (loop port expect-eof? (make-end-para seed) #f (new-para seed)))) (else (let*-values (((seed token) (read-char-data port expect-eof? sig-ws? str-handler seed))) (cond ((eof-object? token) (case expect-eof? ((include #f) (end-para seed)) (else (up (end-para seed))))) (else (case (token-kind token) ((STRING) ;; this is only @-commands that escape ;; characters: @}, @@, @{ -- new para if need-break (let ((seed ((if need-break? end-para identity) seed))) (loop port expect-eof? (if need-break? (make-end-para seed) end-para) #f (str-handler (token-head token) "" ((if need-break? new-para identity) seed))))) ((END) ;; The end will only have a name if it's for an ;; environment (cond ((memq command '(item entry)) (let ((spec (command-spec (token-head token)))) (or (eq? (cadr spec) 'TABLE-ENVIRON) (parser-error port "@item not ended by @end table/enumerate/itemize" token)))) ((eq? expected-content 'ENVIRON) (assert-token token 'END command))) (up (end-para seed))) ((ITEM) (cond ((memq command '(enumerate itemize)) (up (visit 'item port sig-ws? (end-para seed)))) ((eq? expected-content 'TABLE-ENVIRON) (up (visit 'entry port sig-ws? (end-para seed)))) ((memq command '(item entry)) (visit command port sig-ws? (up (end-para seed)))) (else (parser-error port "@item must be within a table environment" command)))) ((PARA) ;; examine valid paragraphs? (loop port expect-eof? end-para (not sig-ws?) seed)) ((INCLUDE) ;; Recurse for include files (let ((seed (call-with-file-and-dir (read-include-file-name port) (lambda (port) (loop port 'include end-para need-break? seed))))) (loop port expect-eof? end-para need-break? seed))) ((START) ; Start of an @-command (let* ((head (token-head token)) (spec (command-spec head)) (head (car spec)) (type (cadr spec)) (inline? (inline-content? type)) (seed ((if (and inline? (not need-break?)) identity end-para) seed)) (end-para (if inline? (if need-break? (make-end-para seed) end-para) identity)) (new-para (if (and inline? need-break?) new-para identity))) (loop port expect-eof? end-para (not inline?) (visit head port sig-ws? (new-para seed))))) (else (parser-error port "Unknown token type" token)))))))))))))))) ;; procedure: reverse-collect-str-drop-ws fragments ;; ;; Given the list of fragments (some of which are text strings), reverse ;; the list and concatenate adjacent text strings. We also drop ;; "unsignificant" whitespace, that is, whitespace in front, behind and ;; between elements. The whitespace that is included in character data ;; is not affected. (define (reverse-collect-str-drop-ws fragments) (cond ((null? fragments) ; a shortcut '()) ((and (string? (car fragments)) ; another shortcut (null? (cdr fragments)) ; remove single ws-only string (string-whitespace? (car fragments))) '()) (else (let loop ((fragments fragments) (result '()) (strs '()) (all-whitespace? #t)) (cond ((null? fragments) (if all-whitespace? result ; remove leading ws (cons (apply string-append strs) result))) ((string? (car fragments)) (loop (cdr fragments) result (cons (car fragments) strs) (and all-whitespace? (string-whitespace? (car fragments))))) (else (loop (cdr fragments) (cons (car fragments) (cond ((null? strs) result) (all-whitespace? (if (null? result) result ; remove trailing whitespace (cons " " result))); replace interstitial ws with ; one space (else (cons (apply string-append strs) result)))) '() #t))))))) (define (parse-inline-text-args port spec text) (let lp ((in text) (cur '()) (out '())) (cond ((null? in) (if (and (pair? cur) (string? (car cur)) (string-whitespace? (car cur))) (lp in (cdr cur) out) (let ((args (reverse (if (null? cur) out (cons (reverse cur) out))))) (arguments->attlist port args (cddr spec))))) ((pair? (car in)) (lp (cdr in) (cons (car in) cur) out)) ((string-index (car in) #\,) (let* ((parts (string-split (car in) #\,)) (head (string-trim-right (car parts))) (rev-tail (reverse (cdr parts))) (last (string-trim (car rev-tail)))) (lp (cdr in) (if (string-null? last) cur (cons last cur)) (append (cdr rev-tail) (cons (reverse (if (string-null? head) cur (cons head cur))) out))))) (else (lp (cdr in) (cons (if (null? cur) (string-trim (car in)) (car in)) cur) out))))) (define (make-dom-parser) (make-command-parser (lambda (command args content seed) ; fdown '()) (lambda (command args parent-seed seed) ; fup (let* ((seed (reverse-collect-str-drop-ws seed)) (spec (command-spec command)) (command (car spec))) (if (eq? (cadr spec) 'INLINE-TEXT-ARGS) (cons (list command (cons '% (parse-inline-text-args #f spec seed))) parent-seed) (acons command (if (null? args) seed (acons '% args seed)) parent-seed)))) (lambda (string1 string2 seed) ; str-handler (if (string-null? string2) (cons string1 seed) (cons* string2 string1 seed))))) (define parse-environment-args (let ((parser (make-dom-parser))) ;; duplicate arguments->attlist to avoid unnecessary splitting (lambda (command port) (let* ((args (cdar (parser '*ENVIRON-ARGS* port '()))) (spec (command-spec command)) (command (car spec)) (arg-names (cddr spec))) (cond ((not arg-names) (if (null? args) '() (parser-error port "@-command doesn't take args" command))) ((eq? arg-names #t) (list (cons 'arguments args))) (else (let loop ((args args) (arg-names arg-names) (out '())) (cond ((null? arg-names) (if (null? args) (reverse! out) (parser-error port "@-command didn't expect more args" command args))) ((symbol? arg-names) (reverse! (acons arg-names args out))) ((null? args) (parser-error port "@-command expects more args" command arg-names)) ((and (string? (car args)) (string-index (car args) #\space)) => (lambda (i) (let ((rest (substring/shared (car args) (1+ i)))) (if (zero? i) (loop (cons rest (cdr args)) arg-names out) (loop (cons rest (cdr args)) (cdr arg-names) (cons (list (car arg-names) (substring (car args) 0 i)) out)))))) (else (loop (cdr args) (cdr arg-names) (if (and (pair? (car args)) (eq? (caar args) '*braces*)) (acons (car arg-names) (cdar args) out) (cons (list (car arg-names) (car args)) out)))))))))))) (define (parse-eol-text-args command port) ;; perhaps parse-environment-args should be named more ;; generically. (parse-environment-args command port)) ;; procedure: texi-fragment->stexi STRING ;; ;; A DOM parser for a texinfo fragment STRING. ;; ;; The procedure returns an SXML tree headed by the special tag, ;; *fragment*. (define (texi-fragment->stexi string-or-port) "Parse the texinfo commands in @var{string-or-port}, and return the resultant stexi tree. The head of the tree will be the special command, @code{*fragment*}." (define (parse port) (postprocess (car ((make-dom-parser) '*fragment* port '())))) (if (input-port? string-or-port) (parse string-or-port) (call-with-input-string string-or-port parse))) ;; procedure: texi->stexi PORT ;; ;; This is an instance of a SSAX parser above that returns an SXML ;; representation of the texinfo document ready to be read at PORT. ;; ;; The procedure returns an SXML tree. The port points to the ;; first character after the @bye, or to the end of the file. (define (texi->stexi port) "Read a full texinfo document from @var{port} and return the parsed stexi tree. The parsing will start at the @code{@@settitle} and end at @code{@@bye} or EOF." (let ((parser (make-dom-parser))) (take-until-settitle port) (postprocess (car (parser 'texinfo port '()))))) (define (car-eq? x y) (and (pair? x) (eq? (car x) y))) (define (make-contents tree) (define (lp in out depth) (cond ((null? in) (values in (cons 'enumerate (reverse! out)))) ((and (pair? (cdr in)) (texi-command-depth (caadr in) 4)) => (lambda (new-depth) (let ((node-name (and (car-eq? (car in) 'node) (cadr (assq 'name (cdadar in)))))) (cond ((< new-depth depth) (values in (cons 'enumerate (reverse! out)))) ((> new-depth depth) (let ((out-cdr (if (null? out) '() (cdr out))) (out-car (if (null? out) (list 'item) (car out)))) (let*-values (((new-in new-out) (lp in '() (1+ depth)))) (lp new-in (cons (append out-car (list new-out)) out-cdr) depth)))) (else ;; same depth (lp (cddr in) (cons `(item (para ,@(if node-name `((ref (% (node ,node-name)))) (cdadr in)))) out) depth)))))) (else (lp (cdr in) out depth)))) (let*-values (((_ contents) (lp tree '() 1))) `((chapheading "Table of Contents") ,contents))) (define (trim-whitespace str trim-left? trim-right?) (let* ((left-space? (and (not trim-left?) (string-prefix? " " str))) (right-space? (and (not trim-right?) (string-suffix? " " str))) (tail (append! (string-tokenize str) (if right-space? '("") '())))) (string-join (if left-space? (cons "" tail) tail)))) (define (postprocess tree) (define (loop in out state first? sig-ws?) (cond ((null? in) (values (reverse! out) state)) ((string? (car in)) (loop (cdr in) (cons (if sig-ws? (car in) (trim-whitespace (car in) first? (null? (cdr in)))) out) state #f sig-ws?)) ((pair? (car in)) (case (caar in) ((set) (if (null? (cdar in)) (error "@set missing arguments" in)) (if (string? (cadar in)) (let ((i (string-index (cadar in) #\space))) (if i (loop (cdr in) out (acons (substring (cadar in) 0 i) (cons (substring (cadar in) (1+ i)) (cddar in)) state) #f sig-ws?) (loop (cdr in) out (acons (cadar in) (cddar in) state) #f sig-ws?))) (error "expected a constant to define for @set" in))) ((value) (loop (fold-right cons (cdr in) (or (and=> (assoc (cadr (assq 'key (cdadar in))) state) cdr) (error "unknown value" (cdadar in) state))) out state #f sig-ws?)) ((copying) (loop (cdr in) out (cons (car in) state) #f sig-ws?)) ((insertcopying) (loop (fold-right cons (cdr in) (or (cdr (assoc 'copying state)) (error "copying isn't set yet"))) out state #f sig-ws?)) ((contents) (loop (cdr in) (fold cons out (make-contents tree)) state #f sig-ws?)) (else (let*-values (((kid-out state) (loop (car in) '() state #t (or sig-ws? (space-significant? (caar in)))))) (loop (cdr in) (cons kid-out out) state #f sig-ws?))))) (else ; a symbol (loop (cdr in) (cons (car in) out) state #t sig-ws?)))) (call-with-values (lambda () (loop tree '() '() #t #f)) (lambda (out state) out))) ;; Replace % with texinfo-arguments. (define (stexi->sxml tree) "Transform the stexi tree @var{tree} into sxml. This involves replacing the @code{%} element that keeps the texinfo arguments with an element for each argument. FIXME: right now it just changes % to @code{texinfo-arguments} -- that doesn't hang with the idea of making a dtd at some point" (pre-post-order tree `((% . ,(lambda (x . t) (cons 'texinfo-arguments t))) (*text* . ,(lambda (x t) t)) (*default* . ,(lambda (x . t) (cons x t)))))) ;;; arch-tag: 73890afa-597c-4264-ae70-46fe7756ffb5 ;;; texinfo.scm ends here PK�����/F2]N �� ����oop/goops/stklos.scmnu�[��������;;;; Copyright (C) 1999,2002, 2006, 2010, 2011 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; (define-module (oop goops stklos) :use-module (oop goops internal) :no-backtrace ) ;;; ;;; This is the stklos compatibility module. ;;; ;;; WARNING: This module is under construction. While we expect to be able ;;; to run most stklos code without problems in the future, this is not the ;;; case now. The current compatibility is only superficial. ;;; ;;; Any comments/complaints/patches are welcome. Tell us about ;;; your incompatibility problems (bug-guile@gnu.org). ;;; ;; Export all bindings that are exported from (oop goops)... (module-for-each (lambda (sym var) (module-add! (module-public-interface (current-module)) sym var)) (resolve-interface '(oop goops))) ;; ...but replace the following bindings: (export define-class define-method) ;; Also export the following (export write-object) ;;; Enable keyword support (*fixme*---currently this has global effect) (read-set! keywords 'prefix) (define-syntax-rule (define-class name supers (slot ...) rest ...) (standard-define-class name supers slot ... rest ...)) (define (toplevel-define! name val) (module-define! (current-module) name val)) (define-syntax define-method (syntax-rules (setter) ((_ (setter name) rest ...) (begin (if (or (not (defined? 'name)) (not (is-a? name <generic-with-setter>))) (toplevel-define! 'name (ensure-accessor (if (defined? 'name) name #f) 'name))) (add-method! (setter name) (method rest ...)))) ((_ name rest ...) (begin (if (or (not (defined? 'name)) (not (or (is-a? name <generic>) (is-a? name <primitive-generic>)))) (toplevel-define! 'name (ensure-generic (if (defined? 'name) name #f) 'name))) (add-method! name (method rest ...)))))) PK�����/F2]&=t������oop/goops/internal.scmnu�[��������;;; installed-scm-file ;;;; Copyright (C) 1999, 2001, 2006 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; (define-module (oop goops internal) :use-module (oop goops)) ;; Export all the bindings that are internal to `(oop goops)'. (let ((public-i (module-public-interface (current-module)))) (module-for-each (lambda (name var) (if (eq? name '%module-public-interface) #t (module-add! public-i name var))) (resolve-module '(oop goops)))) PK�����/F2]D)��)����oop/goops/dispatch.scmnu�[��������;;;; Copyright (C) 1999, 2000, 2001, 2003, 2006, 2009, 2012, 2015 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;; There are circularities here; you can't import (oop goops compile) ;; before (oop goops). So when compiling, make sure that things are ;; kosher. (eval-when (expand) (resolve-module '(oop goops))) (define-module (oop goops dispatch) #:use-module (oop goops) #:use-module (oop goops util) #:use-module (system base target) #:export (memoize-method!) #:no-backtrace) (define *dispatch-module* (current-module)) ;;; ;;; Generic functions have an applicable-methods cache associated with ;;; them. Every distinct set of types that is dispatched through a ;;; generic adds an entry to the cache. This cache gets compiled out to ;;; a dispatch procedure. In steady-state, this dispatch procedure is ;;; never recompiled; but during warm-up there is some churn, both to ;;; the cache and to the dispatch procedure. ;;; ;;; So what is the deal if warm-up happens in a multithreaded context? ;;; There is indeed a window between missing the cache for a certain set ;;; of arguments, and then updating the cache with the newly computed ;;; applicable methods. One of the updaters is liable to lose their new ;;; entry. ;;; ;;; This is actually OK though, because a subsequent cache miss for the ;;; race loser will just cause memoization to try again. The cache will ;;; eventually be consistent. We're not mutating the old part of the ;;; cache, just consing on the new entry. ;;; ;;; It doesn't even matter if the dispatch procedure and the cache are ;;; inconsistent -- most likely the type-set that lost the dispatch ;;; procedure race will simply re-trigger a memoization, but since the ;;; winner isn't in the effective-methods cache, it will likely also ;;; re-trigger a memoization, and the cache will finally be consistent. ;;; As you can see there is a possibility for ping-pong effects, but ;;; it's unlikely given the shortness of the window between slot-set! ;;; invocations. We could add a mutex, but it is strictly unnecessary, ;;; and would add runtime cost and complexity. ;;; (define (emit-linear-dispatch gf-sym nargs methods free rest?) (define (gen-syms n stem) (let lp ((n (1- n)) (syms '())) (if (< n 0) syms (lp (1- n) (cons (gensym stem) syms))))) (let* ((args (gen-syms nargs "a")) (types (gen-syms nargs "t"))) (let lp ((methods methods) (free free) (exp `(cache-miss ,gf-sym ,(if rest? `(cons* ,@args rest) `(list ,@args))))) (cond ((null? methods) (values `(,(if rest? `(,@args . rest) args) (let ,(map (lambda (t a) `(,t (class-of ,a))) types args) ,exp)) free)) (else ;; jeez (let preddy ((free free) (types types) (specs (vector-ref (car methods) 1)) (checks '())) (if (null? types) (let ((m-sym (gensym "p"))) (lp (cdr methods) (acons (vector-ref (car methods) 3) m-sym free) `(if (and . ,checks) ,(if rest? `(apply ,m-sym ,@args rest) `(,m-sym . ,args)) ,exp))) (let ((var (assq-ref free (car specs)))) (if var (preddy free (cdr types) (cdr specs) (cons `(eq? ,(car types) ,var) checks)) (let ((var (gensym "c"))) (preddy (acons (car specs) var free) (cdr types) (cdr specs) (cons `(eq? ,(car types) ,var) checks)))))))))))) (define (compute-dispatch-procedure gf cache) (define (scan) (let lp ((ls cache) (nreq -1) (nrest -1)) (cond ((null? ls) (collate (make-vector (1+ nreq) '()) (make-vector (1+ nrest) '()))) ((vector-ref (car ls) 2) ; rest (lp (cdr ls) nreq (max nrest (vector-ref (car ls) 0)))) (else ; req (lp (cdr ls) (max nreq (vector-ref (car ls) 0)) nrest))))) (define (collate req rest) (let lp ((ls cache)) (cond ((null? ls) (emit req rest)) ((vector-ref (car ls) 2) ; rest (let ((n (vector-ref (car ls) 0))) (vector-set! rest n (cons (car ls) (vector-ref rest n))) (lp (cdr ls)))) (else ; req (let ((n (vector-ref (car ls) 0))) (vector-set! req n (cons (car ls) (vector-ref req n))) (lp (cdr ls))))))) (define (emit req rest) (let ((gf-sym (gensym "g"))) (define (emit-rest n clauses free) (if (< n (vector-length rest)) (let ((methods (vector-ref rest n))) (cond ((null? methods) (emit-rest (1+ n) clauses free)) ;; FIXME: hash dispatch (else (call-with-values (lambda () (emit-linear-dispatch gf-sym n methods free #t)) (lambda (clause free) (emit-rest (1+ n) (cons clause clauses) free)))))) (emit-req (1- (vector-length req)) clauses free))) (define (emit-req n clauses free) (if (< n 0) (comp `(lambda ,(map cdr free) (case-lambda ,@clauses)) (map car free)) (let ((methods (vector-ref req n))) (cond ((null? methods) (emit-req (1- n) clauses free)) ;; FIXME: hash dispatch (else (call-with-values (lambda () (emit-linear-dispatch gf-sym n methods free #f)) (lambda (clause free) (emit-req (1- n) (cons clause clauses) free)))))))) (emit-rest 0 (if (or (zero? (vector-length rest)) (null? (vector-ref rest 0))) (list `(args (cache-miss ,gf-sym args))) '()) (acons gf gf-sym '())))) (define (comp exp vals) ;; When cross-compiling Guile itself, the native Guile must generate ;; code for the host. (with-target %host-type (lambda () (let ((p ((@ (system base compile) compile) exp #:env *dispatch-module* #:from 'scheme #:opts '(#:partial-eval? #f #:cse? #f)))) (apply p vals))))) ;; kick it. (scan)) ;; o/~ ten, nine, eight ;; sometimes that's just how it goes ;; three, two, one ;; ;; get out before it blows o/~ ;; (define timer-init 30) (define (delayed-compile gf) (let ((timer timer-init)) (lambda args (set! timer (1- timer)) (cond ((zero? timer) (let ((dispatch (compute-dispatch-procedure gf (slot-ref gf 'effective-methods)))) (slot-set! gf 'procedure dispatch) (apply dispatch args))) (else ;; interestingly, this catches recursive compilation attempts as ;; well; in that case, timer is negative (cache-dispatch gf args)))))) (define (cache-dispatch gf args) (define (map-until n f ls) (if (or (zero? n) (null? ls)) '() (cons (f (car ls)) (map-until (1- n) f (cdr ls))))) (define (equal? x y) ; can't use the stock equal? because it's a generic... (cond ((pair? x) (and (pair? y) (eq? (car x) (car y)) (equal? (cdr x) (cdr y)))) ((null? x) (null? y)) (else #f))) (if (slot-ref gf 'n-specialized) (let ((types (map-until (slot-ref gf 'n-specialized) class-of args))) (let lp ((cache (slot-ref gf 'effective-methods))) (cond ((null? cache) (cache-miss gf args)) ((equal? (vector-ref (car cache) 1) types) (apply (vector-ref (car cache) 3) args)) (else (lp (cdr cache)))))) (cache-miss gf args))) (define (cache-miss gf args) (apply (memoize-method! gf args) args)) (define (memoize-effective-method! gf args applicable) (define (first-n ls n) (if (or (zero? n) (null? ls)) '() (cons (car ls) (first-n (cdr ls) (- n 1))))) (define (parse n ls) (cond ((null? ls) (memoize n #f (map class-of args))) ((= n (slot-ref gf 'n-specialized)) (memoize n #t (map class-of (first-n args n)))) (else (parse (1+ n) (cdr ls))))) (define (memoize len rest? types) (let* ((cmethod ((@@ (oop goops) compute-cmethod) applicable types)) (cache (cons (vector len types rest? cmethod) (slot-ref gf 'effective-methods)))) (slot-set! gf 'effective-methods cache) (slot-set! gf 'procedure (delayed-compile gf)) cmethod)) (parse 0 args)) ;;; ;;; Memoization ;;; (define (memoize-method! gf args) (let ((applicable ((if (eq? gf compute-applicable-methods) %compute-applicable-methods compute-applicable-methods) gf args))) (cond (applicable (memoize-effective-method! gf args applicable)) (else (no-applicable-method gf args))))) (set-procedure-property! memoize-method! 'system-procedure #t) PK�����/F2]tM` �� ����oop/goops/active-slot.scmnu�[��������;;; installed-scm-file ;;;; Copyright (C) 1999, 2001, 2006, 2009 Free Software Foundation, Inc. ;;;; Copyright (C) 1993-1998 Erick Gallesio - I3S-CNRS/ESSI <eg@unice.fr> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;;; ;;;; This file was based upon active-slot.stklos from the STk distribution ;;;; version 4.0.1 by Erick Gallesio <eg@unice.fr>. ;;;; (define-module (oop goops active-slot) :use-module (oop goops internal) :export (<active-class>)) (define-class <active-class> (<class>)) (define-method (compute-get-n-set (class <active-class>) slot) (if (eq? (slot-definition-allocation slot) #:active) (let* ((index (slot-ref class 'nfields)) (s (cdr slot)) (before-ref (get-keyword #:before-slot-ref s #f)) (after-ref (get-keyword #:after-slot-ref s #f)) (before-set! (get-keyword #:before-slot-set! s #f)) (after-set! (get-keyword #:after-slot-set! s #f)) (unbound (make-unbound))) (slot-set! class 'nfields (+ index 1)) (list (lambda (o) (if before-ref (if (before-ref o) (let ((res (%fast-slot-ref o index))) (and after-ref (not (eqv? res unbound)) (after-ref o)) res) (make-unbound)) (let ((res (%fast-slot-ref o index))) (and after-ref (not (eqv? res unbound)) (after-ref o)) res))) (lambda (o v) (if before-set! (if (before-set! o v) (begin (%fast-slot-set! o index v) (and after-set! (after-set! o v)))) (begin (%fast-slot-set! o index v) (and after-set! (after-set! o v))))))) (next-method))) PK�����/F2]E[��[����oop/goops/describe.scmnu�[��������;;; installed-scm-file ;;;; Copyright (C) 1998, 1999, 2001, 2006, 2008, 2009 Free Software Foundation, Inc. ;;;; Copyright (C) 1993-1998 Erick Gallesio - I3S-CNRS/ESSI <eg@unice.fr> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;;; ;;;; This file was based upon describe.stklos from the STk distribution ;;;; version 4.0.1 by Erick Gallesio <eg@unice.fr>. ;;;; (define-module (oop goops describe) :use-module (oop goops) :use-module (ice-9 session) :use-module (ice-9 format) :export (describe)) ; Export the describe generic function ;;; ;;; describe for simple objects ;;; (define-method (describe (x <top>)) (format #t "~s is " x) (cond ((integer? x) (format #t "an integer")) ((real? x) (format #t "a real")) ((complex? x) (format #t "a complex number")) ((null? x) (format #t "an empty list")) ((boolean? x) (format #t "a boolean value (~s)" (if x 'true 'false))) ((char? x) (format #t "a character, ascii value is ~s" (char->integer x))) ((symbol? x) (format #t "a symbol")) ((list? x) (format #t "a list")) ((pair? x) (if (pair? (cdr x)) (format #t "an improper list") (format #t "a pair"))) ((string? x) (if (eqv? x "") (format #t "an empty string") (format #t "a string of length ~s" (string-length x)))) ((vector? x) (if (eqv? x '#()) (format #t "an empty vector") (format #t "a vector of length ~s" (vector-length x)))) ((eof-object? x) (format #t "the end-of-file object")) (else (format #t "an unknown object (~s)" x))) (format #t ".~%") *unspecified*) (define-method (describe (x <procedure>)) (let ((name (procedure-name x))) (if name (format #t "`~s'" name) (display x)) (display " is ") (display (if name #\a "an anonymous")) (display " procedure") (display " with ") (arity x))) ;;; ;;; describe for GOOPS instances ;;; (define (safe-class-name class) (if (slot-bound? class 'name) (class-name class) class)) (define-method (describe (x <object>)) (format #t "~S is an instance of class ~A~%" x (safe-class-name (class-of x))) ;; print all the instance slots (format #t "Slots are: ~%") (for-each (lambda (slot) (let ((name (slot-definition-name slot))) (format #t " ~S = ~A~%" name (if (slot-bound? x name) (format #f "~S" (slot-ref x name)) "#<unbound>")))) (class-slots (class-of x))) *unspecified*) ;;; ;;; Describe for classes ;;; (define-method (describe (x <class>)) (format #t "~S is a class. It's an instance of ~A~%" (safe-class-name x) (safe-class-name (class-of x))) ;; Super classes (format #t "Superclasses are:~%") (for-each (lambda (class) (format #t " ~A~%" (safe-class-name class))) (class-direct-supers x)) ;; Direct slots (let ((slots (class-direct-slots x))) (if (null? slots) (format #t "(No direct slot)~%") (begin (format #t "Directs slots are:~%") (for-each (lambda (s) (format #t " ~A~%" (slot-definition-name s))) slots)))) ;; Direct subclasses (let ((classes (class-direct-subclasses x))) (if (null? classes) (format #t "(No direct subclass)~%") (begin (format #t "Directs subclasses are:~%") (for-each (lambda (s) (format #t " ~A~%" (safe-class-name s))) classes)))) ;; CPL (format #t "Class Precedence List is:~%") (for-each (lambda (s) (format #t " ~A~%" (safe-class-name s))) (class-precedence-list x)) ;; Direct Methods (let ((methods (class-direct-methods x))) (if (null? methods) (format #t "(No direct method)~%") (begin (format #t "Class direct methods are:~%") (for-each describe methods)))) ; (format #t "~%Field Initializers ~% ") ; (write (slot-ref x 'initializers)) (newline) ; (format #t "~%Getters and Setters~% ") ; (write (slot-ref x 'getters-n-setters)) (newline) ) ;;; ;;; Describe for generic functions ;;; (define-method (describe (x <generic>)) (let ((name (generic-function-name x)) (methods (generic-function-methods x))) ;; Title (format #t "~S is a generic function. It's an instance of ~A.~%" name (safe-class-name (class-of x))) ;; Methods (if (null? methods) (format #t "(No method defined for ~S)~%" name) (begin (format #t "Methods defined for ~S~%" name) (for-each (lambda (x) (describe x #t)) methods))))) ;;; ;;; Describe for methods ;;; (define-method (describe (x <method>) . omit-generic) (letrec ((print-args (lambda (args) ;; take care of dotted arg lists (cond ((null? args) (newline)) ((pair? args) (display #\space) (display (safe-class-name (car args))) (print-args (cdr args))) (else (display #\space) (display (safe-class-name args)) (newline)))))) ;; Title (format #t " Method ~A~%" x) ;; Associated generic (if (null? omit-generic) (let ((gf (method-generic-function x))) (if gf (format #t "\t Generic: ~A~%" (generic-function-name gf)) (format #t "\t(No generic)~%")))) ;; GF specializers (format #t "\tSpecializers:") (print-args (method-specializers x)))) (provide 'describe) PK�����/F2]@" �� ����oop/goops/accessors.scmnu�[��������;;;; Copyright (C) 1999, 2000, 2005, 2006 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; (define-module (oop goops accessors) :use-module (oop goops) :re-export (standard-define-class) :export (define-class-with-accessors define-class-with-accessors-keywords)) (define-macro (define-class-with-accessors name supers . slots) (let ((eat? #f)) `(standard-define-class ,name ,supers ,@(map-in-order (lambda (slot) (cond (eat? (set! eat? #f) slot) ((keyword? slot) (set! eat? #t) slot) ((pair? slot) (if (get-keyword #:accessor (cdr slot) #f) slot (let ((name (car slot))) `(,name #:accessor ,name ,@(cdr slot))))) (else `(,slot #:accessor ,slot)))) slots)))) (define-macro (define-class-with-accessors-keywords name supers . slots) (let ((eat? #f)) `(standard-define-class ,name ,supers ,@(map-in-order (lambda (slot) (cond (eat? (set! eat? #f) slot) ((keyword? slot) (set! eat? #t) slot) ((pair? slot) (let ((slot (if (get-keyword #:accessor (cdr slot) #f) slot (let ((name (car slot))) `(,name #:accessor ,name ,@(cdr slot)))))) (if (get-keyword #:init-keyword (cdr slot) #f) slot (let* ((name (car slot)) (keyword (symbol->keyword name))) `(,name #:init-keyword ,keyword ,@(cdr slot)))))) (else `(,slot #:accessor ,slot #:init-keyword ,(symbol->keyword slot))))) slots)))) PK�����/F2]n������oop/goops/util.scmnu�[��������;;;; Copyright (C) 1999, 2000, 2001, 2003, 2006, 2008 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; (define-module (oop goops util) :export (mapappend find-duplicate map* for-each* length* improper->proper) :use-module (srfi srfi-1) :re-export (any every) :no-backtrace ) ;;; ;;; {Utilities} ;;; (define mapappend append-map) (define (find-duplicate l) ; find a duplicate in a list; #f otherwise (cond ((null? l) #f) ((memv (car l) (cdr l)) (car l)) (else (find-duplicate (cdr l))))) (begin-deprecated (define (top-level-env) (let ((mod (current-module))) (if mod (module-eval-closure mod) '()))) (define (top-level-env? env) (or (null? env) (procedure? (car env)))) (export top-level-env? top-level-env)) (define (map* fn . l) ; A map which accepts dotted lists (arg lists (cond ; must be "isomorph" ((null? (car l)) '()) ((pair? (car l)) (cons (apply fn (map car l)) (apply map* fn (map cdr l)))) (else (apply fn l)))) (define (for-each* fn . l) ; A for-each which accepts dotted lists (arg lists (cond ; must be "isomorph" ((null? (car l)) '()) ((pair? (car l)) (apply fn (map car l)) (apply for-each* fn (map cdr l))) (else (apply fn l)))) (define (length* ls) (do ((n 0 (+ 1 n)) (ls ls (cdr ls))) ((not (pair? ls)) n))) (define (improper->proper ls) (if (pair? ls) (cons (car ls) (improper->proper (cdr ls))) (list ls))) PK�����/F2];ʣ������oop/goops/simple.scmnu�[��������;;; installed-scm-file ;;;; Copyright (C) 2005, 2006, 2010, 2011 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; (define-module (oop goops simple) :use-module (oop goops accessors) :export (define-class) :no-backtrace) (define-syntax-rule (define-class arg ...) (define-class-with-accessors-keywords arg ...)) (module-use! (module-public-interface (current-module)) (resolve-interface '(oop goops))) PK�����/F2]=_��_����oop/goops/save.scmnu�[��������;;; installed-scm-file ;;;; Copyright (C) 2000,2001,2002, 2006, 2009, 2010, 2013 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; (define-module (oop goops save) :use-module (oop goops internal) :use-module (oop goops util) :re-export (make-unbound) :export (save-objects load-objects restore enumerate! enumerate-component! write-readably write-component write-component-procedure literal? readable make-readable)) ;;; ;;; save-objects ALIST PORT [EXCLUDED] [USES] ;;; ;;; ALIST ::= ((NAME . OBJECT) ...) ;;; ;;; Save OBJECT ... to PORT so that when the data is read and evaluated ;;; OBJECT ... are re-created under names NAME ... . ;;; Exclude any references to objects in the list EXCLUDED. ;;; Add a (use-modules . USES) line to the top of the saved text. ;;; ;;; In some instances, when `save-object' doesn't know how to produce ;;; readable syntax for an object, you can explicitly register read ;;; syntax for an object using the special form `readable'. ;;; ;;; Example: ;;; ;;; The function `foo' produces an object of obscure structure. ;;; Only `foo' can construct such objects. Because of this, an ;;; object such as ;;; ;;; (define x (vector 1 (foo))) ;;; ;;; cannot be saved by `save-objects'. But if you instead write ;;; ;;; (define x (vector 1 (readable (foo)))) ;;; ;;; `save-objects' will happily produce the necessary read syntax. ;;; ;;; To add new read syntax, hang methods on `enumerate!' and ;;; `write-readably'. ;;; ;;; enumerate! OBJECT ENV ;;; Should call `enumerate-component!' (which takes same args) on ;;; each component object. Should return #t if the composite object ;;; can be written as a literal. (`enumerate-component!' returns #t ;;; if the component is a literal. ;;; ;;; write-readably OBJECT PORT ENV ;;; Should write a readable representation of OBJECT to PORT. ;;; Should use `write-component' to print each component object. ;;; Use `literal?' to decide if a component is a literal. ;;; ;;; Utilities: ;;; ;;; enumerate-component! OBJECT ENV ;;; ;;; write-component OBJECT PATCHER PORT ENV ;;; PATCHER is an expression which, when evaluated, stores OBJECT ;;; into its current location. ;;; ;;; Example: ;;; ;;; (write-component (car ls) `(set-car! ,ls ,(car ls)) file env) ;;; ;;; write-component is a macro. ;;; ;;; literal? COMPONENT ENV ;;; (define-method (immediate? (o <top>)) #f) (define-method (immediate? (o <null>)) #t) (define-method (immediate? (o <number>)) #t) (define-method (immediate? (o <boolean>)) #t) (define-method (immediate? (o <symbol>)) #t) (define-method (immediate? (o <char>)) #t) (define-method (immediate? (o <keyword>)) #t) ;;; enumerate! OBJECT ENVIRONMENT ;;; ;;; Return #t if object is a literal. ;;; (define-method (enumerate! (o <top>) env) #t) (define-method (write-readably (o <top>) file env) ;;(goops-error "No read-syntax defined for object `~S'" o) (write o file) ;doesn't catch bugs, but is much more flexible ) ;;; ;;; Readables ;;; (define readables (make-weak-key-hash-table 61)) (define-macro (readable exp) `(make-readable ,exp ',(copy-tree exp))) (define (make-readable obj expr) (hashq-set! readables obj expr) obj) (define (readable-expression obj) `(readable ,(hashq-ref readables obj))) ;; FIXME: if obj is nil or false, this can return a false value. OTOH ;; usually this is only for non-immediates. (define (readable? obj) (hashq-ref readables obj)) ;;; ;;; Writer helpers ;;; (define (write-component-procedure o file env) "Return #f if circular reference" (cond ((immediate? o) (write o file) #t) ((readable? o) (write (readable-expression o) file) #t) ((excluded? o env) (display #f file) #t) (else (let ((info (object-info o env))) (cond ((not (binding? info)) (write-readably o file env) #t) ((not (eq? (visiting info) #:defined)) #f) ;forward reference (else (display (binding info) file) #t)))))) ;;; write-component OBJECT PATCHER FILE ENV ;;; (define-macro (write-component object patcher file env) `(or (write-component-procedure ,object ,file ,env) (begin (display #f ,file) (add-patcher! ,patcher ,env)))) ;;; ;;; Strings ;;; (define-method (enumerate! (o <string>) env) #f) ;;; ;;; Vectors ;;; (define-method (enumerate! (o <vector>) env) (or (not (vector? o)) (let ((literal? #t)) (array-for-each (lambda (o) (if (not (enumerate-component! o env)) (set! literal? #f))) o) literal?))) (define-method (write-readably (o <vector>) file env) (if (not (vector? o)) (write o file) (let ((n (vector-length o))) (if (zero? n) (display "#()" file) (let ((not-literal? (not (literal? o env)))) (display (if not-literal? "(vector " "#(") file) (if (and not-literal? (literal? (vector-ref o 0) env)) (display #\' file)) (write-component (vector-ref o 0) `(vector-set! ,o 0 ,(vector-ref o 0)) file env) (do ((i 1 (+ 1 i))) ((= i n)) (display #\space file) (if (and not-literal? (literal? (vector-ref o i) env)) (display #\' file)) (write-component (vector-ref o i) `(vector-set! ,o ,i ,(vector-ref o i)) file env)) (display #\) file)))))) ;;; ;;; Arrays ;;; (define-method (enumerate! (o <array>) env) (enumerate-component! (shared-array-root o) env)) (define (make-mapper array) (let* ((n (array-rank array)) (indices (reverse (if (<= n 11) (list-tail '(t s r q p n m l k j i) (- 11 n)) (let loop ((n n) (ls '())) (if (zero? n) ls (loop (- n 1) (cons (gensym "i") ls)))))))) `(lambda ,indices (+ ,(shared-array-offset array) ,@(map (lambda (ind dim inc) `(* ,inc ,(if (pair? dim) `(- ,ind ,(car dim)) ind))) indices (array-dimensions array) (shared-array-increments array)))))) (define (write-array prefix o not-literal? file env) (letrec ((inner (lambda (n indices) (if (not (zero? n)) (let ((el (apply array-ref o (reverse (cons 0 indices))))) (if (and not-literal? (literal? el env)) (display #\' file)) (write-component el `(array-set! ,o ,el ,@indices) file env))) (do ((i 1 (+ 1 i))) ((= i n)) (display #\space file) (let ((el (apply array-ref o (reverse (cons i indices))))) (if (and not-literal? (literal? el env)) (display #\' file)) (write-component el `(array-set! ,o ,el ,@indices) file env)))))) (display prefix file) (let loop ((dims (array-dimensions o)) (indices '())) (cond ((null? (cdr dims)) (inner (car dims) indices)) (else (let ((n (car dims))) (do ((i 0 (+ 1 i))) ((= i n)) (if (> i 0) (display #\space file)) (display prefix file) (loop (cdr dims) (cons i indices)) (display #\) file)))))) (display #\) file))) (define-method (write-readably (o <array>) file env) (let ((root (shared-array-root o))) (cond ((literal? o env) (if (not (vector? root)) (write o file) (begin (display #\# file) (display (array-rank o) file) (write-array #\( o #f file env)))) ((binding? root env) (display "(make-shared-array " file) (if (literal? root env) (display #\' file)) (write-component root (goops-error "write-readably(<array>): internal error") file env) (display #\space file) (display (make-mapper o) file) (for-each (lambda (dim) (display #\space file) (display dim file)) (array-dimensions o)) (display #\) file)) (else (display "(list->uniform-array " file) (display (array-rank o) file) (display " '() " file) (write-array "(list " o #f file env))))) ;;; ;;; Pairs ;;; ;;; These methods have more complex structure than is required for ;;; most objects, since they take over some of the logic of ;;; `write-component'. ;;; (define-method (enumerate! (o <pair>) env) (let ((literal? (enumerate-component! (car o) env))) (and (enumerate-component! (cdr o) env) literal?))) (define-method (write-readably (o <pair>) file env) (let ((proper? (let loop ((ls o)) (or (null? ls) (and (pair? ls) (not (binding? (cdr ls) env)) (loop (cdr ls)))))) (1? (or (not (pair? (cdr o))) (binding? (cdr o) env))) (not-literal? (not (literal? o env))) (infos '()) (refs (ref-stack env))) (display (cond ((not not-literal?) #\() (proper? "(list ") (1? "(cons ") (else "(cons* ")) file) (if (and not-literal? (literal? (car o) env)) (display #\' file)) (write-component (car o) `(set-car! ,o ,(car o)) file env) (do ((ls (cdr o) (cdr ls)) (prev o ls)) ((or (not (pair? ls)) (binding? ls env)) (if (not (null? ls)) (begin (if (not not-literal?) (display " ." file)) (display #\space file) (if (and not-literal? (literal? ls env)) (display #\' file)) (write-component ls `(set-cdr! ,prev ,ls) file env))) (display #\) file)) (display #\space file) (set! infos (cons (object-info ls env) infos)) (push-ref! ls env) ;*fixme* optimize (set! (visiting? (car infos)) #t) (if (and not-literal? (literal? (car ls) env)) (display #\' file)) (write-component (car ls) `(set-car! ,ls ,(car ls)) file env) ) (for-each (lambda (info) (set! (visiting? info) #f)) infos) (set! (ref-stack env) refs) )) ;;; ;;; Objects ;;; ;;; Doesn't yet handle unbound slots ;; Don't export this function! This is all very temporary. ;; (define (get-set-for-each proc class) (for-each (lambda (slotdef g-n-s) (let ((g-n-s (cddr g-n-s))) (cond ((integer? g-n-s) (proc (standard-get g-n-s) (standard-set g-n-s))) ((not (memq (slot-definition-allocation slotdef) '(#:class #:each-subclass))) (proc (car g-n-s) (cadr g-n-s)))))) (class-slots class) (slot-ref class 'getters-n-setters))) (define (access-for-each proc class) (for-each (lambda (slotdef g-n-s) (let ((g-n-s (cddr g-n-s)) (a (slot-definition-accessor slotdef))) (cond ((integer? g-n-s) (proc (slot-definition-name slotdef) (and a (generic-function-name a)) (standard-get g-n-s) (standard-set g-n-s))) ((not (memq (slot-definition-allocation slotdef) '(#:class #:each-subclass))) (proc (slot-definition-name slotdef) (and a (generic-function-name a)) (car g-n-s) (cadr g-n-s)))))) (class-slots class) (slot-ref class 'getters-n-setters))) (define-macro (restore class slots . exps) "(restore CLASS (SLOT-NAME1 ...) EXP1 ...)" `(let ((o ((@@ (oop goops) %allocate-instance) ,class '()))) (for-each (lambda (name val) (slot-set! o name val)) ',slots (list ,@exps)) o)) (define-method (enumerate! (o <object>) env) (get-set-for-each (lambda (get set) (let ((val (get o))) (if (not (unbound? val)) (enumerate-component! val env)))) (class-of o)) #f) (define-method (write-readably (o <object>) file env) (let ((class (class-of o))) (display "(restore " file) (display (class-name class) file) (display " (" file) (let ((slotdefs (filter (lambda (slotdef) (not (or (memq (slot-definition-allocation slotdef) '(#:class #:each-subclass)) (and (slot-bound? o (slot-definition-name slotdef)) (excluded? (slot-ref o (slot-definition-name slotdef)) env))))) (class-slots class)))) (if (not (null? slotdefs)) (begin (display (slot-definition-name (car slotdefs)) file) (for-each (lambda (slotdef) (display #\space file) (display (slot-definition-name slotdef) file)) (cdr slotdefs))))) (display #\) file) (access-for-each (lambda (name aname get set) (display #\space file) (let ((val (get o))) (cond ((unbound? val) (display '(make-unbound) file)) ((excluded? val env)) (else (if (literal? val env) (display #\' file)) (write-component val (if aname `(set! (,aname ,o) ,val) `(slot-set! ,o ',name ,val)) file env))))) class) (display #\) file))) ;;; ;;; Classes ;;; ;;; Currently, we don't support reading in class objects ;;; (define-method (enumerate! (o <class>) env) #f) (define-method (write-readably (o <class>) file env) (display (class-name o) file)) ;;; ;;; Generics ;;; ;;; Currently, we don't support reading in generic functions ;;; (define-method (enumerate! (o <generic>) env) #f) (define-method (write-readably (o <generic>) file env) (display (generic-function-name o) file)) ;;; ;;; Method ;;; ;;; Currently, we don't support reading in methods ;;; (define-method (enumerate! (o <method>) env) #f) (define-method (write-readably (o <method>) file env) (goops-error "No read-syntax for <method> defined")) ;;; ;;; Environments ;;; (define-class <environment> () (object-info #:accessor object-info #:init-form (make-hash-table 61)) (excluded #:accessor excluded #:init-form (make-hash-table 61)) (pass-2? #:accessor pass-2? #:init-value #f) (ref-stack #:accessor ref-stack #:init-value '()) (objects #:accessor objects #:init-value '()) (pre-defines #:accessor pre-defines #:init-value '()) (locals #:accessor locals #:init-value '()) (stand-ins #:accessor stand-ins #:init-value '()) (post-defines #:accessor post-defines #:init-value '()) (patchers #:accessor patchers #:init-value '()) (multiple-bound #:accessor multiple-bound #:init-value '()) ) (define-method (initialize (env <environment>) initargs) (next-method) (cond ((get-keyword #:excluded initargs #f) => (lambda (excludees) (for-each (lambda (e) (hashq-create-handle! (excluded env) e #f)) excludees))))) (define-method (object-info o env) (hashq-ref (object-info env) o)) (define-method ((setter object-info) o env x) (hashq-set! (object-info env) o x)) (define (excluded? o env) (hashq-get-handle (excluded env) o)) (define (add-patcher! patcher env) (set! (patchers env) (cons patcher (patchers env)))) (define (push-ref! o env) (set! (ref-stack env) (cons o (ref-stack env)))) (define (pop-ref! env) (set! (ref-stack env) (cdr (ref-stack env)))) (define (container env) (car (ref-stack env))) (define-class <object-info> () (visiting #:accessor visiting #:init-value #f) (binding #:accessor binding #:init-value #f) (literal? #:accessor literal? #:init-value #f) ) (define visiting? visiting) (define-method (binding (info <boolean>)) #f) (define-method (binding o env) (binding (object-info o env))) (define binding? binding) (define-method (literal? (info <boolean>)) #t) ;;; Note that this method is intended to be used only during the ;;; writing pass ;;; (define-method (literal? o env) (or (immediate? o) (excluded? o env) (let ((info (object-info o env))) ;; write-component sets all bindings first to #:defining, ;; then to #:defined (and (or (not (binding? info)) ;; we might be using `literal?' in a write-readably method ;; to query about the object being defined (and (eq? (visiting info) #:defining) (null? (cdr (ref-stack env))))) (literal? info))))) ;;; ;;; Enumeration ;;; ;;; Enumeration has two passes. ;;; ;;; Pass 1: Detect common substructure, circular references and order ;;; ;;; Pass 2: Detect literals (define (enumerate-component! o env) (cond ((immediate? o) #t) ((readable? o) #f) ((excluded? o env) #t) ((pass-2? env) (let ((info (object-info o env))) (if (binding? info) ;; if circular reference, we print as a literal ;; (note that during pass-2, circular references are ;; forward references, i.e. *not* yet marked with #:pass-2 (not (eq? (visiting? info) #:pass-2)) (and (enumerate! o env) (begin (set! (literal? info) #t) #t))))) ((object-info o env) => (lambda (info) (set! (binding info) #t) (if (visiting? info) ;; circular reference--mark container (set! (binding (object-info (container env) env)) #t)))) (else (let ((info (make <object-info>))) (set! (object-info o env) info) (push-ref! o env) (set! (visiting? info) #t) (enumerate! o env) (set! (visiting? info) #f) (pop-ref! env) (set! (objects env) (cons o (objects env))))))) ;;; ;;; Main engine ;;; (define binding-name car) (define binding-object cdr) (define (pass-1! alist env) ;; Determine object order and necessary bindings (for-each (lambda (binding) (enumerate-component! (binding-object binding) env)) alist)) (define (make-local i) (string->symbol (string-append "%o" (number->string i)))) (define (name-bindings! alist env) ;; Name top-level bindings (for-each (lambda (b) (let ((o (binding-object b))) (if (not (or (immediate? o) (readable? o) (excluded? o env))) (let ((info (object-info o env))) (if (symbol? (binding info)) ;; already bound to a variable (set! (multiple-bound env) (acons (binding info) (binding-name b) (multiple-bound env))) (set! (binding info) (binding-name b))))))) alist) ;; Name rest of bindings and create stand-in and definition lists (let post-loop ((ls (objects env)) (post-defs '())) (cond ((or (null? ls) (eq? (binding (car ls) env) #t)) (set! (post-defines env) post-defs) (set! (objects env) ls)) ((not (binding (car ls) env)) (post-loop (cdr ls) post-defs)) (else (post-loop (cdr ls) (cons (car ls) post-defs))))) (let pre-loop ((ls (reverse (objects env))) (i 0) (pre-defs '()) (locs '()) (sins '())) (if (null? ls) (begin (set! (pre-defines env) (reverse pre-defs)) (set! (locals env) (reverse locs)) (set! (stand-ins env) (reverse sins))) (let ((info (object-info (car ls) env))) (cond ((not (binding? info)) (pre-loop (cdr ls) i pre-defs locs sins)) ((boolean? (binding info)) ;; local (set! (binding info) (make-local i)) (pre-loop (cdr ls) (+ 1 i) pre-defs (cons (car ls) locs) sins)) ((null? locs) (pre-loop (cdr ls) i (cons (car ls) pre-defs) locs sins)) (else (let ((real-name (binding info))) (set! (binding info) (make-local i)) (pre-loop (cdr ls) (+ 1 i) pre-defs (cons (car ls) locs) (acons (binding info) real-name sins))))))))) (define (pass-2! env) (set! (pass-2? env) #t) (for-each (lambda (o) (let ((info (object-info o env))) (set! (literal? info) (enumerate! o env)) (set! (visiting info) #:pass-2))) (append (pre-defines env) (locals env) (post-defines env)))) (define (write-define! name val literal? file) (display "(define " file) (display name file) (display #\space file) (if literal? (display #\' file)) (write val file) (display ")\n" file)) (define (write-empty-defines! file env) (for-each (lambda (stand-in) (write-define! (cdr stand-in) #f #f file)) (stand-ins env)) (for-each (lambda (o) (write-define! (binding o env) #f #f file)) (post-defines env))) (define (write-definition! prefix o file env) (display prefix file) (let ((info (object-info o env))) (display (binding info) file) (display #\space file) (if (literal? info) (display #\' file)) (push-ref! o env) (set! (visiting info) #:defining) (write-readably o file env) (set! (visiting info) #:defined) (pop-ref! env) (display #\) file))) (define (write-let*-head! file env) (display "(let* (" file) (write-definition! "(" (car (locals env)) file env) (for-each (lambda (o) (write-definition! "\n (" o file env)) (cdr (locals env))) (display ")\n" file)) (define (write-rebindings! prefix bindings file env) (for-each (lambda (patch) (display prefix file) (display (cdr patch) file) (display #\space file) (display (car patch) file) (display ")\n" file)) bindings)) (define (write-definitions! selector prefix file env) (for-each (lambda (o) (write-definition! prefix o file env) (newline file)) (selector env))) (define (write-patches! prefix file env) (for-each (lambda (patch) (display prefix file) (display (let name-objects ((patcher patch)) (cond ((binding patcher env) => (lambda (name) (cond ((assq name (stand-ins env)) => cdr) (else name)))) ((pair? patcher) (cons (name-objects (car patcher)) (name-objects (cdr patcher)))) (else patcher))) file) (newline file)) (reverse (patchers env)))) (define (write-immediates! alist file) (for-each (lambda (b) (if (immediate? (binding-object b)) (write-define! (binding-name b) (binding-object b) #t file))) alist)) (define (write-readables! alist file env) (let ((written '())) (for-each (lambda (b) (cond ((not (readable? (binding-object b)))) ((assq (binding-object b) written) => (lambda (p) (set! (multiple-bound env) (acons (cdr p) (binding-name b) (multiple-bound env))))) (else (write-define! (binding-name b) (readable-expression (binding-object b)) #f file) (set! written (acons (binding-object b) (binding-name b) written))))) alist))) (define-method (save-objects (alist <pair>) (file <string>) . rest) (let ((port (open-output-file file))) (apply save-objects alist port rest) (close-port port) *unspecified*)) (define-method (save-objects (alist <pair>) (file <output-port>) . rest) (let ((excluded (if (>= (length rest) 1) (car rest) '())) (uses (if (>= (length rest) 2) (cadr rest) '()))) (let ((env (make <environment> #:excluded excluded))) (pass-1! alist env) (name-bindings! alist env) (pass-2! env) (if (not (null? uses)) (begin (write `(use-modules ,@uses) file) (newline file))) (write-immediates! alist file) (if (null? (locals env)) (begin (write-definitions! post-defines "(define " file env) (write-patches! "" file env)) (begin (write-definitions! pre-defines "(define " file env) (write-empty-defines! file env) (write-let*-head! file env) (write-rebindings! " (set! " (stand-ins env) file env) (write-definitions! post-defines " (set! " file env) (write-patches! " " file env) (display " )\n" file))) (write-readables! alist file env) (write-rebindings! "(define " (reverse (multiple-bound env)) file env)))) (define-method (load-objects (file <string>)) (let* ((port (open-input-file file)) (objects (load-objects port))) (close-port port) objects)) (define iface (module-public-interface (current-module))) (define-method (load-objects (file <input-port>)) (let ((m (make-module))) (module-use! m the-scm-module) (module-use! m iface) (save-module-excursion (lambda () (set-current-module m) (let loop ((sexp (read file))) (if (not (eof-object? sexp)) (begin (eval sexp m) (loop (read file))))))) (module-map (lambda (name var) (cons name (variable-ref var))) m))) PK�����/F2]B �� ����oop/goops/composite-slot.scmnu�[��������;;; installed-scm-file ;;;; Copyright (C) 1999, 2000, 2001, 2006 Free Software Foundation, Inc. ;;;; Copyright (C) 1993-1998 Erick Gallesio - I3S-CNRS/ESSI <eg@unice.fr> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;;; ;;;; This file was based upon composite-slot.stklos from the STk distribution ;;;; version 4.0.1 by Erick Gallesio <eg@unice.fr>. ;;;; (define-module (oop goops composite-slot) :use-module (oop goops) :export (<composite-class>)) ;;; ;;; (define-class CLASS SUPERS ;;; ... ;;; (OBJECT ...) ;;; ... ;;; (SLOT #:allocation #:propagated ;;; #:propagate-to '(PROPAGATION ...)) ;;; ... ;;; #:metaclass <composite-class>) ;;; ;;; PROPAGATION ::= OBJECT | (OBJECT TARGETSLOT) ;;; ;;; The slot SLOT will be propagated to the slot TARGETSLOT in the object ;;; stored in slot OBJECT. If TARGETSLOT is omitted, assume that the target ;;; slot is named SLOT. ;;; (define-class <composite-class> (<class>)) (define-method (compute-get-n-set (class <composite-class>) slot) (if (eq? (slot-definition-allocation slot) #:propagated) (compute-propagated-get-n-set slot) (next-method))) (define (compute-propagated-get-n-set s) (let ((prop (get-keyword #:propagate-to (cdr s) #f)) (s-name (slot-definition-name s))) (if (not prop) (goops-error "Propagation not specified for slot ~S" s-name)) (if (not (pair? prop)) (goops-error "Bad propagation list for slot ~S" s-name)) (let ((objects (map (lambda (p) (if (pair? p) (car p) p)) prop)) (slots (map (lambda (p) (if (pair? p) (cadr p) s-name)) prop))) (let ((first-object (car objects)) (first-slot (car slots))) (list ;; The getter (lambda (o) (slot-ref (slot-ref o first-object) first-slot)) ;; The setter (if (null? (cdr objects)) (lambda (o v) (slot-set! (slot-ref o first-object) first-slot v)) (lambda (o v) (for-each (lambda (object slot) (slot-set! (slot-ref o object) slot v)) objects slots)))))))) PK�����/F2]0N������oop/goops/compile.scmnu�[��������;;;; Copyright (C) 1999, 2001, 2006, 2009, 2015 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; (define-module (oop goops compile) #:use-module (oop goops internal) #:re-export (compute-cmethod)) PK�����/F2]?O��O�� ��oop/goops.scmnu�[��������;;; installed-scm-file ;;;; Copyright (C) 1998,1999,2000,2001,2002, 2003, 2006, 2009, 2010, 2011, 2014, 2015 Free Software Foundation, Inc. ;;;; Copyright (C) 1993-1998 Erick Gallesio - I3S-CNRS/ESSI <eg@unice.fr> ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;;; ;;;; This file was based upon stklos.stk from the STk distribution ;;;; version 4.0.1 by Erick Gallesio <eg@unice.fr>. ;;;; (define-module (oop goops) #:use-module (srfi srfi-1) #:use-module (ice-9 match) #:use-module (oop goops util) #:export-syntax (define-class class standard-define-class define-generic define-accessor define-method define-extended-generic define-extended-generics method) #:export (is-a? class-of ensure-metaclass ensure-metaclass-with-supers make-class make-generic ensure-generic make-extended-generic make-accessor ensure-accessor add-method! class-slot-ref class-slot-set! slot-unbound slot-missing slot-definition-name slot-definition-options slot-definition-allocation slot-definition-getter slot-definition-setter slot-definition-accessor slot-definition-init-value slot-definition-init-form slot-definition-init-thunk slot-definition-init-keyword slot-init-function class-slot-definition method-source compute-cpl compute-std-cpl compute-get-n-set compute-slots compute-getter-method compute-setter-method allocate-instance initialize make-instance make no-next-method no-applicable-method no-method change-class update-instance-for-different-class shallow-clone deep-clone class-redefinition apply-generic apply-method apply-methods compute-applicable-methods %compute-applicable-methods method-more-specific? sort-applicable-methods class-subclasses class-methods goops-error min-fixnum max-fixnum ;;; *fixme* Should go into goops.c instance? slot-ref-using-class slot-set-using-class! slot-bound-using-class? slot-exists-using-class? slot-ref slot-set! slot-bound? class-name class-direct-supers class-direct-subclasses class-direct-methods class-direct-slots class-precedence-list class-slots generic-function-name generic-function-methods method-generic-function method-specializers method-formals primitive-generic-generic enable-primitive-generic! method-procedure accessor-method-slot-definition slot-exists? make find-method get-keyword)) (define *goops-module* (current-module)) ;; First initialize the builtin part of GOOPS (eval-when (expand load eval) (%init-goops-builtins)) (eval-when (expand load eval) (use-modules ((language tree-il primitives) :select (add-interesting-primitive!))) (add-interesting-primitive! 'class-of)) ;; Then load the rest of GOOPS (use-modules (oop goops dispatch)) ;;; ;;; Compiling next methods into method bodies ;;; ;;; So, for the reader: there basic idea is that, given that the ;;; semantics of `next-method' depend on the concrete types being ;;; dispatched, why not compile a specific procedure to handle each type ;;; combination that we see at runtime. ;;; ;;; In theory we can do much better than a bytecode compilation, because ;;; we know the *exact* types of the arguments. It's ideal for native ;;; compilation. A task for the future. ;;; ;;; I think this whole generic application mess would benefit from a ;;; strict MOP. (define (compute-cmethod methods types) (match methods ((method . methods) (let ((make-procedure (slot-ref method 'make-procedure))) (if make-procedure (make-procedure (if (null? methods) (lambda args (no-next-method (method-generic-function method) args)) (compute-cmethod methods types))) (method-procedure method)))))) (eval-when (expand load eval) (define min-fixnum (- (expt 2 29))) (define max-fixnum (- (expt 2 29) 1))) ;; ;; goops-error ;; (define (goops-error format-string . args) (scm-error 'goops-error #f format-string args '())) ;; ;; is-a? ;; (define (is-a? obj class) (and (memq class (class-precedence-list (class-of obj))) #t)) ;;; ;;; {Meta classes} ;;; (define ensure-metaclass-with-supers (let ((table-of-metas '())) (lambda (meta-supers) (let ((entry (assoc meta-supers table-of-metas))) (if entry ;; Found a previously created metaclass (cdr entry) ;; Create a new meta-class which inherit from "meta-supers" (let ((new (make <class> #:dsupers meta-supers #:slots '() #:name (gensym "metaclass")))) (set! table-of-metas (cons (cons meta-supers new) table-of-metas)) new)))))) (define (ensure-metaclass supers) (if (null? supers) <class> (let* ((all-metas (map (lambda (x) (class-of x)) supers)) (all-cpls (append-map (lambda (m) (cdr (class-precedence-list m))) all-metas)) (needed-metas '())) ;; Find the most specific metaclasses. The new metaclass will be ;; a subclass of these. (for-each (lambda (meta) (if (and (not (member meta all-cpls)) (not (member meta needed-metas))) (set! needed-metas (append needed-metas (list meta))))) all-metas) ;; Now return a subclass of the metaclasses we found. (if (null? (cdr needed-metas)) (car needed-metas) ; If there's only one, just use it. (ensure-metaclass-with-supers needed-metas))))) ;;; ;;; {Classes} ;;; ;;; (define-class NAME (SUPER ...) SLOT-DEFINITION ... OPTION ...) ;;; ;;; SLOT-DEFINITION ::= SLOT-NAME | (SLOT-NAME OPTION ...) ;;; OPTION ::= KEYWORD VALUE ;;; (define (kw-do-map mapper f kwargs) (define (keywords l) (cond ((null? l) '()) ((or (null? (cdr l)) (not (keyword? (car l)))) (goops-error "malformed keyword arguments: ~a" kwargs)) (else (cons (car l) (keywords (cddr l)))))) (define (args l) (if (null? l) '() (cons (cadr l) (args (cddr l))))) ;; let* to check keywords first (let* ((k (keywords kwargs)) (a (args kwargs))) (mapper f k a))) (define (make-class supers slots . options) (let* ((name (get-keyword #:name options (make-unbound))) (supers (if (not (or-map (lambda (class) (memq <object> (class-precedence-list class))) supers)) (append supers (list <object>)) supers)) (metaclass (or (get-keyword #:metaclass options #f) (ensure-metaclass supers)))) ;; Verify that all direct slots are different and that we don't inherit ;; several time from the same class (let ((tmp1 (find-duplicate supers)) (tmp2 (find-duplicate (map slot-definition-name slots)))) (if tmp1 (goops-error "make-class: super class ~S is duplicate in class ~S" tmp1 name)) (if tmp2 (goops-error "make-class: slot ~S is duplicate in class ~S" tmp2 name))) ;; Everything seems correct, build the class (apply make metaclass #:dsupers supers #:slots slots #:name name options))) ;;; (class (SUPER ...) SLOT-DEFINITION ... OPTION ...) ;;; ;;; SLOT-DEFINITION ::= SLOT-NAME | (SLOT-NAME OPTION ...) ;;; OPTION ::= KEYWORD VALUE ;;; (define-macro (class supers . slots) (define (make-slot-definition-forms slots) (map (lambda (def) (cond ((pair? def) `(list ',(car def) ,@(kw-do-map append-map (lambda (kw arg) (case kw ((#:init-form) `(#:init-form ',arg #:init-thunk (lambda () ,arg))) (else (list kw arg)))) (cdr def)))) (else `(list ',def)))) slots)) (if (not (list? supers)) (goops-error "malformed superclass list: ~S" supers)) (let ((slots (take-while (lambda (x) (not (keyword? x))) slots)) (options (or (find-tail keyword? slots) '()))) `(make-class ;; evaluate super class variables (list ,@supers) ;; evaluate slot definitions, except the slot name! (list ,@(make-slot-definition-forms slots)) ;; evaluate class options ,@options))) (define-syntax define-class-pre-definition (lambda (x) (syntax-case x () ((_ (k arg rest ...) out ...) (keyword? (syntax->datum #'k)) (case (syntax->datum #'k) ((#:getter #:setter) #'(define-class-pre-definition (rest ...) out ... (if (or (not (defined? 'arg)) (not (is-a? arg <generic>))) (toplevel-define! 'arg (ensure-generic (if (defined? 'arg) arg #f) 'arg))))) ((#:accessor) #'(define-class-pre-definition (rest ...) out ... (if (or (not (defined? 'arg)) (not (is-a? arg <accessor>))) (toplevel-define! 'arg (ensure-accessor (if (defined? 'arg) arg #f) 'arg))))) (else #'(define-class-pre-definition (rest ...) out ...)))) ((_ () out ...) #'(begin out ...))))) ;; Some slot options require extra definitions to be made. In ;; particular, we want to make sure that the generic function objects ;; which represent accessors exist before `make-class' tries to add ;; methods to them. (define-syntax define-class-pre-definitions (lambda (x) (syntax-case x () ((_ () out ...) #'(begin out ...)) ((_ (slot rest ...) out ...) (keyword? (syntax->datum #'slot)) #'(begin out ...)) ((_ (slot rest ...) out ...) (identifier? #'slot) #'(define-class-pre-definitions (rest ...) out ...)) ((_ ((slotname slotopt ...) rest ...) out ...) #'(define-class-pre-definitions (rest ...) out ... (define-class-pre-definition (slotopt ...))))))) (define-syntax-rule (define-class name supers slot ...) (begin (define-class-pre-definitions (slot ...)) (if (and (defined? 'name) (is-a? name <class>) (memq <object> (class-precedence-list name))) (class-redefinition name (class supers slot ... #:name 'name)) (toplevel-define! 'name (class supers slot ... #:name 'name))))) (define-syntax-rule (standard-define-class arg ...) (define-class arg ...)) ;;; ;;; {Generic functions and accessors} ;;; ;; Apparently the desired semantics are that we extend previous ;; procedural definitions, but that if `name' was already a generic, we ;; overwrite its definition. (define-macro (define-generic name) (if (not (symbol? name)) (goops-error "bad generic function name: ~S" name)) `(define ,name (if (and (defined? ',name) (is-a? ,name <generic>)) (make <generic> #:name ',name) (ensure-generic (if (defined? ',name) ,name #f) ',name)))) (define-macro (define-extended-generic name val) (if (not (symbol? name)) (goops-error "bad generic function name: ~S" name)) `(define ,name (make-extended-generic ,val ',name))) (define-macro (define-extended-generics names . args) (let ((prefixes (get-keyword #:prefix args #f))) (if prefixes `(begin ,@(map (lambda (name) `(define-extended-generic ,name (list ,@(map (lambda (prefix) (symbol-append prefix name)) prefixes)))) names)) (goops-error "no prefixes supplied")))) (define* (make-generic #:optional name) (make <generic> #:name name)) (define* (make-extended-generic gfs #:optional name) (let* ((gfs (if (list? gfs) gfs (list gfs))) (gws? (any (lambda (gf) (is-a? gf <generic-with-setter>)) gfs))) (let ((ans (if gws? (let* ((sname (and name (make-setter-name name))) (setters (append-map (lambda (gf) (if (is-a? gf <generic-with-setter>) (list (ensure-generic (setter gf) sname)) '())) gfs)) (es (make <extended-generic-with-setter> #:name name #:extends gfs #:setter (make <extended-generic> #:name sname #:extends setters)))) (extended-by! setters (setter es)) es) (make <extended-generic> #:name name #:extends gfs)))) (extended-by! gfs ans) ans))) (define (extended-by! gfs eg) (for-each (lambda (gf) (slot-set! gf 'extended-by (cons eg (slot-ref gf 'extended-by)))) gfs) (invalidate-method-cache! eg)) (define (not-extended-by! gfs eg) (for-each (lambda (gf) (slot-set! gf 'extended-by (delq! eg (slot-ref gf 'extended-by)))) gfs) (invalidate-method-cache! eg)) (define* (ensure-generic old-definition #:optional name) (cond ((is-a? old-definition <generic>) old-definition) ((procedure-with-setter? old-definition) (make <generic-with-setter> #:name name #:default (procedure old-definition) #:setter (setter old-definition))) ((procedure? old-definition) (if (generic-capability? old-definition) old-definition (make <generic> #:name name #:default old-definition))) (else (make <generic> #:name name)))) ;; same semantics as <generic> (define-syntax-rule (define-accessor name) (define name (cond ((not (defined? 'name)) (ensure-accessor #f 'name)) ((is-a? name <accessor>) (make <accessor> #:name 'name)) (else (ensure-accessor name 'name))))) (define (make-setter-name name) (string->symbol (string-append "setter:" (symbol->string name)))) (define* (make-accessor #:optional name) (make <accessor> #:name name #:setter (make <generic> #:name (and name (make-setter-name name))))) (define* (ensure-accessor proc #:optional name) (cond ((and (is-a? proc <accessor>) (is-a? (setter proc) <generic>)) proc) ((is-a? proc <generic-with-setter>) (upgrade-accessor proc (setter proc))) ((is-a? proc <generic>) (upgrade-accessor proc (make-generic name))) ((procedure-with-setter? proc) (make <accessor> #:name name #:default (procedure proc) #:setter (ensure-generic (setter proc) name))) ((procedure? proc) (ensure-accessor (if (generic-capability? proc) (make <generic> #:name name #:default proc) (ensure-generic proc name)) name)) (else (make-accessor name)))) (define (upgrade-accessor generic setter) (let ((methods (slot-ref generic 'methods)) (gws (make (if (is-a? generic <extended-generic>) <extended-generic-with-setter> <accessor>) #:name (generic-function-name generic) #:extended-by (slot-ref generic 'extended-by) #:setter setter))) (if (is-a? generic <extended-generic>) (let ((gfs (slot-ref generic 'extends))) (not-extended-by! gfs generic) (slot-set! gws 'extends gfs) (extended-by! gfs gws))) ;; Steal old methods (for-each (lambda (method) (slot-set! method 'generic-function gws)) methods) (slot-set! gws 'methods methods) (invalidate-method-cache! gws) gws)) ;;; ;;; {Methods} ;;; (define (toplevel-define! name val) (module-define! (current-module) name val)) (define-syntax define-method (syntax-rules (setter) ((_ ((setter name) . args) body ...) (begin (if (or (not (defined? 'name)) (not (is-a? name <accessor>))) (toplevel-define! 'name (ensure-accessor (if (defined? 'name) name #f) 'name))) (add-method! (setter name) (method args body ...)))) ((_ (name . args) body ...) (begin ;; FIXME: this code is how it always was, but it's quite cracky: ;; it will only define the generic function if it was undefined ;; before (ok), or *was defined to #f*. The latter is crack. But ;; there are bootstrap issues about fixing this -- change it to ;; (is-a? name <generic>) and see. (if (or (not (defined? 'name)) (not name)) (toplevel-define! 'name (make <generic> #:name 'name))) (add-method! name (method args body ...)))))) (define-syntax method (lambda (x) (define (parse-args args) (let lp ((ls args) (formals '()) (specializers '())) (syntax-case ls () (((f s) . rest) (and (identifier? #'f) (identifier? #'s)) (lp #'rest (cons #'f formals) (cons #'s specializers))) ((f . rest) (identifier? #'f) (lp #'rest (cons #'f formals) (cons #'<top> specializers))) (() (list (reverse formals) (reverse (cons #''() specializers)))) (tail (identifier? #'tail) (list (append (reverse formals) #'tail) (reverse (cons #'<top> specializers))))))) (define (find-free-id exp referent) (syntax-case exp () ((x . y) (or (find-free-id #'x referent) (find-free-id #'y referent))) (x (identifier? #'x) (let ((id (datum->syntax #'x referent))) (and (free-identifier=? #'x id) id))) (_ #f))) (define (compute-procedure formals body) (syntax-case body () ((body0 ...) (with-syntax ((formals formals)) #'(lambda formals body0 ...))))) (define (->proper args) (let lp ((ls args) (out '())) (syntax-case ls () ((x . xs) (lp #'xs (cons #'x out))) (() (reverse out)) (tail (reverse (cons #'tail out)))))) (define (compute-make-procedure formals body next-method) (syntax-case body () ((body ...) (with-syntax ((next-method next-method)) (syntax-case formals () ((formal ...) #'(lambda (real-next-method) (lambda (formal ...) (let ((next-method (lambda args (if (null? args) (real-next-method formal ...) (apply real-next-method args))))) body ...)))) (formals (with-syntax (((formal ...) (->proper #'formals))) #'(lambda (real-next-method) (lambda formals (let ((next-method (lambda args (if (null? args) (apply real-next-method formal ...) (apply real-next-method args))))) body ...)))))))))) (define (compute-procedures formals body) ;; So, our use of this is broken, because it operates on the ;; pre-expansion source code. It's equivalent to just searching ;; for referent in the datums. Ah well. (let ((id (find-free-id body 'next-method))) (if id ;; return a make-procedure (values #'#f (compute-make-procedure formals body id)) (values (compute-procedure formals body) #'#f)))) (syntax-case x () ((_ args) #'(method args (if #f #f))) ((_ args body0 body1 ...) (with-syntax (((formals (specializer ...)) (parse-args #'args))) (call-with-values (lambda () (compute-procedures #'formals #'(body0 body1 ...))) (lambda (procedure make-procedure) (with-syntax ((procedure procedure) (make-procedure make-procedure)) #'(make <method> #:specializers (cons* specializer ...) #:formals 'formals #:body '(body0 body1 ...) #:make-procedure make-procedure #:procedure procedure))))))))) ;;; ;;; {add-method!} ;;; (define (add-method-in-classes! m) ;; Add method in all the classes which appears in its specializers list (for-each* (lambda (x) (let ((dm (class-direct-methods x))) (if (not (memq m dm)) (slot-set! x 'direct-methods (cons m dm))))) (method-specializers m))) (define (remove-method-in-classes! m) ;; Remove method in all the classes which appears in its specializers list (for-each* (lambda (x) (slot-set! x 'direct-methods (delv! m (class-direct-methods x)))) (method-specializers m))) (define (compute-new-list-of-methods gf new) (let ((new-spec (method-specializers new)) (methods (slot-ref gf 'methods))) (let loop ((l methods)) (if (null? l) (cons new methods) (if (equal? (method-specializers (car l)) new-spec) (begin ;; This spec. list already exists. Remove old method from dependents (remove-method-in-classes! (car l)) (set-car! l new) methods) (loop (cdr l))))))) (define (method-n-specializers m) (length* (slot-ref m 'specializers))) (define (calculate-n-specialized gf) (fold (lambda (m n) (max n (method-n-specializers m))) 0 (generic-function-methods gf))) (define (invalidate-method-cache! gf) (%invalidate-method-cache! gf) (slot-set! gf 'n-specialized (calculate-n-specialized gf)) (for-each (lambda (gf) (invalidate-method-cache! gf)) (slot-ref gf 'extended-by))) (define internal-add-method! (method ((gf <generic>) (m <method>)) (slot-set! m 'generic-function gf) (slot-set! gf 'methods (compute-new-list-of-methods gf m)) (invalidate-method-cache! gf) (add-method-in-classes! m) *unspecified*)) (define-generic add-method!) ((method-procedure internal-add-method!) add-method! internal-add-method!) (define-method (add-method! (proc <procedure>) (m <method>)) (if (generic-capability? proc) (begin (enable-primitive-generic! proc) (add-method! proc m)) (next-method))) (define-method (add-method! (pg <primitive-generic>) (m <method>)) (add-method! (primitive-generic-generic pg) m)) (define-method (add-method! obj (m <method>)) (goops-error "~S is not a valid generic function" obj)) ;;; ;;; {Access to meta objects} ;;; ;;; ;;; Methods ;;; (define-method (method-source (m <method>)) (let* ((spec (map* class-name (slot-ref m 'specializers))) (src (procedure-source (slot-ref m 'procedure)))) (and src (let ((args (cadr src)) (body (cddr src))) (cons 'method (cons (map* list args spec) body)))))) (define-method (method-formals (m <method>)) (slot-ref m 'formals)) ;;; ;;; Slots ;;; (define slot-definition-name car) (define slot-definition-options cdr) (define (slot-definition-allocation s) (get-keyword #:allocation (cdr s) #:instance)) (define (slot-definition-getter s) (get-keyword #:getter (cdr s) #f)) (define (slot-definition-setter s) (get-keyword #:setter (cdr s) #f)) (define (slot-definition-accessor s) (get-keyword #:accessor (cdr s) #f)) (define (slot-definition-init-value s) ;; can be #f, so we can't use #f as non-value (get-keyword #:init-value (cdr s) (make-unbound))) (define (slot-definition-init-form s) (get-keyword #:init-form (cdr s) (make-unbound))) (define (slot-definition-init-thunk s) (get-keyword #:init-thunk (cdr s) #f)) (define (slot-definition-init-keyword s) (get-keyword #:init-keyword (cdr s) #f)) (define (class-slot-definition class slot-name) (assq slot-name (class-slots class))) (define (slot-init-function class slot-name) (cadr (assq slot-name (slot-ref class 'getters-n-setters)))) (define (accessor-method-slot-definition obj) "Return the slot definition of the accessor @var{obj}." (slot-ref obj 'slot-definition)) ;;; ;;; {Standard methods used by the C runtime} ;;; ;;; Methods to compare objects ;;; ;; Have to do this in a strange order because equal? is used in the ;; add-method! implementation; we need to make sure that when the ;; primitive is extended, that the generic has a method. = (define g-equal? (make-generic 'equal?)) ;; When this generic gets called, we will have already checked eq? and ;; eqv? -- the purpose of this generic is to extend equality. So by ;; default, there is no extension, thus the #f return. (add-method! g-equal? (method (x y) #f)) (set-primitive-generic! equal? g-equal?) ;;; ;;; methods to display/write an object ;;; ; Code for writing objects must test that the slots they use are ; bound. Otherwise a slot-unbound method will be called and will ; conduct to an infinite loop. ;; Write (define (display-address o file) (display (number->string (object-address o) 16) file)) (define-method (write o file) (display "#<instance " file) (display-address o file) (display #\> file)) (define write-object (primitive-generic-generic write)) (define-method (write (o <object>) file) (let ((class (class-of o))) (if (slot-bound? class 'name) (begin (display "#<" file) (display (class-name class) file) (display #\space file) (display-address o file) (display #\> file)) (next-method)))) (define-method (write (class <class>) file) (let ((meta (class-of class))) (if (and (slot-bound? class 'name) (slot-bound? meta 'name)) (begin (display "#<" file) (display (class-name meta) file) (display #\space file) (display (class-name class) file) (display #\space file) (display-address class file) (display #\> file)) (next-method)))) (define-method (write (gf <generic>) file) (let ((meta (class-of gf))) (if (and (slot-bound? meta 'name) (slot-bound? gf 'methods)) (begin (display "#<" file) (display (class-name meta) file) (let ((name (generic-function-name gf))) (if name (begin (display #\space file) (display name file)))) (display " (" file) (display (length (generic-function-methods gf)) file) (display ")>" file)) (next-method)))) (define-method (write (o <method>) file) (let ((meta (class-of o))) (if (and (slot-bound? meta 'name) (slot-bound? o 'specializers)) (begin (display "#<" file) (display (class-name meta) file) (display #\space file) (display (map* (lambda (spec) (if (slot-bound? spec 'name) (slot-ref spec 'name) spec)) (method-specializers o)) file) (display #\space file) (display-address o file) (display #\> file)) (next-method)))) ;; Display (do the same thing as write by default) (define-method (display o file) (write-object o file)) ;;; ;;; Handling of duplicate bindings in the module system ;;; (define-method (merge-generics (module <module>) (name <symbol>) (int1 <module>) (val1 <top>) (int2 <module>) (val2 <top>) (var <top>) (val <top>)) #f) (define-method (merge-generics (module <module>) (name <symbol>) (int1 <module>) (val1 <generic>) (int2 <module>) (val2 <generic>) (var <top>) (val <boolean>)) (and (not (eq? val1 val2)) (make-variable (make-extended-generic (list val2 val1) name)))) (define-method (merge-generics (module <module>) (name <symbol>) (int1 <module>) (val1 <generic>) (int2 <module>) (val2 <generic>) (var <top>) (gf <extended-generic>)) (and (not (memq val2 (slot-ref gf 'extends))) (begin (slot-set! gf 'extends (cons val2 (delq! val2 (slot-ref gf 'extends)))) (slot-set! val2 'extended-by (cons gf (delq! gf (slot-ref val2 'extended-by)))) (invalidate-method-cache! gf) var))) (module-define! duplicate-handlers 'merge-generics merge-generics) (define-method (merge-accessors (module <module>) (name <symbol>) (int1 <module>) (val1 <top>) (int2 <module>) (val2 <top>) (var <top>) (val <top>)) #f) (define-method (merge-accessors (module <module>) (name <symbol>) (int1 <module>) (val1 <accessor>) (int2 <module>) (val2 <accessor>) (var <top>) (val <top>)) (merge-generics module name int1 val1 int2 val2 var val)) (module-define! duplicate-handlers 'merge-accessors merge-accessors) ;;; ;;; slot access ;;; (define (class-slot-g-n-s class slot-name) (let* ((this-slot (assq slot-name (slot-ref class 'slots))) (g-n-s (cddr (or (assq slot-name (slot-ref class 'getters-n-setters)) (slot-missing class slot-name))))) (if (not (memq (slot-definition-allocation this-slot) '(#:class #:each-subclass))) (slot-missing class slot-name)) g-n-s)) (define (class-slot-ref class slot) (let ((x ((car (class-slot-g-n-s class slot)) #f))) (if (unbound? x) (slot-unbound class slot) x))) (define (class-slot-set! class slot value) ((cadr (class-slot-g-n-s class slot)) #f value)) (define-method (slot-unbound (c <class>) (o <object>) s) (goops-error "Slot `~S' is unbound in object ~S" s o)) (define-method (slot-unbound (c <class>) s) (goops-error "Slot `~S' is unbound in class ~S" s c)) (define-method (slot-unbound (o <object>)) (goops-error "Unbound slot in object ~S" o)) (define-method (slot-missing (c <class>) (o <object>) s) (goops-error "No slot with name `~S' in object ~S" s o)) (define-method (slot-missing (c <class>) s) (goops-error "No class slot with name `~S' in class ~S" s c)) (define-method (slot-missing (c <class>) (o <object>) s value) (slot-missing c o s)) ;;; Methods for the possible error we can encounter when calling a gf (define-method (no-next-method (gf <generic>) args) (goops-error "No next method when calling ~S\nwith arguments ~S" gf args)) (define-method (no-applicable-method (gf <generic>) args) (goops-error "No applicable method for ~S in call ~S" gf (cons (generic-function-name gf) args))) (define-method (no-method (gf <generic>) args) (goops-error "No method defined for ~S" gf)) ;;; ;;; {Cloning functions (from rdeline@CS.CMU.EDU)} ;;; (define-method (shallow-clone (self <object>)) (let ((clone (%allocate-instance (class-of self) '())) (slots (map slot-definition-name (class-slots (class-of self))))) (for-each (lambda (slot) (if (slot-bound? self slot) (slot-set! clone slot (slot-ref self slot)))) slots) clone)) (define-method (deep-clone (self <object>)) (let ((clone (%allocate-instance (class-of self) '())) (slots (map slot-definition-name (class-slots (class-of self))))) (for-each (lambda (slot) (if (slot-bound? self slot) (slot-set! clone slot (let ((value (slot-ref self slot))) (if (instance? value) (deep-clone value) value))))) slots) clone)) ;;; ;;; {Class redefinition utilities} ;;; ;;; (class-redefinition OLD NEW) ;;; ;;; Has correct the following conditions: ;;; Methods ;;; ;;; 1. New accessor specializers refer to new header ;;; ;;; Classes ;;; ;;; 1. New class cpl refers to the new class header ;;; 2. Old class header exists on old super classes direct-subclass lists ;;; 3. New class header exists on new super classes direct-subclass lists (define-method (class-redefinition (old <class>) (new <class>)) ;; Work on direct methods: ;; 1. Remove accessor methods from the old class ;; 2. Patch the occurences of new in the specializers by old ;; 3. Displace the methods from old to new (remove-class-accessors! old) ;; -1- (let ((methods (class-direct-methods new))) (for-each (lambda (m) (update-direct-method! m new old)) ;; -2- methods) (slot-set! new 'direct-methods (append methods (class-direct-methods old)))) ;; Substitute old for new in new cpl (set-car! (slot-ref new 'cpl) old) ;; Remove the old class from the direct-subclasses list of its super classes (for-each (lambda (c) (slot-set! c 'direct-subclasses (delv! old (class-direct-subclasses c)))) (class-direct-supers old)) ;; Replace the new class with the old in the direct-subclasses of the supers (for-each (lambda (c) (slot-set! c 'direct-subclasses (cons old (delv! new (class-direct-subclasses c))))) (class-direct-supers new)) ;; Swap object headers (%modify-class old new) ;; Now old is NEW! ;; Redefine all the subclasses of old to take into account modification (for-each (lambda (c) (update-direct-subclass! c new old)) (class-direct-subclasses new)) ;; Invalidate class so that subsequent instances slot accesses invoke ;; change-object-class (slot-set! new 'redefined old) (%invalidate-class new) ;must come after slot-set! old) ;;; ;;; remove-class-accessors! ;;; (define-method (remove-class-accessors! (c <class>)) (for-each (lambda (m) (if (is-a? m <accessor-method>) (let ((gf (slot-ref m 'generic-function))) ;; remove the method from its GF (slot-set! gf 'methods (delq1! m (slot-ref gf 'methods))) (invalidate-method-cache! gf) ;; remove the method from its specializers (remove-method-in-classes! m)))) (class-direct-methods c))) ;;; ;;; update-direct-method! ;;; (define-method (update-direct-method! (m <method>) (old <class>) (new <class>)) (let loop ((l (method-specializers m))) ;; Note: the <top> in dotted list is never used. ;; So we can work as if we had only proper lists. (if (pair? l) (begin (if (eqv? (car l) old) (set-car! l new)) (loop (cdr l)))))) ;;; ;;; update-direct-subclass! ;;; (define-method (update-direct-subclass! (c <class>) (old <class>) (new <class>)) (class-redefinition c (make-class (class-direct-supers c) (class-direct-slots c) #:name (class-name c) #:metaclass (class-of c)))) ;;; ;;; {Utilities for INITIALIZE methods} ;;; ;;; compute-slot-accessors ;;; (define (compute-slot-accessors class slots) (for-each (lambda (s g-n-s) (let ((getter-function (slot-definition-getter s)) (setter-function (slot-definition-setter s)) (accessor (slot-definition-accessor s))) (if getter-function (add-method! getter-function (compute-getter-method class g-n-s))) (if setter-function (add-method! setter-function (compute-setter-method class g-n-s))) (if accessor (begin (add-method! accessor (compute-getter-method class g-n-s)) (add-method! (setter accessor) (compute-setter-method class g-n-s)))))) slots (slot-ref class 'getters-n-setters))) (define-method (compute-getter-method (class <class>) g-n-s) (let ((init-thunk (cadr g-n-s)) (g-n-s (cddr g-n-s))) (make <accessor-method> #:specializers (list class) #:procedure (cond ((pair? g-n-s) (make-generic-bound-check-getter (car g-n-s))) (init-thunk (standard-get g-n-s)) (else (bound-check-get g-n-s))) #:slot-definition g-n-s))) (define-method (compute-setter-method (class <class>) g-n-s) (let ((init-thunk (cadr g-n-s)) (g-n-s (cddr g-n-s))) (make <accessor-method> #:specializers (list class <top>) #:procedure (if (pair? g-n-s) (cadr g-n-s) (standard-set g-n-s)) #:slot-definition g-n-s))) (define (make-generic-bound-check-getter proc) (lambda (o) (assert-bound (proc o) o))) ;; the idea is to compile the index into the procedure, for fastest ;; lookup. (eval-when (expand load eval) (define num-standard-pre-cache 20)) (define-macro (define-standard-accessor-method form . body) (let ((name (caar form)) (n-var (cadar form)) (args (cdr form))) (define (make-one x) (define (body-trans form) (cond ((not (pair? form)) form) ((eq? (car form) 'struct-ref) `(,(car form) ,(cadr form) ,x)) ((eq? (car form) 'struct-set!) `(,(car form) ,(cadr form) ,x ,(cadddr form))) (else (map body-trans form)))) `(lambda ,args ,@(map body-trans body))) `(define ,name (let ((cache (vector ,@(map make-one (iota num-standard-pre-cache))))) (lambda (n) (if (< n ,num-standard-pre-cache) (vector-ref cache n) ((lambda (,n-var) (lambda ,args ,@body)) n))))))) (define-standard-accessor-method ((bound-check-get n) o) (let ((x (struct-ref o n))) (if (unbound? x) (slot-unbound o) x))) (define-standard-accessor-method ((standard-get n) o) (struct-ref o n)) (define-standard-accessor-method ((standard-set n) o v) (struct-set! o n v)) ;;; compute-getters-n-setters ;;; (define (compute-getters-n-setters class slots) (define (compute-slot-init-function name s) (or (let ((thunk (slot-definition-init-thunk s))) (and thunk (if (thunk? thunk) thunk (goops-error "Bad init-thunk for slot `~S' in ~S: ~S" name class thunk)))) (let ((init (slot-definition-init-value s))) (and (not (unbound? init)) (lambda () init))))) (define (verify-accessors slot l) (cond ((integer? l)) ((not (and (list? l) (= (length l) 2))) (goops-error "Bad getter and setter for slot `~S' in ~S: ~S" slot class l)) (else (let ((get (car l)) (set (cadr l))) (if (not (procedure? get)) (goops-error "Bad getter closure for slot `~S' in ~S: ~S" slot class get)) (if (not (procedure? set)) (goops-error "Bad setter closure for slot `~S' in ~S: ~S" slot class set)))))) (map (lambda (s) ;; The strange treatment of nfields is due to backward compatibility. (let* ((index (slot-ref class 'nfields)) (g-n-s (compute-get-n-set class s)) (size (- (slot-ref class 'nfields) index)) (name (slot-definition-name s))) ;; NOTE: The following is interdependent with C macros ;; defined above goops.c:scm_sys_prep_layout_x. ;; ;; For simple instance slots, we have the simplest form ;; '(name init-function . index) ;; For other slots we have ;; '(name init-function getter setter . alloc) ;; where alloc is: ;; '(index size) for instance allocated slots ;; '() for other slots (verify-accessors name g-n-s) (case (slot-definition-allocation s) ((#:each-subclass #:class) (unless (and (zero? size) (pair? g-n-s)) (error "Class-allocated slots should not reserve fields")) ;; Don't initialize the slot; that's handled when the slot ;; is allocated, in compute-get-n-set. (cons name (cons #f g-n-s))) (else (cons name (cons (compute-slot-init-function name s) (if (or (integer? g-n-s) (zero? size)) g-n-s (append g-n-s (list index size))))))))) slots)) ;;; compute-cpl ;;; ;;; Correct behaviour: ;;; ;;; (define-class food ()) ;;; (define-class fruit (food)) ;;; (define-class spice (food)) ;;; (define-class apple (fruit)) ;;; (define-class cinnamon (spice)) ;;; (define-class pie (apple cinnamon)) ;;; => cpl (pie) = pie apple fruit cinnamon spice food object top ;;; ;;; (define-class d ()) ;;; (define-class e ()) ;;; (define-class f ()) ;;; (define-class b (d e)) ;;; (define-class c (e f)) ;;; (define-class a (b c)) ;;; => cpl (a) = a b d c e f object top ;;; (define-method (compute-cpl (class <class>)) (compute-std-cpl class class-direct-supers)) ;; Support (define (only-non-null lst) (filter (lambda (l) (not (null? l))) lst)) (define (compute-std-cpl c get-direct-supers) (let ((c-direct-supers (get-direct-supers c))) (merge-lists (list c) (only-non-null (append (map class-precedence-list c-direct-supers) (list c-direct-supers)))))) (define (merge-lists reversed-partial-result inputs) (cond ((every null? inputs) (reverse! reversed-partial-result)) (else (let* ((candidate (lambda (c) (and (not (any (lambda (l) (memq c (cdr l))) inputs)) c))) (candidate-car (lambda (l) (and (not (null? l)) (candidate (car l))))) (next (any candidate-car inputs))) (if (not next) (goops-error "merge-lists: Inconsistent precedence graph")) (let ((remove-next (lambda (l) (if (eq? (car l) next) (cdr l) l)))) (merge-lists (cons next reversed-partial-result) (only-non-null (map remove-next inputs)))))))) ;; Modified from TinyClos: ;; ;; A simple topological sort. ;; ;; It's in this file so that both TinyClos and Objects can use it. ;; ;; This is a fairly modified version of code I originally got from Anurag ;; Mendhekar <anurag@moose.cs.indiana.edu>. ;; (define (compute-clos-cpl c get-direct-supers) (top-sort ((build-transitive-closure get-direct-supers) c) ((build-constraints get-direct-supers) c) (std-tie-breaker get-direct-supers))) (define (top-sort elements constraints tie-breaker) (let loop ((elements elements) (constraints constraints) (result '())) (if (null? elements) result (let ((can-go-in-now (filter (lambda (x) (every (lambda (constraint) (or (not (eq? (cadr constraint) x)) (memq (car constraint) result))) constraints)) elements))) (if (null? can-go-in-now) (goops-error "top-sort: Invalid constraints") (let ((choice (if (null? (cdr can-go-in-now)) (car can-go-in-now) (tie-breaker result can-go-in-now)))) (loop (filter (lambda (x) (not (eq? x choice))) elements) constraints (append result (list choice))))))))) (define (std-tie-breaker get-supers) (lambda (partial-cpl min-elts) (let loop ((pcpl (reverse partial-cpl))) (let ((current-elt (car pcpl))) (let ((ds-of-ce (get-supers current-elt))) (let ((common (filter (lambda (x) (memq x ds-of-ce)) min-elts))) (if (null? common) (if (null? (cdr pcpl)) (goops-error "std-tie-breaker: Nothing valid") (loop (cdr pcpl))) (car common)))))))) (define (build-transitive-closure get-follow-ons) (lambda (x) (let track ((result '()) (pending (list x))) (if (null? pending) result (let ((next (car pending))) (if (memq next result) (track result (cdr pending)) (track (cons next result) (append (get-follow-ons next) (cdr pending))))))))) (define (build-constraints get-follow-ons) (lambda (x) (let loop ((elements ((build-transitive-closure get-follow-ons) x)) (this-one '()) (result '())) (if (or (null? this-one) (null? (cdr this-one))) (if (null? elements) result (loop (cdr elements) (cons (car elements) (get-follow-ons (car elements))) result)) (loop elements (cdr this-one) (cons (list (car this-one) (cadr this-one)) result)))))) ;;; compute-get-n-set ;;; (define-method (compute-get-n-set (class <class>) s) (define (class-slot-init-value) (let ((thunk (slot-definition-init-thunk s))) (if thunk (thunk) (slot-definition-init-value s)))) (case (slot-definition-allocation s) ((#:instance) ;; Instance slot ;; get-n-set is just its offset (let ((already-allocated (slot-ref class 'nfields))) (slot-set! class 'nfields (+ already-allocated 1)) already-allocated)) ((#:class) ;; Class slot ;; Class-slots accessors are implemented as 2 closures around ;; a Scheme variable. As instance slots, class slots must be ;; unbound at init time. (let ((name (slot-definition-name s))) (if (memq name (map slot-definition-name (class-direct-slots class))) ;; This slot is direct; create a new shared variable (make-closure-variable class (class-slot-init-value)) ;; Slot is inherited. Find its definition in superclass (let loop ((l (cdr (class-precedence-list class)))) (let ((r (assoc name (slot-ref (car l) 'getters-n-setters)))) (if r (cddr r) (loop (cdr l)))))))) ((#:each-subclass) ;; slot shared by instances of direct subclass. ;; (Thomas Buerger, April 1998) (make-closure-variable class (class-slot-init-value))) ((#:virtual) ;; No allocation ;; slot-ref and slot-set! function must be given by the user (let ((get (get-keyword #:slot-ref (slot-definition-options s) #f)) (set (get-keyword #:slot-set! (slot-definition-options s) #f))) (if (not (and get set)) (goops-error "You must supply a #:slot-ref and a #:slot-set! in ~S" s)) (list get set))) (else (next-method)))) (define (make-closure-variable class value) (list (lambda (o) value) (lambda (o v) (set! value v)))) (define-method (compute-get-n-set (o <object>) s) (goops-error "Allocation \"~S\" is unknown" (slot-definition-allocation s))) (define-method (compute-slots (class <class>)) (%compute-slots class)) ;;; ;;; {Initialize} ;;; (define-method (initialize (object <object>) initargs) (%initialize-object object initargs)) (define-method (initialize (class <class>) initargs) (next-method) (let ((dslots (get-keyword #:slots initargs '())) (supers (get-keyword #:dsupers initargs '()))) (slot-set! class 'name (get-keyword #:name initargs '???)) (slot-set! class 'direct-supers supers) (slot-set! class 'direct-slots dslots) (slot-set! class 'direct-subclasses '()) (slot-set! class 'direct-methods '()) (slot-set! class 'cpl (compute-cpl class)) (slot-set! class 'redefined #f) (let ((slots (compute-slots class))) (slot-set! class 'slots slots) (slot-set! class 'nfields 0) (slot-set! class 'getters-n-setters (compute-getters-n-setters class slots)) ;; Build getters - setters - accessors (compute-slot-accessors class slots)) ;; Update the "direct-subclasses" of each inherited classes (for-each (lambda (x) (slot-set! x 'direct-subclasses (cons class (slot-ref x 'direct-subclasses)))) supers) ;; Support for the underlying structs: ;; Set the layout slot (%prep-layout! class) ;; Inherit class flags (invisible on scheme level) from supers (%inherit-magic! class supers))) (define (initialize-object-procedure object initargs) (let ((proc (get-keyword #:procedure initargs #f))) (cond ((not proc)) ((pair? proc) (apply slot-set! object 'procedure proc)) (else (slot-set! object 'procedure proc))))) (define-method (initialize (applicable-struct <applicable-struct>) initargs) (next-method) (initialize-object-procedure applicable-struct initargs)) (define-method (initialize (generic <generic>) initargs) (let ((previous-definition (get-keyword #:default initargs #f)) (name (get-keyword #:name initargs #f))) (next-method) (slot-set! generic 'methods (if (is-a? previous-definition <procedure>) (list (method args (apply previous-definition args))) '())) (if name (set-procedure-property! generic 'name name)) )) (define-method (initialize (gws <generic-with-setter>) initargs) (next-method) (%set-object-setter! gws (get-keyword #:setter initargs #f))) (define-method (initialize (eg <extended-generic>) initargs) (next-method) (slot-set! eg 'extends (get-keyword #:extends initargs '()))) (define dummy-procedure (lambda args *unspecified*)) (define-method (initialize (method <method>) initargs) (next-method) (slot-set! method 'generic-function (get-keyword #:generic-function initargs #f)) (slot-set! method 'specializers (get-keyword #:specializers initargs '())) (slot-set! method 'procedure (get-keyword #:procedure initargs #f)) (slot-set! method 'formals (get-keyword #:formals initargs '())) (slot-set! method 'body (get-keyword #:body initargs '())) (slot-set! method 'make-procedure (get-keyword #:make-procedure initargs #f))) ;;; ;;; {Change-class} ;;; (define (change-object-class old-instance old-class new-class) (let ((new-instance (allocate-instance new-class '()))) ;; Initialize the slots of the new instance (for-each (lambda (slot) (if (and (slot-exists-using-class? old-class old-instance slot) (eq? (slot-definition-allocation (class-slot-definition old-class slot)) #:instance) (slot-bound-using-class? old-class old-instance slot)) ;; Slot was present and allocated in old instance; copy it (slot-set-using-class! new-class new-instance slot (slot-ref-using-class old-class old-instance slot)) ;; slot was absent; initialize it with its default value (let ((init (slot-init-function new-class slot))) (if init (slot-set-using-class! new-class new-instance slot (apply init '())))))) (map slot-definition-name (class-slots new-class))) ;; Exchange old and new instance in place to keep pointers valid (%modify-instance old-instance new-instance) ;; Allow class specific updates of instances (which now are swapped) (update-instance-for-different-class new-instance old-instance) old-instance)) (define-method (update-instance-for-different-class (old-instance <object>) (new-instance <object>)) ;;not really important what we do, we just need a default method new-instance) (define-method (change-class (old-instance <object>) (new-class <class>)) (change-object-class old-instance (class-of old-instance) new-class)) ;;; ;;; {make} ;;; ;;; A new definition which overwrites the previous one which was built-in ;;; (define-method (allocate-instance (class <class>) initargs) (%allocate-instance class initargs)) (define-method (make-instance (class <class>) . initargs) (let ((instance (allocate-instance class initargs))) (initialize instance initargs) instance)) (define make make-instance) ;;; ;;; {apply-generic} ;;; ;;; Protocol for calling standard generic functions. This protocol is ;;; not used for real <generic> functions (in this case we use a ;;; completely C hard-coded protocol). Apply-generic is used by ;;; goops for calls to subclasses of <generic> and <generic-with-setter>. ;;; The code below is similar to the first MOP described in AMOP. In ;;; particular, it doesn't used the currified approach to gf ;;; call. There are 2 reasons for that: ;;; - the protocol below is exposed to mimic completely the one written in C ;;; - the currified protocol would be imho inefficient in C. ;;; (define-method (apply-generic (gf <generic>) args) (if (null? (slot-ref gf 'methods)) (no-method gf args)) (let ((methods (compute-applicable-methods gf args))) (if methods (apply-methods gf (sort-applicable-methods gf methods args) args) (no-applicable-method gf args)))) ;; compute-applicable-methods is bound to %compute-applicable-methods. ;; *fixme* use let (define %%compute-applicable-methods (make <generic> #:name 'compute-applicable-methods)) (define-method (%%compute-applicable-methods (gf <generic>) args) (%compute-applicable-methods gf args)) (set! compute-applicable-methods %%compute-applicable-methods) (define-method (sort-applicable-methods (gf <generic>) methods args) (let ((targs (map class-of args))) (sort methods (lambda (m1 m2) (method-more-specific? m1 m2 targs))))) (define-method (method-more-specific? (m1 <method>) (m2 <method>) targs) (%method-more-specific? m1 m2 targs)) (define-method (apply-method (gf <generic>) methods build-next args) (apply (method-procedure (car methods)) (build-next (cdr methods) args) args)) (define-method (apply-methods (gf <generic>) (l <list>) args) (letrec ((next (lambda (procs args) (lambda new-args (let ((a (if (null? new-args) args new-args))) (if (null? procs) (no-next-method gf a) (apply-method gf procs next a))))))) (apply-method gf l next args))) ;; We don't want the following procedure to turn up in backtraces: (for-each (lambda (proc) (set-procedure-property! proc 'system-procedure #t)) (list slot-unbound slot-missing no-next-method no-applicable-method no-method )) ;;; ;;; {<composite-metaclass> and <active-metaclass>} ;;; ;(autoload "active-slot" <active-metaclass>) ;(autoload "composite-slot" <composite-metaclass>) ;(export <composite-metaclass> <active-metaclass>) ;;; ;;; {Tools} ;;; ;; list2set ;; ;; duplicate the standard list->set function but using eq instead of ;; eqv which really sucks a lot, uselessly here ;; (define (list2set l) (let loop ((l l) (res '())) (cond ((null? l) res) ((memq (car l) res) (loop (cdr l) res)) (else (loop (cdr l) (cons (car l) res)))))) (define (class-subclasses c) (letrec ((allsubs (lambda (c) (cons c (mapappend allsubs (class-direct-subclasses c)))))) (list2set (cdr (allsubs c))))) (define (class-methods c) (list2set (mapappend class-direct-methods (cons c (class-subclasses c))))) ;;; ;;; {Final initialization} ;;; ;; Tell C code that the main bulk of Goops has been loaded (%goops-loaded) PK�����/F2]mS������sxml/ssax/input-parse.scmnu�[��������;;;; (sxml ssax input-parse) -- a simple lexer ;;;; ;;;; Copyright (C) 2009 Free Software Foundation, Inc. ;;;; Modified 2004 by Andy Wingo <wingo at pobox dot com>. ;;;; Written 2003 by Oleg Kiselyov <oleg at pobox dot com> as input-parse.scm. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;; A simple lexer. ;; ;; The procedures in this module surprisingly often suffice to parse an ;; input stream. They either skip, or build and return tokens, according ;; to inclusion or delimiting semantics. The list of characters to ;; expect, include, or to break at may vary from one invocation of a ;; function to another. This allows the functions to easily parse even ;; context-sensitive languages. ;; ;; EOF is generally frowned on, and thrown up upon if encountered. ;; Exceptions are mentioned specifically. The list of expected ;; characters (characters to skip until, or break-characters) may ;; include an EOF "character", which is to be coded as the symbol, ;; @code{*eof*}. ;; ;; The input stream to parse is specified as a @dfn{port}, which is ;; usually the last (and optional) argument. It defaults to the current ;; input port if omitted. ;; ;; If the parser encounters an error, it will throw an exception to the ;; key @code{parser-error}. The arguments will be of the form ;; @code{(@var{port} @var{message} @var{specialising-msg}*)}. ;; ;; The first argument is a port, which typically points to the offending ;; character or its neighborhood. You can then use @code{port-column} ;; and @code{port-line} to query the current position. @var{message} is ;; the description of the error. Other arguments supply more details ;; about the problem. ;; ;;; Code: (define-module (sxml ssax input-parse) #:use-module (ice-9 rdelim) #:export (peek-next-char assert-curr-char skip-until skip-while next-token next-token-of read-text-line read-string find-string-from-port?)) (define ascii->char integer->char) (define char->ascii char->integer) (define char-newline #\newline) (define char-return #\return) (define inc 1+) (define dec 1-) ;; rewrite oleg's define-opt into define* style (define-macro (define-opt bindings body . body-rest) (let* ((rev-bindings (reverse bindings)) (opt-bindings (and (pair? rev-bindings) (pair? (car rev-bindings)) (eq? 'optional (caar rev-bindings)) (cdar rev-bindings)))) (if opt-bindings `(define* ,(append (reverse (cons #:optional (cdr rev-bindings))) opt-bindings) ,body ,@body-rest) `(define* ,bindings ,body ,@body-rest)))) (define (parser-error port message . rest) (apply throw 'parser-error port message rest)) (include-from-path "sxml/upstream/input-parse.scm") ;; This version for guile is quite speedy, due to read-delimited (which ;; is implemented in C). (define-opt (next-token prefix-skipped-chars break-chars (optional (comment "") (port (current-input-port))) ) (let ((delims (list->string (delete '*eof* break-chars)))) (if (eof-object? (if (null? prefix-skipped-chars) (peek-char port) (skip-while prefix-skipped-chars port))) (if (memq '*eof* break-chars) "" (parser-error port "EOF while reading a token " comment)) (let ((token (read-delimited delims port 'peek))) (if (and (eof-object? (peek-char port)) (not (memq '*eof* break-chars))) (parser-error port "EOF while reading a token " comment) token))))) (define-opt (read-text-line (optional (port (current-input-port))) ) (read-line port)) ;; Written 1995, 1996 by Oleg Kiselyov (oleg@acm.org) ;; Modified 1996, 1997, 1998, 2001 by A. Jaffer (agj@alum.mit.edu) ;; Modified 2003 by Steve VanDevender (stevev@hexadecimal.uoregon.edu) ;; Modified 2004 Andy Wingo <wingo at pobox dot com> ;; This function is from SLIB's strsrch.scm, and is in the public domain. (define (find-string-from-port? str <input-port> . max-no-char) "Looks for @var{str} in @var{<input-port>}, optionally within the first @var{max-no-char} characters." (set! max-no-char (if (null? max-no-char) #f (car max-no-char))) (letrec ((no-chars-read 0) (peeked? #f) (my-peek-char ; Return a peeked char or #f (lambda () (and (or (not (number? max-no-char)) (< no-chars-read max-no-char)) (let ((c (peek-char <input-port>))) (cond (peeked? c) ((eof-object? c) #f) ((procedure? max-no-char) (set! peeked? #t) (if (max-no-char c) #f c)) ((eqv? max-no-char c) #f) (else c)))))) (next-char (lambda () (set! peeked? #f) (read-char <input-port>) (set! no-chars-read (+ 1 no-chars-read)))) (match-1st-char ; of the string str (lambda () (let ((c (my-peek-char))) (and c (begin (next-char) (if (char=? c (string-ref str 0)) (match-other-chars 1) (match-1st-char))))))) ;; There has been a partial match, up to the point pos-to-match ;; (for example, str[0] has been found in the stream) ;; Now look to see if str[pos-to-match] for would be found, too (match-other-chars (lambda (pos-to-match) (if (>= pos-to-match (string-length str)) no-chars-read ; the entire string has matched (let ((c (my-peek-char))) (and c (if (not (char=? c (string-ref str pos-to-match))) (backtrack 1 pos-to-match) (begin (next-char) (match-other-chars (+ 1 pos-to-match))))))))) ;; There had been a partial match, but then a wrong char showed up. ;; Before discarding previously read (and matched) characters, we check ;; to see if there was some smaller partial match. Note, characters read ;; so far (which matter) are those of str[0..matched-substr-len - 1] ;; In other words, we will check to see if there is such i>0 that ;; substr(str,0,j) = substr(str,i,matched-substr-len) ;; where j=matched-substr-len - i (backtrack (lambda (i matched-substr-len) (let ((j (- matched-substr-len i))) (if (<= j 0) ;; backed off completely to the begining of str (match-1st-char) (let loop ((k 0)) (if (>= k j) (match-other-chars j) ; there was indeed a shorter match (if (char=? (string-ref str k) (string-ref str (+ i k))) (loop (+ 1 k)) (backtrack (+ 1 i) matched-substr-len)))))))) ) (match-1st-char))) PK�����/F2]BU/��U/����sxml/transform.scmnu�[��������;;;; (sxml transform) -- pre- and post-order sxml transformation ;;;; ;;;; Copyright (C) 2009 Free Software Foundation, Inc. ;;;; Modified 2004 by Andy Wingo <wingo at pobox dot com>. ;;;; Written 2003 by Oleg Kiselyov <oleg at pobox dot com> as SXML-tree-trans.scm. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;@heading SXML expression tree transformers ; ;@subheading Pre-Post-order traversal of a tree and creation of a new tree ;@smallexample ;pre-post-order:: <tree> x <bindings> -> <new-tree> ;@end smallexample ; where ;@smallexample ; <bindings> ::= (<binding> ...) ; <binding> ::= (<trigger-symbol> *preorder* . <handler>) | ; (<trigger-symbol> *macro* . <handler>) | ; (<trigger-symbol> <new-bindings> . <handler>) | ; (<trigger-symbol> . <handler>) ; <trigger-symbol> ::= XMLname | *text* | *default* ; <handler> :: <trigger-symbol> x [<tree>] -> <new-tree> ;@end smallexample ; ; The pre-post-order function visits the nodes and nodelists ; pre-post-order (depth-first). For each @code{<Node>} of the form ; @code{(@var{name} <Node> ...)}, it looks up an association with the ; given @var{name} among its @var{<bindings>}. If failed, ; @code{pre-post-order} tries to locate a @code{*default*} binding. It's ; an error if the latter attempt fails as well. Having found a binding, ; the @code{pre-post-order} function first checks to see if the binding ; is of the form ;@smallexample ; (<trigger-symbol> *preorder* . <handler>) ;@end smallexample ; ; If it is, the handler is 'applied' to the current node. Otherwise, the ; pre-post-order function first calls itself recursively for each child ; of the current node, with @var{<new-bindings>} prepended to the ; @var{<bindings>} in effect. The result of these calls is passed to the ; @var{<handler>} (along with the head of the current @var{<Node>}). To ; be more precise, the handler is _applied_ to the head of the current ; node and its processed children. The result of the handler, which ; should also be a @code{<tree>}, replaces the current @var{<Node>}. If ; the current @var{<Node>} is a text string or other atom, a special ; binding with a symbol @code{*text*} is looked up. ; ; A binding can also be of a form ;@smallexample ; (<trigger-symbol> *macro* . <handler>) ;@end smallexample ; This is equivalent to @code{*preorder*} described above. However, the ; result is re-processed again, with the current stylesheet. ;; ;;; Code: (define-module (sxml transform) #:export (SRV:send-reply foldts post-order pre-post-order replace-range)) ;; Upstream version: ; $Id: SXML-tree-trans.scm,v 1.8 2003/04/24 19:39:53 oleg Exp oleg $ ; Like let* but allowing for multiple-value bindings (define-macro (let*-values bindings . body) (if (null? bindings) (cons 'begin body) (apply (lambda (vars initializer) (let ((cont (cons 'let*-values (cons (cdr bindings) body)))) (cond ((not (pair? vars)) ; regular let case, a single var `(let ((,vars ,initializer)) ,cont)) ((null? (cdr vars)) ; single var, see the prev case `(let ((,(car vars) ,initializer)) ,cont)) (else ; the most generic case `(call-with-values (lambda () ,initializer) (lambda ,vars ,cont)))))) (car bindings)))) (define (SRV:send-reply . fragments) "Output the @var{fragments} to the current output port. The fragments are a list of strings, characters, numbers, thunks, @code{#f}, @code{#t} -- and other fragments. The function traverses the tree depth-first, writes out strings and characters, executes thunks, and ignores @code{#f} and @code{'()}. The function returns @code{#t} if anything was written at all; otherwise the result is @code{#f} If @code{#t} occurs among the fragments, it is not written out but causes the result of @code{SRV:send-reply} to be @code{#t}." (let loop ((fragments fragments) (result #f)) (cond ((null? fragments) result) ((not (car fragments)) (loop (cdr fragments) result)) ((null? (car fragments)) (loop (cdr fragments) result)) ((eq? #t (car fragments)) (loop (cdr fragments) #t)) ((pair? (car fragments)) (loop (cdr fragments) (loop (car fragments) result))) ((procedure? (car fragments)) ((car fragments)) (loop (cdr fragments) #t)) (else (display (car fragments)) (loop (cdr fragments) #t))))) ;------------------------------------------------------------------------ ; Traversal of an SXML tree or a grove: ; a <Node> or a <Nodelist> ; ; A <Node> and a <Nodelist> are mutually-recursive datatypes that ; underlie the SXML tree: ; <Node> ::= (name . <Nodelist>) | "text string" ; An (ordered) set of nodes is just a list of the constituent nodes: ; <Nodelist> ::= (<Node> ...) ; Nodelists, and Nodes other than text strings are both lists. A ; <Nodelist> however is either an empty list, or a list whose head is ; not a symbol (an atom in general). A symbol at the head of a node is ; either an XML name (in which case it's a tag of an XML element), or ; an administrative name such as '@'. ; See SXPath.scm and SSAX.scm for more information on SXML. ;; see the commentary for docs (define (pre-post-order tree bindings) (let* ((default-binding (assq '*default* bindings)) (text-binding (or (assq '*text* bindings) default-binding)) (text-handler ; Cache default and text bindings (and text-binding (if (procedure? (cdr text-binding)) (cdr text-binding) (cddr text-binding))))) (let loop ((tree tree)) (cond ((null? tree) '()) ((not (pair? tree)) (let ((trigger '*text*)) (if text-handler (text-handler trigger tree) (error "Unknown binding for " trigger " and no default")))) ((not (symbol? (car tree))) (map loop tree)) ; tree is a nodelist (else ; tree is an SXML node (let* ((trigger (car tree)) (binding (or (assq trigger bindings) default-binding))) (cond ((not binding) (error "Unknown binding for " trigger " and no default")) ((not (pair? (cdr binding))) ; must be a procedure: handler (apply (cdr binding) trigger (map loop (cdr tree)))) ((eq? '*preorder* (cadr binding)) (apply (cddr binding) tree)) ((eq? '*macro* (cadr binding)) (loop (apply (cddr binding) tree))) (else ; (cadr binding) is a local binding (apply (cddr binding) trigger (pre-post-order (cdr tree) (append (cadr binding) bindings))) )))))))) ; post-order is a strict subset of pre-post-order without *preorder* ; (let alone *macro*) traversals. ; Now pre-post-order is actually faster than the old post-order. ; The function post-order is deprecated and is aliased below for ; backward compatibility. (define post-order pre-post-order) ;------------------------------------------------------------------------ ; Extended tree fold ; tree = atom | (node-name tree ...) ; ; foldts fdown fup fhere seed (Leaf str) = fhere seed str ; foldts fdown fup fhere seed (Nd kids) = ; fup seed $ foldl (foldts fdown fup fhere) (fdown seed) kids ; procedure fhere: seed -> atom -> seed ; procedure fdown: seed -> node -> seed ; procedure fup: parent-seed -> last-kid-seed -> node -> seed ; foldts returns the final seed (define (foldts fdown fup fhere seed tree) (cond ((null? tree) seed) ((not (pair? tree)) ; An atom (fhere seed tree)) (else (let loop ((kid-seed (fdown seed tree)) (kids (cdr tree))) (if (null? kids) (fup seed kid-seed tree) (loop (foldts fdown fup fhere kid-seed (car kids)) (cdr kids))))))) ;------------------------------------------------------------------------ ; Traverse a forest depth-first and cut/replace ranges of nodes. ; ; The nodes that define a range don't have to have the same immediate ; parent, don't have to be on the same level, and the end node of a ; range doesn't even have to exist. A replace-range procedure removes ; nodes from the beginning node of the range up to (but not including) ; the end node of the range. In addition, the beginning node of the ; range can be replaced by a node or a list of nodes. The range of ; nodes is cut while depth-first traversing the forest. If all ; branches of the node are cut a node is cut as well. The procedure ; can cut several non-overlapping ranges from a forest. ; replace-range:: BEG-PRED x END-PRED x FOREST -> FOREST ; where ; type FOREST = (NODE ...) ; type NODE = Atom | (Name . FOREST) | FOREST ; ; The range of nodes is specified by two predicates, beg-pred and end-pred. ; beg-pred:: NODE -> #f | FOREST ; end-pred:: NODE -> #f | FOREST ; The beg-pred predicate decides on the beginning of the range. The node ; for which the predicate yields non-#f marks the beginning of the range ; The non-#f value of the predicate replaces the node. The value can be a ; list of nodes. The replace-range procedure then traverses the tree and skips ; all the nodes, until the end-pred yields non-#f. The value of the end-pred ; replaces the end-range node. The new end node and its brothers will be ; re-scanned. ; The predicates are evaluated pre-order. We do not descend into a node that ; is marked as the beginning of the range. (define (replace-range beg-pred end-pred forest) ; loop forest keep? new-forest ; forest is the forest to traverse ; new-forest accumulates the nodes we will keep, in the reverse ; order ; If keep? is #t, keep the curr node if atomic. If the node is not atomic, ; traverse its children and keep those that are not in the skip range. ; If keep? is #f, skip the current node if atomic. Otherwise, ; traverse its children. If all children are skipped, skip the node ; as well. (define (loop forest keep? new-forest) (if (null? forest) (values (reverse new-forest) keep?) (let ((node (car forest))) (if keep? (cond ; accumulate mode ((beg-pred node) => ; see if the node starts the skip range (lambda (repl-branches) ; if so, skip/replace the node (loop (cdr forest) #f (append (reverse repl-branches) new-forest)))) ((not (pair? node)) ; it's an atom, keep it (loop (cdr forest) keep? (cons node new-forest))) (else (let*-values (((node?) (symbol? (car node))) ; or is it a nodelist? ((new-kids keep?) ; traverse its children (loop (if node? (cdr node) node) #t '()))) (loop (cdr forest) keep? (cons (if node? (cons (car node) new-kids) new-kids) new-forest))))) ; skip mode (cond ((end-pred node) => ; end the skip range (lambda (repl-branches) ; repl-branches will be re-scanned (loop (append repl-branches (cdr forest)) #t new-forest))) ((not (pair? node)) ; it's an atom, skip it (loop (cdr forest) keep? new-forest)) (else (let*-values (((node?) (symbol? (car node))) ; or is it a nodelist? ((new-kids keep?) ; traverse its children (loop (if node? (cdr node) node) #f '()))) (loop (cdr forest) keep? (if (or keep? (pair? new-kids)) (cons (if node? (cons (car node) new-kids) new-kids) new-forest) new-forest) ; if all kids are skipped )))))))) ; skip the node too (let*-values (((new-forest keep?) (loop forest #t '()))) new-forest)) ;;; arch-tag: 6c814f4b-38f7-42c1-b8ef-ce3447edefc7 ;;; transform.scm ends here PK�����/F2]PzM��zM����sxml/xpath.scmnu�[��������;;;; (sxml xpath) -- SXPath ;;;; ;;;; Copyright (C) 2009 Free Software Foundation, Inc. ;;;; Modified 2004 by Andy Wingo <wingo at pobox dot com>. ;;;; Written 2001 by Oleg Kiselyov <oleg at pobox dot com> SXPath.scm. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;;@heading SXPath: SXML Query Language ;; ;; SXPath is a query language for SXML, an instance of XML Information ;; set (Infoset) in the form of s-expressions. See @code{(sxml ssax)} ;; for the definition of SXML and more details. SXPath is also a ;; translation into Scheme of an XML Path Language, ;; @uref{http://www.w3.org/TR/xpath,XPath}. XPath and SXPath describe ;; means of selecting a set of Infoset's items or their properties. ;; ;; To facilitate queries, XPath maps the XML Infoset into an explicit ;; tree, and introduces important notions of a location path and a ;; current, context node. A location path denotes a selection of a set of ;; nodes relative to a context node. Any XPath tree has a distinguished, ;; root node -- which serves as the context node for absolute location ;; paths. Location path is recursively defined as a location step joined ;; with a location path. A location step is a simple query of the ;; database relative to a context node. A step may include expressions ;; that further filter the selected set. Each node in the resulting set ;; is used as a context node for the adjoining location path. The result ;; of the step is a union of the sets returned by the latter location ;; paths. ;; ;; The SXML representation of the XML Infoset (see SSAX.scm) is rather ;; suitable for querying as it is. Bowing to the XPath specification, ;; we will refer to SXML information items as 'Nodes': ;;@example ;; <Node> ::= <Element> | <attributes-coll> | <attrib> ;; | "text string" | <PI> ;;@end example ;; This production can also be described as ;;@example ;; <Node> ::= (name . <Nodeset>) | "text string" ;;@end example ;; An (ordered) set of nodes is just a list of the constituent nodes: ;;@example ;; <Nodeset> ::= (<Node> ...) ;;@end example ;; Nodesets, and Nodes other than text strings are both lists. A ;; <Nodeset> however is either an empty list, or a list whose head is not ;; a symbol. A symbol at the head of a node is either an XML name (in ;; which case it's a tag of an XML element), or an administrative name ;; such as '@@'. This uniform list representation makes processing rather ;; simple and elegant, while avoiding confusion. The multi-branch tree ;; structure formed by the mutually-recursive datatypes <Node> and ;; <Nodeset> lends itself well to processing by functional languages. ;; ;; A location path is in fact a composite query over an XPath tree or ;; its branch. A singe step is a combination of a projection, selection ;; or a transitive closure. Multiple steps are combined via join and ;; union operations. This insight allows us to @emph{elegantly} ;; implement XPath as a sequence of projection and filtering primitives ;; -- converters -- joined by @dfn{combinators}. Each converter takes a ;; node and returns a nodeset which is the result of the corresponding ;; query relative to that node. A converter can also be called on a set ;; of nodes. In that case it returns a union of the corresponding ;; queries over each node in the set. The union is easily implemented as ;; a list append operation as all nodes in a SXML tree are considered ;; distinct, by XPath conventions. We also preserve the order of the ;; members in the union. Query combinators are high-order functions: ;; they take converter(s) (which is a Node|Nodeset -> Nodeset function) ;; and compose or otherwise combine them. We will be concerned with only ;; relative location paths [XPath]: an absolute location path is a ;; relative path applied to the root node. ;; ;; Similarly to XPath, SXPath defines full and abbreviated notations ;; for location paths. In both cases, the abbreviated notation can be ;; mechanically expanded into the full form by simple rewriting ;; rules. In case of SXPath the corresponding rules are given as ;; comments to a sxpath function, below. The regression test suite at ;; the end of this file shows a representative sample of SXPaths in ;; both notations, juxtaposed with the corresponding XPath ;; expressions. Most of the samples are borrowed literally from the ;; XPath specification, while the others are adjusted for our running ;; example, tree1. ;; ;;; Code: (define-module (sxml xpath) #:use-module (ice-9 pretty-print) #:export (nodeset? node-typeof? node-eq? node-equal? node-pos filter take-until take-after map-union node-reverse node-trace select-kids node-self node-join node-reduce node-or node-closure node-parent sxpath)) ;; Upstream version: ; $Id: SXPath.scm,v 3.5 2001/01/12 23:20:35 oleg Exp oleg $ (define (nodeset? x) (or (and (pair? x) (not (symbol? (car x)))) (null? x))) ;------------------------- ; Basic converters and applicators ; A converter is a function ; type Converter = Node|Nodeset -> Nodeset ; A converter can also play a role of a predicate: in that case, if a ; converter, applied to a node or a nodeset, yields a non-empty ; nodeset, the converter-predicate is deemed satisfied. Throughout ; this file a nil nodeset is equivalent to #f in denoting a failure. ; The following function implements a 'Node test' as defined in ; Sec. 2.3 of XPath document. A node test is one of the components of a ; location step. It is also a converter-predicate in SXPath. ; ; The function node-typeof? takes a type criterion and returns a function, ; which, when applied to a node, will tell if the node satisfies ; the test. ; node-typeof? :: Crit -> Node -> Boolean ; ; The criterion 'crit' is a symbol, one of the following: ; id - tests if the Node has the right name (id) ; @ - tests if the Node is an <attributes-coll> ; * - tests if the Node is an <Element> ; *text* - tests if the Node is a text node ; *PI* - tests if the Node is a PI node ; *any* - #t for any type of Node (define (node-typeof? crit) (lambda (node) (case crit ((*) (and (pair? node) (not (memq (car node) '(@ *PI*))))) ((*any*) #t) ((*text*) (string? node)) (else (and (pair? node) (eq? crit (car node)))) ))) ; Curried equivalence converter-predicates (define (node-eq? other) (lambda (node) (eq? other node))) (define (node-equal? other) (lambda (node) (equal? other node))) ; node-pos:: N -> Nodeset -> Nodeset, or ; node-pos:: N -> Converter ; Select the N'th element of a Nodeset and return as a singular Nodeset; ; Return an empty nodeset if the Nth element does not exist. ; ((node-pos 1) Nodeset) selects the node at the head of the Nodeset, ; if exists; ((node-pos 2) Nodeset) selects the Node after that, if ; exists. ; N can also be a negative number: in that case the node is picked from ; the tail of the list. ; ((node-pos -1) Nodeset) selects the last node of a non-empty nodeset; ; ((node-pos -2) Nodeset) selects the last but one node, if exists. (define (node-pos n) (lambda (nodeset) (cond ((not (nodeset? nodeset)) '()) ((null? nodeset) nodeset) ((eqv? n 1) (list (car nodeset))) ((negative? n) ((node-pos (+ n 1 (length nodeset))) nodeset)) (else (or (positive? n) (error "yikes!")) ((node-pos (1- n)) (cdr nodeset)))))) ; filter:: Converter -> Converter ; A filter applicator, which introduces a filtering context. The argument ; converter is considered a predicate, with either #f or nil result meaning ; failure. (define (filter pred?) (lambda (lst) ; a nodeset or a node (will be converted to a singleton nset) (let loop ((lst (if (nodeset? lst) lst (list lst))) (res '())) (if (null? lst) (reverse res) (let ((pred-result (pred? (car lst)))) (loop (cdr lst) (if (and pred-result (not (null? pred-result))) (cons (car lst) res) res))))))) ; take-until:: Converter -> Converter, or ; take-until:: Pred -> Node|Nodeset -> Nodeset ; Given a converter-predicate and a nodeset, apply the predicate to ; each element of the nodeset, until the predicate yields anything but #f or ; nil. Return the elements of the input nodeset that have been processed ; till that moment (that is, which fail the predicate). ; take-until is a variation of the filter above: take-until passes ; elements of an ordered input set till (but not including) the first ; element that satisfies the predicate. ; The nodeset returned by ((take-until (not pred)) nset) is a subset -- ; to be more precise, a prefix -- of the nodeset returned by ; ((filter pred) nset) (define (take-until pred?) (lambda (lst) ; a nodeset or a node (will be converted to a singleton nset) (let loop ((lst (if (nodeset? lst) lst (list lst)))) (if (null? lst) lst (let ((pred-result (pred? (car lst)))) (if (and pred-result (not (null? pred-result))) '() (cons (car lst) (loop (cdr lst))))) )))) ; take-after:: Converter -> Converter, or ; take-after:: Pred -> Node|Nodeset -> Nodeset ; Given a converter-predicate and a nodeset, apply the predicate to ; each element of the nodeset, until the predicate yields anything but #f or ; nil. Return the elements of the input nodeset that have not been processed: ; that is, return the elements of the input nodeset that follow the first ; element that satisfied the predicate. ; take-after along with take-until partition an input nodeset into three ; parts: the first element that satisfies a predicate, all preceding ; elements and all following elements. (define (take-after pred?) (lambda (lst) ; a nodeset or a node (will be converted to a singleton nset) (let loop ((lst (if (nodeset? lst) lst (list lst)))) (if (null? lst) lst (let ((pred-result (pred? (car lst)))) (if (and pred-result (not (null? pred-result))) (cdr lst) (loop (cdr lst)))) )))) ; Apply proc to each element of lst and return the list of results. ; if proc returns a nodeset, splice it into the result ; ; From another point of view, map-union is a function Converter->Converter, ; which places an argument-converter in a joining context. (define (map-union proc lst) (if (null? lst) lst (let ((proc-res (proc (car lst)))) ((if (nodeset? proc-res) append cons) proc-res (map-union proc (cdr lst)))))) ; node-reverse :: Converter, or ; node-reverse:: Node|Nodeset -> Nodeset ; Reverses the order of nodes in the nodeset ; This basic converter is needed to implement a reverse document order ; (see the XPath Recommendation). (define node-reverse (lambda (node-or-nodeset) (if (not (nodeset? node-or-nodeset)) (list node-or-nodeset) (reverse node-or-nodeset)))) ; node-trace:: String -> Converter ; (node-trace title) is an identity converter. In addition it prints out ; a node or nodeset it is applied to, prefixed with the 'title'. ; This converter is very useful for debugging. (define (node-trace title) (lambda (node-or-nodeset) (display "\n-->") (display title) (display " :") (pretty-print node-or-nodeset) node-or-nodeset)) ;------------------------- ; Converter combinators ; ; Combinators are higher-order functions that transmogrify a converter ; or glue a sequence of converters into a single, non-trivial ; converter. The goal is to arrive at converters that correspond to ; XPath location paths. ; ; From a different point of view, a combinator is a fixed, named ; _pattern_ of applying converters. Given below is a complete set of ; such patterns that together implement XPath location path ; specification. As it turns out, all these combinators can be built ; from a small number of basic blocks: regular functional composition, ; map-union and filter applicators, and the nodeset union. ; select-kids:: Pred -> Node -> Nodeset ; Given a Node, return an (ordered) subset its children that satisfy ; the Pred (a converter, actually) ; select-kids:: Pred -> Nodeset -> Nodeset ; The same as above, but select among children of all the nodes in ; the Nodeset ; ; More succinctly, the signature of this function is ; select-kids:: Converter -> Converter (define (select-kids test-pred?) (lambda (node) ; node or node-set (cond ((null? node) node) ((not (pair? node)) '()) ; No children ((symbol? (car node)) ((filter test-pred?) (cdr node))) ; it's a single node (else (map-union (select-kids test-pred?) node))))) ; node-self:: Pred -> Node -> Nodeset, or ; node-self:: Converter -> Converter ; Similar to select-kids but apply to the Node itself rather ; than to its children. The resulting Nodeset will contain either one ; component, or will be empty (if the Node failed the Pred). (define node-self filter) ; node-join:: [LocPath] -> Node|Nodeset -> Nodeset, or ; node-join:: [Converter] -> Converter ; join the sequence of location steps or paths as described ; in the title comments above. (define (node-join . selectors) (lambda (nodeset) ; Nodeset or node (let loop ((nodeset nodeset) (selectors selectors)) (if (null? selectors) nodeset (loop (if (nodeset? nodeset) (map-union (car selectors) nodeset) ((car selectors) nodeset)) (cdr selectors)))))) ; node-reduce:: [LocPath] -> Node|Nodeset -> Nodeset, or ; node-reduce:: [Converter] -> Converter ; A regular functional composition of converters. ; From a different point of view, ; ((apply node-reduce converters) nodeset) ; is equivalent to ; (foldl apply nodeset converters) ; i.e., folding, or reducing, a list of converters with the nodeset ; as a seed. (define (node-reduce . converters) (lambda (nodeset) ; Nodeset or node (let loop ((nodeset nodeset) (converters converters)) (if (null? converters) nodeset (loop ((car converters) nodeset) (cdr converters)))))) ; node-or:: [Converter] -> Converter ; This combinator applies all converters to a given node and ; produces the union of their results. ; This combinator corresponds to a union, '|' operation for XPath ; location paths. ; (define (node-or . converters) ; (lambda (node-or-nodeset) ; (if (null? converters) node-or-nodeset ; (append ; ((car converters) node-or-nodeset) ; ((apply node-or (cdr converters)) node-or-nodeset))))) ; More optimal implementation follows (define (node-or . converters) (lambda (node-or-nodeset) (let loop ((result '()) (converters converters)) (if (null? converters) result (loop (append result (or ((car converters) node-or-nodeset) '())) (cdr converters)))))) ; node-closure:: Converter -> Converter ; Select all _descendants_ of a node that satisfy a converter-predicate. ; This combinator is similar to select-kids but applies to ; grand... children as well. ; This combinator implements the "descendant::" XPath axis ; Conceptually, this combinator can be expressed as ; (define (node-closure f) ; (node-or ; (select-kids f) ; (node-reduce (select-kids (node-typeof? '*)) (node-closure f)))) ; This definition, as written, looks somewhat like a fixpoint, and it ; will run forever. It is obvious however that sooner or later ; (select-kids (node-typeof? '*)) will return an empty nodeset. At ; this point further iterations will no longer affect the result and ; can be stopped. (define (node-closure test-pred?) (lambda (node) ; Nodeset or node (let loop ((parent node) (result '())) (if (null? parent) result (loop ((select-kids (node-typeof? '*)) parent) (append result ((select-kids test-pred?) parent))) )))) ; node-parent:: RootNode -> Converter ; (node-parent rootnode) yields a converter that returns a parent of a ; node it is applied to. If applied to a nodeset, it returns the list ; of parents of nodes in the nodeset. The rootnode does not have ; to be the root node of the whole SXML tree -- it may be a root node ; of a branch of interest. ; Given the notation of Philip Wadler's paper on semantics of XSLT, ; parent(x) = { y | y=subnode*(root), x=subnode(y) } ; Therefore, node-parent is not the fundamental converter: it can be ; expressed through the existing ones. Yet node-parent is a rather ; convenient converter. It corresponds to a parent:: axis of SXPath. ; Note that the parent:: axis can be used with an attribute node as well! (define (node-parent rootnode) (lambda (node) ; Nodeset or node (if (nodeset? node) (map-union (node-parent rootnode) node) (let ((pred (node-or (node-reduce (node-self (node-typeof? '*)) (select-kids (node-eq? node))) (node-join (select-kids (node-typeof? '@)) (select-kids (node-eq? node)))))) ((node-or (node-self pred) (node-closure pred)) rootnode))))) ;------------------------- ; Evaluate an abbreviated SXPath ; sxpath:: AbbrPath -> Converter, or ; sxpath:: AbbrPath -> Node|Nodeset -> Nodeset ; AbbrPath is a list. It is translated to the full SXPath according ; to the following rewriting rules ; (sxpath '()) -> (node-join) ; (sxpath '(path-component ...)) -> ; (node-join (sxpath1 path-component) (sxpath '(...))) ; (sxpath1 '//) -> (node-or ; (node-self (node-typeof? '*any*)) ; (node-closure (node-typeof? '*any*))) ; (sxpath1 '(equal? x)) -> (select-kids (node-equal? x)) ; (sxpath1 '(eq? x)) -> (select-kids (node-eq? x)) ; (sxpath1 ?symbol) -> (select-kids (node-typeof? ?symbol) ; (sxpath1 procedure) -> procedure ; (sxpath1 '(?symbol ...)) -> (sxpath1 '((?symbol) ...)) ; (sxpath1 '(path reducer ...)) -> ; (node-reduce (sxpath path) (sxpathr reducer) ...) ; (sxpathr number) -> (node-pos number) ; (sxpathr path-filter) -> (filter (sxpath path-filter)) (define (sxpath path) (lambda (nodeset) (let loop ((nodeset nodeset) (path path)) (cond ((null? path) nodeset) ((nodeset? nodeset) (map-union (sxpath path) nodeset)) ((procedure? (car path)) (loop ((car path) nodeset) (cdr path))) ((eq? '// (car path)) (loop ((if (nodeset? nodeset) append cons) nodeset ((node-closure (node-typeof? '*any*)) nodeset)) (cdr path))) ((symbol? (car path)) (loop ((select-kids (node-typeof? (car path))) nodeset) (cdr path))) ((and (pair? (car path)) (eq? 'equal? (caar path))) (loop ((select-kids (apply node-equal? (cdar path))) nodeset) (cdr path))) ((and (pair? (car path)) (eq? 'eq? (caar path))) (loop ((select-kids (apply node-eq? (cdar path))) nodeset) (cdr path))) ((pair? (car path)) (let reducer ((nodeset (if (symbol? (caar path)) ((select-kids (node-typeof? (caar path))) nodeset) (loop nodeset (caar path)))) (reducing-path (cdar path))) (cond ((null? reducing-path) (loop nodeset (cdr path))) ((number? (car reducing-path)) (reducer ((node-pos (car reducing-path)) nodeset) (cdr reducing-path))) (else (reducer ((filter (sxpath (car reducing-path))) nodeset) (cdr reducing-path)))))) (else (error "Invalid path step: " (car path))))))) ;;; arch-tag: c4e57abf-6b61-4612-a6aa-d1536d440774 ;;; xpath.scm ends here PK�����/F2] � ���sxml/sxml-match.ssnu�[��������;; Library: sxml-match ;; Author: Jim Bender ;; Version: 1.1, version for PLT Scheme ;; ;; Copyright 2005-9, Jim Bender ;; sxml-match is released under the MIT License ;; (module sxml-match mzscheme (provide sxml-match sxml-match-let sxml-match-let*) (require (rename (lib "fold.ss" "srfi" "1") fold-right fold-right) (rename (lib "filter.ss" "srfi" "1") filter filter)) (define (nodeset? x) (or (and (pair? x) (not (symbol? (car x)))) (null? x))) (define (xml-element-tag s) (if (and (pair? s) (symbol? (car s))) (car s) (error 'xml-element-tag "expected an xml-element, given" s))) (define (xml-element-attributes s) (if (and (pair? s) (symbol? (car s))) (fold-right (lambda (a b) (if (and (pair? a) (eq? '@ (car a))) (if (null? b) (filter (lambda (i) (not (and (pair? i) (eq? '@ (car i))))) (cdr a)) (fold-right (lambda (c d) (if (and (pair? c) (eq? '@ (car c))) d (cons c d))) b (cdr a))) b)) '() (cdr s)) (error 'xml-element-attributes "expected an xml-element, given" s))) (define (xml-element-contents s) (if (and (pair? s) (symbol? (car s))) (filter (lambda (i) (not (and (pair? i) (eq? '@ (car i))))) (cdr s)) (error 'xml-element-contents "expected an xml-element, given" s))) (define (match-xml-attribute key l) (if (not (pair? l)) #f (if (eq? (car (car l)) key) (car l) (match-xml-attribute key (cdr l))))) (define (filter-attributes keys lst) (if (null? lst) '() (if (member (caar lst) keys) (filter-attributes keys (cdr lst)) (cons (car lst) (filter-attributes keys (cdr lst)))))) (define-syntax compile-clause (lambda (stx) (letrec ([sxml-match-syntax-error (lambda (msg exp sub) (raise-syntax-error #f msg (with-syntax ([s exp]) (syntax (sxml-match s))) sub))] [ellipsis? (lambda (stx) (and (identifier? stx) (eq? '... (syntax-object->datum stx))))] [literal? (lambda (stx) (let ([x (syntax-object->datum stx)]) (or (string? x) (char? x) (number? x) (boolean? x))))] [keyword? (lambda (stx) (and (identifier? stx) (let ([str (symbol->string (syntax-object->datum stx))]) (char=? #\: (string-ref str (- (string-length str) 1))))))] [extract-cata-fun (lambda (cf) (syntax-case cf () [#f #f] [other cf]))] [add-pat-var (lambda (pvar pvar-lst) (define (check-pvar lst) (if (null? lst) (void) (if (bound-identifier=? (car lst) pvar) (sxml-match-syntax-error "duplicate pattern variable not allowed" stx pvar) (check-pvar (cdr lst))))) (check-pvar pvar-lst) (cons pvar pvar-lst))] [add-cata-def (lambda (depth cvars cfun ctemp cdefs) (cons (list depth cvars cfun ctemp) cdefs))] [process-cata-exp (lambda (depth cfun ctemp) (if (= depth 0) (with-syntax ([cf cfun] [ct ctemp]) (syntax (cf ct))) (let ([new-ctemp (car (generate-temporaries (list ctemp)))]) (with-syntax ([ct ctemp] [nct new-ctemp] [body (process-cata-exp (- depth 1) cfun new-ctemp)]) (syntax (map (lambda (nct) body) ct))))))] [process-cata-defs (lambda (cata-defs body) (if (null? cata-defs) body (with-syntax ([(cata-binding ...) (map (lambda (def) (with-syntax ([bvar (cadr def)] [bval (process-cata-exp (car def) (caddr def) (cadddr def))]) (syntax (bvar bval)))) cata-defs)] [body-stx body]) (syntax (let-values (cata-binding ...) body-stx)))))] [cata-defs->pvar-lst (lambda (lst) (if (null? lst) '() (let iter ([items (cadr (car lst))]) (syntax-case items () [() (cata-defs->pvar-lst (cdr lst))] [(fst . rst) (cons (syntax fst) (iter (syntax rst)))]))))] [process-output-action (lambda (action dotted-vars) (define (finite-lst? lst) (syntax-case lst () (item (identifier? (syntax item)) #f) (() #t) ((fst dots . rst) (ellipsis? (syntax dots)) #f) ((fst . rst) (finite-lst? (syntax rst))))) (define (expand-lst lst) (syntax-case lst () [() (syntax '())] [item (identifier? (syntax item)) (syntax item)] [(fst dots . rst) (ellipsis? (syntax dots)) (with-syntax ([exp-lft (expand-dotted-item (process-output-action (syntax fst) dotted-vars))] [exp-rgt (expand-lst (syntax rst))]) (syntax (append exp-lft exp-rgt)))] [(fst . rst) (with-syntax ([exp-lft (process-output-action (syntax fst) dotted-vars)] [exp-rgt (expand-lst (syntax rst))]) (syntax (cons exp-lft exp-rgt)))])) (define (member-var? var lst) (let iter ([lst lst]) (if (null? lst) #f (if (or (bound-identifier=? var (car lst)) (free-identifier=? var (car lst))) #t (iter (cdr lst)))))) (define (dotted-var? var) (member-var? var dotted-vars)) (define (merge-pvars lst1 lst2) (if (null? lst1) lst2 (if (member-var? (car lst1) lst2) (merge-pvars (cdr lst1) lst2) (cons (car lst1) (merge-pvars (cdr lst1) lst2))))) (define (select-dotted-vars x) (define (walk-quasi-body y) (syntax-case y (unquote unquote-splicing) [((unquote a) . rst) (merge-pvars (select-dotted-vars (syntax a)) (walk-quasi-body (syntax rst)))] [((unquote-splicing a) . rst) (merge-pvars (select-dotted-vars (syntax a)) (walk-quasi-body (syntax rst)))] [(fst . rst) (merge-pvars (walk-quasi-body (syntax fst)) (walk-quasi-body (syntax rst)))] [other '()])) (syntax-case x (quote quasiquote) [(quote . rst) '()] [(quasiquote . rst) (walk-quasi-body (syntax rst))] [(fst . rst) (merge-pvars (select-dotted-vars (syntax fst)) (select-dotted-vars (syntax rst)))] [item (and (identifier? (syntax item)) (dotted-var? (syntax item))) (list (syntax item))] [item '()])) (define (expand-dotted-item item) (let ([dvars (select-dotted-vars item)]) (syntax-case item () [x (identifier? (syntax x)) (syntax x)] [x (with-syntax ([(dv ...) dvars]) (syntax (map (lambda (dv ...) x) dv ...)))]))) (define (expand-quasiquote-body x) (syntax-case x (unquote unquote-splicing quasiquote) [(quasiquote . rst) (process-quasiquote x)] [(unquote item) (with-syntax ([expanded-item (process-output-action (syntax item) dotted-vars)]) (syntax (unquote expanded-item)))] [(unquote-splicing item) (with-syntax ([expanded-item (process-output-action (syntax item) dotted-vars)]) (syntax (unquote-splicing expanded-item)))] [((unquote item) dots . rst) (ellipsis? (syntax dots)) (with-syntax ([expanded-item (expand-dotted-item (process-output-action (syntax item) dotted-vars))] [expanded-rst (expand-quasiquote-body (syntax rst))]) (syntax ((unquote-splicing expanded-item) . expanded-rst)))] [(item dots . rst) (ellipsis? (syntax dots)) (with-syntax ([expanded-item (expand-dotted-item (process-output-action (syntax (quasiquote item)) dotted-vars))] [expanded-rst (expand-quasiquote-body (syntax rst))]) (syntax ((unquote-splicing expanded-item) . expanded-rst)))] [(fst . rst) (with-syntax ([expanded-fst (expand-quasiquote-body (syntax fst))] [expanded-rst (expand-quasiquote-body (syntax rst))]) (syntax (expanded-fst . expanded-rst)))] [other x])) (define (process-quasiquote x) (syntax-case x () [(quasiquote term) (with-syntax ([expanded-body (expand-quasiquote-body (syntax term))]) (syntax (quasiquote expanded-body)))] [else (sxml-match-syntax-error "bad quasiquote-form" stx x)])) (syntax-case action (quote quasiquote) [(quote . rst) action] [(quasiquote . rst) (process-quasiquote action)] [(fst . rst) (if (finite-lst? action) (with-syntax ([exp-lft (process-output-action (syntax fst) dotted-vars)] [exp-rgt (process-output-action (syntax rst) dotted-vars)]) (syntax (exp-lft . exp-rgt))) (with-syntax ([exp-lft (process-output-action (syntax fst) dotted-vars)] [exp-rgt (expand-lst (syntax rst))]) (syntax (apply exp-lft exp-rgt))))] [item action]))] [compile-element-pat (lambda (ele exp nextp fail-k pvar-lst depth cata-fun cata-defs dotted-vars) (syntax-case ele (@) [(tag (@ . attr-items) . items) (identifier? (syntax tag)) (let ([attr-exp (car (generate-temporaries (list exp)))] [body-exp (car (generate-temporaries (list exp)))]) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax attr-items) (syntax items) attr-exp body-exp '() nextp fail-k pvar-lst depth cata-fun cata-defs dotted-vars)]) (values (with-syntax ([x exp] [ax attr-exp] [bx body-exp] [body tests] [fail-to fail-k]) (syntax (if (and (pair? x) (eq? 'tag (xml-element-tag x))) (let ([ax (xml-element-attributes x)] [bx (xml-element-contents x)]) body) (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)))] [(tag . items) (identifier? (syntax tag)) (let ([body-exp (car (generate-temporaries (list exp)))]) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-item-list (syntax items) body-exp nextp fail-k #t pvar-lst depth cata-fun cata-defs dotted-vars)]) (values (with-syntax ([x exp] [bx body-exp] [body tests] [fail-to fail-k]) (syntax (if (and (pair? x) (eq? 'tag (xml-element-tag x))) (let ([bx (xml-element-contents x)]) body) (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)))]))] [compile-end-element (lambda (exp nextp fail-k pvar-lst cata-defs dotted-vars) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp pvar-lst cata-defs dotted-vars)]) (values (with-syntax ([x exp] [body next-tests] [fail-to fail-k]) (syntax (if (null? x) body (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)))] [compile-attr-list (lambda (attr-lst body-lst attr-exp body-exp attr-key-lst nextp fail-k pvar-lst depth cata-fun cata-defs dotted-vars) (syntax-case attr-lst (unquote ->) [(unquote var) (identifier? (syntax var)) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-item-list body-lst body-exp nextp fail-k #t (add-pat-var (syntax var) pvar-lst) depth cata-fun cata-defs dotted-vars)]) (values (with-syntax ([ax attr-exp] [matched-attrs attr-key-lst] [body tests]) (syntax (let ([var (filter-attributes 'matched-attrs ax)]) body))) new-pvar-lst new-cata-defs new-dotted-vars))] [((atag [(unquote [cata -> cvar ...]) default]) . rst) (identifier? (syntax atag)) (let ([ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax rst) body-lst attr-exp body-exp (cons (syntax atag) attr-key-lst) nextp fail-k (add-pat-var ctemp pvar-lst) depth cata-fun (add-cata-def depth (syntax [cvar ...]) (syntax cata) ctemp cata-defs) dotted-vars)]) (values (with-syntax ([ax attr-exp] [ct ctemp] [body tests]) (syntax (let ([binding (match-xml-attribute 'atag ax)]) (let ([ct (if binding (cadr binding) default)]) body)))) new-pvar-lst new-cata-defs new-dotted-vars)))] [((atag [(unquote [cvar ...]) default]) . rst) (identifier? (syntax atag)) (let ([ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (if (not cata-fun) (sxml-match-syntax-error "sxml-match pattern: catamorphism not allowed in this context" stx (syntax [cvar ...]))) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax rst) body-lst attr-exp body-exp (cons (syntax atag) attr-key-lst) nextp fail-k (add-pat-var ctemp pvar-lst) depth cata-fun (add-cata-def depth (syntax [cvar ...]) cata-fun ctemp cata-defs) dotted-vars)]) (values (with-syntax ([ax attr-exp] [ct ctemp] [body tests]) (syntax (let ([binding (match-xml-attribute 'atag ax)]) (let ([ct (if binding (cadr binding) default)]) body)))) new-pvar-lst new-cata-defs new-dotted-vars)))] [((atag [(unquote var) default]) . rst) (and (identifier? (syntax atag)) (identifier? (syntax var))) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax rst) body-lst attr-exp body-exp (cons (syntax atag) attr-key-lst) nextp fail-k (add-pat-var (syntax var) pvar-lst) depth cata-fun cata-defs dotted-vars)]) (values (with-syntax ([ax attr-exp] [body tests]) (syntax (let ([binding (match-xml-attribute 'atag ax)]) (let ([var (if binding (cadr binding) default)]) body)))) new-pvar-lst new-cata-defs new-dotted-vars))] [((atag (unquote [cata -> cvar ...])) . rst) (identifier? (syntax atag)) (let ([ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax rst) body-lst attr-exp body-exp (cons (syntax atag) attr-key-lst) nextp fail-k (add-pat-var ctemp pvar-lst) depth cata-fun (add-cata-def depth (syntax [cvar ...]) (syntax cata) ctemp cata-defs) dotted-vars)]) (values (with-syntax ([ax attr-exp] [ct ctemp] [body tests] [fail-to fail-k]) (syntax (let ([binding (match-xml-attribute 'atag ax)]) (if binding (let ([ct (cadr binding)]) body) (fail-to))))) new-pvar-lst new-cata-defs new-dotted-vars)))] [((atag (unquote [cvar ...])) . rst) (identifier? (syntax atag)) (let ([ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (if (not cata-fun) (sxml-match-syntax-error "sxml-match pattern: catamorphism not allowed in this context" stx (syntax [cvar ...]))) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax rst) body-lst attr-exp body-exp (cons (syntax atag) attr-key-lst) nextp fail-k (add-pat-var ctemp pvar-lst) depth cata-fun (add-cata-def depth (syntax [cvar ...]) cata-fun ctemp cata-defs) dotted-vars)]) (values (with-syntax ([ax attr-exp] [ct ctemp] [body tests] [fail-to fail-k]) (syntax (let ([binding (match-xml-attribute 'atag ax)]) (if binding (let ([ct (cadr binding)]) body) (fail-to))))) new-pvar-lst new-cata-defs new-dotted-vars)))] [((atag (unquote var)) . rst) (and (identifier? (syntax atag)) (identifier? (syntax var))) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax rst) body-lst attr-exp body-exp (cons (syntax atag) attr-key-lst) nextp fail-k (add-pat-var (syntax var) pvar-lst) depth cata-fun cata-defs dotted-vars)]) (values (with-syntax ([ax attr-exp] [body tests] [fail-to fail-k]) (syntax (let ([binding (match-xml-attribute 'atag ax)]) (if binding (let ([var (cadr binding)]) body) (fail-to))))) new-pvar-lst new-cata-defs new-dotted-vars))] [((atag (i ...)) . rst) (identifier? (syntax atag)) (sxml-match-syntax-error "bad attribute pattern" stx (syntax (kwd (i ...))))] [((atag i) . rst) (and (identifier? (syntax atag)) (identifier? (syntax i))) (sxml-match-syntax-error "bad attribute pattern" stx (syntax (kwd i)))] [((atag literal) . rst) (and (identifier? (syntax atag)) (literal? (syntax literal))) (let-values ([(tests new-pvar-lst new-cata-defs new-dotted-vars) (compile-attr-list (syntax rst) body-lst attr-exp body-exp (cons (syntax atag) attr-key-lst) nextp fail-k pvar-lst depth cata-fun cata-defs dotted-vars)]) (values (with-syntax ([ax attr-exp] [body tests] [fail-to fail-k]) (syntax (let ([binding (match-xml-attribute 'atag ax)]) (if binding (if (equal? (cadr binding) literal) body (fail-to)) (fail-to))))) new-pvar-lst new-cata-defs new-dotted-vars))] [() (compile-item-list body-lst body-exp nextp fail-k #t pvar-lst depth cata-fun cata-defs dotted-vars)]))] [compile-item-list (lambda (lst exp nextp fail-k ellipsis-allowed? pvar-lst depth cata-fun cata-defs dotted-vars) (syntax-case lst (unquote ->) [() (compile-end-element exp nextp fail-k pvar-lst cata-defs dotted-vars)] [(unquote var) (identifier? (syntax var)) (if (not ellipsis-allowed?) (sxml-match-syntax-error "improper list pattern not allowed in this context" stx (syntax dots)) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp (add-pat-var (syntax var) pvar-lst) cata-defs dotted-vars)]) (values (with-syntax ([x exp] [body next-tests]) (syntax (let ([var x]) body))) new-pvar-lst new-cata-defs new-dotted-vars)))] [(unquote [cata -> cvar ...]) (if (not ellipsis-allowed?) (sxml-match-syntax-error "improper list pattern not allowed in this context" stx (syntax dots)) (let ([ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp (add-pat-var ctemp pvar-lst) (add-cata-def depth (syntax [cvar ...]) (syntax cata) ctemp cata-defs) dotted-vars)]) (values (with-syntax ([ct ctemp] [x exp] [body next-tests]) (syntax (let ([ct x]) body))) new-pvar-lst new-cata-defs new-dotted-vars))))] [(unquote [cvar ...]) (let ([ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (if (not cata-fun) (sxml-match-syntax-error "sxml-match pattern: catamorphism not allowed in this context" stx (syntax [cvar ...]))) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp (add-pat-var ctemp pvar-lst) (add-cata-def depth (syntax [cvar ...]) cata-fun ctemp cata-defs) dotted-vars)]) (values (with-syntax ([ct ctemp] [x exp] [body next-tests]) (syntax (let ([ct x]) body))) new-pvar-lst new-cata-defs new-dotted-vars)))] [(item dots . rst) (ellipsis? (syntax dots)) (if (not ellipsis-allowed?) (sxml-match-syntax-error "ellipses not allowed in this context" stx (syntax dots)) (compile-dotted-pattern-list (syntax item) (syntax rst) exp nextp fail-k pvar-lst depth cata-fun cata-defs dotted-vars))] [(item . rst) (compile-item (syntax item) exp (lambda (new-exp new-pvar-lst new-cata-defs new-dotted-vars) (compile-item-list (syntax rst) new-exp nextp fail-k ellipsis-allowed? new-pvar-lst depth cata-fun new-cata-defs new-dotted-vars)) fail-k pvar-lst depth cata-fun cata-defs dotted-vars)]))] [compile-dotted-pattern-list (lambda (item tail exp nextp fail-k pvar-lst depth cata-fun cata-defs dotted-vars) (let-values ([(tail-tests tail-pvar-lst tail-cata-defs tail-dotted-vars) (compile-item-list tail (syntax lst) (lambda (new-pvar-lst new-cata-defs new-dotted-vars) (values (with-syntax ([(npv ...) new-pvar-lst]) (syntax (values #t npv ...))) new-pvar-lst new-cata-defs new-dotted-vars)) (syntax fail) #f '() depth '() '() dotted-vars)] [(item-tests item-pvar-lst item-cata-defs item-dotted-vars) (compile-item item (syntax lst) (lambda (new-exp new-pvar-lst new-cata-defs new-dotted-vars) (values (with-syntax ([(npv ...) new-pvar-lst]) (syntax (values #t (cdr lst) npv ...))) new-pvar-lst new-cata-defs new-dotted-vars)) (syntax fail) '() (+ 1 depth) cata-fun '() dotted-vars)]) ; more here: check for duplicate pat-vars, cata-defs (let-values ([(final-tests final-pvar-lst final-cata-defs final-dotted-vars) (nextp (append tail-pvar-lst item-pvar-lst pvar-lst) (append tail-cata-defs item-cata-defs cata-defs) (append item-pvar-lst (cata-defs->pvar-lst item-cata-defs) tail-dotted-vars dotted-vars))]) (let ([temp-item-pvar-lst (generate-temporaries item-pvar-lst)]) (values (with-syntax ([x exp] [fail-to fail-k] [tail-body tail-tests] [item-body item-tests] [final-body final-tests] [(ipv ...) item-pvar-lst] [(gpv ...) temp-item-pvar-lst] [(tpv ...) tail-pvar-lst] [(item-void ...) (map (lambda (i) (syntax (void))) item-pvar-lst)] [(tail-void ...) (map (lambda (i) (syntax (void))) tail-pvar-lst)] [(item-null ...) (map (lambda (i) (syntax '())) item-pvar-lst)] [(item-cons ...) (map (lambda (a b) (with-syntax ([xa a] [xb b]) (syntax (cons xa xb)))) item-pvar-lst temp-item-pvar-lst)]) (syntax (letrec ([match-tail (lambda (lst fail) tail-body)] [match-item (lambda (lst) (let ([fail (lambda () (values #f lst item-void ...))]) item-body))] [match-dotted (lambda (x) (let-values ([(tail-res tpv ...) (match-tail x (lambda () (values #f tail-void ...)))]) (if tail-res (values item-null ... tpv ...) (let-values ([(res new-x ipv ...) (match-item x)]) (if res (let-values ([(gpv ... tpv ...) (match-dotted new-x)]) (values item-cons ... tpv ...)) (let-values ([(last-tail-res tpv ...) (match-tail x fail-to)]) (values item-null ... tpv ...)))))))]) (let-values ([(ipv ... tpv ...) (match-dotted x)]) final-body)))) final-pvar-lst final-cata-defs final-dotted-vars)))))] [compile-item (lambda (item exp nextp fail-k pvar-lst depth cata-fun cata-defs dotted-vars) (syntax-case item (unquote ->) ; normal pattern var [(unquote var) (identifier? (syntax var)) (let ([new-exp (car (generate-temporaries (list exp)))]) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp new-exp (add-pat-var (syntax var) pvar-lst) cata-defs dotted-vars)]) (values (with-syntax ([x exp] [nx new-exp] [body next-tests] [fail-to fail-k]) (syntax (if (pair? x) (let ([nx (cdr x)] [var (car x)]) body) (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)))] ; named catamorphism [(unquote [cata -> cvar ...]) (let ([new-exp (car (generate-temporaries (list exp)))] [ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp new-exp (add-pat-var ctemp pvar-lst) (add-cata-def depth (syntax [cvar ...]) (syntax cata) ctemp cata-defs) dotted-vars)]) (values (with-syntax ([x exp] [nx new-exp] [ct ctemp] [body next-tests] [fail-to fail-k]) (syntax (if (pair? x) (let ([nx (cdr x)] [ct (car x)]) body) (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)))] ; basic catamorphism [(unquote [cvar ...]) (let ([new-exp (car (generate-temporaries (list exp)))] [ctemp (car (generate-temporaries (syntax ([cvar ...]))))]) (if (not cata-fun) (sxml-match-syntax-error "sxml-match pattern: catamorphism not allowed in this context" stx (syntax [cvar ...]))) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp new-exp (add-pat-var ctemp pvar-lst) (add-cata-def depth (syntax [cvar ...]) cata-fun ctemp cata-defs) dotted-vars)]) (values (with-syntax ([x exp] [nx new-exp] [ct ctemp] [body next-tests] [fail-to fail-k]) (syntax (if (pair? x) (let ([nx (cdr x)] [ct (car x)]) body) (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)))] [(tag item ...) (identifier? (syntax tag)) (let ([new-exp (car (generate-temporaries (list exp)))]) (let-values ([(after-tests after-pvar-lst after-cata-defs after-dotted-vars) (compile-element-pat (syntax (tag item ...)) (with-syntax ([x exp]) (syntax (car x))) (lambda (more-pvar-lst more-cata-defs more-dotted-vars) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp new-exp more-pvar-lst more-cata-defs more-dotted-vars)]) (values (with-syntax ([x exp] [nx new-exp] [body next-tests]) (syntax (let ([nx (cdr x)]) body))) new-pvar-lst new-cata-defs new-dotted-vars))) fail-k pvar-lst depth cata-fun cata-defs dotted-vars)]) ; test that we are not at the end of an item-list, BEFORE ; entering tests for the element pattern (against the 'car' of the item-list) (values (with-syntax ([x exp] [body after-tests] [fail-to fail-k]) (syntax (if (pair? x) body (fail-to)))) after-pvar-lst after-cata-defs after-dotted-vars)))] [(i ...) (sxml-match-syntax-error "bad pattern syntax (not an element pattern)" stx (syntax (i ...)))] [i (identifier? (syntax i)) (sxml-match-syntax-error "bad pattern syntax (symbol not allowed in this context)" stx (syntax i))] [literal (literal? (syntax literal)) (let ([new-exp (car (generate-temporaries (list exp)))]) (let-values ([(next-tests new-pvar-lst new-cata-defs new-dotted-vars) (nextp new-exp pvar-lst cata-defs dotted-vars)]) (values (with-syntax ([x exp] [nx new-exp] [body next-tests] [fail-to fail-k]) (syntax (if (and (pair? x) (equal? literal (car x))) (let ([nx (cdr x)]) body) (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)))]))]) (let ([fail-k (syntax failure)]) (syntax-case stx (unquote guard ->) [(compile-clause ((unquote var) (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (identifier? (syntax var)) (syntax (let ([var exp]) (if (and gexp ...) (begin action0 action ...) (fail-exp))))] [(compile-clause ((unquote [cata -> cvar ...]) (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (syntax (if (and gexp ...) (let-values ([(cvar ...) (cata exp)]) (begin action0 action ...)) (fail-exp)))] [(compile-clause ((unquote [cvar ...]) (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (if (not (extract-cata-fun (syntax cata-fun))) (sxml-match-syntax-error "sxml-match pattern: catamorphism not allowed in this context" stx (syntax [cvar ...])) (syntax (if (and gexp ...) (let-values ([(cvar ...) (cata-fun exp)]) (begin action0 action ...)) (fail-exp))))] [(compile-clause ((unquote var) action0 action ...) exp cata-fun fail-exp) (identifier? (syntax var)) (syntax (let ([var exp]) action0 action ...))] [(compile-clause ((unquote [cata -> cvar ...]) action0 action ...) exp cata-fun fail-exp) (syntax (let-values ([(cvar ...) (cata exp)]) action0 action ...))] [(compile-clause ((unquote [cvar ...]) action0 action ...) exp cata-fun fail-exp) (if (not (extract-cata-fun (syntax cata-fun))) (sxml-match-syntax-error "sxml-match pattern: catamorphism not allowed in this context" stx (syntax [cvar ...])) (syntax (let-values ([(cvar ...) (cata-fun exp)]) action0 action ...)))] [(compile-clause ((lst . rst) (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (and (identifier? (syntax lst)) (eq? 'list (syntax-object->datum (syntax lst)))) (let-values ([(result pvar-lst cata-defs dotted-vars) (compile-item-list (syntax rst) (syntax exp) (lambda (new-pvar-lst new-cata-defs new-dotted-vars) (values (with-syntax ([exp-body (process-cata-defs new-cata-defs (process-output-action (syntax (begin action0 action ...)) new-dotted-vars))] [fail-to fail-k]) (syntax (if (and gexp ...) exp-body (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)) fail-k #t '() 0 (extract-cata-fun (syntax cata-fun)) '() '())]) (with-syntax ([fail-to fail-k] [body result]) (syntax (let ([fail-to fail-exp]) (if (nodeset? exp) body (fail-to))))))] [(compile-clause ((lst . rst) action0 action ...) exp cata-fun fail-exp) (and (identifier? (syntax lst)) (eq? 'list (syntax-object->datum (syntax lst)))) (let-values ([(result pvar-lst cata-defs dotted-vars) (compile-item-list (syntax rst) (syntax exp) (lambda (new-pvar-lst new-cata-defs new-dotted-vars) (values (process-cata-defs new-cata-defs (process-output-action (syntax (begin action0 action ...)) new-dotted-vars)) new-pvar-lst new-cata-defs new-dotted-vars)) fail-k #t '() 0 (extract-cata-fun (syntax cata-fun)) '() '())]) (with-syntax ([body result] [fail-to fail-k]) (syntax (let ([fail-to fail-exp]) (if (nodeset? exp) body (fail-to))))))] [(compile-clause ((fst . rst) (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (identifier? (syntax fst)) (let-values ([(result pvar-lst cata-defs dotted-vars) (compile-element-pat (syntax (fst . rst)) (syntax exp) (lambda (new-pvar-lst new-cata-defs new-dotted-vars) (values (with-syntax ([body (process-cata-defs new-cata-defs (process-output-action (syntax (begin action0 action ...)) new-dotted-vars))] [fail-to fail-k]) (syntax (if (and gexp ...) body (fail-to)))) new-pvar-lst new-cata-defs new-dotted-vars)) fail-k '() 0 (extract-cata-fun (syntax cata-fun)) '() '())]) (with-syntax ([fail-to fail-k] [body result]) (syntax (let ([fail-to fail-exp]) body))))] [(compile-clause ((fst . rst) action0 action ...) exp cata-fun fail-exp) (identifier? (syntax fst)) (let-values ([(result pvar-lst cata-defs dotted-vars) (compile-element-pat (syntax (fst . rst)) (syntax exp) (lambda (new-pvar-lst new-cata-defs new-dotted-vars) (values (process-cata-defs new-cata-defs (process-output-action (syntax (begin action0 action ...)) new-dotted-vars)) new-pvar-lst new-cata-defs new-dotted-vars)) fail-k '() 0 (extract-cata-fun (syntax cata-fun)) '() '())]) (with-syntax ([fail-to fail-k] [body result]) (syntax (let ([fail-to fail-exp]) body))))] [(compile-clause ((i ...) (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (sxml-match-syntax-error "bad pattern syntax (not an element pattern)" stx (syntax (i ...)))] [(compile-clause ((i ...) action0 action ...) exp cata-fun fail-exp) (sxml-match-syntax-error "bad pattern syntax (not an element pattern)" stx (syntax (i ...)))] [(compile-clause (pat (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (identifier? (syntax pat)) (sxml-match-syntax-error "bad pattern syntax (symbol not allowed in this context)" stx (syntax pat))] [(compile-clause (pat action0 action ...) exp cata-fun fail-exp) (identifier? (syntax pat)) (sxml-match-syntax-error "bad pattern syntax (symbol not allowed in this context)" stx (syntax pat))] [(compile-clause (literal (guard gexp ...) action0 action ...) exp cata-fun fail-exp) (literal? (syntax literal)) (syntax (if (and (equal? literal exp) (and gexp ...)) (begin action0 action ...) (fail-exp)))] [(compile-clause (literal action0 action ...) exp cata-fun fail-exp) (literal? (syntax literal)) (syntax (if (equal? literal exp) (begin action0 action ...) (fail-exp)))]))))) (define-syntax sxml-match1 (syntax-rules () [(sxml-match1 exp cata-fun clause) (compile-clause clause exp cata-fun (lambda () (error 'sxml-match "no matching clause found")))] [(sxml-match1 exp cata-fun clause0 clause ...) (let/ec escape (compile-clause clause0 exp cata-fun (lambda () (call-with-values (lambda () (sxml-match1 exp cata-fun clause ...)) escape))))])) (define-syntax sxml-match (syntax-rules () ((sxml-match val clause0 clause ...) (letrec ([cfun (lambda (exp) (sxml-match1 exp cfun clause0 clause ...))]) (cfun val))))) (define-syntax sxml-match-let1 (syntax-rules () [(sxml-match-let1 syntag synform () body0 body ...) (let () body0 body ...)] [(sxml-match-let1 syntag synform ([pat exp]) body0 body ...) (compile-clause (pat (let () body0 body ...)) exp #f (lambda () (error 'syntag "could not match pattern ~s" 'pat)))] [(sxml-match-let1 syntag synform ([pat0 exp0] [pat exp] ...) body0 body ...) (compile-clause (pat0 (sxml-match-let1 syntag synform ([pat exp] ...) body0 body ...)) exp0 #f (lambda () (error 'syntag "could not match pattern ~s" 'pat0)))])) (define-syntax sxml-match-let-help (lambda (stx) (syntax-case stx () [(sxml-match-let-help syntag synform ([pat exp] ...) body0 body ...) (with-syntax ([(temp-name ...) (generate-temporaries (syntax (exp ...)))]) (syntax (let ([temp-name exp] ...) (sxml-match-let1 syntag synform ([pat temp-name] ...) body0 body ...))))]))) (define-syntax sxml-match-let (lambda (stx) (syntax-case stx () [(sxml-match-let ([pat exp] ...) body0 body ...) (with-syntax ([synform stx]) (syntax (sxml-match-let-help sxml-match-let synform ([pat exp] ...) body0 body ...)))]))) (define-syntax sxml-match-let* (lambda (stx) (syntax-case stx () [(sxml-match-let* () body0 body ...) (syntax (let () body0 body ...))] [(sxml-match-let* ([pat0 exp0] [pat exp] ...) body0 body ...) (with-syntax ([synform stx]) (syntax (sxml-match-let-help sxml-match-let* synform ([pat0 exp0]) (sxml-match-let* ([pat exp] ...) body0 body ...))))]))) ) PK�����/F2]P+��+�� ��sxml/ssax.scmnu�[��������;;;; (sxml ssax) -- the SSAX parser ;;;; ;;;; Copyright (C) 2009, 2010,2012,2013 Free Software Foundation, Inc. ;;;; Modified 2004 by Andy Wingo <wingo at pobox dot com>. ;;;; Written 2001,2002,2003,2004 by Oleg Kiselyov <oleg at pobox dot com> as SSAX.scm. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;@subheading Functional XML parsing framework ;@subsubheading SAX/DOM and SXML parsers with support for XML Namespaces and validation ; ; This is a package of low-to-high level lexing and parsing procedures ; that can be combined to yield a SAX, a DOM, a validating parser, or ; a parser intended for a particular document type. The procedures in ; the package can be used separately to tokenize or parse various ; pieces of XML documents. The package supports XML Namespaces, ; internal and external parsed entities, user-controlled handling of ; whitespace, and validation. This module therefore is intended to be ; a framework, a set of "Lego blocks" you can use to build a parser ; following any discipline and performing validation to any degree. As ; an example of the parser construction, this file includes a ; semi-validating SXML parser. ; The present XML framework has a "sequential" feel of SAX yet a ; "functional style" of DOM. Like a SAX parser, the framework scans the ; document only once and permits incremental processing. An application ; that handles document elements in order can run as efficiently as ; possible. @emph{Unlike} a SAX parser, the framework does not require ; an application register stateful callbacks and surrender control to ; the parser. Rather, it is the application that can drive the framework ; -- calling its functions to get the current lexical or syntax element. ; These functions do not maintain or mutate any state save the input ; port. Therefore, the framework permits parsing of XML in a pure ; functional style, with the input port being a monad (or a linear, ; read-once parameter). ; Besides the @var{port}, there is another monad -- @var{seed}. Most of ; the middle- and high-level parsers are single-threaded through the ; @var{seed}. The functions of this framework do not process or affect ; the @var{seed} in any way: they simply pass it around as an instance ; of an opaque datatype. User functions, on the other hand, can use the ; seed to maintain user's state, to accumulate parsing results, etc. A ; user can freely mix his own functions with those of the framework. On ; the other hand, the user may wish to instantiate a high-level parser: ; @code{SSAX:make-elem-parser} or @code{SSAX:make-parser}. In the latter ; case, the user must provide functions of specific signatures, which ; are called at predictable moments during the parsing: to handle ; character data, element data, or processing instructions (PI). The ; functions are always given the @var{seed}, among other parameters, and ; must return the new @var{seed}. ; From a functional point of view, XML parsing is a combined ; pre-post-order traversal of a "tree" that is the XML document ; itself. This down-and-up traversal tells the user about an element ; when its start tag is encountered. The user is notified about the ; element once more, after all element's children have been ; handled. The process of XML parsing therefore is a fold over the ; raw XML document. Unlike a fold over trees defined in [1], the ; parser is necessarily single-threaded -- obviously as elements ; in a text XML document are laid down sequentially. The parser ; therefore is a tree fold that has been transformed to accept an ; accumulating parameter [1,2]. ; Formally, the denotational semantics of the parser can be expressed ; as ;@smallexample ; parser:: (Start-tag -> Seed -> Seed) -> ; (Start-tag -> Seed -> Seed -> Seed) -> ; (Char-Data -> Seed -> Seed) -> ; XML-text-fragment -> Seed -> Seed ; parser fdown fup fchar "<elem attrs> content </elem>" seed ; = fup "<elem attrs>" seed ; (parser fdown fup fchar "content" (fdown "<elem attrs>" seed)) ; ; parser fdown fup fchar "char-data content" seed ; = parser fdown fup fchar "content" (fchar "char-data" seed) ; ; parser fdown fup fchar "elem-content content" seed ; = parser fdown fup fchar "content" ( ; parser fdown fup fchar "elem-content" seed) ;@end smallexample ; Compare the last two equations with the left fold ;@smallexample ; fold-left kons elem:list seed = fold-left kons list (kons elem seed) ;@end smallexample ; The real parser created by @code{SSAX:make-parser} is slightly more ; complicated, to account for processing instructions, entity ; references, namespaces, processing of document type declaration, etc. ; The XML standard document referred to in this module is ; @uref{http://www.w3.org/TR/1998/REC-xml-19980210.html} ; ; The present file also defines a procedure that parses the text of an ; XML document or of a separate element into SXML, an S-expression-based ; model of an XML Information Set. SXML is also an Abstract Syntax Tree ; of an XML document. SXML is similar but not identical to DOM; SXML is ; particularly suitable for Scheme-based XML/HTML authoring, SXPath ; queries, and tree transformations. See SXML.html for more details. ; SXML is a term implementation of evaluation of the XML document [3]. ; The other implementation is context-passing. ; The present frameworks fully supports the XML Namespaces Recommendation: ; @uref{http://www.w3.org/TR/REC-xml-names/} ; Other links: ;@table @asis ;@item [1] ; Jeremy Gibbons, Geraint Jones, "The Under-appreciated Unfold," ; Proc. ICFP'98, 1998, pp. 273-279. ;@item [2] ; Richard S. Bird, The promotion and accumulation strategies in ; transformational programming, ACM Trans. Progr. Lang. Systems, ; 6(4):487-504, October 1984. ;@item [3] ; Ralf Hinze, "Deriving Backtracking Monad Transformers," ; Functional Pearl. Proc ICFP'00, pp. 186-197. ;@end table ;; ;;; Code: (define-module (sxml ssax) #:use-module (sxml ssax input-parse) #:use-module (srfi srfi-1) #:use-module (srfi srfi-13) #:export (current-ssax-error-port with-ssax-error-to-port xml-token? xml-token-kind xml-token-head make-empty-attlist attlist-add attlist-null? attlist-remove-top attlist->alist attlist-fold define-parsed-entity! reset-parsed-entity-definitions! ssax:uri-string->symbol ssax:skip-internal-dtd ssax:read-pi-body-as-string ssax:reverse-collect-str-drop-ws ssax:read-markup-token ssax:read-cdata-body ssax:read-char-ref ssax:read-attributes ssax:complete-start-tag ssax:read-external-id ssax:read-char-data ssax:xml->sxml ssax:make-parser ssax:make-pi-parser ssax:make-elem-parser)) (define (parser-error port message . rest) (apply throw 'parser-error port message rest)) (define ascii->char integer->char) (define char->ascii char->integer) (define current-ssax-error-port (make-parameter (current-error-port))) (define *current-ssax-error-port* (parameter-fluid current-ssax-error-port)) (define (with-ssax-error-to-port port thunk) (parameterize ((current-ssax-error-port port)) (thunk))) (define (ssax:warn port . args) (with-output-to-port (current-ssax-error-port) (lambda () (display ";;; SSAX warning: ") (for-each display args) (newline)))) (define (ucscode->string codepoint) (string (integer->char codepoint))) (define char-newline #\newline) (define char-return #\return) (define char-tab #\tab) (define nl "\n") ;; This isn't a great API, but a more proper fix will involve hacking ;; SSAX. (define (reset-parsed-entity-definitions!) "Restore the set of parsed entity definitions to its initial state." (set! ssax:predefined-parsed-entities '((amp . "&") (lt . "<") (gt . ">") (apos . "'") (quot . "\"")))) (define (define-parsed-entity! entity str) "Define a new parsed entity. @var{entity} should be a symbol. Instances of &@var{entity}; in XML text will be replaced with the string @var{str}, which will then be parsed." (set! ssax:predefined-parsed-entities (acons entity str ssax:predefined-parsed-entities))) ;; Execute a sequence of forms and return the result of the _first_ one. ;; Like PROG1 in Lisp. Typically used to evaluate one or more forms with ;; side effects and return a value that must be computed before some or ;; all of the side effects happen. (define-syntax begin0 (syntax-rules () ((begin0 form form1 ... ) (let ((val form)) form1 ... val)))) ; Like let* but allowing for multiple-value bindings (define-syntax let*-values (syntax-rules () ((let*-values () . bodies) (begin . bodies)) ((let*-values (((var) initializer) . rest) . bodies) (let ((var initializer)) ; a single var optimization (let*-values rest . bodies))) ((let*-values ((vars initializer) . rest) . bodies) (call-with-values (lambda () initializer) ; the most generic case (lambda vars (let*-values rest . bodies)))))) ;; needed for some dumb reason (define inc 1+) (define dec 1-) (define-syntax include-from-path/filtered (lambda (x) (define (read-filtered accept-list file) (with-input-from-file (%search-load-path file) (lambda () (let loop ((sexp (read)) (out '())) (cond ((eof-object? sexp) (reverse out)) ((and (pair? sexp) (memq (car sexp) accept-list)) (loop (read) (cons sexp out))) (else (loop (read) out))))))) (syntax-case x () ((_ accept-list file) (with-syntax (((exp ...) (datum->syntax x (read-filtered (syntax->datum #'accept-list) (syntax->datum #'file))))) #'(begin exp ...)))))) (include-from-path "sxml/upstream/assert.scm") (include-from-path/filtered (define define-syntax ssax:define-labeled-arg-macro) "sxml/upstream/SSAX.scm") PK�����/F2]:(��(����sxml/upstream/SXPath-old.scmnu�[��������; XML processing in Scheme ; SXPath -- SXML Query Language ; ; SXPath is a query language for SXML, an instance of XML Information ; set (Infoset) in the form of s-expressions. See SSAX.scm for the ; definition of SXML and more details. SXPath is also a translation into ; Scheme of an XML Path Language, XPath: ; http://www.w3.org/TR/xpath ; XPath and SXPath describe means of selecting a set of Infoset's items ; or their properties. ; ; To facilitate queries, XPath maps the XML Infoset into an explicit ; tree, and introduces important notions of a location path and a ; current, context node. A location path denotes a selection of a set of ; nodes relative to a context node. Any XPath tree has a distinguished, ; root node -- which serves as the context node for absolute location ; paths. Location path is recursively defined as a location step joined ; with a location path. A location step is a simple query of the ; database relative to a context node. A step may include expressions ; that further filter the selected set. Each node in the resulting set ; is used as a context node for the adjoining location path. The result ; of the step is a union of the sets returned by the latter location ; paths. ; ; The SXML representation of the XML Infoset (see SSAX.scm) is rather ; suitable for querying as it is. Bowing to the XPath specification, ; we will refer to SXML information items as 'Nodes': ; <Node> ::= <Element> | <attributes-coll> | <attrib> ; | "text string" | <PI> ; This production can also be described as ; <Node> ::= (name . <Nodeset>) | "text string" ; An (ordered) set of nodes is just a list of the constituent nodes: ; <Nodeset> ::= (<Node> ...) ; Nodesets, and Nodes other than text strings are both lists. A ; <Nodeset> however is either an empty list, or a list whose head is not ; a symbol. A symbol at the head of a node is either an XML name (in ; which case it's a tag of an XML element), or an administrative name ; such as '@'. This uniform list representation makes processing rather ; simple and elegant, while avoiding confusion. The multi-branch tree ; structure formed by the mutually-recursive datatypes <Node> and ; <Nodeset> lends itself well to processing by functional languages. ; ; A location path is in fact a composite query over an XPath tree or ; its branch. A singe step is a combination of a projection, selection ; or a transitive closure. Multiple steps are combined via join and ; union operations. This insight allows us to _elegantly_ implement ; XPath as a sequence of projection and filtering primitives -- ; converters -- joined by _combinators_. Each converter takes a node ; and returns a nodeset which is the result of the corresponding query ; relative to that node. A converter can also be called on a set of ; nodes. In that case it returns a union of the corresponding queries over ; each node in the set. The union is easily implemented as a list ; append operation as all nodes in a SXML tree are considered ; distinct, by XPath conventions. We also preserve the order of the ; members in the union. Query combinators are high-order functions: ; they take converter(s) (which is a Node|Nodeset -> Nodeset function) ; and compose or otherwise combine them. We will be concerned with ; only relative location paths [XPath]: an absolute location path is a ; relative path applied to the root node. ; ; Similarly to XPath, SXPath defines full and abbreviated notations ; for location paths. In both cases, the abbreviated notation can be ; mechanically expanded into the full form by simple rewriting ; rules. In case of SXPath the corresponding rules are given as ; comments to a sxpath function, below. The regression test suite at ; the end of this file shows a representative sample of SXPaths in ; both notations, juxtaposed with the corresponding XPath ; expressions. Most of the samples are borrowed literally from the ; XPath specification, while the others are adjusted for our running ; example, tree1. ; ; To do: ; Rename filter to node-filter or ns-filter ; Use ;=== for chapters, ;--- for sections, and ;^^^ for end sections ; ; $Id: SXPath-old.scm,v 1.4 2004/07/07 16:02:31 sperber Exp $ ; See http://pobox.com/~oleg/ftp/Scheme/myenv.scm ; See http://pobox.com/~oleg/ftp/Scheme/myenv-scm.scm ; See http://pobox.com/~oleg/ftp/Scheme/myenv-bigloo.scm ;(module SXPath ; (include "myenv-bigloo.scm")) ; For use with Bigloo 2.2b ;(load "myenv-scm.scm") ; For use with SCM v5d2 ;(include "myenv.scm") ; For use with Gambit-C 3.0 (define (nodeset? x) (or (and (pair? x) (not (symbol? (car x)))) (null? x))) ;------------------------- ; Basic converters and applicators ; A converter is a function ; type Converter = Node|Nodeset -> Nodeset ; A converter can also play a role of a predicate: in that case, if a ; converter, applied to a node or a nodeset, yields a non-empty ; nodeset, the converter-predicate is deemed satisfied. Throughout ; this file a nil nodeset is equivalent to #f in denoting a failure. ; The following function implements a 'Node test' as defined in ; Sec. 2.3 of XPath document. A node test is one of the components of a ; location step. It is also a converter-predicate in SXPath. ; ; The function node-typeof? takes a type criterion and returns a function, ; which, when applied to a node, will tell if the node satisfies ; the test. ; node-typeof? :: Crit -> Node -> Boolean ; ; The criterion 'crit' is a symbol, one of the following: ; id - tests if the Node has the right name (id) ; @ - tests if the Node is an <attributes-coll> ; * - tests if the Node is an <Element> ; *text* - tests if the Node is a text node ; *PI* - tests if the Node is a PI node ; *any* - #t for any type of Node (define (node-typeof? crit) (lambda (node) (case crit ((*) (and (pair? node) (not (memq (car node) '(@ *PI*))))) ((*any*) #t) ((*text*) (string? node)) (else (and (pair? node) (eq? crit (car node)))) ))) ; Curried equivalence converter-predicates (define (node-eq? other) (lambda (node) (eq? other node))) (define (node-equal? other) (lambda (node) (equal? other node))) ; node-pos:: N -> Nodeset -> Nodeset, or ; node-pos:: N -> Converter ; Select the N'th element of a Nodeset and return as a singular Nodeset; ; Return an empty nodeset if the Nth element does not exist. ; ((node-pos 1) Nodeset) selects the node at the head of the Nodeset, ; if exists; ((node-pos 2) Nodeset) selects the Node after that, if ; exists. ; N can also be a negative number: in that case the node is picked from ; the tail of the list. ; ((node-pos -1) Nodeset) selects the last node of a non-empty nodeset; ; ((node-pos -2) Nodeset) selects the last but one node, if exists. (define (node-pos n) (lambda (nodeset) (cond ((not (nodeset? nodeset)) '()) ((null? nodeset) nodeset) ((eqv? n 1) (list (car nodeset))) ((negative? n) ((node-pos (+ n 1 (length nodeset))) nodeset)) (else (assert (positive? n)) ((node-pos (dec n)) (cdr nodeset)))))) ; filter:: Converter -> Converter ; A filter applicator, which introduces a filtering context. The argument ; converter is considered a predicate, with either #f or nil result meaning ; failure. (define (filter pred?) (lambda (lst) ; a nodeset or a node (will be converted to a singleton nset) (let loop ((lst (if (nodeset? lst) lst (list lst))) (res '())) (if (null? lst) (reverse res) (let ((pred-result (pred? (car lst)))) (loop (cdr lst) (if (and pred-result (not (null? pred-result))) (cons (car lst) res) res))))))) ; take-until:: Converter -> Converter, or ; take-until:: Pred -> Node|Nodeset -> Nodeset ; Given a converter-predicate and a nodeset, apply the predicate to ; each element of the nodeset, until the predicate yields anything but #f or ; nil. Return the elements of the input nodeset that have been processed ; till that moment (that is, which fail the predicate). ; take-until is a variation of the filter above: take-until passes ; elements of an ordered input set till (but not including) the first ; element that satisfies the predicate. ; The nodeset returned by ((take-until (not pred)) nset) is a subset -- ; to be more precise, a prefix -- of the nodeset returned by ; ((filter pred) nset) (define (take-until pred?) (lambda (lst) ; a nodeset or a node (will be converted to a singleton nset) (let loop ((lst (if (nodeset? lst) lst (list lst)))) (if (null? lst) lst (let ((pred-result (pred? (car lst)))) (if (and pred-result (not (null? pred-result))) '() (cons (car lst) (loop (cdr lst))))) )))) ; take-after:: Converter -> Converter, or ; take-after:: Pred -> Node|Nodeset -> Nodeset ; Given a converter-predicate and a nodeset, apply the predicate to ; each element of the nodeset, until the predicate yields anything but #f or ; nil. Return the elements of the input nodeset that have not been processed: ; that is, return the elements of the input nodeset that follow the first ; element that satisfied the predicate. ; take-after along with take-until partition an input nodeset into three ; parts: the first element that satisfies a predicate, all preceding ; elements and all following elements. (define (take-after pred?) (lambda (lst) ; a nodeset or a node (will be converted to a singleton nset) (let loop ((lst (if (nodeset? lst) lst (list lst)))) (if (null? lst) lst (let ((pred-result (pred? (car lst)))) (if (and pred-result (not (null? pred-result))) (cdr lst) (loop (cdr lst)))) )))) ; Apply proc to each element of lst and return the list of results. ; if proc returns a nodeset, splice it into the result ; ; From another point of view, map-union is a function Converter->Converter, ; which places an argument-converter in a joining context. (define (map-union proc lst) (if (null? lst) lst (let ((proc-res (proc (car lst)))) ((if (nodeset? proc-res) append cons) proc-res (map-union proc (cdr lst)))))) ; node-reverse :: Converter, or ; node-reverse:: Node|Nodeset -> Nodeset ; Reverses the order of nodes in the nodeset ; This basic converter is needed to implement a reverse document order ; (see the XPath Recommendation). (define node-reverse (lambda (node-or-nodeset) (if (not (nodeset? node-or-nodeset)) (list node-or-nodeset) (reverse node-or-nodeset)))) ; node-trace:: String -> Converter ; (node-trace title) is an identity converter. In addition it prints out ; a node or nodeset it is applied to, prefixed with the 'title'. ; This converter is very useful for debugging. (define (node-trace title) (lambda (node-or-nodeset) (cout nl "-->") (display title) (display " :") (pretty-print node-or-nodeset) node-or-nodeset)) ;------------------------- ; Converter combinators ; ; Combinators are higher-order functions that transmogrify a converter ; or glue a sequence of converters into a single, non-trivial ; converter. The goal is to arrive at converters that correspond to ; XPath location paths. ; ; From a different point of view, a combinator is a fixed, named ; _pattern_ of applying converters. Given below is a complete set of ; such patterns that together implement XPath location path ; specification. As it turns out, all these combinators can be built ; from a small number of basic blocks: regular functional composition, ; map-union and filter applicators, and the nodeset union. ; select-kids:: Pred -> Node -> Nodeset ; Given a Node, return an (ordered) subset its children that satisfy ; the Pred (a converter, actually) ; select-kids:: Pred -> Nodeset -> Nodeset ; The same as above, but select among children of all the nodes in ; the Nodeset ; ; More succinctly, the signature of this function is ; select-kids:: Converter -> Converter (define (select-kids test-pred?) (lambda (node) ; node or node-set (cond ((null? node) node) ((not (pair? node)) '()) ; No children ((symbol? (car node)) ((filter test-pred?) (cdr node))) ; it's a single node (else (map-union (select-kids test-pred?) node))))) ; node-self:: Pred -> Node -> Nodeset, or ; node-self:: Converter -> Converter ; Similar to select-kids but apply to the Node itself rather ; than to its children. The resulting Nodeset will contain either one ; component, or will be empty (if the Node failed the Pred). (define node-self filter) ; node-join:: [LocPath] -> Node|Nodeset -> Nodeset, or ; node-join:: [Converter] -> Converter ; join the sequence of location steps or paths as described ; in the title comments above. (define (node-join . selectors) (lambda (nodeset) ; Nodeset or node (let loop ((nodeset nodeset) (selectors selectors)) (if (null? selectors) nodeset (loop (if (nodeset? nodeset) (map-union (car selectors) nodeset) ((car selectors) nodeset)) (cdr selectors)))))) ; node-reduce:: [LocPath] -> Node|Nodeset -> Nodeset, or ; node-reduce:: [Converter] -> Converter ; A regular functional composition of converters. ; From a different point of view, ; ((apply node-reduce converters) nodeset) ; is equivalent to ; (foldl apply nodeset converters) ; i.e., folding, or reducing, a list of converters with the nodeset ; as a seed. (define (node-reduce . converters) (lambda (nodeset) ; Nodeset or node (let loop ((nodeset nodeset) (converters converters)) (if (null? converters) nodeset (loop ((car converters) nodeset) (cdr converters)))))) ; node-or:: [Converter] -> Converter ; This combinator applies all converters to a given node and ; produces the union of their results. ; This combinator corresponds to a union, '|' operation for XPath ; location paths. ; (define (node-or . converters) ; (lambda (node-or-nodeset) ; (if (null? converters) node-or-nodeset ; (append ; ((car converters) node-or-nodeset) ; ((apply node-or (cdr converters)) node-or-nodeset))))) ; More optimal implementation follows (define (node-or . converters) (lambda (node-or-nodeset) (let loop ((result '()) (converters converters)) (if (null? converters) result (loop (append result (or ((car converters) node-or-nodeset) '())) (cdr converters)))))) ; node-closure:: Converter -> Converter ; Select all _descendants_ of a node that satisfy a converter-predicate. ; This combinator is similar to select-kids but applies to ; grand... children as well. ; This combinator implements the "descendant::" XPath axis ; Conceptually, this combinator can be expressed as ; (define (node-closure f) ; (node-or ; (select-kids f) ; (node-reduce (select-kids (node-typeof? '*)) (node-closure f)))) ; This definition, as written, looks somewhat like a fixpoint, and it ; will run forever. It is obvious however that sooner or later ; (select-kids (node-typeof? '*)) will return an empty nodeset. At ; this point further iterations will no longer affect the result and ; can be stopped. (define (node-closure test-pred?) (lambda (node) ; Nodeset or node (let loop ((parent node) (result '())) (if (null? parent) result (loop ((select-kids (node-typeof? '*)) parent) (append result ((select-kids test-pred?) parent))) )))) ; node-parent:: RootNode -> Converter ; (node-parent rootnode) yields a converter that returns a parent of a ; node it is applied to. If applied to a nodeset, it returns the list ; of parents of nodes in the nodeset. The rootnode does not have ; to be the root node of the whole SXML tree -- it may be a root node ; of a branch of interest. ; Given the notation of Philip Wadler's paper on semantics of XSLT, ; parent(x) = { y | y=subnode*(root), x=subnode(y) } ; Therefore, node-parent is not the fundamental converter: it can be ; expressed through the existing ones. Yet node-parent is a rather ; convenient converter. It corresponds to a parent:: axis of SXPath. ; Note that the parent:: axis can be used with an attribute node as well! (define (node-parent rootnode) (lambda (node) ; Nodeset or node (if (nodeset? node) (map-union (node-parent rootnode) node) (let ((pred (node-or (node-reduce (node-self (node-typeof? '*)) (select-kids (node-eq? node))) (node-join (select-kids (node-typeof? '@)) (select-kids (node-eq? node)))))) ((node-or (node-self pred) (node-closure pred)) rootnode))))) ;------------------------- ; Evaluate an abbreviated SXPath ; sxpath:: AbbrPath -> Converter, or ; sxpath:: AbbrPath -> Node|Nodeset -> Nodeset ; AbbrPath is a list. It is translated to the full SXPath according ; to the following rewriting rules ; (sxpath '()) -> (node-join) ; (sxpath '(path-component ...)) -> ; (node-join (sxpath1 path-component) (sxpath '(...))) ; (sxpath1 '//) -> (node-or ; (node-self (node-typeof? '*any*)) ; (node-closure (node-typeof? '*any*))) ; (sxpath1 '(equal? x)) -> (select-kids (node-equal? x)) ; (sxpath1 '(eq? x)) -> (select-kids (node-eq? x)) ; (sxpath1 ?symbol) -> (select-kids (node-typeof? ?symbol) ; (sxpath1 procedure) -> procedure ; (sxpath1 '(?symbol ...)) -> (sxpath1 '((?symbol) ...)) ; (sxpath1 '(path reducer ...)) -> ; (node-reduce (sxpath path) (sxpathr reducer) ...) ; (sxpathr number) -> (node-pos number) ; (sxpathr path-filter) -> (filter (sxpath path-filter)) (define (sxpath path) (lambda (nodeset) (let loop ((nodeset nodeset) (path path)) (cond ((null? path) nodeset) ((nodeset? nodeset) (map-union (sxpath path) nodeset)) ((procedure? (car path)) (loop ((car path) nodeset) (cdr path))) ((eq? '// (car path)) (loop ((if (nodeset? nodeset) append cons) nodeset ((node-closure (node-typeof? '*any*)) nodeset)) (cdr path))) ((symbol? (car path)) (loop ((select-kids (node-typeof? (car path))) nodeset) (cdr path))) ((and (pair? (car path)) (eq? 'equal? (caar path))) (loop ((select-kids (apply node-equal? (cdar path))) nodeset) (cdr path))) ((and (pair? (car path)) (eq? 'eq? (caar path))) (loop ((select-kids (apply node-eq? (cdar path))) nodeset) (cdr path))) ((pair? (car path)) (let reducer ((nodeset (if (symbol? (caar path)) ((select-kids (node-typeof? (caar path))) nodeset) (loop nodeset (caar path)))) (reducing-path (cdar path))) (cond ((null? reducing-path) (loop nodeset (cdr path))) ((number? (car reducing-path)) (reducer ((node-pos (car reducing-path)) nodeset) (cdr reducing-path))) (else (reducer ((filter (sxpath (car reducing-path))) nodeset) (cdr reducing-path)))))) (else (error "Invalid path step: " (car path))) )))) ;------------------------------------------------------------------------ ; Sample XPath/SXPath expressions: regression test suite for the ; implementation above. ; A running example (define tree1 '(html (head (title "Slides")) (body (p (@ (align "center")) (table (@ (style "font-size: x-large")) (tr (td (@ (align "right")) "Talks ") (td (@ (align "center")) " = ") (td " slides + transition")) (tr (td) (td (@ (align "center")) " = ") (td " data + control")) (tr (td) (td (@ (align "center")) " = ") (td " programs")))) (ul (li (a (@ (href "slides/slide0001.gif")) "Introduction")) (li (a (@ (href "slides/slide0010.gif")) "Summary"))) ))) ; Example from a posting "Re: DrScheme and XML", ; Shriram Krishnamurthi, comp.lang.scheme, Nov. 26. 1999. ; http://www.deja.com/getdoc.xp?AN=553507805 (define tree3 '(poem (@ (title "The Lovesong of J. Alfred Prufrock") (poet "T. S. Eliot")) (stanza (line "Let us go then, you and I,") (line "When the evening is spread out against the sky") (line "Like a patient etherized upon a table:")) (stanza (line "In the room the women come and go") (line "Talking of Michaelangelo.")))) ; Validation Test harness (define-syntax run-test (syntax-rules (define) ((run-test "scan-exp" (define vars body)) (define vars (run-test "scan-exp" body))) ((run-test "scan-exp" ?body) (letrec-syntax ((scan-exp ; (scan-exp body k) (syntax-rules (quote quasiquote !) ((scan-exp '() (k-head ! . args)) (k-head '() . args)) ((scan-exp (quote (hd . tl)) k) (scan-lit-lst (hd . tl) (do-wrap ! quasiquote k))) ((scan-exp (quasiquote (hd . tl)) k) (scan-lit-lst (hd . tl) (do-wrap ! quasiquote k))) ((scan-exp (quote x) (k-head ! . args)) (k-head (if (string? (quote x)) (string->symbol (quote x)) (quote x)) . args)) ((scan-exp (hd . tl) k) (scan-exp hd (do-tl ! scan-exp tl k))) ((scan-exp x (k-head ! . args)) (k-head x . args)))) (do-tl (syntax-rules (!) ((do-tl processed-hd fn () (k-head ! . args)) (k-head (processed-hd) . args)) ((do-tl processed-hd fn old-tl k) (fn old-tl (do-cons ! processed-hd k))))) (do-cons (syntax-rules (!) ((do-cons processed-tl processed-hd (k-head ! . args)) (k-head (processed-hd . processed-tl) . args)))) (do-wrap (syntax-rules (!) ((do-wrap val fn (k-head ! . args)) (k-head (fn val) . args)))) (do-finish (syntax-rules () ((do-finish new-body) new-body))) (scan-lit-lst ; scan literal list (syntax-rules (quote unquote unquote-splicing !) ((scan-lit-lst '() (k-head ! . args)) (k-head '() . args)) ((scan-lit-lst (quote (hd . tl)) k) (do-tl quote scan-lit-lst ((hd . tl)) k)) ((scan-lit-lst (unquote x) k) (scan-exp x (do-wrap ! unquote k))) ((scan-lit-lst (unquote-splicing x) k) (scan-exp x (do-wrap ! unquote-splicing k))) ((scan-lit-lst (quote x) (k-head ! . args)) (k-head ,(if (string? (quote x)) (string->symbol (quote x)) (quote x)) . args)) ((scan-lit-lst (hd . tl) k) (scan-lit-lst hd (do-tl ! scan-lit-lst tl k))) ((scan-lit-lst x (k-head ! . args)) (k-head x . args)))) ) (scan-exp ?body (do-finish !)))) ((run-test body ...) (begin (run-test "scan-exp" body) ...)) )) ; Overwrite the above macro to switch the tests off ; (define-macro (run-test selector node expected-result) #f) ; Location path, full form: child::para ; Location path, abbreviated form: para ; selects the para element children of the context node (let ((tree '(elem (@) (para (@) "para") (br (@)) "cdata" (para (@) "second par")) ) (expected '((para (@) "para") (para (@) "second par"))) ) (run-test (select-kids (node-typeof? 'para)) tree expected) (run-test (sxpath '(para)) tree expected) ) ; Location path, full form: child::* ; Location path, abbreviated form: * ; selects all element children of the context node (let ((tree '(elem (@) (para (@) "para") (br (@)) "cdata" (para "second par")) ) (expected '((para (@) "para") (br (@)) (para "second par"))) ) (run-test (select-kids (node-typeof? '*)) tree expected) (run-test (sxpath '(*)) tree expected) ) ; Location path, full form: child::text() ; Location path, abbreviated form: text() ; selects all text node children of the context node (let ((tree '(elem (@) (para (@) "para") (br (@)) "cdata" (para "second par")) ) (expected '("cdata")) ) (run-test (select-kids (node-typeof? '*text*)) tree expected) (run-test (sxpath '(*text*)) tree expected) ) ; Location path, full form: child::node() ; Location path, abbreviated form: node() ; selects all the children of the context node, whatever their node type (let* ((tree '(elem (@) (para (@) "para") (br (@)) "cdata" (para "second par")) ) (expected (cdr tree)) ) (run-test (select-kids (node-typeof? '*any*)) tree expected) (run-test (sxpath '(*any*)) tree expected) ) ; Location path, full form: child::*/child::para ; Location path, abbreviated form: */para ; selects all para grandchildren of the context node (let ((tree '(elem (@) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para "third para"))) ) (expected '((para "third para"))) ) (run-test (node-join (select-kids (node-typeof? '*)) (select-kids (node-typeof? 'para))) tree expected) (run-test (sxpath '(* para)) tree expected) ) ; Location path, full form: attribute::name ; Location path, abbreviated form: @name ; selects the 'name' attribute of the context node (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para (@) "second par") (div (@ (name "aa")) (para (@) "third para"))) ) (expected '((name "elem"))) ) (run-test (node-join (select-kids (node-typeof? '@)) (select-kids (node-typeof? 'name))) tree expected) (run-test (sxpath '(@ name)) tree expected) ) ; Location path, full form: attribute::* ; Location path, abbreviated form: @* ; selects all the attributes of the context node (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) (expected '((name "elem") (id "idz"))) ) (run-test (node-join (select-kids (node-typeof? '@)) (select-kids (node-typeof? '*))) tree expected) (run-test (sxpath '(@ *)) tree expected) ) ; Location path, full form: descendant::para ; Location path, abbreviated form: .//para ; selects the para element descendants of the context node (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) (expected '((para (@) "para") (para "second par") (para (@) "third para"))) ) (run-test (node-closure (node-typeof? 'para)) tree expected) (run-test (sxpath '(// para)) tree expected) ) ; Location path, full form: self::para ; Location path, abbreviated form: _none_ ; selects the context node if it is a para element; otherwise selects nothing (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) ) (run-test (node-self (node-typeof? 'para)) tree '()) (run-test (node-self (node-typeof? 'elem)) tree (list tree)) ) ; Location path, full form: descendant-or-self::node() ; Location path, abbreviated form: // ; selects the context node, all the children (including attribute nodes) ; of the context node, and all the children of all the (element) ; descendants of the context node. ; This is _almost_ a powerset of the context node. (let* ((tree '(para (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) (expected (cons tree (append (cdr tree) '((@) "para" (@) "second par" (@ (name "aa")) (para (@) "third para") (@) "third para")))) ) (run-test (node-or (node-self (node-typeof? '*any*)) (node-closure (node-typeof? '*any*))) tree expected) (run-test (sxpath '(//)) tree expected) ) ; Location path, full form: ancestor::div ; Location path, abbreviated form: _none_ ; selects all div ancestors of the context node ; This Location expression is equivalent to the following: ; /descendant-or-self::div[descendant::node() = curr_node] ; This shows that the ancestor:: axis is actually redundant. Still, ; it can be emulated as the following SXPath expression demonstrates. ; The insight behind "ancestor::div" -- selecting all "div" ancestors ; of the current node -- is ; S[ancestor::div] context_node = ; { y | y=subnode*(root), context_node=subnode(subnode*(y)), ; isElement(y), name(y) = "div" } ; We observe that ; { y | y=subnode*(root), pred(y) } ; can be expressed in SXPath as ; ((node-or (node-self pred) (node-closure pred)) root-node) ; The composite predicate 'isElement(y) & name(y) = "div"' corresponds to ; (node-self (node-typeof? 'div)) in SXPath. Finally, filter ; context_node=subnode(subnode*(y)) is tantamount to ; (node-closure (node-eq? context-node)), whereas node-reduce denotes the ; the composition of converters-predicates in the filtering context. (let* ((root '(div (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para (@) "second par") (div (@ (name "aa")) (para (@) "third para")))) (context-node ; /descendant::any()[child::text() == "third para"] (car ((node-closure (select-kids (node-equal? "third para"))) root))) (pred (node-reduce (node-self (node-typeof? 'div)) (node-closure (node-eq? context-node)) )) ) (run-test (node-or (node-self pred) (node-closure pred)) root (cons root '((div (@ (name "aa")) (para (@) "third para"))))) ) ; Location path, full form: child::div/descendant::para ; Location path, abbreviated form: div//para ; selects the para element descendants of the div element ; children of the context node (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para") (div (para "fourth para")))) ) (expected '((para (@) "third para") (para "fourth para"))) ) (run-test (node-join (select-kids (node-typeof? 'div)) (node-closure (node-typeof? 'para))) tree expected) (run-test (sxpath '(div // para)) tree expected) ) ; Location path, full form: /descendant::olist/child::item ; Location path, abbreviated form: //olist/item ; selects all the item elements that have an olist parent (which is not root) ; and that are in the same document as the context node ; See the following test. ; Location path, full form: /descendant::td/attribute::align ; Location path, abbreviated form: //td/@align ; Selects 'align' attributes of all 'td' elements in tree1 (let ((tree tree1) (expected '((align "right") (align "center") (align "center") (align "center")) )) (run-test (node-join (node-closure (node-typeof? 'td)) (select-kids (node-typeof? '@)) (select-kids (node-typeof? 'align))) tree expected) (run-test (sxpath '(// td @ align)) tree expected) ) ; Location path, full form: /descendant::td[attribute::align] ; Location path, abbreviated form: //td[@align] ; Selects all td elements that have an attribute 'align' in tree1 (let ((tree tree1) (expected '((td (@ (align "right")) "Talks ") (td (@ (align "center")) " = ") (td (@ (align "center")) " = ") (td (@ (align "center")) " = ")) )) (run-test (node-reduce (node-closure (node-typeof? 'td)) (filter (node-join (select-kids (node-typeof? '@)) (select-kids (node-typeof? 'align))))) tree expected) (run-test (sxpath `(// td ,(node-self (sxpath '(@ align))))) tree expected) (run-test (sxpath '(// (td (@ align)))) tree expected) (run-test (sxpath '(// ((td) (@ align)))) tree expected) ; note! (sxpath ...) is a converter. Therefore, it can be used ; as any other converter, for example, in the full-form SXPath. ; Thus we can mix the full and abbreviated form SXPath's freely. (run-test (node-reduce (node-closure (node-typeof? 'td)) (filter (sxpath '(@ align)))) tree expected) ) ; Location path, full form: /descendant::td[attribute::align = "right"] ; Location path, abbreviated form: //td[@align = "right"] ; Selects all td elements that have an attribute align = "right" in tree1 (let ((tree tree1) (expected '((td (@ (align "right")) "Talks ")) )) (run-test (node-reduce (node-closure (node-typeof? 'td)) (filter (node-join (select-kids (node-typeof? '@)) (select-kids (node-equal? '(align "right")))))) tree expected) (run-test (sxpath '(// (td (@ (equal? (align "right")))))) tree expected) ) ; Location path, full form: child::para[position()=1] ; Location path, abbreviated form: para[1] ; selects the first para child of the context node (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) (expected '((para (@) "para")) )) (run-test (node-reduce (select-kids (node-typeof? 'para)) (node-pos 1)) tree expected) (run-test (sxpath '((para 1))) tree expected) ) ; Location path, full form: child::para[position()=last()] ; Location path, abbreviated form: para[last()] ; selects the last para child of the context node (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) (expected '((para "second par")) )) (run-test (node-reduce (select-kids (node-typeof? 'para)) (node-pos -1)) tree expected) (run-test (sxpath '((para -1))) tree expected) ) ; Illustrating the following Note of Sec 2.5 of XPath: ; "NOTE: The location path //para[1] does not mean the same as the ; location path /descendant::para[1]. The latter selects the first ; descendant para element; the former selects all descendant para ; elements that are the first para children of their parents." (let ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) ) (run-test (node-reduce ; /descendant::para[1] in SXPath (node-closure (node-typeof? 'para)) (node-pos 1)) tree '((para (@) "para"))) (run-test (sxpath '(// (para 1))) tree '((para (@) "para") (para (@) "third para"))) ) ; Location path, full form: parent::node() ; Location path, abbreviated form: .. ; selects the parent of the context node. The context node may be ; an attribute node! ; For the last test: ; Location path, full form: parent::*/attribute::name ; Location path, abbreviated form: ../@name ; Selects the name attribute of the parent of the context node (let* ((tree '(elem (@ (name "elem") (id "idz")) (para (@) "para") (br (@)) "cdata" (para "second par") (div (@ (name "aa")) (para (@) "third para"))) ) (para1 ; the first para node (car ((sxpath '(para)) tree))) (para3 ; the third para node (car ((sxpath '(div para)) tree))) (div ; div node (car ((sxpath '(// div)) tree))) ) (run-test (node-parent tree) para1 (list tree)) (run-test (node-parent tree) para3 (list div)) (run-test ; checking the parent of an attribute node (node-parent tree) ((sxpath '(@ name)) div) (list div)) (run-test (node-join (node-parent tree) (select-kids (node-typeof? '@)) (select-kids (node-typeof? 'name))) para3 '((name "aa"))) (run-test (sxpath `(,(node-parent tree) @ name)) para3 '((name "aa"))) ) ; Location path, full form: following-sibling::chapter[position()=1] ; Location path, abbreviated form: none ; selects the next chapter sibling of the context node ; The path is equivalent to ; let cnode = context-node ; in ; parent::* / child::chapter [take-after node_eq(self::*,cnode)] ; [position()=1] (let* ((tree '(document (preface "preface") (chapter (@ (id "one")) "Chap 1 text") (chapter (@ (id "two")) "Chap 2 text") (chapter (@ (id "three")) "Chap 3 text") (chapter (@ (id "four")) "Chap 4 text") (epilogue "Epilogue text") (appendix (@ (id "A")) "App A text") (References "References")) ) (a-node ; to be used as a context node (car ((sxpath '(// (chapter (@ (equal? (id "two")))))) tree))) (expected '((chapter (@ (id "three")) "Chap 3 text"))) ) (run-test (node-reduce (node-join (node-parent tree) (select-kids (node-typeof? 'chapter))) (take-after (node-eq? a-node)) (node-pos 1) ) a-node expected) ) ; preceding-sibling::chapter[position()=1] ; selects the previous chapter sibling of the context node ; The path is equivalent to ; let cnode = context-node ; in ; parent::* / child::chapter [take-until node_eq(self::*,cnode)] ; [position()=-1] (let* ((tree '(document (preface "preface") (chapter (@ (id "one")) "Chap 1 text") (chapter (@ (id "two")) "Chap 2 text") (chapter (@ (id "three")) "Chap 3 text") (chapter (@ (id "four")) "Chap 4 text") (epilogue "Epilogue text") (appendix (@ (id "A")) "App A text") (References "References")) ) (a-node ; to be used as a context node (car ((sxpath '(// (chapter (@ (equal? (id "three")))))) tree))) (expected '((chapter (@ (id "two")) "Chap 2 text"))) ) (run-test (node-reduce (node-join (node-parent tree) (select-kids (node-typeof? 'chapter))) (take-until (node-eq? a-node)) (node-pos -1) ) a-node expected) ) ; /descendant::figure[position()=42] ; selects the forty-second figure element in the document ; See the next example, which is more general. ; Location path, full form: ; child::table/child::tr[position()=2]/child::td[position()=3] ; Location path, abbreviated form: table/tr[2]/td[3] ; selects the third td of the second tr of the table (let ((tree ((node-closure (node-typeof? 'p)) tree1)) (expected '((td " data + control")) )) (run-test (node-join (select-kids (node-typeof? 'table)) (node-reduce (select-kids (node-typeof? 'tr)) (node-pos 2)) (node-reduce (select-kids (node-typeof? 'td)) (node-pos 3))) tree expected) (run-test (sxpath '(table (tr 2) (td 3))) tree expected) ) ; Location path, full form: ; child::para[attribute::type='warning'][position()=5] ; Location path, abbreviated form: para[@type='warning'][5] ; selects the fifth para child of the context node that has a type ; attribute with value warning (let ((tree '(chapter (para "para1") (para (@ (type "warning")) "para 2") (para (@ (type "warning")) "para 3") (para (@ (type "warning")) "para 4") (para (@ (type "warning")) "para 5") (para (@ (type "warning")) "para 6")) ) (expected '((para (@ (type "warning")) "para 6")) )) (run-test (node-reduce (select-kids (node-typeof? 'para)) (filter (node-join (select-kids (node-typeof? '@)) (select-kids (node-equal? '(type "warning"))))) (node-pos 5)) tree expected) (run-test (sxpath '( (((para (@ (equal? (type "warning"))))) 5 ) )) tree expected) (run-test (sxpath '( (para (@ (equal? (type "warning"))) 5 ) )) tree expected) ) ; Location path, full form: ; child::para[position()=5][attribute::type='warning'] ; Location path, abbreviated form: para[5][@type='warning'] ; selects the fifth para child of the context node if that child has a 'type' ; attribute with value warning (let ((tree '(chapter (para "para1") (para (@ (type "warning")) "para 2") (para (@ (type "warning")) "para 3") (para (@ (type "warning")) "para 4") (para (@ (type "warning")) "para 5") (para (@ (type "warning")) "para 6")) ) (expected '((para (@ (type "warning")) "para 5")) )) (run-test (node-reduce (select-kids (node-typeof? 'para)) (node-pos 5) (filter (node-join (select-kids (node-typeof? '@)) (select-kids (node-equal? '(type "warning")))))) tree expected) (run-test (sxpath '( (( (para 5)) (@ (equal? (type "warning")))))) tree expected) (run-test (sxpath '( (para 5 (@ (equal? (type "warning")))) )) tree expected) ) ; Location path, full form: ; child::*[self::chapter or self::appendix] ; Location path, semi-abbreviated form: *[self::chapter or self::appendix] ; selects the chapter and appendix children of the context node (let ((tree '(document (preface "preface") (chapter (@ (id "one")) "Chap 1 text") (chapter (@ (id "two")) "Chap 2 text") (chapter (@ (id "three")) "Chap 3 text") (epilogue "Epilogue text") (appendix (@ (id "A")) "App A text") (References "References")) ) (expected '((chapter (@ (id "one")) "Chap 1 text") (chapter (@ (id "two")) "Chap 2 text") (chapter (@ (id "three")) "Chap 3 text") (appendix (@ (id "A")) "App A text")) )) (run-test (node-join (select-kids (node-typeof? '*)) (filter (node-or (node-self (node-typeof? 'chapter)) (node-self (node-typeof? 'appendix))))) tree expected) (run-test (sxpath `(* ,(node-or (node-self (node-typeof? 'chapter)) (node-self (node-typeof? 'appendix))))) tree expected) ) ; Location path, full form: child::chapter[child::title='Introduction'] ; Location path, abbreviated form: chapter[title = 'Introduction'] ; selects the chapter children of the context node that have one or more ; title children with string-value equal to Introduction ; See a similar example: //td[@align = "right"] above. ; Location path, full form: child::chapter[child::title] ; Location path, abbreviated form: chapter[title] ; selects the chapter children of the context node that have one or ; more title children ; See a similar example //td[@align] above. (cerr nl "Example with tree3: extracting the first lines of every stanza" nl) (let ((tree tree3) (expected '("Let us go then, you and I," "In the room the women come and go") )) (run-test (node-join (node-closure (node-typeof? 'stanza)) (node-reduce (select-kids (node-typeof? 'line)) (node-pos 1)) (select-kids (node-typeof? '*text*))) tree expected) (run-test (sxpath '(// stanza (line 1) *text*)) tree expected) ) PK�����/F2];Aa'��a'��!��sxml/upstream/SXML-tree-trans.scmnu�[��������; XML/HTML processing in Scheme ; SXML expression tree transformers ; ; IMPORT ; A prelude appropriate for your Scheme system ; (myenv-bigloo.scm, myenv-mit.scm, etc.) ; ; EXPORT ; (provide SRV:send-reply ; post-order pre-post-order replace-range) ; ; See vSXML-tree-trans.scm for the validation code, which also ; serves as usage examples. ; ; $Id: SXML-tree-trans.scm,v 1.6 2003/04/25 19:16:15 oleg Exp $ ; Output the 'fragments' ; The fragments are a list of strings, characters, ; numbers, thunks, #f, #t -- and other fragments. ; The function traverses the tree depth-first, writes out ; strings and characters, executes thunks, and ignores ; #f and '(). ; The function returns #t if anything was written at all; ; otherwise the result is #f ; If #t occurs among the fragments, it is not written out ; but causes the result of SRV:send-reply to be #t (define (SRV:send-reply . fragments) (let loop ((fragments fragments) (result #f)) (cond ((null? fragments) result) ((not (car fragments)) (loop (cdr fragments) result)) ((null? (car fragments)) (loop (cdr fragments) result)) ((eq? #t (car fragments)) (loop (cdr fragments) #t)) ((pair? (car fragments)) (loop (cdr fragments) (loop (car fragments) result))) ((procedure? (car fragments)) ((car fragments)) (loop (cdr fragments) #t)) (else (display (car fragments)) (loop (cdr fragments) #t))))) ;------------------------------------------------------------------------ ; Traversal of an SXML tree or a grove: ; a <Node> or a <Nodelist> ; ; A <Node> and a <Nodelist> are mutually-recursive datatypes that ; underlie the SXML tree: ; <Node> ::= (name . <Nodelist>) | "text string" ; An (ordered) set of nodes is just a list of the constituent nodes: ; <Nodelist> ::= (<Node> ...) ; Nodelists, and Nodes other than text strings are both lists. A ; <Nodelist> however is either an empty list, or a list whose head is ; not a symbol (an atom in general). A symbol at the head of a node is ; either an XML name (in which case it's a tag of an XML element), or ; an administrative name such as '@'. ; See SXPath.scm and SSAX.scm for more information on SXML. ; Pre-Post-order traversal of a tree and creation of a new tree: ; pre-post-order:: <tree> x <bindings> -> <new-tree> ; where ; <bindings> ::= (<binding> ...) ; <binding> ::= (<trigger-symbol> *preorder* . <handler>) | ; (<trigger-symbol> *macro* . <handler>) | ; (<trigger-symbol> <new-bindings> . <handler>) | ; (<trigger-symbol> . <handler>) ; <trigger-symbol> ::= XMLname | *text* | *default* ; <handler> :: <trigger-symbol> x [<tree>] -> <new-tree> ; ; The pre-post-order function visits the nodes and nodelists ; pre-post-order (depth-first). For each <Node> of the form (name ; <Node> ...) it looks up an association with the given 'name' among ; its <bindings>. If failed, pre-post-order tries to locate a ; *default* binding. It's an error if the latter attempt fails as ; well. Having found a binding, the pre-post-order function first ; checks to see if the binding is of the form ; (<trigger-symbol> *preorder* . <handler>) ; If it is, the handler is 'applied' to the current node. Otherwise, ; the pre-post-order function first calls itself recursively for each ; child of the current node, with <new-bindings> prepended to the ; <bindings> in effect. The result of these calls is passed to the ; <handler> (along with the head of the current <Node>). To be more ; precise, the handler is _applied_ to the head of the current node ; and its processed children. The result of the handler, which should ; also be a <tree>, replaces the current <Node>. If the current <Node> ; is a text string or other atom, a special binding with a symbol ; *text* is looked up. ; ; A binding can also be of a form ; (<trigger-symbol> *macro* . <handler>) ; This is equivalent to *preorder* described above. However, the result ; is re-processed again, with the current stylesheet. (define (pre-post-order tree bindings) (let* ((default-binding (assq '*default* bindings)) (text-binding (or (assq '*text* bindings) default-binding)) (text-handler ; Cache default and text bindings (and text-binding (if (procedure? (cdr text-binding)) (cdr text-binding) (cddr text-binding))))) (let loop ((tree tree)) (cond ((null? tree) '()) ((not (pair? tree)) (let ((trigger '*text*)) (if text-handler (text-handler trigger tree) (error "Unknown binding for " trigger " and no default")))) ((not (symbol? (car tree))) (map loop tree)) ; tree is a nodelist (else ; tree is an SXML node (let* ((trigger (car tree)) (binding (or (assq trigger bindings) default-binding))) (cond ((not binding) (error "Unknown binding for " trigger " and no default")) ((not (pair? (cdr binding))) ; must be a procedure: handler (apply (cdr binding) trigger (map loop (cdr tree)))) ((eq? '*preorder* (cadr binding)) (apply (cddr binding) tree)) ((eq? '*macro* (cadr binding)) (loop (apply (cddr binding) tree))) (else ; (cadr binding) is a local binding (apply (cddr binding) trigger (pre-post-order (cdr tree) (append (cadr binding) bindings))) )))))))) ; post-order is a strict subset of pre-post-order without *preorder* ; (let alone *macro*) traversals. ; Now pre-post-order is actually faster than the old post-order. ; The function post-order is deprecated and is aliased below for ; backward compatibility. (define post-order pre-post-order) ;------------------------------------------------------------------------ ; Extended tree fold ; tree = atom | (node-name tree ...) ; ; foldts fdown fup fhere seed (Leaf str) = fhere seed str ; foldts fdown fup fhere seed (Nd kids) = ; fup seed $ foldl (foldts fdown fup fhere) (fdown seed) kids ; procedure fhere: seed -> atom -> seed ; procedure fdown: seed -> node -> seed ; procedure fup: parent-seed -> last-kid-seed -> node -> seed ; foldts returns the final seed (define (foldts fdown fup fhere seed tree) (cond ((null? tree) seed) ((not (pair? tree)) ; An atom (fhere seed tree)) (else (let loop ((kid-seed (fdown seed tree)) (kids (cdr tree))) (if (null? kids) (fup seed kid-seed tree) (loop (foldts fdown fup fhere kid-seed (car kids)) (cdr kids))))))) ;------------------------------------------------------------------------ ; Traverse a forest depth-first and cut/replace ranges of nodes. ; ; The nodes that define a range don't have to have the same immediate ; parent, don't have to be on the same level, and the end node of a ; range doesn't even have to exist. A replace-range procedure removes ; nodes from the beginning node of the range up to (but not including) ; the end node of the range. In addition, the beginning node of the ; range can be replaced by a node or a list of nodes. The range of ; nodes is cut while depth-first traversing the forest. If all ; branches of the node are cut a node is cut as well. The procedure ; can cut several non-overlapping ranges from a forest. ; replace-range:: BEG-PRED x END-PRED x FOREST -> FOREST ; where ; type FOREST = (NODE ...) ; type NODE = Atom | (Name . FOREST) | FOREST ; ; The range of nodes is specified by two predicates, beg-pred and end-pred. ; beg-pred:: NODE -> #f | FOREST ; end-pred:: NODE -> #f | FOREST ; The beg-pred predicate decides on the beginning of the range. The node ; for which the predicate yields non-#f marks the beginning of the range ; The non-#f value of the predicate replaces the node. The value can be a ; list of nodes. The replace-range procedure then traverses the tree and skips ; all the nodes, until the end-pred yields non-#f. The value of the end-pred ; replaces the end-range node. The new end node and its brothers will be ; re-scanned. ; The predicates are evaluated pre-order. We do not descend into a node that ; is marked as the beginning of the range. (define (replace-range beg-pred end-pred forest) ; loop forest keep? new-forest ; forest is the forest to traverse ; new-forest accumulates the nodes we will keep, in the reverse ; order ; If keep? is #t, keep the curr node if atomic. If the node is not atomic, ; traverse its children and keep those that are not in the skip range. ; If keep? is #f, skip the current node if atomic. Otherwise, ; traverse its children. If all children are skipped, skip the node ; as well. (define (loop forest keep? new-forest) (if (null? forest) (values (reverse new-forest) keep?) (let ((node (car forest))) (if keep? (cond ; accumulate mode ((beg-pred node) => ; see if the node starts the skip range (lambda (repl-branches) ; if so, skip/replace the node (loop (cdr forest) #f (append (reverse repl-branches) new-forest)))) ((not (pair? node)) ; it's an atom, keep it (loop (cdr forest) keep? (cons node new-forest))) (else (let*-values (((node?) (symbol? (car node))) ; or is it a nodelist? ((new-kids keep?) ; traverse its children (loop (if node? (cdr node) node) #t '()))) (loop (cdr forest) keep? (cons (if node? (cons (car node) new-kids) new-kids) new-forest))))) ; skip mode (cond ((end-pred node) => ; end the skip range (lambda (repl-branches) ; repl-branches will be re-scanned (loop (append repl-branches (cdr forest)) #t new-forest))) ((not (pair? node)) ; it's an atom, skip it (loop (cdr forest) keep? new-forest)) (else (let*-values (((node?) (symbol? (car node))) ; or is it a nodelist? ((new-kids keep?) ; traverse its children (loop (if node? (cdr node) node) #f '()))) (loop (cdr forest) keep? (if (or keep? (pair? new-kids)) (cons (if node? (cons (car node) new-kids) new-kids) new-forest) new-forest) ; if all kids are skipped )))))))) ; skip the node too (let*-values (((new-forest keep?) (loop forest #t '()))) new-forest)) PK�����/F2];7Qs��s����sxml/upstream/assert.scmnu�[��������; ; syntax: assert ?expr ?expr ... [report: ?r-exp ?r-exp ...] ; ; If (and ?expr ?expr ...) evaluates to anything but #f, the result ; is the value of that expression. ; If (and ?expr ?expr ...) evaluates to #f, an error is reported. ; The error message will show the failed expressions, as well ; as the values of selected variables (or expressions, in general). ; The user may explicitly specify the expressions whose ; values are to be printed upon assertion failure -- as ?r-exp that ; follow the identifier 'report:' ; Typically, ?r-exp is either a variable or a string constant. ; If the user specified no ?r-exp, the values of variables that are ; referenced in ?expr will be printed upon the assertion failure. (define-syntax assert (syntax-rules (report:) ((assert "doit" (expr ...) (r-exp ...)) (cond ((and expr ...) => (lambda (x) x)) (else (error "assertion failure: ~a" (list '(and expr ...) r-exp ...))))) ((assert "collect" (expr ...)) (assert "doit" (expr ...) ())) ((assert "collect" (expr ...) report: r-exp ...) (assert "doit" (expr ...) (r-exp ...))) ((assert "collect" (expr ...) expr1 stuff ...) (assert "collect" (expr ... expr1) stuff ...)) ((assert stuff ...) (assert "collect" () stuff ...)))) (define-syntax assure (syntax-rules () ((assure exp error-msg) (assert exp report: error-msg))))PK�����/F2]jQ5��Q5����sxml/upstream/input-parse.scmnu�[��������;**************************************************************************** ; Simple Parsing of input ; ; The following simple functions surprisingly often suffice to parse ; an input stream. They either skip, or build and return tokens, ; according to inclusion or delimiting semantics. The list of ; characters to expect, include, or to break at may vary from one ; invocation of a function to another. This allows the functions to ; easily parse even context-sensitive languages. ; ; EOF is generally frowned on, and thrown up upon if encountered. ; Exceptions are mentioned specifically. The list of expected characters ; (characters to skip until, or break-characters) may include an EOF ; "character", which is to be coded as symbol *eof* ; ; The input stream to parse is specified as a PORT, which is usually ; the last (and optional) argument. It defaults to the current input ; port if omitted. ; ; IMPORT ; This package relies on a function parser-error, which must be defined ; by a user of the package. The function has the following signature: ; parser-error PORT MESSAGE SPECIALISING-MSG* ; Many procedures of this package call parser-error to report a parsing ; error. The first argument is a port, which typically points to the ; offending character or its neighborhood. Most of the Scheme systems ; let the user query a PORT for the current position. MESSAGE is the ; description of the error. Other arguments supply more details about ; the problem. ; myenv.scm, myenv-bigloo.scm or a similar prelude is assumed. ; From SRFI-13, string-concatenate-reverse ; If a particular implementation lacks SRFI-13 support, please ; include the file srfi-13-local.scm ; ; $Id: input-parse.scm,v 1.7 2004/07/07 16:02:31 sperber Exp $ ;------------------------------------------------------------------------ ; -- procedure+: peek-next-char [PORT] ; advances to the next character in the PORT and peeks at it. ; This function is useful when parsing LR(1)-type languages ; (one-char-read-ahead). ; The optional argument PORT defaults to the current input port. (define-opt (peek-next-char (optional (port (current-input-port)))) (read-char port) (peek-char port)) ;------------------------------------------------------------------------ ; -- procedure+: assert-curr-char CHAR-LIST STRING [PORT] ; Reads a character from the PORT and looks it up ; in the CHAR-LIST of expected characters ; If the read character was found among expected, it is returned ; Otherwise, the procedure writes a nasty message using STRING ; as a comment, and quits. ; The optional argument PORT defaults to the current input port. ; (define-opt (assert-curr-char expected-chars comment (optional (port (current-input-port)))) (let ((c (read-char port))) (if (memv c expected-chars) c (parser-error port "Wrong character " c " (0x" (if (eof-object? c) "*eof*" (number->string (char->integer c) 16)) ") " comment ". " expected-chars " expected")))) ; -- procedure+: skip-until CHAR-LIST [PORT] ; Reads and skips characters from the PORT until one of the break ; characters is encountered. This break character is returned. ; The break characters are specified as the CHAR-LIST. This list ; may include EOF, which is to be coded as a symbol *eof* ; ; -- procedure+: skip-until NUMBER [PORT] ; Skips the specified NUMBER of characters from the PORT and returns #f ; ; The optional argument PORT defaults to the current input port. (define-opt (skip-until arg (optional (port (current-input-port))) ) (cond ((number? arg) ; skip 'arg' characters (do ((i arg (dec i))) ((not (positive? i)) #f) (if (eof-object? (read-char port)) (parser-error port "Unexpected EOF while skipping " arg " characters")))) (else ; skip until break-chars (=arg) (let loop ((c (read-char port))) (cond ((memv c arg) c) ((eof-object? c) (if (memq '*eof* arg) c (parser-error port "Unexpected EOF while skipping until " arg))) (else (loop (read-char port)))))))) ; -- procedure+: skip-while CHAR-LIST [PORT] ; Reads characters from the PORT and disregards them, ; as long as they are mentioned in the CHAR-LIST. ; The first character (which may be EOF) peeked from the stream ; that is NOT a member of the CHAR-LIST is returned. This character ; is left on the stream. ; The optional argument PORT defaults to the current input port. (define-opt (skip-while skip-chars (optional (port (current-input-port))) ) (do ((c (peek-char port) (peek-char port))) ((not (memv c skip-chars)) c) (read-char port))) ; whitespace const ;------------------------------------------------------------------------ ; Stream tokenizers ; -- procedure+: ; next-token PREFIX-CHAR-LIST BREAK-CHAR-LIST [COMMENT-STRING] [PORT] ; skips any number of the prefix characters (members of the ; PREFIX-CHAR-LIST), if any, and reads the sequence of characters ; up to (but not including) a break character, one of the ; BREAK-CHAR-LIST. ; The string of characters thus read is returned. ; The break character is left on the input stream ; The list of break characters may include EOF, which is to be coded as ; a symbol *eof*. Otherwise, EOF is fatal, generating an error message ; including a specified COMMENT-STRING (if any) ; ; The optional argument PORT defaults to the current input port. ; ; Note: since we can't tell offhand how large the token being read is ; going to be, we make a guess, pre-allocate a string, and grow it by ; quanta if necessary. The quantum is always the length of the string ; before it was extended the last time. Thus the algorithm does ; a Fibonacci-type extension, which has been proven optimal. ; Note, explicit port specification in read-char, peek-char helps. ; Procedure: input-parse:init-buffer ; returns an initial buffer for next-token* procedures. ; The input-parse:init-buffer may allocate a new buffer per each invocation: ; (define (input-parse:init-buffer) (make-string 32)) ; Size 32 turns out to be fairly good, on average. ; That policy is good only when a Scheme system is multi-threaded with ; preemptive scheduling, or when a Scheme system supports shared substrings. ; In all the other cases, it's better for input-parse:init-buffer to ; return the same static buffer. next-token* functions return a copy ; (a substring) of accumulated data, so the same buffer can be reused. ; We shouldn't worry about an incoming token being too large: ; next-token will use another chunk automatically. Still, ; the best size for the static buffer is to allow most of the tokens to fit in. ; Using a static buffer _dramatically_ reduces the amount of produced garbage ; (e.g., during XML parsing). (define input-parse:init-buffer (let ((buffer (make-string 512))) (lambda () buffer))) ; See a better version below (define-opt (next-token-old prefix-skipped-chars break-chars (optional (comment "") (port (current-input-port))) ) (let* ((buffer (input-parse:init-buffer)) (curr-buf-len (string-length buffer)) (quantum curr-buf-len)) (let loop ((i 0) (c (skip-while prefix-skipped-chars port))) (cond ((memv c break-chars) (substring buffer 0 i)) ((eof-object? c) (if (memq '*eof* break-chars) (substring buffer 0 i) ; was EOF expected? (parser-error port "EOF while reading a token " comment))) (else (if (>= i curr-buf-len) ; make space for i-th char in buffer (begin ; -> grow the buffer by the quantum (set! buffer (string-append buffer (make-string quantum))) (set! quantum curr-buf-len) (set! curr-buf-len (string-length buffer)))) (string-set! buffer i c) (read-char port) ; move to the next char (loop (inc i) (peek-char port)) ))))) ; A better version of next-token, which accumulates the characters ; in chunks, and later on reverse-concatenates them, using ; SRFI-13 if available. ; The overhead of copying characters is only 100% (or even smaller: bulk ; string copying might be well-optimised), compared to the (hypothetical) ; circumstance if we had known the size of the token beforehand. ; For small tokens, the code performs just as above. For large ; tokens, we expect an improvement. Note, the code also has no ; assignments. ; See next-token-comp.scm (define-opt (next-token prefix-skipped-chars break-chars (optional (comment "") (port (current-input-port))) ) (let outer ((buffer (input-parse:init-buffer)) (filled-buffer-l '()) (c (skip-while prefix-skipped-chars port))) (let ((curr-buf-len (string-length buffer))) (let loop ((i 0) (c c)) (cond ((memv c break-chars) (if (null? filled-buffer-l) (substring buffer 0 i) (string-concatenate-reverse filled-buffer-l buffer i))) ((eof-object? c) (if (memq '*eof* break-chars) ; was EOF expected? (if (null? filled-buffer-l) (substring buffer 0 i) (string-concatenate-reverse filled-buffer-l buffer i)) (parser-error port "EOF while reading a token " comment))) ((>= i curr-buf-len) (outer (make-string curr-buf-len) (cons buffer filled-buffer-l) c)) (else (string-set! buffer i c) (read-char port) ; move to the next char (loop (inc i) (peek-char port)))))))) ; -- procedure+: next-token-of INC-CHARSET [PORT] ; Reads characters from the PORT that belong to the list of characters ; INC-CHARSET. The reading stops at the first character which is not ; a member of the set. This character is left on the stream. ; All the read characters are returned in a string. ; ; -- procedure+: next-token-of PRED [PORT] ; Reads characters from the PORT for which PRED (a procedure of one ; argument) returns non-#f. The reading stops at the first character ; for which PRED returns #f. That character is left on the stream. ; All the results of evaluating of PRED up to #f are returned in a ; string. ; ; PRED is a procedure that takes one argument (a character ; or the EOF object) and returns a character or #f. The returned ; character does not have to be the same as the input argument ; to the PRED. For example, ; (next-token-of (lambda (c) ; (cond ((eof-object? c) #f) ; ((char-alphabetic? c) (char-downcase c)) ; (else #f)))) ; will try to read an alphabetic token from the current ; input port, and return it in lower case. ; ; The optional argument PORT defaults to the current input port. ; ; This procedure is similar to next-token but only it implements ; an inclusion rather than delimiting semantics. (define-opt (next-token-of incl-list/pred (optional (port (current-input-port))) ) (let* ((buffer (input-parse:init-buffer)) (curr-buf-len (string-length buffer))) (if (procedure? incl-list/pred) (let outer ((buffer buffer) (filled-buffer-l '())) (let loop ((i 0)) (if (>= i curr-buf-len) ; make sure we have space (outer (make-string curr-buf-len) (cons buffer filled-buffer-l)) (let ((c (incl-list/pred (peek-char port)))) (if c (begin (string-set! buffer i c) (read-char port) ; move to the next char (loop (inc i))) ; incl-list/pred decided it had had enough (if (null? filled-buffer-l) (substring buffer 0 i) (string-concatenate-reverse filled-buffer-l buffer i))))))) ; incl-list/pred is a list of allowed characters (let outer ((buffer buffer) (filled-buffer-l '())) (let loop ((i 0)) (if (>= i curr-buf-len) ; make sure we have space (outer (make-string curr-buf-len) (cons buffer filled-buffer-l)) (let ((c (peek-char port))) (cond ((not (memv c incl-list/pred)) (if (null? filled-buffer-l) (substring buffer 0 i) (string-concatenate-reverse filled-buffer-l buffer i))) (else (string-set! buffer i c) (read-char port) ; move to the next char (loop (inc i)))))))) ))) ; -- procedure+: read-text-line [PORT] ; Reads one line of text from the PORT, and returns it as a string. ; A line is a (possibly empty) sequence of characters terminated ; by CR, CRLF or LF (or even the end of file). ; The terminating character (or CRLF combination) is removed from ; the input stream. The terminating character(s) is not a part ; of the return string either. ; If EOF is encountered before any character is read, the return ; value is EOF. ; ; The optional argument PORT defaults to the current input port. (define *read-line-breaks* (list char-newline char-return '*eof*)) (define-opt (read-text-line (optional (port (current-input-port))) ) (if (eof-object? (peek-char port)) (peek-char port) (let* ((line (next-token '() *read-line-breaks* "reading a line" port)) (c (read-char port))) ; must be either \n or \r or EOF (and (eqv? c char-return) (eqv? (peek-char port) #\newline) (read-char port)) ; skip \n that follows \r line))) ; -- procedure+: read-string N [PORT] ; Reads N characters from the PORT, and returns them in a string. ; If EOF is encountered before N characters are read, a shorter string ; will be returned. ; If N is not positive, an empty string will be returned. ; The optional argument PORT defaults to the current input port. (define-opt (read-string n (optional (port (current-input-port))) ) (if (not (positive? n)) "" (let ((buffer (make-string n))) (let loop ((i 0) (c (read-char port))) (if (eof-object? c) (substring buffer 0 i) (let ((i1 (inc i))) (string-set! buffer i c) (if (= i1 n) buffer (loop i1 (read-char port))))))))) PK�����/F2]ߕȄ[�[���sxml/upstream/SSAX.scmnu�[��������; Functional XML parsing framework: SAX/DOM and SXML parsers ; with support for XML Namespaces and validation ; ; This is a package of low-to-high level lexing and parsing procedures ; that can be combined to yield a SAX, a DOM, a validating parsers, or ; a parser intended for a particular document type. The procedures in ; the package can be used separately to tokenize or parse various ; pieces of XML documents. The package supports XML Namespaces, ; internal and external parsed entities, user-controlled handling of ; whitespace, and validation. This module therefore is intended to be ; a framework, a set of "Lego blocks" you can use to build a parser ; following any discipline and performing validation to any degree. As ; an example of the parser construction, this file includes a ; semi-validating SXML parser. ; The present XML framework has a "sequential" feel of SAX yet a ; "functional style" of DOM. Like a SAX parser, the framework scans ; the document only once and permits incremental processing. An ; application that handles document elements in order can run as ; efficiently as possible. _Unlike_ a SAX parser, the framework does ; not require an application register stateful callbacks and surrender ; control to the parser. Rather, it is the application that can drive ; the framework -- calling its functions to get the current lexical or ; syntax element. These functions do not maintain or mutate any state ; save the input port. Therefore, the framework permits parsing of XML ; in a pure functional style, with the input port being a monad (or a ; linear, read-once parameter). ; Besides the PORT, there is another monad -- SEED. Most of the ; middle- and high-level parsers are single-threaded through the ; seed. The functions of this framework do not process or affect the ; SEED in any way: they simply pass it around as an instance of an ; opaque datatype. User functions, on the other hand, can use the ; seed to maintain user's state, to accumulate parsing results, etc. A ; user can freely mix his own functions with those of the ; framework. On the other hand, the user may wish to instantiate a ; high-level parser: ssax:make-elem-parser or ssax:make-parser. In ; the latter case, the user must provide functions of specific ; signatures, which are called at predictable moments during the ; parsing: to handle character data, element data, or processing ; instructions (PI). The functions are always given the SEED, among ; other parameters, and must return the new SEED. ; From a functional point of view, XML parsing is a combined ; pre-post-order traversal of a "tree" that is the XML document ; itself. This down-and-up traversal tells the user about an element ; when its start tag is encountered. The user is notified about the ; element once more, after all element's children have been ; handled. The process of XML parsing therefore is a fold over the ; raw XML document. Unlike a fold over trees defined in [1], the ; parser is necessarily single-threaded -- obviously as elements ; in a text XML document are laid down sequentially. The parser ; therefore is a tree fold that has been transformed to accept an ; accumulating parameter [1,2]. ; Formally, the denotational semantics of the parser can be expressed ; as ; parser:: (Start-tag -> Seed -> Seed) -> ; (Start-tag -> Seed -> Seed -> Seed) -> ; (Char-Data -> Seed -> Seed) -> ; XML-text-fragment -> Seed -> Seed ; parser fdown fup fchar "<elem attrs> content </elem>" seed ; = fup "<elem attrs>" seed ; (parser fdown fup fchar "content" (fdown "<elem attrs>" seed)) ; ; parser fdown fup fchar "char-data content" seed ; = parser fdown fup fchar "content" (fchar "char-data" seed) ; ; parser fdown fup fchar "elem-content content" seed ; = parser fdown fup fchar "content" ( ; parser fdown fup fchar "elem-content" seed) ; Compare the last two equations with the left fold ; fold-left kons elem:list seed = fold-left kons list (kons elem seed) ; The real parser created my ssax:make-parser is slightly more complicated, ; to account for processing instructions, entity references, namespaces, ; processing of document type declaration, etc. ; The XML standard document referred to in this module is ; http://www.w3.org/TR/1998/REC-xml-19980210.html ; ; The present file also defines a procedure that parses the text of an ; XML document or of a separate element into SXML, an ; S-expression-based model of an XML Information Set. SXML is also an ; Abstract Syntax Tree of an XML document. SXML is similar ; but not identical to DOM; SXML is particularly suitable for ; Scheme-based XML/HTML authoring, SXPath queries, and tree ; transformations. See SXML.html for more details. ; SXML is a term implementation of evaluation of the XML document [3]. ; The other implementation is context-passing. ; The present frameworks fully supports the XML Namespaces Recommendation: ; http://www.w3.org/TR/REC-xml-names/ ; Other links: ; [1] Jeremy Gibbons, Geraint Jones, "The Under-appreciated Unfold," ; Proc. ICFP'98, 1998, pp. 273-279. ; [2] Richard S. Bird, The promotion and accumulation strategies in ; transformational programming, ACM Trans. Progr. Lang. Systems, ; 6(4):487-504, October 1984. ; [3] Ralf Hinze, "Deriving Backtracking Monad Transformers," ; Functional Pearl. Proc ICFP'00, pp. 186-197. ; IMPORT ; parser-error ssax:warn, see Handling of errors, below ; functions declared in files util.scm, input-parse.scm and look-for-str.scm ; char-encoding.scm for various platform-specific character-encoding functions. ; From SRFI-13: string-concatenate/shared and string-concatenate-reverse/shared ; If a particular implementation lacks SRFI-13 support, please ; include the file srfi-13-local.scm ; Handling of errors ; This package relies on a function parser-error, which must be defined ; by a user of the package. The function has the following signature: ; parser-error PORT MESSAGE SPECIALISING-MSG* ; Many procedures of this package call 'parser-error' whenever a ; parsing, well-formedness or validation error is encountered. The ; first argument is a port, which typically points to the offending ; character or its neighborhood. Most of the Scheme systems let the ; user query a PORT for the current position. The MESSAGE argument ; indicates a failed XML production or a failed XML constraint. The ; latter is referred to by its anchor name in the XML Recommendation ; or XML Namespaces Recommendation. The parsing library (e.g., ; next-token, assert-curr-char) invoke 'parser-error' as well, in ; exactly the same way. See input-parse.scm for more details. ; See ; http://pair.com/lisovsky/download/parse-error.scm ; for an excellent example of such a redefined parser-error function. ; ; In addition, the present code invokes a function ssax:warn ; ssax:warn PORT MESSAGE SPECIALISING-MSG* ; to notify the user about warnings that are NOT errors but still ; may alert the user. ; ; Again, parser-error and ssax:warn are supposed to be defined by the ; user. However, if a run-test macro below is set to include ; self-tests, this present code does provide the definitions for these ; functions to allow tests to run. ; Misc notes ; It seems it is highly desirable to separate tests out in a dedicated ; file. ; ; Jim Bender wrote on Mon, 9 Sep 2002 20:03:42 EDT on the SSAX-SXML ; mailing list (message A fine-grained "lego") ; The task was to record precise source location information, as PLT ; does with its current XML parser. That parser records the start and ; end location (filepos, line#, column#) for pi, elements, attributes, ; chuncks of "pcdata". ; As suggested above, though, in some cases I needed to be able force ; open an interface that did not yet exist. For instance, I added an ; "end-char-data-hook", which would be called at the end of char-data ; fragment. This returns a function of type (seed -> seed) which is ; invoked on the current seed only if read-char-data has indeed reached ; the end of a block of char data (after reading a new token. ; But the deepest interface that I needed to expose was that of reading ; attributes. In the official distribution, this is not even a separate ; function. Instead, it is embedded within SSAX:read-attributes. This ; required some small re-structuring as well. ; This definitely will not be to everyone's taste (nor needed by most). ; Certainly, the existing make-parser interface addresses most custom ; needs. And likely 80-90 lines of a "link specification" to create a ; parser from many tiny little lego blocks may please only a few, while ; appalling others. ; The code is available at http://celtic.benderweb.net/ssax-lego.plt or ; http://celtic.benderweb.net/ssax-lego.tar.gz ; In the examples directory, I provide: ; - a unit version of the make-parser interface, ; - a simple SXML parser using that interface, ; - an SXML parser which directly uses the "new lego", ; - a pseudo-SXML parser, which records source location information ; - and lastly a parser which returns the structures used in PLT's xml ; collection, with source location information ; $Id: SSAX.scm,v 5.1 2004/07/07 16:02:30 sperber Exp $ ;^^^^^^^^^ ; See the Makefile in the ../tests directory ; (in particular, the rule vSSAX) for an example of how ; to run this code on various Scheme systems. ; See SSAX examples for many samples of using this code, ; again, on a variety of Scheme systems. ; See http://ssax.sf.net/ ; The following macro runs built-in test cases -- or does not run, ; depending on which of the two cases below you commented out ; Case 1: no tests: ;(define-macro run-test (lambda body '(begin #f))) ;(define-syntax run-test (syntax-rules () ((run-test . args) (begin #f)))) ; Case 2: with tests. ; The following macro could've been defined just as ; (define-macro run-test (lambda body `(begin (display "\n-->Test\n") ,@body))) ; ; Instead, it's more involved, to make up for case-insensitivity of ; symbols on some Scheme systems. In Gambit, symbols are case ; sensitive: (eq? 'A 'a) is #f and (eq? 'Aa (string->symbol "Aa")) is ; #t. On some systems, symbols are case-insensitive and just the ; opposite is true. Therefore, we introduce a notation '"ASymbol" (a ; quoted string) that stands for a case-_sensitive_ ASymbol -- on any ; R5RS Scheme system. This notation is valid only within the body of ; run-test. ; The notation is implemented by scanning the run-test's ; body and replacing every occurrence of (quote "str") with the result ; of (string->symbol "str"). We can do such a replacement at macro-expand ; time (rather than at run time). ; Here's the previous version of run-test, implemented as a low-level ; macro. ; (define-macro run-test ; (lambda body ; (define (re-write body) ; (cond ; ((vector? body) ; (list->vector (re-write (vector->list body)))) ; ((not (pair? body)) body) ; ((and (eq? 'quote (car body)) (pair? (cdr body)) ; (string? (cadr body))) ; (string->symbol (cadr body))) ; (else (cons (re-write (car body)) (re-write (cdr body)))))) ; (cons 'begin (re-write body)))) ; ; For portability, it is re-written as syntax-rules. The syntax-rules ; version is less powerful: for example, it can't handle ; (case x (('"Foo") (do-on-Foo))) whereas the low-level macro ; could correctly place a case-sensitive symbol at the right place. ; We also do not scan vectors (because we don't use them here). ; Twice-deep quasiquotes aren't handled either. ; Still, the syntax-rules version satisfies our immediate needs. ; Incidentally, I originally didn't believe that the macro below ; was at all possible. ; ; The macro is written in a continuation-passing style. A continuation ; typically has the following structure: (k-head ! . args) ; When the continuation is invoked, we expand into ; (k-head <computed-result> . arg). That is, the dedicated symbol ! ; is the placeholder for the result. ; ; It seems that the most modular way to write the run-test macro would ; be the following ; ; (define-syntax run-test ; (syntax-rules () ; ((run-test . ?body) ; (letrec-syntax ; ((scan-exp ; (scan-exp body k) ; (syntax-rules (quote quasiquote !) ; ((scan-exp (quote (hd . tl)) k) ; (scan-lit-lst (hd . tl) (do-wrap ! quasiquote k))) ; ((scan-exp (quote x) (k-head ! . args)) ; (k-head ; (if (string? (quote x)) (string->symbol (quote x)) (quote x)) ; . args)) ; ((scan-exp (hd . tl) k) ; (scan-exp hd (do-tl ! scan-exp tl k))) ; ((scan-exp x (k-head ! . args)) ; (k-head x . args)))) ; (do-tl ; (syntax-rules (!) ; ((do-tl processed-hd fn () (k-head ! . args)) ; (k-head (processed-hd) . args)) ; ((do-tl processed-hd fn old-tl k) ; (fn old-tl (do-cons ! processed-hd k))))) ; ... ; (do-finish ; (syntax-rules () ; ((do-finish (new-body)) new-body) ; ((do-finish new-body) (begin . new-body)))) ; ... ; (scan-exp ?body (do-finish !)) ; )))) ; ; Alas, that doesn't work on all systems. We hit yet another dark ; corner of the R5RS macros. The reason is that run-test is used in ; the code below to introduce definitions. For example: ; (run-test ; (define (ssax:warn port msg . other-msg) ; (apply cerr (cons* nl "Warning: " msg other-msg))) ; ) ; This code expands to ; (begin ; (define (ssax:warn port msg . other-msg) ...)) ; so the definition gets spliced in into the top level. Right? ; Well, On Petite Chez Scheme it is so. However, many other systems ; don't like this approach. The reason is that the invocation of ; (run-test (define (ssax:warn port msg . other-msg) ...)) ; first expands into ; (letrec-syntax (...) ; (scan-exp ((define (ssax:warn port msg . other-msg) ...)) ...)) ; because of the presence of (letrec-syntax ...), the begin form that ; is generated eventually is no longer at the top level! The begin ; form in Scheme is an overloading of two distinct forms: top-level ; begin and the other begin. The forms have different rules: for example, ; (begin (define x 1)) is OK for a top-level begin but not OK for ; the other begin. Some Scheme systems see the that the macro ; (run-test ...) expands into (letrec-syntax ...) and decide right there ; that any further (begin ...) forms are NOT top-level begin forms. ; The only way out is to make sure all our macros are top-level. ; The best approach <sigh> seems to be to make run-test one huge ; top-level macro. (define-syntax run-test (syntax-rules (define) ((run-test "scan-exp" (define vars body)) (define vars (run-test "scan-exp" body))) ((run-test "scan-exp" ?body) (letrec-syntax ((scan-exp ; (scan-exp body k) (syntax-rules (quote quasiquote !) ((scan-exp '() (k-head ! . args)) (k-head '() . args)) ((scan-exp (quote (hd . tl)) k) (scan-lit-lst (hd . tl) (do-wrap ! quasiquote k))) ((scan-exp (quasiquote (hd . tl)) k) (scan-lit-lst (hd . tl) (do-wrap ! quasiquote k))) ((scan-exp (quote x) (k-head ! . args)) (k-head (if (string? (quote x)) (string->symbol (quote x)) (quote x)) . args)) ((scan-exp (hd . tl) k) (scan-exp hd (do-tl ! scan-exp tl k))) ((scan-exp x (k-head ! . args)) (k-head x . args)))) (do-tl (syntax-rules (!) ((do-tl processed-hd fn () (k-head ! . args)) (k-head (processed-hd) . args)) ((do-tl processed-hd fn old-tl k) (fn old-tl (do-cons ! processed-hd k))))) (do-cons (syntax-rules (!) ((do-cons processed-tl processed-hd (k-head ! . args)) (k-head (processed-hd . processed-tl) . args)))) (do-wrap (syntax-rules (!) ((do-wrap val fn (k-head ! . args)) (k-head (fn val) . args)))) (do-finish (syntax-rules () ((do-finish new-body) new-body))) (scan-lit-lst ; scan literal list (syntax-rules (quote unquote unquote-splicing !) ((scan-lit-lst '() (k-head ! . args)) (k-head '() . args)) ((scan-lit-lst (quote (hd . tl)) k) (do-tl quote scan-lit-lst ((hd . tl)) k)) ((scan-lit-lst (unquote x) k) (scan-exp x (do-wrap ! unquote k))) ((scan-lit-lst (unquote-splicing x) k) (scan-exp x (do-wrap ! unquote-splicing k))) ((scan-lit-lst (quote x) (k-head ! . args)) (k-head ,(if (string? (quote x)) (string->symbol (quote x)) (quote x)) . args)) ((scan-lit-lst (hd . tl) k) (scan-lit-lst hd (do-tl ! scan-lit-lst tl k))) ((scan-lit-lst x (k-head ! . args)) (k-head x . args)))) ) (scan-exp ?body (do-finish !)))) ((run-test body ...) (begin (run-test "scan-exp" body) ...)) )) ;======================================================================== ; Data Types ; TAG-KIND ; a symbol 'START, 'END, 'PI, 'DECL, 'COMMENT, 'CDSECT ; or 'ENTITY-REF that identifies a markup token ; UNRES-NAME ; a name (called GI in the XML Recommendation) as given in an xml ; document for a markup token: start-tag, PI target, attribute name. ; If a GI is an NCName, UNRES-NAME is this NCName converted into ; a Scheme symbol. If a GI is a QName, UNRES-NAME is a pair of ; symbols: (PREFIX . LOCALPART) ; RES-NAME ; An expanded name, a resolved version of an UNRES-NAME. ; For an element or an attribute name with a non-empty namespace URI, ; RES-NAME is a pair of symbols, (URI-SYMB . LOCALPART). ; Otherwise, it's a single symbol. ; ELEM-CONTENT-MODEL ; A symbol: ; ANY - anything goes, expect an END tag. ; EMPTY-TAG - no content, and no END-tag is coming ; EMPTY - no content, expect the END-tag as the next token ; PCDATA - expect character data only, and no children elements ; MIXED ; ELEM-CONTENT ; URI-SYMB ; A symbol representing a namespace URI -- or other symbol chosen ; by the user to represent URI. In the former case, ; URI-SYMB is created by %-quoting of bad URI characters and ; converting the resulting string into a symbol. ; NAMESPACES ; A list representing namespaces in effect. An element of the list ; has one of the following forms: ; (PREFIX URI-SYMB . URI-SYMB) or ; (PREFIX USER-PREFIX . URI-SYMB) ; USER-PREFIX is a symbol chosen by the user ; to represent the URI. ; (#f USER-PREFIX . URI-SYMB) ; Specification of the user-chosen prefix and a URI-SYMBOL. ; (*DEFAULT* USER-PREFIX . URI-SYMB) ; Declaration of the default namespace ; (*DEFAULT* #f . #f) ; Un-declaration of the default namespace. This notation ; represents overriding of the previous declaration ; A NAMESPACES list may contain several elements for the same PREFIX. ; The one closest to the beginning of the list takes effect. ; ATTLIST ; An ordered collection of (NAME . VALUE) pairs, where NAME is ; a RES-NAME or an UNRES-NAME. The collection is an ADT ; STR-HANDLER ; A procedure of three arguments: STRING1 STRING2 SEED ; returning a new SEED ; The procedure is supposed to handle a chunk of character data ; STRING1 followed by a chunk of character data STRING2. ; STRING2 is a short string, often "\n" and even "" ; ENTITIES ; An assoc list of pairs: ; (named-entity-name . named-entity-body) ; where named-entity-name is a symbol under which the entity was ; declared, named-entity-body is either a string, or ; (for an external entity) a thunk that will return an ; input port (from which the entity can be read). ; named-entity-body may also be #f. This is an indication that a ; named-entity-name is currently being expanded. A reference to ; this named-entity-name will be an error: violation of the ; WFC nonrecursion. ; ; As an extension to the original SSAX, Guile allows a ; named-entity-name of *DEFAULT* to indicate a fallback procedure, ; called as (FALLBACK PORT NAME). The procedure should return a ; string. ; XML-TOKEN -- a record ; In Gambit, you can use the following declaration: ; (define-structure xml-token kind head) ; The following declaration is "standard" as it follows SRFI-9: ;;(define-record-type xml-token (make-xml-token kind head) xml-token? ;; (kind xml-token-kind) ;; (head xml-token-head) ) ; No field mutators are declared as SSAX is a pure functional parser ; ; But to make the code more portable, we define xml-token simply as ; a pair. It suffices for us. Furthermore, xml-token-kind and xml-token-head ; can be defined as simple procedures. However, they are declared as ; macros below for efficiency. (define (make-xml-token kind head) (cons kind head)) (define xml-token? pair?) (define-syntax xml-token-kind (syntax-rules () ((xml-token-kind token) (car token)))) (define-syntax xml-token-head (syntax-rules () ((xml-token-head token) (cdr token)))) ; (define-macro xml-token-kind (lambda (token) `(car ,token))) ; (define-macro xml-token-head (lambda (token) `(cdr ,token))) ; This record represents a markup, which is, according to the XML ; Recommendation, "takes the form of start-tags, end-tags, empty-element tags, ; entity references, character references, comments, CDATA section delimiters, ; document type declarations, and processing instructions." ; ; kind -- a TAG-KIND ; head -- an UNRES-NAME. For xml-tokens of kinds 'COMMENT and ; 'CDSECT, the head is #f ; ; For example, ; <P> => kind='START, head='P ; </P> => kind='END, head='P ; <BR/> => kind='EMPTY-EL, head='BR ; <!DOCTYPE OMF ...> => kind='DECL, head='DOCTYPE ; <?xml version="1.0"?> => kind='PI, head='xml ; &my-ent; => kind = 'ENTITY-REF, head='my-ent ; ; Character references are not represented by xml-tokens as these references ; are transparently resolved into the corresponding characters. ; ; XML-DECL -- a record ; The following is Gambit-specific, see below for a portable declaration ;(define-structure xml-decl elems entities notations) ; The record represents a datatype of an XML document: the list of ; declared elements and their attributes, declared notations, list of ; replacement strings or loading procedures for parsed general ; entities, etc. Normally an xml-decl record is created from a DTD or ; an XML Schema, although it can be created and filled in in many other ; ways (e.g., loaded from a file). ; ; elems: an (assoc) list of decl-elem or #f. The latter instructs ; the parser to do no validation of elements and attributes. ; ; decl-elem: declaration of one element: ; (elem-name elem-content decl-attrs) ; elem-name is an UNRES-NAME for the element. ; elem-content is an ELEM-CONTENT-MODEL. ; decl-attrs is an ATTLIST, of (ATTR-NAME . VALUE) associations ; !!!This element can declare a user procedure to handle parsing of an ; element (e.g., to do a custom validation, or to build a hash of ; IDs as they're encountered). ; ; decl-attr: an element of an ATTLIST, declaration of one attribute ; (attr-name content-type use-type default-value) ; attr-name is an UNRES-NAME for the declared attribute ; content-type is a symbol: CDATA, NMTOKEN, NMTOKENS, ... ; or a list of strings for the enumerated type. ; use-type is a symbol: REQUIRED, IMPLIED, FIXED ; default-value is a string for the default value, or #f if not given. ; ; ; see a function make-empty-xml-decl to make a XML declaration entry ; suitable for a non-validating parsing. ;------------------------- ; Utilities ; ssax:warn PORT MESSAGE SPECIALISING-MSG* ; to notify the user about warnings that are NOT errors but still ; may alert the user. ; Result is unspecified. ; We need to define the function to allow the self-tests to run. ; Normally the definition of ssax:warn is to be provided by the user. (run-test (define (ssax:warn port msg . other-msg) (apply cerr (cons* nl "Warning: " msg other-msg))) ) ; parser-error PORT MESSAGE SPECIALISING-MSG* ; to let the user know of a syntax error or a violation of a ; well-formedness or validation constraint. ; Result is unspecified. ; We need to define the function to allow the self-tests to run. ; Normally the definition of parser-error is to be provided by the user. (run-test (define (parser-error port msg . specializing-msgs) (apply error (cons msg specializing-msgs))) ) ; The following is a function that is often used in validation tests, ; to make sure that the computed result matches the expected one. ; This function is a standard equal? predicate with one exception. ; On Scheme systems where (string->symbol "A") and a symbol A ; are the same, equal_? is precisely equal? ; On other Scheme systems, we compare symbols disregarding their case. ; Since this function is used only in tests, we don't have to ; strive to make it efficient. (run-test (define (equal_? e1 e2) (if (eq? 'A (string->symbol "A")) (equal? e1 e2) (cond ((symbol? e1) (and (symbol? e2) (string-ci=? (symbol->string e1) (symbol->string e2)))) ((pair? e1) (and (pair? e2) (equal_? (car e1) (car e2)) (equal_? (cdr e1) (cdr e2)))) ((vector? e1) (and (vector? e2) (equal_? (vector->list e1) (vector->list e2)))) (else (equal? e1 e2))))) ) ; The following function, which is often used in validation tests, ; lets us conveniently enter newline, CR and tab characters in a character ; string. ; unesc-string: ESC-STRING -> STRING ; where ESC-STRING is a character string that may contain ; %n -- for #\newline ; %r -- for #\return ; %t -- for #\tab ; %% -- for #\% ; ; The result of unesc-string is a character string with all %-combinations ; above replaced with their character equivalents (run-test (define (unesc-string str) (call-with-input-string str (lambda (port) (let loop ((frags '())) (let* ((token (next-token '() '(#\% *eof*) "unesc-string" port)) (cterm (read-char port)) (frags (cons token frags))) (if (eof-object? cterm) (string-concatenate-reverse/shared frags) (let ((cchar (read-char port))) ; char after #\% (if (eof-object? cchar) (error "unexpected EOF after reading % in unesc-string:" str) (loop (cons (case cchar ((#\n) (string #\newline)) ((#\r) (string char-return)) ((#\t) (string char-tab)) ((#\%) "%") (else (error "bad %-char in unesc-string:" cchar))) frags)))))))))) ) ; Test if a string is made of only whitespace ; An empty string is considered made of whitespace as well (define (string-whitespace? str) (let ((len (string-length str))) (cond ((zero? len) #t) ((= 1 len) (char-whitespace? (string-ref str 0))) ((= 2 len) (and (char-whitespace? (string-ref str 0)) (char-whitespace? (string-ref str 1)))) (else (let loop ((i 0)) (or (>= i len) (and (char-whitespace? (string-ref str i)) (loop (inc i))))))))) ; Find val in alist ; Return (values found-el remaining-alist) or ; (values #f alist) (define (assq-values val alist) (let loop ((alist alist) (scanned '())) (cond ((null? alist) (values #f scanned)) ((equal? val (caar alist)) (values (car alist) (append scanned (cdr alist)))) (else (loop (cdr alist) (cons (car alist) scanned)))))) ; From SRFI-1 (define (fold-right kons knil lis1) (let recur ((lis lis1)) (if (null? lis) knil (let ((head (car lis))) (kons head (recur (cdr lis))))))) ; Left fold combinator for a single list (define (fold kons knil lis1) (let lp ((lis lis1) (ans knil)) (if (null? lis) ans (lp (cdr lis) (kons (car lis) ans))))) ;======================================================================== ; Lower-level parsers and scanners ; ; They deal with primitive lexical units (Names, whitespaces, tags) ; and with pieces of more generic productions. Most of these parsers ; must be called in appropriate context. For example, ssax:complete-start-tag ; must be called only when the start-tag has been detected and its GI ; has been read. ;------------------------------------------------------------------------ ; Low-level parsing code ; Skip the S (whitespace) production as defined by ; [3] S ::= (#x20 | #x9 | #xD | #xA) ; The procedure returns the first not-whitespace character it ; encounters while scanning the PORT. This character is left ; on the input stream. (define ssax:S-chars (map ascii->char '(32 10 9 13))) (define (ssax:skip-S port) (skip-while ssax:S-chars port)) ; Read a Name lexem and return it as string ; [4] NameChar ::= Letter | Digit | '.' | '-' | '_' | ':' ; | CombiningChar | Extender ; [5] Name ::= (Letter | '_' | ':') (NameChar)* ; ; This code supports the XML Namespace Recommendation REC-xml-names, ; which modifies the above productions as follows: ; ; [4] NCNameChar ::= Letter | Digit | '.' | '-' | '_' ; | CombiningChar | Extender ; [5] NCName ::= (Letter | '_') (NCNameChar)* ; As the Rec-xml-names says, ; "An XML document conforms to this specification if all other tokens ; [other than element types and attribute names] in the document which ; are required, for XML conformance, to match the XML production for ; Name, match this specification's production for NCName." ; Element types and attribute names must match the production QName, ; defined below. ; Check to see if a-char may start a NCName (define (ssax:ncname-starting-char? a-char) (and (char? a-char) (or (char-alphabetic? a-char) (char=? #\_ a-char)))) ; Read a NCName starting from the current position in the PORT and ; return it as a symbol. (define (ssax:read-NCName port) (let ((first-char (peek-char port))) (or (ssax:ncname-starting-char? first-char) (parser-error port "XMLNS [4] for '" first-char "'"))) (string->symbol (next-token-of (lambda (c) (cond ((eof-object? c) #f) ((char-alphabetic? c) c) ((string-index "0123456789.-_" c) c) (else #f))) port))) ; Read a (namespace-) Qualified Name, QName, from the current ; position in the PORT. ; From REC-xml-names: ; [6] QName ::= (Prefix ':')? LocalPart ; [7] Prefix ::= NCName ; [8] LocalPart ::= NCName ; Return: an UNRES-NAME (define (ssax:read-QName port) (let ((prefix-or-localpart (ssax:read-NCName port))) (case (peek-char port) ((#\:) ; prefix was given after all (read-char port) ; consume the colon (cons prefix-or-localpart (ssax:read-NCName port))) (else prefix-or-localpart) ; Prefix was omitted ))) ; The prefix of the pre-defined XML namespace (define ssax:Prefix-XML (string->symbol "xml")) (run-test (assert (eq? '_ (call-with-input-string "_" ssax:read-NCName))) (assert (eq? '_ (call-with-input-string "_" ssax:read-QName))) (assert (eq? (string->symbol "_abc_") (call-with-input-string "_abc_;" ssax:read-NCName))) (assert (eq? (string->symbol "_abc_") (call-with-input-string "_abc_;" ssax:read-QName))) (assert (eq? (string->symbol "_a.b") (call-with-input-string "_a.b " ssax:read-QName))) (assert (equal? (cons (string->symbol "_a.b") (string->symbol "d.1-ef-")) (call-with-input-string "_a.b:d.1-ef-;" ssax:read-QName))) (assert (equal? (cons (string->symbol "a") (string->symbol "b")) (call-with-input-string "a:b:c" ssax:read-QName))) (assert (failed? (call-with-input-string ":abc" ssax:read-NCName))) (assert (failed? (call-with-input-string "1:bc" ssax:read-NCName))) ) ; Compare one RES-NAME or an UNRES-NAME with the other. ; Return a symbol '<, '>, or '= depending on the result of ; the comparison. ; Names without PREFIX are always smaller than those with the PREFIX. (define name-compare (letrec ((symbol-compare (lambda (symb1 symb2) (cond ((eq? symb1 symb2) '=) ((string<? (symbol->string symb1) (symbol->string symb2)) '<) (else '>))))) (lambda (name1 name2) (cond ((symbol? name1) (if (symbol? name2) (symbol-compare name1 name2) '<)) ((symbol? name2) '>) ((eq? name2 ssax:largest-unres-name) '<) ((eq? name1 ssax:largest-unres-name) '>) ((eq? (car name1) (car name2)) ; prefixes the same (symbol-compare (cdr name1) (cdr name2))) (else (symbol-compare (car name1) (car name2))))))) ; An UNRES-NAME that is postulated to be larger than anything that can occur in ; a well-formed XML document. ; name-compare enforces this postulate. (define ssax:largest-unres-name (cons (string->symbol "#LARGEST-SYMBOL") (string->symbol "#LARGEST-SYMBOL"))) (run-test (assert (eq? '= (name-compare 'ABC 'ABC))) (assert (eq? '< (name-compare 'ABC 'ABCD))) (assert (eq? '> (name-compare 'XB 'ABCD))) (assert (eq? '> (name-compare '(HTML . PRE) 'PRE))) (assert (eq? '< (name-compare 'HTML '(HTML . PRE)))) (assert (eq? '= (name-compare '(HTML . PRE) '(HTML . PRE)))) (assert (eq? '< (name-compare '(HTML . PRE) '(XML . PRE)))) (assert (eq? '> (name-compare '(HTML . PRE) '(HTML . P)))) (assert (eq? '< (name-compare '(HTML . PRE) ssax:largest-unres-name))) (assert (eq? '< (name-compare '(ZZZZ . ZZZ) ssax:largest-unres-name))) (assert (eq? '> (name-compare ssax:largest-unres-name '(ZZZZ . ZZZ) ))) ) ; procedure: ssax:read-markup-token PORT ; This procedure starts parsing of a markup token. The current position ; in the stream must be #\<. This procedure scans enough of the input stream ; to figure out what kind of a markup token it is seeing. The procedure returns ; an xml-token structure describing the token. Note, generally reading ; of the current markup is not finished! In particular, no attributes of ; the start-tag token are scanned. ; ; Here's a detailed break out of the return values and the position in the PORT ; when that particular value is returned: ; PI-token: only PI-target is read. ; To finish the Processing Instruction and disregard it, ; call ssax:skip-pi. ssax:read-attributes may be useful ; as well (for PIs whose content is attribute-value ; pairs) ; END-token: The end tag is read completely; the current position ; is right after the terminating #\> character. ; COMMENT is read and skipped completely. The current position ; is right after "-->" that terminates the comment. ; CDSECT The current position is right after "<!CDATA[" ; Use ssax:read-cdata-body to read the rest. ; DECL We have read the keyword (the one that follows "<!") ; identifying this declaration markup. The current ; position is after the keyword (usually a ; whitespace character) ; ; START-token We have read the keyword (GI) of this start tag. ; No attributes are scanned yet. We don't know if this ; tag has an empty content either. ; Use ssax:complete-start-tag to finish parsing of ; the token. (define ssax:read-markup-token ; procedure ssax:read-markup-token port (let () ; we have read "<!-". Skip through the rest of the comment ; Return the 'COMMENT token as an indication we saw a comment ; and skipped it. (define (skip-comment port) (assert-curr-char '(#\-) "XML [15], second dash" port) (if (not (find-string-from-port? "-->" port)) (parser-error port "XML [15], no -->")) (make-xml-token 'COMMENT #f)) ; we have read "<![" that must begin a CDATA section (define (read-cdata port) (assert (string=? "CDATA[" (read-string 6 port))) (make-xml-token 'CDSECT #f)) (lambda (port) (assert-curr-char '(#\<) "start of the token" port) (case (peek-char port) ((#\/) (read-char port) (begin0 (make-xml-token 'END (ssax:read-QName port)) (ssax:skip-S port) (assert-curr-char '(#\>) "XML [42]" port))) ((#\?) (read-char port) (make-xml-token 'PI (ssax:read-NCName port))) ((#\!) (case (peek-next-char port) ((#\-) (read-char port) (skip-comment port)) ((#\[) (read-char port) (read-cdata port)) (else (make-xml-token 'DECL (ssax:read-NCName port))))) (else (make-xml-token 'START (ssax:read-QName port))))) )) ; The current position is inside a PI. Skip till the rest of the PI (define (ssax:skip-pi port) (if (not (find-string-from-port? "?>" port)) (parser-error port "Failed to find ?> terminating the PI"))) ; The current position is right after reading the PITarget. We read the ; body of PI and return is as a string. The port will point to the ; character right after '?>' combination that terminates PI. ; [16] PI ::= '<?' PITarget (S (Char* - (Char* '?>' Char*)))? '?>' (define (ssax:read-pi-body-as-string port) (ssax:skip-S port) ; skip WS after the PI target name (string-concatenate/shared (let loop () (let ((pi-fragment (next-token '() '(#\?) "reading PI content" port))) (if (eqv? #\> (peek-next-char port)) (begin (read-char port) (cons pi-fragment '())) (cons* pi-fragment "?" (loop))))))) (run-test (assert (equal? "p1 content " (call-with-input-string "<?pi1 p1 content ?>" (lambda (port) (ssax:read-markup-token port) (ssax:read-pi-body-as-string port))))) (assert (equal? "pi2? content? ?" (call-with-input-string "<?pi2 pi2? content? ??>" (lambda (port) (ssax:read-markup-token port) (ssax:read-pi-body-as-string port))))) ) ;(define (ssax:read-pi-body-as-name-values port) ; The current pos in the port is inside an internal DTD subset ; (e.g., after reading #\[ that begins an internal DTD subset) ; Skip until the "]>" combination that terminates this DTD (define (ssax:skip-internal-dtd port) (if (not (find-string-from-port? "]>" port)) (parser-error port "Failed to find ]> terminating the internal DTD subset"))) ; procedure+: ssax:read-cdata-body PORT STR-HANDLER SEED ; ; This procedure must be called after we have read a string "<![CDATA[" ; that begins a CDATA section. The current position must be the first ; position of the CDATA body. This function reads _lines_ of the CDATA ; body and passes them to a STR-HANDLER, a character data consumer. ; ; The str-handler is a STR-HANDLER, a procedure STRING1 STRING2 SEED. ; The first STRING1 argument to STR-HANDLER never contains a newline. ; The second STRING2 argument often will. On the first invocation of ; the STR-HANDLER, the seed is the one passed to ssax:read-cdata-body ; as the third argument. The result of this first invocation will be ; passed as the seed argument to the second invocation of the line ; consumer, and so on. The result of the last invocation of the ; STR-HANDLER is returned by the ssax:read-cdata-body. Note a ; similarity to the fundamental 'fold' iterator. ; ; Within a CDATA section all characters are taken at their face value, ; with only three exceptions: ; CR, LF, and CRLF are treated as line delimiters, and passed ; as a single #\newline to the STR-HANDLER ; "]]>" combination is the end of the CDATA section. ; > is treated as an embedded #\> character ; Note, < and & are not specially recognized (and are not expanded)! (define ssax:read-cdata-body (let ((cdata-delimiters (list char-return #\newline #\] #\&))) (lambda (port str-handler seed) (let loop ((seed seed)) (let ((fragment (next-token '() cdata-delimiters "reading CDATA" port))) ; that is, we're reading the char after the 'fragment' (case (read-char port) ((#\newline) (loop (str-handler fragment nl seed))) ((#\]) (if (not (eqv? (peek-char port) #\])) (loop (str-handler fragment "]" seed)) (let check-after-second-braket ((seed (if (string-null? fragment) seed (str-handler fragment "" seed)))) (case (peek-next-char port) ; after the second bracket ((#\>) (read-char port) seed) ; we have read "]]>" ((#\]) (check-after-second-braket (str-handler "]" "" seed))) (else (loop (str-handler "]]" "" seed))))))) ((#\&) ; Note that #\& within CDATA may stand for itself (let ((ent-ref ; it does not have to start an entity ref (next-token-of (lambda (c) (and (not (eof-object? c)) (char-alphabetic? c) c)) port))) (cond ; ">" is to be replaced with #\> ((and (string=? "gt" ent-ref) (eqv? (peek-char port) #\;)) (read-char port) (loop (str-handler fragment ">" seed))) (else (loop (str-handler ent-ref "" (str-handler fragment "&" seed))))))) (else ; Must be CR: if the next char is #\newline, skip it (if (eqv? (peek-char port) #\newline) (read-char port)) (loop (str-handler fragment nl seed))) )))))) ; a few lines of validation code (run-test (letrec ((consumer (lambda (fragment foll-fragment seed) (cons* (if (equal? foll-fragment (string #\newline)) " NL" foll-fragment) fragment seed))) (test (lambda (str expected-result) (newline) (display "body: ") (write str) (newline) (display "Result: ") (let ((result (reverse (call-with-input-string (unesc-string str) (lambda (port) (ssax:read-cdata-body port consumer '())) )))) (write result) (assert (equal? result expected-result))))) ) (test "]]>" '()) (test "abcd]]>" '("abcd" "")) (test "abcd]]]>" '("abcd" "" "]" "")) (test "abcd]]]]>" '("abcd" "" "]" "" "]" "")) (test "abcd]]]]]>" '("abcd" "" "]" "" "]" "" "]" "")) (test "abcd]]]a]]>" '("abcd" "" "]" "" "]]" "" "a" "")) (test "abc%r%ndef%n]]>" '("abc" " NL" "def" " NL")) (test "%r%n%r%n]]>" '("" " NL" "" " NL")) (test "%r%n%r%na]]>" '("" " NL" "" " NL" "a" "")) (test "%r%r%r%na]]>" '("" " NL" "" " NL" "" " NL" "a" "")) (test "abc&!!!]]>" '("abc" "&" "" "" "!!!" "")) (test "abc]]>>&]]]>and]]>" '("abc" "" "]]" "" "" ">" "" "&" "gt" "" "" "&" "amp" "" ";" "" "]" "" "]]" "" "" ">" "and" "")) )) ; procedure+: ssax:read-char-ref PORT ; ; [66] CharRef ::= '&#' [0-9]+ ';' ; | '&#x' [0-9a-fA-F]+ ';' ; ; This procedure must be called after we we have read "&#" ; that introduces a char reference. ; The procedure reads this reference and returns the corresponding char ; The current position in PORT will be after ";" that terminates ; the char reference ; Faults detected: ; WFC: XML-Spec.html#wf-Legalchar ; ; According to Section "4.1 Character and Entity References" ; of the XML Recommendation: ; "[Definition: A character reference refers to a specific character ; in the ISO/IEC 10646 character set, for example one not directly ; accessible from available input devices.]" ; Therefore, we use a ucscode->string function to convert a character ; code into the character -- *regardless* of the current character ; encoding of the input stream. (define (ssax:read-char-ref port) (let* ((base (cond ((eqv? (peek-char port) #\x) (read-char port) 16) (else 10))) (name (next-token '() '(#\;) "XML [66]" port)) (char-code (string->number name base))) (read-char port) ; read the terminating #\; char (if (integer? char-code) (ucscode->string char-code) (parser-error port "[wf-Legalchar] broken for '" name "'")))) ; procedure+: ssax:handle-parsed-entity PORT NAME ENTITIES ; CONTENT-HANDLER STR-HANDLER SEED ; ; Expand and handle a parsed-entity reference ; port - a PORT ; name - the name of the parsed entity to expand, a symbol ; entities - see ENTITIES ; content-handler -- procedure PORT ENTITIES SEED ; that is supposed to return a SEED ; str-handler - a STR-HANDLER. It is called if the entity in question ; turns out to be a pre-declared entity ; ; The result is the one returned by CONTENT-HANDLER or STR-HANDLER ; Faults detected: ; WFC: XML-Spec.html#wf-entdeclared ; WFC: XML-Spec.html#norecursion (define ssax:predefined-parsed-entities `( (,(string->symbol "amp") . "&") (,(string->symbol "lt") . "<") (,(string->symbol "gt") . ">") (,(string->symbol "apos") . "'") (,(string->symbol "quot") . "\""))) (define (ssax:handle-parsed-entity port name entities content-handler str-handler seed) (cond ; First we check the list of the declared entities ((assq name entities) => (lambda (decl-entity) (let ((ent-body (cdr decl-entity)) ; mark the list to prevent recursion (new-entities (cons (cons name #f) entities))) (cond ((string? ent-body) (call-with-input-string ent-body (lambda (port) (content-handler port new-entities seed)))) ((procedure? ent-body) (let ((port (ent-body))) (begin0 (content-handler port new-entities seed) (close-input-port port)))) (else (parser-error port "[norecursion] broken for " name)))))) ((assq name ssax:predefined-parsed-entities) => (lambda (decl-entity) (str-handler (cdr decl-entity) "" seed))) ((assq '*DEFAULT* entities) => (lambda (decl-entity) (let ((fallback (cdr decl-entity)) (new-entities (cons (cons name #f) entities))) (cond ((procedure? fallback) (call-with-input-string (fallback port name) (lambda (port) (content-handler port new-entities seed)))) (else (parser-error port "[norecursion] broken for " name)))))) (else (parser-error port "[wf-entdeclared] broken for " name)))) ; The ATTLIST Abstract Data Type ; Currently is implemented as an assoc list sorted in the ascending ; order of NAMES. (define (make-empty-attlist) '()) ; Add a name-value pair to the existing attlist preserving the order ; Return the new list, in the sorted ascending order. ; Return #f if a pair with the same name already exists in the attlist (define (attlist-add attlist name-value) (if (null? attlist) (cons name-value attlist) (case (name-compare (car name-value) (caar attlist)) ((=) #f) ((<) (cons name-value attlist)) (else (cons (car attlist) (attlist-add (cdr attlist) name-value))) ))) (define attlist-null? null?) ; Given an non-null attlist, return a pair of values: the top and the rest (define (attlist-remove-top attlist) (values (car attlist) (cdr attlist))) (define (attlist->alist attlist) attlist) (define attlist-fold fold) ; procedure+: ssax:read-attributes PORT ENTITIES ; ; This procedure reads and parses a production Attribute* ; [41] Attribute ::= Name Eq AttValue ; [10] AttValue ::= '"' ([^<&"] | Reference)* '"' ; | "'" ([^<&'] | Reference)* "'" ; [25] Eq ::= S? '=' S? ; ; ; The procedure returns an ATTLIST, of Name (as UNRES-NAME), Value (as string) ; pairs. The current character on the PORT is a non-whitespace character ; that is not an ncname-starting character. ; ; Note the following rules to keep in mind when reading an 'AttValue' ; "Before the value of an attribute is passed to the application ; or checked for validity, the XML processor must normalize it as follows: ; - a character reference is processed by appending the referenced ; character to the attribute value ; - an entity reference is processed by recursively processing the ; replacement text of the entity [see ENTITIES] ; [named entities amp lt gt quot apos are assumed pre-declared] ; - a whitespace character (#x20, #xD, #xA, #x9) is processed by appending #x20 ; to the normalized value, except that only a single #x20 is appended for a ; "#xD#xA" sequence that is part of an external parsed entity or the ; literal entity value of an internal parsed entity ; - other characters are processed by appending them to the normalized value " ; ; ; Faults detected: ; WFC: XML-Spec.html#CleanAttrVals ; WFC: XML-Spec.html#uniqattspec (define ssax:read-attributes ; ssax:read-attributes port entities (let ((value-delimeters (append ssax:S-chars '(#\< #\&)))) ; Read the AttValue from the PORT up to the delimiter ; (which can be a single or double-quote character, ; or even a symbol *eof*) ; 'prev-fragments' is the list of string fragments, accumulated ; so far, in reverse order. ; Return the list of fragments with newly read fragments ; prepended. (define (read-attrib-value delimiter port entities prev-fragments) (let* ((new-fragments (cons (next-token '() (cons delimiter value-delimeters) "XML [10]" port) prev-fragments)) (cterm (read-char port))) (cond ((or (eof-object? cterm) (eqv? cterm delimiter)) new-fragments) ((eqv? cterm char-return) ; treat a CR and CRLF as a LF (if (eqv? (peek-char port) #\newline) (read-char port)) (read-attrib-value delimiter port entities (cons " " new-fragments))) ((memv cterm ssax:S-chars) (read-attrib-value delimiter port entities (cons " " new-fragments))) ((eqv? cterm #\&) (cond ((eqv? (peek-char port) #\#) (read-char port) (read-attrib-value delimiter port entities (cons (ssax:read-char-ref port) new-fragments))) (else (read-attrib-value delimiter port entities (read-named-entity port entities new-fragments))))) (else (parser-error port "[CleanAttrVals] broken"))))) ; we have read "&" that introduces a named entity reference. ; read this reference and return the result of ; normalizing of the corresponding string ; (that is, read-attrib-value is applied to the replacement ; text of the entity) ; The current position will be after ";" that terminates ; the entity reference (define (read-named-entity port entities fragments) (let ((name (ssax:read-NCName port))) (assert-curr-char '(#\;) "XML [68]" port) (ssax:handle-parsed-entity port name entities (lambda (port entities fragments) (read-attrib-value '*eof* port entities fragments)) (lambda (str1 str2 fragments) (if (equal? "" str2) (cons str1 fragments) (cons* str2 str1 fragments))) fragments))) (lambda (port entities) (let loop ((attr-list (make-empty-attlist))) (if (not (ssax:ncname-starting-char? (ssax:skip-S port))) attr-list (let ((name (ssax:read-QName port))) (ssax:skip-S port) (assert-curr-char '(#\=) "XML [25]" port) (ssax:skip-S port) (let ((delimiter (assert-curr-char '(#\' #\" ) "XML [10]" port))) (loop (or (attlist-add attr-list (cons name (string-concatenate-reverse/shared (read-attrib-value delimiter port entities '())))) (parser-error port "[uniqattspec] broken for " name)))))))) )) ; a few lines of validation code (run-test (letrec ((test (lambda (str decl-entities expected-res) (newline) (display "input: ") (write str) (newline) (display "Result: ") (let ((result (call-with-input-string (unesc-string str) (lambda (port) (ssax:read-attributes port decl-entities))))) (write result) (newline) (assert (equal? result expected-res)))))) (test "" '() '()) (test "href='http://a%tb%r%n%r%n%nc'" '() `((,(string->symbol "href") . "http://a b c"))) (test "href='http://a%tb%r%r%n%rc'" '() `((,(string->symbol "href") . "http://a b c"))) (test "_1 ='12&' _2= \"%r%n%t12 3\">" '() `((_1 . "12&") (_2 . ,(unesc-string " 12%n3")))) (test "%tAbc='<&> '%nNext='12&ent;34' />" '((ent . "<xx>")) `((,(string->symbol "Abc") . ,(unesc-string "<&>%n")) (,(string->symbol "Next") . "12<xx>34"))) (test "%tAbc='<&> '%nNext='12&ent;34' />" '((ent . "<xx>")) `((,(string->symbol "Abc") . ,(unesc-string "<&>%r")) (,(string->symbol "Next") . "12<xx>34"))) (test "%tAbc='<&> '%nNext='12&en;34' />" `((en . ,(lambda () (open-input-string ""xx'")))) `((,(string->symbol "Abc") . ,(unesc-string "<&>%n")) (,(string->symbol "Next") . "12\"xx'34"))) (test "%tAbc='<&> '%nNext='12&ent;34' />" '((ent . "<&ent1;T;>") (ent1 . "&")) `((,(string->symbol "Abc") . ,(unesc-string "<&>%n")) (,(string->symbol "Next") . "12<&T;>34"))) (test "%tAbc='<&> '%nNext='12&ent;34' />" `((*DEFAULT* . ,(lambda (port name) (case name ((ent) "<&ent1;T;>") ((ent1) "&") (else (error "unrecognized" name)))))) `((,(string->symbol "Abc") . ,(unesc-string "<&>%n")) (,(string->symbol "Next") . "12<&T;>34"))) (assert (failed? (test "%tAbc='<&> '%nNext='12&ent;34' />" '((ent . "<&ent1;T;>") (ent1 . "&")) '()))) (assert (failed? (test "%tAbc='<&> '%nNext='12&ent;34' />" '((ent . "<&ent;T;>") (ent1 . "&")) '()))) (assert (failed? (test "%tAbc='<&> '%nNext='12&ent;34' />" '((ent . "<&ent1;T;>") (ent1 . "&ent;")) '()))) (test "html:href='http://a%tb%r%n%r%n%nc'" '() `(((,(string->symbol "html") . ,(string->symbol "href")) . "http://a b c"))) (test "html:href='ref1' html:src='ref2'" '() `(((,(string->symbol "html") . ,(string->symbol "href")) . "ref1") ((,(string->symbol "html") . ,(string->symbol "src")) . "ref2"))) (test "html:href='ref1' xml:html='ref2'" '() `(((,(string->symbol "html") . ,(string->symbol "href")) . "ref1") ((,ssax:Prefix-XML . ,(string->symbol "html")) . "ref2"))) (assert (failed? (test "html:href='ref1' html:href='ref2'" '() '()))) (assert (failed? (test "html:href='<' html:href='ref2'" '() '()))) (assert (failed? (test "html:href='ref1' html:href='&ref2;'" '() '()))) )) ; ssax:resolve-name PORT UNRES-NAME NAMESPACES apply-default-ns? ; ; Convert an UNRES-NAME to a RES-NAME given the appropriate NAMESPACES ; declarations. ; the last parameter apply-default-ns? determines if the default ; namespace applies (for instance, it does not for attribute names) ; ; Per REC-xml-names/#nsc-NSDeclared, "xml" prefix is considered pre-declared ; and bound to the namespace name "http://www.w3.org/XML/1998/namespace". ; ; This procedure tests for the namespace constraints: ; http://www.w3.org/TR/REC-xml-names/#nsc-NSDeclared (define (ssax:resolve-name port unres-name namespaces apply-default-ns?) (cond ((pair? unres-name) ; it's a QNAME (cons (cond ((assq (car unres-name) namespaces) => cadr) ((eq? (car unres-name) ssax:Prefix-XML) ssax:Prefix-XML) (else (parser-error port "[nsc-NSDeclared] broken; prefix " (car unres-name)))) (cdr unres-name))) (apply-default-ns? ; Do apply the default namespace, if any (let ((default-ns (assq '*DEFAULT* namespaces))) (if (and default-ns (cadr default-ns)) (cons (cadr default-ns) unres-name) unres-name))) ; no default namespace declared (else unres-name))) ; no prefix, don't apply the default-ns (run-test (let* ((namespaces '((HTML UHTML . URN-HTML) (HTML UHTML-1 . URN-HTML) (A UHTML . URN-HTML))) (namespaces-def (cons '(*DEFAULT* DEF . URN-DEF) namespaces)) (namespaces-undef (cons '(*DEFAULT* #f . #f) namespaces-def)) (port (current-input-port))) (assert (equal? 'ABC (ssax:resolve-name port 'ABC namespaces #t))) (assert (equal? '(DEF . ABC) (ssax:resolve-name port 'ABC namespaces-def #t))) (assert (equal? 'ABC (ssax:resolve-name port 'ABC namespaces-def #f))) (assert (equal? 'ABC (ssax:resolve-name port 'ABC namespaces-undef #t))) (assert (equal? '(UHTML . ABC) (ssax:resolve-name port '(HTML . ABC) namespaces-def #t))) (assert (equal? '(UHTML . ABC) (ssax:resolve-name port '(HTML . ABC) namespaces-def #f))) (assert (equal? `(,ssax:Prefix-XML . space) (ssax:resolve-name port `(,(string->symbol "xml") . space) namespaces-def #f))) (assert (failed? (ssax:resolve-name port '(XXX . ABC) namespaces-def #f))) )) ; procedure+: ssax:uri-string->symbol URI-STR ; Convert a URI-STR to an appropriate symbol (define (ssax:uri-string->symbol uri-str) (string->symbol uri-str)) ; procedure+: ssax:complete-start-tag TAG PORT ELEMS ENTITIES NAMESPACES ; ; This procedure is to complete parsing of a start-tag markup. The ; procedure must be called after the start tag token has been ; read. TAG is an UNRES-NAME. ELEMS is an instance of xml-decl::elems; ; it can be #f to tell the function to do _no_ validation of elements ; and their attributes. ; ; This procedure returns several values: ; ELEM-GI: a RES-NAME. ; ATTRIBUTES: element's attributes, an ATTLIST of (RES-NAME . STRING) ; pairs. The list does NOT include xmlns attributes. ; NAMESPACES: the input list of namespaces amended with namespace ; (re-)declarations contained within the start-tag under parsing ; ELEM-CONTENT-MODEL ; On exit, the current position in PORT will be the first character after ; #\> that terminates the start-tag markup. ; ; Faults detected: ; VC: XML-Spec.html#enum ; VC: XML-Spec.html#RequiredAttr ; VC: XML-Spec.html#FixedAttr ; VC: XML-Spec.html#ValueType ; WFC: XML-Spec.html#uniqattspec (after namespaces prefixes are resolved) ; VC: XML-Spec.html#elementvalid ; WFC: REC-xml-names/#dt-NSName ; Note, although XML Recommendation does not explicitly say it, ; xmlns and xmlns: attributes don't have to be declared (although they ; can be declared, to specify their default value) ; Procedure: ssax:complete-start-tag tag-head port elems entities namespaces (define ssax:complete-start-tag (let ((xmlns (string->symbol "xmlns")) (largest-dummy-decl-attr (list ssax:largest-unres-name #f #f #f))) ; Scan through the attlist and validate it, against decl-attrs ; Return an assoc list with added fixed or implied attrs. ; Note that both attlist and decl-attrs are ATTLISTs, and therefore, ; sorted (define (validate-attrs port attlist decl-attrs) ; Check to see decl-attr is not of use type REQUIRED. Add ; the association with the default value, if any declared (define (add-default-decl decl-attr result) (let*-values (((attr-name content-type use-type default-value) (apply values decl-attr))) (and (eq? use-type 'REQUIRED) (parser-error port "[RequiredAttr] broken for" attr-name)) (if default-value (cons (cons attr-name default-value) result) result))) (let loop ((attlist attlist) (decl-attrs decl-attrs) (result '())) (if (attlist-null? attlist) (attlist-fold add-default-decl result decl-attrs) (let*-values (((attr attr-others) (attlist-remove-top attlist)) ((decl-attr other-decls) (if (attlist-null? decl-attrs) (values largest-dummy-decl-attr decl-attrs) (attlist-remove-top decl-attrs))) ) (case (name-compare (car attr) (car decl-attr)) ((<) (if (or (eq? xmlns (car attr)) (and (pair? (car attr)) (eq? xmlns (caar attr)))) (loop attr-others decl-attrs (cons attr result)) (parser-error port "[ValueType] broken for " attr))) ((>) (loop attlist other-decls (add-default-decl decl-attr result))) (else ; matched occurrence of an attr with its declaration (let*-values (((attr-name content-type use-type default-value) (apply values decl-attr))) ; Run some tests on the content of the attribute (cond ((eq? use-type 'FIXED) (or (equal? (cdr attr) default-value) (parser-error port "[FixedAttr] broken for " attr-name))) ((eq? content-type 'CDATA) #t) ; everything goes ((pair? content-type) (or (member (cdr attr) content-type) (parser-error port "[enum] broken for " attr-name "=" (cdr attr)))) (else (ssax:warn port "declared content type " content-type " not verified yet"))) (loop attr-others other-decls (cons attr result))))) )))) ; Add a new namespace declaration to namespaces. ; First we convert the uri-str to a uri-symbol and search namespaces for ; an association (_ user-prefix . uri-symbol). ; If found, we return the argument namespaces with an association ; (prefix user-prefix . uri-symbol) prepended. ; Otherwise, we prepend (prefix uri-symbol . uri-symbol) (define (add-ns port prefix uri-str namespaces) (and (equal? "" uri-str) (parser-error port "[dt-NSName] broken for " prefix)) (let ((uri-symbol (ssax:uri-string->symbol uri-str))) (let loop ((nss namespaces)) (cond ((null? nss) (cons (cons* prefix uri-symbol uri-symbol) namespaces)) ((eq? uri-symbol (cddar nss)) (cons (cons* prefix (cadar nss) uri-symbol) namespaces)) (else (loop (cdr nss))))))) ; partition attrs into proper attrs and new namespace declarations ; return two values: proper attrs and the updated namespace declarations (define (adjust-namespace-decl port attrs namespaces) (let loop ((attrs attrs) (proper-attrs '()) (namespaces namespaces)) (cond ((null? attrs) (values proper-attrs namespaces)) ((eq? xmlns (caar attrs)) ; re-decl of the default namespace (loop (cdr attrs) proper-attrs (if (equal? "" (cdar attrs)) ; un-decl of the default ns (cons (cons* '*DEFAULT* #f #f) namespaces) (add-ns port '*DEFAULT* (cdar attrs) namespaces)))) ((and (pair? (caar attrs)) (eq? xmlns (caaar attrs))) (loop (cdr attrs) proper-attrs (add-ns port (cdaar attrs) (cdar attrs) namespaces))) (else (loop (cdr attrs) (cons (car attrs) proper-attrs) namespaces))))) ; The body of the function (lambda (tag-head port elems entities namespaces) (let*-values (((attlist) (ssax:read-attributes port entities)) ((empty-el-tag?) (begin (ssax:skip-S port) (and (eqv? #\/ (assert-curr-char '(#\> #\/) "XML [40], XML [44], no '>'" port)) (assert-curr-char '(#\>) "XML [44], no '>'" port)))) ((elem-content decl-attrs) ; see xml-decl for their type (if elems ; elements declared: validate! (cond ((assoc tag-head elems) => (lambda (decl-elem) ; of type xml-decl::decl-elem (values (if empty-el-tag? 'EMPTY-TAG (cadr decl-elem)) (caddr decl-elem)))) (else (parser-error port "[elementvalid] broken, no decl for " tag-head))) (values ; non-validating parsing (if empty-el-tag? 'EMPTY-TAG 'ANY) #f) ; no attributes declared )) ((merged-attrs) (if decl-attrs (validate-attrs port attlist decl-attrs) (attlist->alist attlist))) ((proper-attrs namespaces) (adjust-namespace-decl port merged-attrs namespaces)) ) ;(cerr "proper attrs: " proper-attrs nl) ; build the return value (values (ssax:resolve-name port tag-head namespaces #t) (fold-right (lambda (name-value attlist) (or (attlist-add attlist (cons (ssax:resolve-name port (car name-value) namespaces #f) (cdr name-value))) (parser-error port "[uniqattspec] after NS expansion broken for " name-value))) (make-empty-attlist) proper-attrs) namespaces elem-content))))) (run-test (let* ((urn-a (string->symbol "urn:a")) (urn-b (string->symbol "urn:b")) (urn-html (string->symbol "http://w3c.org/html")) (namespaces `((#f '"UHTML" . ,urn-html) ('"A" '"UA" . ,urn-a))) (test (lambda (tag-head-name elems str) (call-with-input-string str (lambda (port) (call-with-values (lambda () (ssax:complete-start-tag (call-with-input-string tag-head-name (lambda (port) (ssax:read-QName port))) port elems '() namespaces)) list)))))) ; First test with no validation of elements ;(test "TAG1" #f "") (assert (equal? `('"TAG1" () ,namespaces ANY) (test "TAG1" #f ">"))) (assert (equal? `('"TAG1" () ,namespaces EMPTY-TAG) (test "TAG1" #f "/>"))) (assert (equal? `('"TAG1" (('"HREF" . "a")) ,namespaces EMPTY-TAG) (test "TAG1" #f "HREF='a'/>"))) (assert (equal? `(('"UA" . '"TAG1") (('"HREF" . "a")) ,(cons `(*DEFAULT* '"UA" . ,urn-a) namespaces) ANY) (test "TAG1" #f "HREF='a' xmlns='urn:a'>"))) (assert (equal? `('"TAG1" (('"HREF" . "a")) ,(cons '(*DEFAULT* #f . #f) namespaces) ANY) (test "TAG1" #f "HREF='a' xmlns=''>"))) (assert (failed? (test "UA:TAG1" #f "HREF='a' xmlns=''/>"))) (assert (equal? `(('"UA" . '"TAG1") ((('"UA" . '"HREF") . "a")) ,(cons '(*DEFAULT* #f . #f) namespaces) ANY) (test "A:TAG1" #f "A:HREF='a' xmlns=''>"))) (assert (equal? `(('"UA" . '"TAG1") ((('"UA" . '"HREF") . "a")) ,(cons `(*DEFAULT* ,urn-b . ,urn-b) namespaces) ANY) (test "A:TAG1" #f "A:HREF='a' xmlns='urn:b'>"))) (assert (failed? (test "B:TAG1" #f "A:HREF='a' xmlns:b=''/>"))) (assert (equal? `((,urn-b . '"TAG1") ((('"UA" . '"HREF") . "a")) ,(cons `('"B" ,urn-b . ,urn-b) namespaces) ANY) (test "B:TAG1" #f "A:HREF='a' xmlns:B='urn:b'>"))) (assert (equal? `((,urn-b . '"TAG1") ((('"UA" . '"HREF") . "a") ((,urn-b . '"SRC") . "b")) ,(cons `('"B" ,urn-b . ,urn-b) namespaces) ANY) (test "B:TAG1" #f "B:SRC='b' A:HREF='a' xmlns:B='urn:b'>"))) (assert (equal? `((,urn-b . '"TAG1") ((('"UA" . '"HREF") . "a") ((,urn-b . '"HREF") . "b")) ,(cons `('"B" ,urn-b . ,urn-b) namespaces) ANY) (test "B:TAG1" #f "B:HREF=\"b\" A:HREF='a' xmlns:B='urn:b'>"))) ; must be an error! Duplicate attr (assert (failed? (test "B:TAG1" #f "HREF=\"b\" HREF='a' xmlns:B='urn:a'/>"))) ; must be an error! Duplicate attr after ns expansion (assert (failed? (test "B:TAG1" #f "B:HREF=\"b\" A:HREF='a' xmlns:B='urn:a'/>"))) (assert (equal? `(('"UA" . '"TAG1") (('"HREF" . "a") (('"UA" . '"HREF") . "b")) ,(cons `(*DEFAULT* '"UA" . ,urn-a) namespaces) ANY) (test "TAG1" #f "A:HREF=\"b\" HREF='a' xmlns='urn:a'>"))) (assert (equal? `('"TAG1" ((('"UHTML" . '"HREF") . "a") ((,urn-b . '"HREF") . "b")) ,(append `( ('"HTML" '"UHTML" . ,urn-html) ('"B" ,urn-b . ,urn-b)) namespaces) ANY) (test "TAG1" #f "B:HREF=\"b\" xmlns:B='urn:b' xmlns:HTML='http://w3c.org/html' HTML:HREF='a' >"))) ; Now test the validating parsing ; No decl for tag1 (assert (failed? (test "TAG1" '((TAG2 ANY ())) "B:HREF='b' xmlns:B='urn:b'>"))) ; No decl for HREF elem ;; (cond-expand ;; ((not (or scm mit-scheme)) ; Regretfully, SCM treats '() as #f ;; (assert (failed? ;; (test "TAG1" '(('"TAG1" ANY ())) ;; "B:HREF='b' xmlns:B='urn:b'>")))) ;; (else #t)) ; No decl for HREF elem (assert (failed? (test "TAG1" '(('"TAG1" ANY (('"HREF1" CDATA IMPLIED #f)))) "B:HREF='b' xmlns:B='urn:b'>"))) (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces EMPTY-TAG) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f)))) "HREF='b'/>"))) (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f)))) "HREF='b'>"))) ; Req'd attribute not given error (assert (failed? (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f)))) ">"))) ; Wrong content-type of the attribute (assert (failed? (test "TAG1" '(('"TAG1" PCDATA (('"HREF" ("c") REQUIRED #f)))) "HREF='b'>"))) (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" ("c" "b") IMPLIED #f)))) "HREF='b'>"))) (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA IMPLIED "c")))) "HREF='b'>"))) ; Bad fixed attribute (assert (failed? (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA FIXED "c")))) "HREF='b'>"))) (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA FIXED "b")))) "HREF='b'>"))) (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA FIXED "b")))) ">"))) (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA IMPLIED "b")))) ">"))) (assert (equal? `('"TAG1" () ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA IMPLIED #f)))) ">"))) ; Undeclared attr (assert (failed? (test "TAG1" '(('"TAG1" PCDATA ((('"A" . '"HREF") CDATA IMPLIED "c")))) "HREF='b'>"))) (assert (equal? `('"TAG1" (('"HREF" . "b") (('"UA" . '"HREF") . "c")) ,namespaces PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f) (('"A" . '"HREF") CDATA IMPLIED "c")))) "HREF='b'>"))) (assert (equal? `(('"UA" . '"TAG1") (('"HREF" . "b") (('"UA" . '"HREF") . "c")) ,namespaces PCDATA) (test "A:TAG1" '((('"A" . '"TAG1") PCDATA (('"HREF" NMTOKEN REQUIRED #f) (('"A" . '"HREF") CDATA IMPLIED "c")))) "HREF='b'>"))) (assert (equal? `((,urn-b . '"TAG1") (('"HREF" . "b")) ,(cons `('"B" ,urn-b . ,urn-b) namespaces) PCDATA) (test "B:TAG1" '((('"B" . '"TAG1") PCDATA (('"HREF" CDATA REQUIRED #f) (('"xmlns" . '"B") CDATA IMPLIED "urn:b")))) "HREF='b'>"))) (assert (equal? `((,urn-b . '"TAG1") (((,urn-b . '"HREF") . "b")) ,(cons `('"B" ,urn-b . ,urn-b) namespaces) PCDATA) (test "B:TAG1" '((('"B" . '"TAG1") PCDATA ((('"B" . '"HREF") CDATA REQUIRED #f) (('"xmlns" . '"B") CDATA IMPLIED "urn:b")))) "B:HREF='b'>"))) (assert (equal? `((,urn-b . '"TAG1") (('"HREF" . "b")) ,(cons `(*DEFAULT* ,urn-b . ,urn-b) namespaces) PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f) ('"xmlns" CDATA IMPLIED "urn:b")))) "HREF='b'>"))) ; xmlns not declared (assert (equal? `((,urn-b . '"TAG1") (('"HREF" . "b")) ,(cons `(*DEFAULT* ,urn-b . ,urn-b) namespaces) PCDATA) (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f) ))) "HREF='b' xmlns='urn:b'>"))) ; xmlns:B not declared (assert (equal? `((,urn-b . '"TAG1") (((,urn-b . '"HREF") . "b")) ,(cons `('"B" ,urn-b . ,urn-b) namespaces) PCDATA) (test "B:TAG1" '((('"B" . '"TAG1") PCDATA ((('"B" . '"HREF") CDATA REQUIRED #f) ))) "B:HREF='b' xmlns:B='urn:b'>"))) )) ; procedure+: ssax:read-external-id PORT ; ; This procedure parses an ExternalID production: ; [75] ExternalID ::= 'SYSTEM' S SystemLiteral ; | 'PUBLIC' S PubidLiteral S SystemLiteral ; [11] SystemLiteral ::= ('"' [^"]* '"') | ("'" [^']* "'") ; [12] PubidLiteral ::= '"' PubidChar* '"' | "'" (PubidChar - "'")* "'" ; [13] PubidChar ::= #x20 | #xD | #xA | [a-zA-Z0-9] ; | [-'()+,./:=?;!*#@$_%] ; ; This procedure is supposed to be called when an ExternalID is expected; ; that is, the current character must be either #\S or #\P that start ; correspondingly a SYSTEM or PUBLIC token. This procedure returns the ; SystemLiteral as a string. A PubidLiteral is disregarded if present. (define (ssax:read-external-id port) (let ((discriminator (ssax:read-NCName port))) (assert-curr-char ssax:S-chars "space after SYSTEM or PUBLIC" port) (ssax:skip-S port) (let ((delimiter (assert-curr-char '(#\' #\" ) "XML [11], XML [12]" port))) (cond ((eq? discriminator (string->symbol "SYSTEM")) (begin0 (next-token '() (list delimiter) "XML [11]" port) (read-char port) ; reading the closing delim )) ((eq? discriminator (string->symbol "PUBLIC")) (skip-until (list delimiter) port) (assert-curr-char ssax:S-chars "space after PubidLiteral" port) (ssax:skip-S port) (let* ((delimiter (assert-curr-char '(#\' #\" ) "XML [11]" port)) (systemid (next-token '() (list delimiter) "XML [11]" port))) (read-char port) ; reading the closing delim systemid)) (else (parser-error port "XML [75], " discriminator " rather than SYSTEM or PUBLIC")))))) ;----------------------------------------------------------------------------- ; Higher-level parsers and scanners ; ; They parse productions corresponding to the whole (document) entity ; or its higher-level pieces (prolog, root element, etc). ; Scan the Misc production in the context ; [1] document ::= prolog element Misc* ; [22] prolog ::= XMLDecl? Misc* (doctypedec l Misc*)? ; [27] Misc ::= Comment | PI | S ; ; The following function should be called in the prolog or epilog contexts. ; In these contexts, whitespaces are completely ignored. ; The return value from ssax:scan-Misc is either a PI-token, ; a DECL-token, a START token, or EOF. ; Comments are ignored and not reported. (define (ssax:scan-Misc port) (let loop ((c (ssax:skip-S port))) (cond ((eof-object? c) c) ((not (char=? c #\<)) (parser-error port "XML [22], char '" c "' unexpected")) (else (let ((token (ssax:read-markup-token port))) (case (xml-token-kind token) ((COMMENT) (loop (ssax:skip-S port))) ((PI DECL START) token) (else (parser-error port "XML [22], unexpected token of kind " (xml-token-kind token) )))))))) ; procedure+: ssax:read-char-data PORT EXPECT-EOF? STR-HANDLER SEED ; ; This procedure is to read the character content of an XML document ; or an XML element. ; [43] content ::= ; (element | CharData | Reference | CDSect | PI ; | Comment)* ; To be more precise, the procedure reads CharData, expands CDSect ; and character entities, and skips comments. The procedure stops ; at a named reference, EOF, at the beginning of a PI or a start/end tag. ; ; port ; a PORT to read ; expect-eof? ; a boolean indicating if EOF is normal, i.e., the character ; data may be terminated by the EOF. EOF is normal ; while processing a parsed entity. ; str-handler ; a STR-HANDLER ; seed ; an argument passed to the first invocation of STR-HANDLER. ; ; The procedure returns two results: SEED and TOKEN. ; The SEED is the result of the last invocation of STR-HANDLER, or the ; original seed if STR-HANDLER was never called. ; ; TOKEN can be either an eof-object (this can happen only if ; expect-eof? was #t), or: ; - an xml-token describing a START tag or an END-tag; ; For a start token, the caller has to finish reading it. ; - an xml-token describing the beginning of a PI. It's up to an ; application to read or skip through the rest of this PI; ; - an xml-token describing a named entity reference. ; ; CDATA sections and character references are expanded inline and ; never returned. Comments are silently disregarded. ; ; As the XML Recommendation requires, all whitespace in character data ; must be preserved. However, a CR character (#xD) must be disregarded ; if it appears before a LF character (#xA), or replaced by a #xA character ; otherwise. See Secs. 2.10 and 2.11 of the XML Recommendation. See also ; the canonical XML Recommendation. ; ssax:read-char-data port expect-eof? str-handler seed (define ssax:read-char-data (let ((terminators-usual (list #\< #\& char-return)) (terminators-usual-eof (list #\< '*eof* #\& char-return)) (handle-fragment (lambda (fragment str-handler seed) (if (string-null? fragment) seed (str-handler fragment "" seed)))) ) (lambda (port expect-eof? str-handler seed) ; Very often, the first character we encounter is #\< ; Therefore, we handle this case in a special, fast path (if (eqv? #\< (peek-char port)) ; The fast path (let ((token (ssax:read-markup-token port))) (case (xml-token-kind token) ((START END) ; The most common case (values seed token)) ((CDSECT) (let ((seed (ssax:read-cdata-body port str-handler seed))) (ssax:read-char-data port expect-eof? str-handler seed))) ((COMMENT) (ssax:read-char-data port expect-eof? str-handler seed)) (else (values seed token)))) ; The slow path (let ((char-data-terminators (if expect-eof? terminators-usual-eof terminators-usual))) (let loop ((seed seed)) (let* ((fragment (next-token '() char-data-terminators "reading char data" port)) (term-char (peek-char port)) ; one of char-data-terminators ) (if (eof-object? term-char) (values (handle-fragment fragment str-handler seed) term-char) (case term-char ((#\<) (let ((token (ssax:read-markup-token port))) (case (xml-token-kind token) ((CDSECT) (loop (ssax:read-cdata-body port str-handler (handle-fragment fragment str-handler seed)))) ((COMMENT) (loop (handle-fragment fragment str-handler seed))) (else (values (handle-fragment fragment str-handler seed) token))))) ((#\&) (case (peek-next-char port) ((#\#) (read-char port) (loop (str-handler fragment (ssax:read-char-ref port) seed))) (else (let ((name (ssax:read-NCName port))) (assert-curr-char '(#\;) "XML [68]" port) (values (handle-fragment fragment str-handler seed) (make-xml-token 'ENTITY-REF name)))))) (else ; This must be a CR character (if (eqv? (peek-next-char port) #\newline) (read-char port)) (loop (str-handler fragment (string #\newline) seed)))) )))))))) ; a few lines of validation code (run-test (letrec ((a-tag (make-xml-token 'START (string->symbol "BR"))) (a-ref (make-xml-token 'ENTITY-REF (string->symbol "lt"))) (eof-object (lambda () eof-object)) ; a unique value (str-handler (lambda (fragment foll-fragment seed) (if (string-null? foll-fragment) (cons fragment seed) (cons* foll-fragment fragment seed)))) (test (lambda (str expect-eof? expected-data expected-token) (newline) (display "body: ") (write str) (newline) (display "Result: ") (let*-values (((seed token) (call-with-input-string (unesc-string str) (lambda (port) (ssax:read-char-data port expect-eof? str-handler '())))) ((result) (reverse seed))) (write result) (display " ") (display token) (assert (equal? result (map unesc-string expected-data)) (if (eq? expected-token eof-object) (eof-object? token) (equal? token expected-token)))))) ) (test "" #t '() eof-object) (assert (failed? (test "" #f '() eof-object))) (test " " #t '(" ") eof-object) (test "<BR/>" #f '() a-tag) (test " <BR />" #f '(" ") a-tag) (test " <" #f '(" ") a-ref) (test " a<" #f '(" a") a-ref) (test " a <" #f '(" a ") a-ref) (test " <!-- comment--> a a<BR/>" #f '(" " " a a") a-tag) (test " <!-- comment-->%ra a<BR/>" #f '(" " "" "%n" "a a") a-tag) (test " <!-- comment-->%r%na a<BR/>" #f '(" " "" "%n" "a a") a-tag) (test " <!-- comment-->%r%na%t%r%r%na<BR/>" #f '(" " "" "%n" "a%t" "%n" "" "%n" "a") a-tag) (test "a<!-- comment--> a a<BR/>" #f '("a" " a a") a-tag) (test "!<BR/>" #f '("" "!") a-tag) (test "!%n<BR/>" #f '("" "!" "%n") a-tag) (test "%t!%n<BR/>" #f '("%t" "!" "%n") a-tag) (test "%t!%na a<BR/>" #f '("%t" "!" "%na a") a-tag) (test "%t!%ra a<BR/>" #f '("%t" "!" "" "%n" "a a") a-tag) (test "%t!%r%na a<BR/>" #f '("%t" "!" "" "%n" "a a") a-tag) (test " %ta ! b <BR/>" #f '(" %ta " "!" " b ") a-tag) (test " %ta b <BR/>" #f '(" %ta " " " " b ") a-tag) (test "<![CDATA[<]]><BR/>" #f '("<") a-tag) (test "<![CDATA[]]]><BR/>" #f '("]") a-tag) (test "%t<![CDATA[<]]><BR/>" #f '("%t" "<") a-tag) (test "%t<![CDATA[<]]>a b<BR/>" #f '("%t" "<" "a b") a-tag) (test "%t<![CDATA[<]]> a b<BR/>" #f '("%t" "<" " a b") a-tag) (test "%td <![CDATA[ <%r%r%n]]> a b<BR/>" #f '("%td " " <" "%n" "" "%n" " a b") a-tag) )) ; procedure+: ssax:assert-token TOKEN KIND GI ; Make sure that TOKEN is of anticipated KIND and has anticipated GI ; Note GI argument may actually be a pair of two symbols, Namespace ; URI or the prefix, and of the localname. ; If the assertion fails, error-cont is evaluated by passing it ; three arguments: token kind gi. The result of error-cont is returned. (define (ssax:assert-token token kind gi error-cont) (or (and (xml-token? token) (eq? kind (xml-token-kind token)) (equal? gi (xml-token-head token))) (error-cont token kind gi))) ;======================================================================== ; Highest-level parsers: XML to SXML ; These parsers are a set of syntactic forms to instantiate a SSAX parser. ; A user can instantiate the parser to do the full validation, or ; no validation, or any particular validation. The user specifies ; which PI he wants to be notified about. The user tells what to do ; with the parsed character and element data. The latter handlers ; determine if the parsing follows a SAX or a DOM model. ; syntax: ssax:make-pi-parser my-pi-handlers ; Create a parser to parse and process one Processing Element (PI). ; my-pi-handlers ; An assoc list of pairs (PI-TAG . PI-HANDLER) ; where PI-TAG is an NCName symbol, the PI target, and ; PI-HANDLER is a procedure PORT PI-TAG SEED ; where PORT points to the first symbol after the PI target. ; The handler should read the rest of the PI up to and including ; the combination '?>' that terminates the PI. The handler should ; return a new seed. ; One of the PI-TAGs may be the symbol *DEFAULT*. The corresponding ; handler will handle PIs that no other handler will. If the ; *DEFAULT* PI-TAG is not specified, ssax:make-pi-parser will assume ; the default handler that skips the body of the PI ; ; The output of the ssax:make-pi-parser is a procedure ; PORT PI-TAG SEED ; that will parse the current PI according to the user-specified handlers. ; ; The previous version of ssax:make-pi-parser was a low-level macro: ; (define-macro ssax:make-pi-parser ; (lambda (my-pi-handlers) ; `(lambda (port target seed) ; (case target ; ; Generate the body of the case statement ; ,@(let loop ((pi-handlers my-pi-handlers) (default #f)) ; (cond ; ((null? pi-handlers) ; (if default `((else (,default port target seed))) ; '((else ; (ssax:warn port "Skipping PI: " target nl) ; (ssax:skip-pi port) ; seed)))) ; ((eq? '*DEFAULT* (caar pi-handlers)) ; (loop (cdr pi-handlers) (cdar pi-handlers))) ; (else ; (cons ; `((,(caar pi-handlers)) (,(cdar pi-handlers) port target seed)) ; (loop (cdr pi-handlers) default))))))))) (define-syntax ssax:make-pi-parser (syntax-rules () ((ssax:make-pi-parser orig-handlers) (letrec-syntax ; Generate the clauses of the case statement ((loop (syntax-rules (*DEFAULT*) ((loop () #f accum port target seed) ; no default (make-case ((else (ssax:warn port "Skipping PI: " target nl) (ssax:skip-pi port) seed) . accum) () target)) ((loop () default accum port target seed) (make-case ((else (default port target seed)) . accum) () target)) ((loop ((*DEFAULT* . default) . handlers) old-def accum port target seed) (loop handlers default accum port target seed)) ((loop ((tag . handler) . handlers) default accum port target seed) (loop handlers default (((tag) (handler port target seed)) . accum) port target seed)) )) (make-case ; Reverse the clauses, make the 'case' (syntax-rules () ((make-case () clauses target) (case target . clauses)) ((make-case (clause . clauses) accum target) (make-case clauses (clause . accum) target))) )) (lambda (port target seed) (loop orig-handlers #f () port target seed)) )))) (run-test (pp (ssax:make-pi-parser ())) (pp (ssax:make-pi-parser ((xml . (lambda (port target seed) seed))))) (pp (ssax:make-pi-parser ((xml . (lambda (port target seed) seed)) (html . list) (*DEFAULT* . ssax:warn)))) ) ; syntax: ssax:make-elem-parser my-new-level-seed my-finish-element ; my-char-data-handler my-pi-handlers ; Create a parser to parse and process one element, including its ; character content or children elements. The parser is typically ; applied to the root element of a document. ; my-new-level-seed ; procedure ELEM-GI ATTRIBUTES NAMESPACES EXPECTED-CONTENT SEED ; where ELEM-GI is a RES-NAME of the element ; about to be processed. ; This procedure is to generate the seed to be passed ; to handlers that process the content of the element. ; This is the function identified as 'fdown' in the denotational ; semantics of the XML parser given in the title comments to this ; file. ; ; my-finish-element ; procedure ELEM-GI ATTRIBUTES NAMESPACES PARENT-SEED SEED ; This procedure is called when parsing of ELEM-GI is finished. ; The SEED is the result from the last content parser (or ; from my-new-level-seed if the element has the empty content). ; PARENT-SEED is the same seed as was passed to my-new-level-seed. ; The procedure is to generate a seed that will be the result ; of the element parser. ; This is the function identified as 'fup' in the denotational ; semantics of the XML parser given in the title comments to this ; file. ; ; my-char-data-handler ; A STR-HANDLER ; ; my-pi-handlers ; See ssax:make-pi-handler above ; ; The generated parser is a ; procedure START-TAG-HEAD PORT ELEMS ENTITIES ; NAMESPACES PRESERVE-WS? SEED ; The procedure must be called after the start tag token has been ; read. START-TAG-HEAD is an UNRES-NAME from the start-element tag. ; ELEMS is an instance of xml-decl::elems. ; See ssax:complete-start-tag::preserve-ws? ; Faults detected: ; VC: XML-Spec.html#elementvalid ; WFC: XML-Spec.html#GIMatch (define-syntax ssax:make-elem-parser (syntax-rules () ((ssax:make-elem-parser my-new-level-seed my-finish-element my-char-data-handler my-pi-handlers) (lambda (start-tag-head port elems entities namespaces preserve-ws? seed) (define xml-space-gi (cons ssax:Prefix-XML (string->symbol "space"))) (let handle-start-tag ((start-tag-head start-tag-head) (port port) (entities entities) (namespaces namespaces) (preserve-ws? preserve-ws?) (parent-seed seed)) (let*-values (((elem-gi attributes namespaces expected-content) (ssax:complete-start-tag start-tag-head port elems entities namespaces)) ((seed) (my-new-level-seed elem-gi attributes namespaces expected-content parent-seed))) (case expected-content ((EMPTY-TAG) (my-finish-element elem-gi attributes namespaces parent-seed seed)) ((EMPTY) ; The end tag must immediately follow (ssax:assert-token (and (eqv? #\< (ssax:skip-S port)) (ssax:read-markup-token port)) 'END start-tag-head (lambda (token exp-kind exp-head) (parser-error port "[elementvalid] broken for " token " while expecting " exp-kind exp-head))) (my-finish-element elem-gi attributes namespaces parent-seed seed)) (else ; reading the content... (let ((preserve-ws? ; inherit or set the preserve-ws? flag (cond ((assoc xml-space-gi attributes) => (lambda (name-value) (equal? "preserve" (cdr name-value)))) (else preserve-ws?)))) (let loop ((port port) (entities entities) (expect-eof? #f) (seed seed)) (let*-values (((seed term-token) (ssax:read-char-data port expect-eof? my-char-data-handler seed))) (if (eof-object? term-token) seed (case (xml-token-kind term-token) ((END) (ssax:assert-token term-token 'END start-tag-head (lambda (token exp-kind exp-head) (parser-error port "[GIMatch] broken for " term-token " while expecting " exp-kind exp-head))) (my-finish-element elem-gi attributes namespaces parent-seed seed)) ((PI) (let ((seed ((ssax:make-pi-parser my-pi-handlers) port (xml-token-head term-token) seed))) (loop port entities expect-eof? seed))) ((ENTITY-REF) (let ((seed (ssax:handle-parsed-entity port (xml-token-head term-token) entities (lambda (port entities seed) (loop port entities #t seed)) my-char-data-handler seed))) ; keep on reading the content after ent (loop port entities expect-eof? seed))) ((START) ; Start of a child element (if (eq? expected-content 'PCDATA) (parser-error port "[elementvalid] broken for " elem-gi " with char content only; unexpected token " term-token)) ; Do other validation of the element content (let ((seed (handle-start-tag (xml-token-head term-token) port entities namespaces preserve-ws? seed))) (loop port entities expect-eof? seed))) (else (parser-error port "XML [43] broken for " term-token)))))))) ))) )))) ; syntax: ssax:make-parser user-handler-tag user-handler-proc ... ; ; Create an XML parser, an instance of the XML parsing framework. ; This will be a SAX, a DOM, or a specialized parser depending ; on the supplied user-handlers. ; user-handler-tag is a symbol that identifies a procedural expression ; that follows the tag. Given below are tags and signatures of the ; corresponding procedures. Not all tags have to be specified. If some ; are omitted, reasonable defaults will apply. ; ; tag: DOCTYPE ; handler-procedure: PORT DOCNAME SYSTEMID INTERNAL-SUBSET? SEED ; If internal-subset? is #t, the current position in the port ; is right after we have read #\[ that begins the internal DTD subset. ; We must finish reading of this subset before we return ; (or must call skip-internal-subset if we aren't interested in reading it). ; The port at exit must be at the first symbol after the whole ; DOCTYPE declaration. ; The handler-procedure must generate four values: ; ELEMS ENTITIES NAMESPACES SEED ; See xml-decl::elems for ELEMS. It may be #f to switch off the validation. ; NAMESPACES will typically contain USER-PREFIXes for selected URI-SYMBs. ; The default handler-procedure skips the internal subset, ; if any, and returns (values #f '() '() seed) ; tag: UNDECL-ROOT ; handler-procedure: ELEM-GI SEED ; where ELEM-GI is an UNRES-NAME of the root element. This procedure ; is called when an XML document under parsing contains _no_ DOCTYPE ; declaration. ; The handler-procedure, as a DOCTYPE handler procedure above, ; must generate four values: ; ELEMS ENTITIES NAMESPACES SEED ; The default handler-procedure returns (values #f '() '() seed) ; tag: DECL-ROOT ; handler-procedure: ELEM-GI SEED ; where ELEM-GI is an UNRES-NAME of the root element. This procedure ; is called when an XML document under parsing does contains the DOCTYPE ; declaration. ; The handler-procedure must generate a new SEED (and verify ; that the name of the root element matches the doctype, if the handler ; so wishes). ; The default handler-procedure is the identity function. ; tag: NEW-LEVEL-SEED ; handler-procedure: see ssax:make-elem-parser, my-new-level-seed ; tag: FINISH-ELEMENT ; handler-procedure: see ssax:make-elem-parser, my-finish-element ; tag: CHAR-DATA-HANDLER ; handler-procedure: see ssax:make-elem-parser, my-char-data-handler ; tag: PI ; handler-procedure: see ssax:make-pi-parser ; The default value is '() ; The generated parser is a ; procedure PORT SEED ; This procedure parses the document prolog and then exits to ; an element parser (created by ssax:make-elem-parser) to handle ; the rest. ; ; [1] document ::= prolog element Misc* ; [22] prolog ::= XMLDecl? Misc* (doctypedec | Misc*)? ; [27] Misc ::= Comment | PI | S ; ; [28] doctypedecl ::= '<!DOCTYPE' S Name (S ExternalID)? S? ; ('[' (markupdecl | PEReference | S)* ']' S?)? '>' ; [29] markupdecl ::= elementdecl | AttlistDecl ; | EntityDecl ; | NotationDecl | PI ; | Comment ; ; This is ssax:make-parser with all the (specialization) handlers given ; as positional arguments. It is called by ssax:make-parser, see below (define-syntax ssax:make-parser/positional-args (syntax-rules () ((ssax:make-parser/positional-args *handler-DOCTYPE *handler-UNDECL-ROOT *handler-DECL-ROOT *handler-NEW-LEVEL-SEED *handler-FINISH-ELEMENT *handler-CHAR-DATA-HANDLER *handler-PI) (lambda (port seed) ; We must've just scanned the DOCTYPE token ; Handle the doctype declaration and exit to ; scan-for-significant-prolog-token-2, and eventually, to the ; element parser. (define (handle-decl port token-head seed) (or (eq? (string->symbol "DOCTYPE") token-head) (parser-error port "XML [22], expected DOCTYPE declaration, found " token-head)) (assert-curr-char ssax:S-chars "XML [28], space after DOCTYPE" port) (ssax:skip-S port) (let*-values (((docname) (ssax:read-QName port)) ((systemid) (and (ssax:ncname-starting-char? (ssax:skip-S port)) (ssax:read-external-id port))) ((internal-subset?) (begin (ssax:skip-S port) (eqv? #\[ (assert-curr-char '(#\> #\[) "XML [28], end-of-DOCTYPE" port)))) ((elems entities namespaces seed) (*handler-DOCTYPE port docname systemid internal-subset? seed)) ) (scan-for-significant-prolog-token-2 port elems entities namespaces seed))) ; Scan the leading PIs until we encounter either a doctype declaration ; or a start token (of the root element) ; In the latter two cases, we exit to the appropriate continuation (define (scan-for-significant-prolog-token-1 port seed) (let ((token (ssax:scan-Misc port))) (if (eof-object? token) (parser-error port "XML [22], unexpected EOF") (case (xml-token-kind token) ((PI) (let ((seed ((ssax:make-pi-parser *handler-PI) port (xml-token-head token) seed))) (scan-for-significant-prolog-token-1 port seed))) ((DECL) (handle-decl port (xml-token-head token) seed)) ((START) (let*-values (((elems entities namespaces seed) (*handler-UNDECL-ROOT (xml-token-head token) seed))) (element-parser (xml-token-head token) port elems entities namespaces #f seed))) (else (parser-error port "XML [22], unexpected markup " token)))))) ; Scan PIs after the doctype declaration, till we encounter ; the start tag of the root element. After that we exit ; to the element parser (define (scan-for-significant-prolog-token-2 port elems entities namespaces seed) (let ((token (ssax:scan-Misc port))) (if (eof-object? token) (parser-error port "XML [22], unexpected EOF") (case (xml-token-kind token) ((PI) (let ((seed ((ssax:make-pi-parser *handler-PI) port (xml-token-head token) seed))) (scan-for-significant-prolog-token-2 port elems entities namespaces seed))) ((START) (element-parser (xml-token-head token) port elems entities namespaces #f (*handler-DECL-ROOT (xml-token-head token) seed))) (else (parser-error port "XML [22], unexpected markup " token)))))) ; A procedure start-tag-head port elems entities namespaces ; preserve-ws? seed (define element-parser (ssax:make-elem-parser *handler-NEW-LEVEL-SEED *handler-FINISH-ELEMENT *handler-CHAR-DATA-HANDLER *handler-PI)) ; Get the ball rolling ... (scan-for-significant-prolog-token-1 port seed) )))) ; The following meta-macro turns a regular macro (with positional ; arguments) into a form with keyword (labeled) arguments. We later ; use the meta-macro to convert ssax:make-parser/positional-args into ; ssax:make-parser. The latter provides a prettier (with labeled ; arguments and defaults) interface to ; ssax:make-parser/positional-args ; ; ssax:define-labeled-arg-macro LABELED-ARG-MACRO-NAME ; (POS-MACRO-NAME ARG-DESCRIPTOR ...) ; expands into the definition of a macro ; LABELED-ARG-MACRO-NAME KW-NAME KW-VALUE KW-NAME1 KW-VALUE1 ... ; which, in turn, expands into ; POS-MACRO-NAME ARG1 ARG2 ... ; where each ARG1 etc. comes either from KW-VALUE or from ; the deafult part of ARG-DESCRIPTOR. ARG1 corresponds to the first ; ARG-DESCRIPTOR, ARG2 corresponds to the second descriptor, etc. ; Here ARG-DESCRIPTOR describes one argument of the positional macro. ; It has the form ; (ARG-NAME DEFAULT-VALUE) ; or ; (ARG-NAME) ; In the latter form, the default value is not given, so that the invocation of ; LABELED-ARG-MACRO-NAME must mention the corresponding parameter. ; ARG-NAME can be anything: an identifier, a string, or even a number. (define-syntax ssax:define-labeled-arg-macro (syntax-rules () ((ssax:define-labeled-arg-macro labeled-arg-macro-name (positional-macro-name (arg-name . arg-def) ...)) (define-syntax labeled-arg-macro-name (syntax-rules () ((labeled-arg-macro-name . kw-val-pairs) (letrec-syntax ((find (syntax-rules (arg-name ...) ((find k-args (arg-name . default) arg-name val . others) ; found arg-name among kw-val-pairs (next val . k-args)) ... ((find k-args key arg-no-match-name val . others) (find k-args key . others)) ((find k-args (arg-name default)) ; default must be here (next default . k-args)) ... )) (next ; pack the continuation to find (syntax-rules () ((next val vals key . keys) (find ((val . vals) . keys) key . kw-val-pairs)) ((next val vals) ; processed all arg-descriptors (rev-apply (val) vals)))) (rev-apply (syntax-rules () ((rev-apply form (x . xs)) (rev-apply (x . form) xs)) ((rev-apply form ()) form)))) (next positional-macro-name () (arg-name . arg-def) ...)))))))) ; The definition of ssax:make-parser (ssax:define-labeled-arg-macro ssax:make-parser (ssax:make-parser/positional-args (DOCTYPE (lambda (port docname systemid internal-subset? seed) (when internal-subset? (ssax:warn port "Internal DTD subset is not currently handled ") (ssax:skip-internal-dtd port)) (ssax:warn port "DOCTYPE DECL " docname " " systemid " found and skipped") (values #f '() '() seed) )) (UNDECL-ROOT (lambda (elem-gi seed) (values #f '() '() seed))) (DECL-ROOT (lambda (elem-gi seed) seed)) (NEW-LEVEL-SEED) ; required (FINISH-ELEMENT) ; required (CHAR-DATA-HANDLER) ; required (PI ()) )) (run-test (letrec ((simple-parser (lambda (str doctype-fn) (call-with-input-string str (lambda (port) ((ssax:make-parser NEW-LEVEL-SEED (lambda (elem-gi attributes namespaces expected-content seed) '()) FINISH-ELEMENT (lambda (elem-gi attributes namespaces parent-seed seed) (let ((seed (if (null? namespaces) (reverse seed) (cons (list '*NAMESPACES* namespaces) (reverse seed))))) (let ((seed (if (attlist-null? attributes) seed (cons (cons '@ (map (lambda (attr) (list (car attr) (cdr attr))) (attlist->alist attributes))) seed)))) (cons (cons elem-gi seed) parent-seed)))) CHAR-DATA-HANDLER (lambda (string1 string2 seed) (if (string-null? string2) (cons string1 seed) (cons* string2 string1 seed))) DOCTYPE (lambda (port docname systemid internal-subset? seed) (when internal-subset? (ssax:warn port "Internal DTD subset is not currently handled ") (ssax:skip-internal-dtd port)) (ssax:warn port "DOCTYPE DECL " docname " " systemid " found and skipped") (doctype-fn docname seed)) UNDECL-ROOT (lambda (elem-gi seed) (doctype-fn elem-gi seed)) ) port '()))))) (dummy-doctype-fn (lambda (elem-gi seed) (values #f '() '() seed))) (test (lambda (str doctype-fn expected) (cout nl "Parsing: " str nl) (let ((result (simple-parser (unesc-string str) doctype-fn))) (write result) (assert (equal? result expected))))) ) (test "<BR/>" dummy-doctype-fn '(('"BR"))) (assert (failed? (test "<BR>" dummy-doctype-fn '()))) (test "<BR></BR>" dummy-doctype-fn '(('"BR"))) (assert (failed? (test "<BR></BB>" dummy-doctype-fn '()))) (test " <A HREF='URL'> link <I>itlink </I> &amp;</A>" dummy-doctype-fn '(('"A" (@ ('"HREF" "URL")) " link " ('"I" "itlink ") " " "&" "amp;"))) (test " <A HREF='URL' xml:space='preserve'> link <I>itlink </I> &amp;</A>" dummy-doctype-fn '(('"A" (@ ('"HREF" "URL") (('"xml" . '"space") "preserve")) " link " ('"I" "itlink ") " " "&" "amp;"))) (test " <A HREF='URL' xml:space='preserve'> link <I xml:space='default'>itlink </I> &amp;</A>" dummy-doctype-fn '(('"A" (@ ('"HREF" "URL") (('"xml" . '"space") "preserve")) " link " ('"I" (@ (('"xml" . '"space") "default")) "itlink ") " " "&" "amp;"))) (test "<itemize><item>This is item 1 </item>%n<!-- Just:a comment --><item>Item 2</item>%n </itemize>" dummy-doctype-fn `(('"itemize" ('"item" "This is item 1 ") ,(unesc-string "%n") ('"item" "Item 2") ,(unesc-string "%n ")))) (test " <P><![CDATA[<BR>%n<![CDATA[<BR>]]>]]></P>" dummy-doctype-fn `(('"P" "<BR>" ,nl "<![CDATA[<BR>" "]]" "" ">"))) (test " <P><![CDATA[<BR>%r<![CDATA[<BR>]]>]]></P>" dummy-doctype-fn `(('"P" "<BR>" ,nl "<![CDATA[<BR>" "]]" "" ">"))) (test "<?xml version='1.0'?>%n%n<Reports TStamp='1'></Reports>" dummy-doctype-fn '(('"Reports" (@ ('"TStamp" "1"))))) (test "%n<?PI xxx?><!-- Comment %n -%r-->%n<?PI1 zzz?><T/>" dummy-doctype-fn '(('"T"))) (test "<!DOCTYPE T SYSTEM 'system1' ><!-- comment -->%n<T/>" (lambda (elem-gi seed) (assert (equal? elem-gi ''"T")) (values #f '() '() seed)) '(('"T"))) (test "<!DOCTYPE T PUBLIC '//EN/T' \"system1\" [ <!ELEMENT a 'aa'> ]>%n<?pi?><T/>" (lambda (elem-gi seed) (assert (equal? elem-gi ''"T")) (values #f '() '() seed)) '(('"T"))) (test "<BR/>" (lambda (elem-gi seed) (values '(('"BR" EMPTY ())) '() '() seed)) '(('"BR"))) (test "<BR></BR>" (lambda (elem-gi seed) (values '(('"BR" EMPTY ())) '() '() seed)) '(('"BR"))) (assert (failed? (test "<BR>aa</BR>" (lambda (elem-gi seed) (values '(('"BR" EMPTY ())) '() '() seed)) '()))) (test "<BR>aa</BR>" (lambda (elem-gi seed) (values '(('"BR" PCDATA ())) '() '() seed)) '(('"BR" "aa"))) (assert (failed? (test "<BR>a<I>a</I></BR>" (lambda (elem-gi seed) (values '(('"BR" PCDATA ())) '() '() seed)) '()))) (test "<BR>a<I>a</I></BR>" (lambda (elem-gi seed) (values '(('"BR" ANY ()) ('"I" PCDATA ())) '() '() seed)) '(('"BR" "a" ('"I" "a")))) (test "<DIV>Example: \"&example;\"</DIV>" (lambda (elem-gi seed) (values #f '((example . "<P>An ampersand (&) may be escaped numerically (&#38;) or with a general entity (&amp;).</P>")) '() seed)) '(('"DIV" "Example: \"" ('"P" "An ampersand (" "&" ") may be escaped numerically (" "&" "#38;) or with a general entity (" "&" "amp;).") "\""))) (test "<DIV>Example: \"&example;\" <P/></DIV>" (lambda (elem-gi seed) (values #f '(('"quote" . "<I>example:</I> ex") ('"example" . "<Q>"e;!</Q>?")) '() seed)) '(('"DIV" "Example: \"" ('"Q" ('"I" "example:") " ex" "!") "?" "\" " ('"P")))) (assert (failed? (test "<DIV>Example: \"&example;\" <P/></DIV>" (lambda (elem-gi seed) (values #f '(('"quote" . "<I>example:") ('"example" . "<Q>"e;</I>!</Q>?")) '() seed)) '()))) (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" (lambda (elem-gi seed) (values #f '() '() seed)) '((('"URI1" . '"DIV") (@ ('"B" "B") (('"URI1" . '"B") "A")) (*NAMESPACES* (('"A" '"URI1" . '"URI1") (*DEFAULT* '"URI1" . '"URI1"))) (('"URI1" . '"P") (*NAMESPACES* ((*DEFAULT* #f . #f) ('"A" '"URI1" . '"URI1") (*DEFAULT* '"URI1" . '"URI1"))) ('"BR" (*NAMESPACES* ((*DEFAULT* #f . #f) ('"A" '"URI1" . '"URI1") (*DEFAULT* '"URI1" . '"URI1")))))))) (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" (lambda (elem-gi seed) (values #f '() '((#f '"UA" . '"URI1")) seed)) '((('"UA" . '"DIV") (@ ('"B" "B") (('"UA" . '"B") "A")) (*NAMESPACES* (('"A" '"UA" . '"URI1") (*DEFAULT* '"UA" . '"URI1") (#f '"UA" . '"URI1"))) (('"UA" . '"P") (*NAMESPACES* ((*DEFAULT* #f . #f) ('"A" '"UA" . '"URI1") (*DEFAULT* '"UA" . '"URI1") (#f '"UA" . '"URI1"))) ('"BR" (*NAMESPACES* ((*DEFAULT* #f . #f) ('"A" '"UA" . '"URI1") (*DEFAULT* '"UA" . '"URI1") (#f '"UA" . '"URI1")))))))) ; uniqattr should fail (assert (failed? (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" (lambda (elem-gi seed) (values `(('"DIV" ANY (('"B" CDATA IMPLIED #f) (('"A" . '"B") CDATA IMPLIED #f) (('"C" . '"B") CDATA IMPLIED "xx") (('"xmlns" . '"C") CDATA IMPLIED "URI1") )) (('"A" . '"P") ANY ()) ('"BR" '"EMPTY" ())) '() '((#f '"UA" . '"URI1")) seed)) '()))) ; prefix C undeclared (assert (failed? (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" (lambda (elem-gi seed) (values '(('"DIV" ANY (('"B" CDATA IMPLIED #f) ('"xmlns" CDATA IMPLIED "URI1") (('"A" . '"B") CDATA IMPLIED #f) (('"C" . '"B") CDATA IMPLIED "xx") )) (('"A" . '"P") ANY ()) ('"BR" EMPTY ())) '() '((#f '"UA" . '"URI1")) seed)) '()))) ; contradiction to xmlns declaration (assert (failed? (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" (lambda (elem-gi seed) (values '(('"DIV" ANY (('"B" CDATA IMPLIED #f) ('"xmlns" CDATA FIXED "URI2") (('"A" . '"B") CDATA IMPLIED #f) )) (('"A" . '"P") ANY ()) ('"BR" EMPTY ())) '() '((#f '"UA" . '"URI1")) seed)) '()))) (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" (lambda (elem-gi seed) (values '(('"DIV" ANY (('"B" CDATA IMPLIED #f) ('"xmlns" CDATA FIXED "URI1") (('"A" . '"B") CDATA IMPLIED #f) )) (('"A" . '"P") ANY ()) ('"BR" EMPTY ())) '() '((#f '"UA" . '"URI1")) seed)) '((('"UA" . '"DIV") (@ ('"B" "B") (('"UA" . '"B") "A")) (*NAMESPACES* ((*DEFAULT* '"UA" . '"URI1") ('"A" '"UA" . '"URI1") (#f '"UA" . '"URI1"))) (('"UA" . '"P") (*NAMESPACES* ((*DEFAULT* #f . #f) (*DEFAULT* '"UA" . '"URI1") ('"A" '"UA" . '"URI1") (#f '"UA" . '"URI1"))) ('"BR" (*NAMESPACES* ((*DEFAULT* #f . #f) (*DEFAULT* '"UA" . '"URI1") ('"A" '"UA" . '"URI1") (#f '"UA" . '"URI1")))))))) (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" (lambda (elem-gi seed) (values '(('"DIV" ANY (('"B" CDATA IMPLIED #f) (('"A" . '"B") CDATA IMPLIED #f) (('"C" . '"B") CDATA IMPLIED "xx") (('"xmlns" . '"C") CDATA IMPLIED "URI2") )) (('"A" . '"P") ANY ()) ('"BR" EMPTY ())) '() '((#f '"UA" . '"URI1")) seed)) '((('"UA" . '"DIV") (@ ('"B" "B") (('"UA" . '"B") "A") (('"URI2" . '"B") "xx")) (*NAMESPACES* ((*DEFAULT* '"UA" . '"URI1") ('"A" '"UA" . '"URI1") ('"C" '"URI2" . '"URI2") (#f '"UA" . '"URI1"))) (('"UA" . '"P") (*NAMESPACES* ((*DEFAULT* #f . #f) (*DEFAULT* '"UA" . '"URI1") ('"A" '"UA" . '"URI1") ('"C" '"URI2" . '"URI2") (#f '"UA" . '"URI1"))) ('"BR" (*NAMESPACES* ((*DEFAULT* #f . #f) (*DEFAULT* '"UA" . '"URI1") ('"A" '"UA" . '"URI1") ('"C" '"URI2" . '"URI2") (#f '"UA" . '"URI1")))))))) )) ;======================================================================== ; Highest-level parsers: XML to SXML ; ; First, a few utility procedures that turned out useful ; ssax:reverse-collect-str LIST-OF-FRAGS -> LIST-OF-FRAGS ; given the list of fragments (some of which are text strings) ; reverse the list and concatenate adjacent text strings. ; We can prove from the general case below that if LIST-OF-FRAGS ; has zero or one element, the result of the procedure is equal? ; to its argument. This fact justifies the shortcut evaluation below. (define (ssax:reverse-collect-str fragments) (cond ((null? fragments) '()) ; a shortcut ((null? (cdr fragments)) fragments) ; see the comment above (else (let loop ((fragments fragments) (result '()) (strs '())) (cond ((null? fragments) (if (null? strs) result (cons (string-concatenate/shared strs) result))) ((string? (car fragments)) (loop (cdr fragments) result (cons (car fragments) strs))) (else (loop (cdr fragments) (cons (car fragments) (if (null? strs) result (cons (string-concatenate/shared strs) result))) '()))))))) ; ssax:reverse-collect-str-drop-ws LIST-OF-FRAGS -> LIST-OF-FRAGS ; given the list of fragments (some of which are text strings) ; reverse the list and concatenate adjacent text strings. ; We also drop "unsignificant" whitespace, that is, whitespace ; in front, behind and between elements. The whitespace that ; is included in character data is not affected. ; We use this procedure to "intelligently" drop "insignificant" ; whitespace in the parsed SXML. If the strict compliance with ; the XML Recommendation regarding the whitespace is desired, please ; use the ssax:reverse-collect-str procedure instead. (define (ssax:reverse-collect-str-drop-ws fragments) (cond ((null? fragments) '()) ; a shortcut ((null? (cdr fragments)) ; another shortcut (if (and (string? (car fragments)) (string-whitespace? (car fragments))) '() fragments)) ; remove trailing ws (else (let loop ((fragments fragments) (result '()) (strs '()) (all-whitespace? #t)) (cond ((null? fragments) (if all-whitespace? result ; remove leading ws (cons (string-concatenate/shared strs) result))) ((string? (car fragments)) (loop (cdr fragments) result (cons (car fragments) strs) (and all-whitespace? (string-whitespace? (car fragments))))) (else (loop (cdr fragments) (cons (car fragments) (if all-whitespace? result (cons (string-concatenate/shared strs) result))) '() #t))))))) ; procedure: ssax:xml->sxml PORT NAMESPACE-PREFIX-ASSIG ; ; This is an instance of a SSAX parser above that returns an SXML ; representation of the XML document to be read from PORT. ; NAMESPACE-PREFIX-ASSIG is a list of (USER-PREFIX . URI-STRING) ; that assigns USER-PREFIXes to certain namespaces identified by ; particular URI-STRINGs. It may be an empty list. ; The procedure returns an SXML tree. The port points out to the ; first character after the root element. (define (ssax:xml->sxml port namespace-prefix-assig) (letrec ((namespaces (map (lambda (el) (cons* #f (car el) (ssax:uri-string->symbol (cdr el)))) namespace-prefix-assig)) (RES-NAME->SXML (lambda (res-name) (string->symbol (string-append (symbol->string (car res-name)) ":" (symbol->string (cdr res-name)))))) ) (let ((result (reverse ((ssax:make-parser NEW-LEVEL-SEED (lambda (elem-gi attributes namespaces expected-content seed) '()) FINISH-ELEMENT (lambda (elem-gi attributes namespaces parent-seed seed) (let ((seed (ssax:reverse-collect-str seed)) (attrs (attlist-fold (lambda (attr accum) (cons (list (if (symbol? (car attr)) (car attr) (RES-NAME->SXML (car attr))) (cdr attr)) accum)) '() attributes))) (cons (cons (if (symbol? elem-gi) elem-gi (RES-NAME->SXML elem-gi)) (if (null? attrs) seed (cons (cons '@ attrs) seed))) parent-seed))) CHAR-DATA-HANDLER (lambda (string1 string2 seed) (if (string-null? string2) (cons string1 seed) (cons* string2 string1 seed))) DOCTYPE (lambda (port docname systemid internal-subset? seed) (when internal-subset? (ssax:warn port "Internal DTD subset is not currently handled ") (ssax:skip-internal-dtd port)) (ssax:warn port "DOCTYPE DECL " docname " " systemid " found and skipped") (values #f '() namespaces seed)) UNDECL-ROOT (lambda (elem-gi seed) (values #f '() namespaces seed)) PI ((*DEFAULT* . (lambda (port pi-tag seed) (cons (list '*PI* pi-tag (ssax:read-pi-body-as-string port)) seed)))) ) port '())))) (cons '*TOP* (if (null? namespace-prefix-assig) result (cons (list '@ (cons '*NAMESPACES* (map (lambda (ns) (list (car ns) (cdr ns))) namespace-prefix-assig))) result))) ))) ; For backwards compatibility (define SSAX:XML->SXML ssax:xml->sxml) ; a few lines of validation code (run-test (letrec ((test (lambda (str namespace-assig expected-res) (newline) (display "input: ") (write (unesc-string str)) (newline) (display "Result: ") (let ((result (call-with-input-string (unesc-string str) (lambda (port) (ssax:xml->sxml port namespace-assig))))) (pp result) (assert (equal_? result expected-res)))))) (test " <BR/>" '() '(*TOP* (BR))) (test "<BR></BR>" '() '(*TOP* (BR))) (test " <BR CLEAR='ALL'%nCLASS='Class1'/>" '() '(*TOP* (BR (@ (CLEAR "ALL") (CLASS "Class1"))))) (test " <A HREF='URL'> link <I>itlink </I> &amp;</A>" '() '(*TOP* (A (@ (HREF "URL")) " link " (I "itlink ") " &"))) (test " <A HREF='URL' xml:space='preserve'> link <I>itlink </I> &amp;</A>" '() '(*TOP* (A (@ (xml:space "preserve") (HREF "URL")) " link " (I "itlink ") " &"))) (test " <A HREF='URL' xml:space='preserve'> link <I xml:space='default'>itlink </I> &amp;</A>" '() '(*TOP* (A (@ (xml:space "preserve") (HREF "URL")) " link " (I (@ (xml:space "default")) "itlink ") " &"))) (test " <P><?pi1 p1 content ?>?<?pi2 pi2? content? ??></P>" '() '(*TOP* (P (*PI* pi1 "p1 content ") "?" (*PI* pi2 "pi2? content? ?")))) (test " <P>some text <![CDATA[<]]>1%n"<B>strong</B>"%r</P>" '() `(*TOP* (P ,(unesc-string "some text <1%n\"") (B "strong") ,(unesc-string "\"%n")))) (test " <P><![CDATA[<BR>%n<![CDATA[<BR>]]>]]></P>" '() `(*TOP* (P ,(unesc-string "<BR>%n<![CDATA[<BR>]]>")))) ; (test "<T1><T2>it's%r%nand that%n</T2>%r%n%r%n%n</T1>" '() ; '(*TOP* (T1 (T2 "it's%nand that%n") "%n%n%n"))) (test "<T1><T2>it's%r%nand that%n</T2>%r%n%r%n%n</T1>" '() `(*TOP* (T1 (T2 ,(unesc-string "it's%nand that%n")) ,(unesc-string "%n%n%n")))) (test "<T1><T2>it's%rand that%n</T2>%r%n%r%n%n</T1>" '() `(*TOP* (T1 (T2 ,(unesc-string "it's%nand that%n")) ,(unesc-string "%n%n%n")))) (test "<!DOCTYPE T SYSTEM 'system1' ><!-- comment -->%n<T/>" '() '(*TOP* (T))) (test "<?xml version='1.0'?>%n<WEIGHT unit=\"pound\">%n<NET certified='certified'> 67 </NET>%n<GROSS> 95 </GROSS>%n</WEIGHT>" '() `(*TOP* (*PI* xml "version='1.0'") (WEIGHT (@ (unit "pound")) ,nl (NET (@ (certified "certified")) " 67 ") ,nl (GROSS " 95 ") ,nl) )) ; (test "<?xml version='1.0'?>%n<WEIGHT unit=\"pound\">%n<NET certified='certified'> 67 </NET>%n<GROSS> 95 </GROSS>%n</WEIGHT>" '() ; '(*TOP* (*PI* xml "version='1.0'") (WEIGHT (@ (unit "pound")) ; "%n" (NET (@ (certified "certified")) " 67 ") ; "%n" (GROSS " 95 ") "%n") ; )) (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" '() '(*TOP* (URI1:DIV (@ (URI1:B "A") (B "B")) (URI1:P (BR))))) (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" '((UA . "URI1")) '(*TOP* (@ (*NAMESPACES* (UA "URI1"))) (UA:DIV (@ (UA:B "A") (B "B")) (UA:P (BR))))) ; A few tests from XML Namespaces Recommendation (test (string-append "<x xmlns:edi='http://ecommerce.org/schema'>" "<!-- the 'taxClass' attribute's ns http://ecommerce.org/schema -->" "<lineItem edi:taxClass='exempt'>Baby food</lineItem>" nl "</x>") '() `(*TOP* (x (lineItem (@ (http://ecommerce.org/schema:taxClass "exempt")) "Baby food") ,nl))) (test (string-append "<x xmlns:edi='http://ecommerce.org/schema'>" "<!-- the 'taxClass' attribute's ns http://ecommerce.org/schema -->" "<lineItem edi:taxClass='exempt'>Baby food</lineItem>" "</x>") '((EDI . "http://ecommerce.org/schema")) '(*TOP* (@ (*NAMESPACES* (EDI "http://ecommerce.org/schema"))) (x (lineItem (@ (EDI:taxClass "exempt")) "Baby food")))) (test (string-append "<bk:book xmlns:bk='urn:loc.gov:books' " "xmlns:isbn='urn:ISBN:0-395-36341-6'>" "<bk:title>Cheaper by the Dozen</bk:title>" "<isbn:number>1568491379</isbn:number></bk:book>") '() '(*TOP* (urn:loc.gov:books:book (urn:loc.gov:books:title "Cheaper by the Dozen") (urn:ISBN:0-395-36341-6:number "1568491379")))) (test (string-append "<!-- initially, the default namespace is 'books' -->" "<book xmlns='urn:loc.gov:books' " "xmlns:isbn='urn:ISBN:0-395-36341-6'>" "<title>Cheaper by the Dozen" "1568491379" "" "" "

" "This is a funny book!" "

" "
" "") '() '(*TOP* (urn:loc.gov:books:book (urn:loc.gov:books:title "Cheaper by the Dozen") (urn:ISBN:0-395-36341-6:number "1568491379") (urn:loc.gov:books:notes (urn:w3-org-ns:HTML:p "This is a " (urn:w3-org-ns:HTML:i "funny") " book!"))))) (test (string-append "" "" "" "" "" "" "" "" "" "" "
NameOriginDescription
HuntsmanBath, UK" "
BitterFuggles" "Wonderful hop, light alcohol, good summer beer" "Fragile; excessive variance pub to pub" "
" "
" "
") '((html . "http://www.w3.org/TR/REC-html40")) '(*TOP* (@ (*NAMESPACES* (html "http://www.w3.org/TR/REC-html40"))) (Beers (html:table (html:th (html:td "Name") (html:td "Origin") (html:td "Description")) (html:tr (html:td (brandName "Huntsman")) (html:td (origin "Bath, UK")) (html:td (details (class "Bitter") (hop "Fuggles") (pro "Wonderful hop, light alcohol, good summer beer") (con "Fragile; excessive variance pub to pub")))))))) (test (string-append "" "Layman, A" "33B" "Check Status" "1997-05-24T07:55:00+1") '((HTML . "http://www.w3.org/TR/REC-html40")) '(*TOP* (@ (*NAMESPACES* (HTML "http://www.w3.org/TR/REC-html40"))) (RESERVATION (NAME (@ (HTML:CLASS "largeSansSerif")) "Layman, A") (SEAT (@ (HTML:CLASS "largeMonotype") (CLASS "Y")) "33B") (HTML:A (@ (HREF "/cgi-bin/ResStatus")) "Check Status") (DEPARTURE "1997-05-24T07:55:00+1")))) ; Part of RDF from the XML Infoset (test (string-concatenate/shared '( "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "")) '((RDF . "http://www.w3.org/1999/02/22-rdf-syntax-ns#") (RDFS . "http://www.w3.org/2000/01/rdf-schema#") (ISET . "http://www.w3.org/2001/02/infoset#")) '(*TOP* (@ (*NAMESPACES* (RDF "http://www.w3.org/1999/02/22-rdf-syntax-ns#") (RDFS "http://www.w3.org/2000/01/rdf-schema#") (ISET "http://www.w3.org/2001/02/infoset#"))) (*PI* xml "version='1.0' encoding='utf-8' standalone='yes'") (RDF:RDF (RDFS:Class (@ (ID "Boolean"))) (ISET:Boolean (@ (ID "Boolean.true"))) (ISET:Boolean (@ (ID "Boolean.false"))) (RDFS:Class (@ (ID "InfoItem"))) (RDFS:Class (@ (RDFS:subClassOf "#InfoItem") (ID "Document"))) (RDFS:Class (@ (RDFS:subClassOf "#InfoItem") (ID "Element"))) (RDFS:Class (@ (RDFS:subClassOf "#InfoItem") (ID "Attribute"))) (RDFS:Class (@ (RDFS:subClassOf "http://www.w3.org/1999/02/22-rdf-syntax-ns#Bag") (ID "InfoItemSet"))) (RDFS:Class (@ (RDFS:subClassOf "#InfoItemSet") (ID "AttributeSet"))) (RDFS:Property (@ (ID "allDeclarationsProcessed")) (RDFS:domain (@ (resource "#Document"))) (RDFS:range (@ (resource "#Boolean")))) (RDFS:Property (@ (ID "attributes")) (RDFS:domain (@ (resource "#Element"))) (RDFS:range (@ (resource "#AttributeSet"))))))) ; Part of RDF from RSS of the Daemon News Mall (test (string-concatenate/shared (list-intersperse '( "" "" "Daemon News Mall" "http://mall.daemonnews.org/" "Central source for all your BSD needs" "" "" "Daemon News Jan/Feb Issue NOW Available! Subscribe $24.95" "http://mall.daemonnews.org/?page=shop/flypage&product_id=880" "" "" "The Design and Implementation of the 4.4BSD Operating System $54.95" "http://mall.daemonnews.org/?page=shop/flypage&product_id=912&category_id=1761" "" "") (string #\newline) )) '((RDF . "http://www.w3.org/1999/02/22-rdf-syntax-ns#") (RSS . "http://my.netscape.com/rdf/simple/0.9/") (ISET . "http://www.w3.org/2001/02/infoset#")) `(*TOP* (@ (*NAMESPACES* (RDF "http://www.w3.org/1999/02/22-rdf-syntax-ns#") (RSS "http://my.netscape.com/rdf/simple/0.9/") (ISET "http://www.w3.org/2001/02/infoset#"))) (*PI* xml "version='1.0'") (RDF:RDF ,nl (RSS:channel ,nl (RSS:title "Daemon News Mall") ,nl (RSS:link "http://mall.daemonnews.org/") ,nl (RSS:description "Central source for all your BSD needs") ,nl) ,nl (RSS:item ,nl (RSS:title "Daemon News Jan/Feb Issue NOW Available! Subscribe $24.95") ,nl (RSS:link "http://mall.daemonnews.org/?page=shop/flypage&product_id=880") ,nl) ,nl (RSS:item ,nl (RSS:title "The Design and Implementation of the 4.4BSD Operating System $54.95") ,nl (RSS:link "http://mall.daemonnews.org/?page=shop/flypage&product_id=912&category_id=1761") ,nl) ,nl))) (test (string-concatenate/shared '("" "" "111730Z 111818" "" "31010KT P6SM FEW030" "" "" "29016KT P6SM FEW040" "" "" "29010KT P6SM SCT200" "VRB05KT" "" "")) '() '(*TOP* (Forecasts (@ (TStamp "958082142")) (TAF (@ (TStamp "958066200") (SName "KMRY, MONTEREY PENINSULA") (LatLon "36.583, -121.850") (BId "724915")) (VALID (@ (TRange "958068000, 958154400")) "111730Z 111818") (PERIOD (@ (TRange "958068000, 958078800")) (PREVAILING "31010KT P6SM FEW030")) (PERIOD (@ (Title "FM2100") (TRange "958078800, 958104000")) (PREVAILING "29016KT P6SM FEW040")) (PERIOD (@ (Title "FM0400") (TRange "958104000, 958154400")) (PREVAILING "29010KT P6SM SCT200") (VAR (@ (Title "BECMG 0708") (TRange "958114800, 958118400")) "VRB05KT")))))) )) (run-test (newline) (display "All tests passed") (newline) ) PK/F2]iyoosxml/apply-templates.scmnu[;;;; (sxml apply-templates) -- xslt-like transformation for sxml ;;;; ;;;; Copyright (C) 2009 Free Software Foundation, Inc. ;;;; Copyright 2004 by Andy Wingo . ;;;; Written 2003 by Oleg Kiselyov as apply-templates.scm. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public ;;;; License as published by the Free Software Foundation; either ;;;; version 3 of the License, or (at your option) any later version. ;;;; ;;;; This library is distributed in the hope that it will be useful, ;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;;;; Lesser General Public License for more details. ;;;; ;;;; You should have received a copy of the GNU Lesser General Public ;;;; License along with this library; if not, write to the Free Software ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ;;;; ;;; Commentary: ;; ;; Pre-order traversal of a tree and creation of a new tree: ;; ;;@smallexample ;; apply-templates:: tree x -> ;;@end smallexample ;; where ;;@smallexample ;; ::= (